Neural Network Implementation Using CUDA

Abstract

With the invention of cheaper GPU hardware and easy to use Application Programming Interface (API) like NVIDIA CUDA, which is an API developed by NVIDIA for the parallel computation on the GPU. Since for the large neural network, there is a need of powerful machines which will perform computations. But those high-performance machines are not available easily in day to day life. Any programmer who wish to compute a large neural network will seldom find work time on the high-performance machines. So, the Graphics Processing Unit which comes with the personal computer is not use for graphics purpose only. We can harness the power of that GPU to perform the complex computations using parallel programming. NVIDIA provides the API for the programming on GPU which follow the CUDA (Compute Unified Device Architecture). In this thesis an Artificial Neural network consisting of Linear Layer, Sigmoid Layer and RELU Layer is implemented using NVIDIA CUDA programming model. This Artificial Neural Network is developed for Binary Classification Problem which classifies the two-dimensional data points. A total of 21 batches of two-dimensional data points with 100 data points in each batch is fed to the network. Here, 20 batches are used to train the network and last batch is used for testing. This Artificial Neural Network running on NVIDIA GeForce RTX 2080Ti GPU gives better performance from equivalent CPU. The results demonstrate that the CUDA improves performance compared to the equivalent CPU