Text Detection and Character Recognition in Images with Neural Networks

Abstract

This dissertation presents a Deep Learning approach for recognizing the handwritten Indian Devanagari characters from images. With the advancement in the technology and massive computational power of the GPUs, it has become possible to achieve human-level intelligence in machines to some extent. Devanagari script is written from left to right and a horizontal line at the top, which is known as the Shirorekha, joins all the characters in a word together. The Devanagari script is a constituent of Sanskrit script, which is used in other languages in India such as Nepali, Gujarati, and Marathi. Artificial Neural Networks are designed based on the biological neural networks in humans. Although researchers have already achieved a lot of success in English character recognition but nowadays Devanagari character recognition task has also gained the attention of many researchers. In this dissertation, Convolutional Neural Network (CNN) is used for feature extraction and offline Devanagari handwritten character recognition. Convolutional Neural Networks can be trained to learn the complex patterns from the images. Handwritten character recognition task is somewhat difficult due to the different writing styles of each person. The features of the input image samples are extracted with the help of convolutional and pooling layers. Softmax classifier is used to generate the class scores for final classification. The error i.e. the difference between the ground truth label and the predicted label is computed with a categorical cross-entropy loss function. The proposed approach for character recognition achieves an accuracy of 98.74% on handwritten Devanagari character dataset. Next, we prepared a database of Devanagari words and their corresponding labels. We have segmented the Devanagari words of our database into characters. Then, we used our CNN model trained on the handwritten Devanagari characters to generate predictions on the segmented Devanagari characters of our database. The CNN output is then used as an input sequence for the Language Transliteration system and the target sequence are the corresponding labels of our Devanagari words. RNN based Encoder-Decoder model is used for the Language Transliteration task. The performance of the Language Transliteration system is evaluated with different RNNs viz. SimpleRNN, LSTM and GRU.