Image Translation and Super Resolution using Generative Adversarial Networks

Abstract

With plethora of development in the field of artificial intelligence, the use of forged images has increased manifold in deep learning algorithms. Previously, they were widely used in cyber-crimes and treachery. However, the use of forged images has gone beyond malpractices and thievery. Forged images are now produced for marketing of products, testing of algorithms, training of deep learning algorithms for self-driven vehicles and drones, etc. For generation of forged images, generative adversarial networks (GAN) are the most widely used algorithm today. Introduced by Ian Goodfellow in the year 2014, GANs have seen rapid development in the past five years. Various new models of GAN have been produced for different applications like conditional image generation, image-to-image translation, single image super-resolution, etc. With so many models GANs have been applied to different fields and expanding their scope beyond image generation. One such application of GAN is image translation, which has proven useful in analysis of new models, translation of medical images from one domain to another and generation of new datasets. The proposed work in this dissertation is based on two applications of GAN. The first application performs image translation on disjoint and unpaired images of different bird species from CBNWI-50. CBNWI-50 is deep learning bird image dataset comprising 50 species and 5102 original images of birds commonly found in north western region of Indian subcontinent. The resolution of the translated images was improved using SR GAN which was pre-trained on CBNWI-20, DIV2k and Flickr2k datasets. The second application was developed to obtain CT scan images of breast cancer tumours using PET scan images and vice-versa. Cross domain, unpaired images from ACRIN FLT Breast dataset were used for translation using Cycle GAN. To improve the resolution of translated images, SR GAN was used. In order to preserve the size and position of cancer tumours in the translated image, a U-Net structure was used as a feature extractor. The simulations were carried out on python 3.6 environment using the Tensorflow and Pytorch backend on NVIDIA CUDA 9.1 graphics driver. The simulation results on both bird species and medical images were evaluated using performance parameters like PSNR, SSIM, MSE, MAE and FID score. An improvement of 12%-13% in FID score was observed on bird species in comparison to existing work. The images generated by the algorithm were also evaluated from a human perspective in order to determine their precision. However, an improvement of 5% in PSNR was achieved in medical image translation. In future, the image translation model can be further improved by using progressively growing GAN (Pro GAN).