Development of an Efficient Defogging Framework for Road Safety

Abstract

In the early morning and the late night of winter season, the majority of road accidents occur every year despite less traffic, due to the poor visibility of drivers in the presence of air pollutants such as fog, haze, smog, etc. By improving the visibility of drivers, precious lives can be saved. The existing visibility restoration algorithms available in the literature suffer from one or more poor qualitative measures such as edge distortion, color distortion, texture distortion, halo effects, excessive lighting, gradient artefacts, etc. Present thesis is an attempt to propose new visibility restoration algorithms and improvement in existing algorithms, for enhancement of drivers' visual range to save human lives. Initially, a new real time roadside database called hazy unpaired dataset for road safety (HUDRS) has been proposed. After the dataset acquisition, four visibility restoration algorithms namely, Wavelet based multiscale convolution neural network (Hybrid CNN); a combination of deep learning architecture called residual regression network (RRNet) and morphological erosion; a combination of gush enhancer-based autoencoder (Aethra-Net) and vessel enhancement-based filter; and a bounding function for gray-world kernel prior (BFGKP), have been proposed for the estimation of image restoration parameters, i.e. atmospheric light and transmission map. A conventional optimization technique called grey wolf optimization has also been utilized for single image dehazing. Moreover, a hardware set-up has been presented to accomplish real-time video dehazing using Aethra-Net. In addition to that, BFGKP has been used to obtain results for real-time dehazing using MATLAB Mobile Application designed by Mathwork, Inc. Various visibility restoration challenges such as computation time, fog density, and parameter tuning have been considered while implementing this framework for real-time application. Most of the existing hazy datasets such as FRIDA, FRIDA2, 0-HAZE, I-HAZE, NH-HAZE, RESIDE, etc., consists of synthetic foggy images produced using either software or artificial fog generating machine. A real-time database called HUDRS has been prepared which consists of foggy and fog-free images captured during morning/evening and day time, respectively. All the images have been captured with manual adjustment of Canon Power Shot SX400 IS camera under natural environmental conditions. It depicts 522 hazy and 1050 haze-free real-time roadside images with a resolution of 4608x3456. Further, each clear image has been processed with haze of different intensities, producing a total of 2088 synthetic hazy images. The performance of the existing and proposed datasets has been evaluated in terms of several parameters namely, blind/referenceless image spatial quality evaluator (BRISQUE), brightness (/3), percentage of saturated pixels (I?), structure similarity index metric (SSIM), perceptionbased image quality evaluator (PIQE), peak signal-to-noise ratio (PSNR), and image entropy (1E), by using existing prior-based, metaheuristic-based (i.e., grey-wolf optimization), and deep learning-based dehazing algorithms. Experimental results reveal that they work differently in the real-time scenarios due to the presence of uncertainty such as area of frame and fog density in the real-world. A novel single image dehazing algorithm called hybrid CNN has been proposed that uses a multi-scale convolution neural network and various filters namely, retinex, wavelet, and inverted wavelet, to process hazy images. The wavelet filter has been used to enhance the features of non-sky regions. Similarly, the inverted-wavelet filter has been used to enhance the features of sky regions. The wavelet-transformed and inverted wavelet-transformed images have been processed with fine-scale network and coarsescale network, respectively, to obtain two separate feature maps. The airlight has been estimated by using 2D order statistic filter. The combined maps and estimated airlight have been used to produce the optimized transmission map for hazy image restoration. The proposed approach has been found superior to existing dehazing techniques in terms of fog aware density evaluator (FADE) and naturalness image quality evaluator (NIQE). A new software-based video dehazing architecture called Aethra-Net has been proposed. It consists of gush enhancer-based autoencoder and a vessel enhancementbased filter, to estimate the transmission map. The multiple blocks of ResNet-101 have been employed to overcome vanishing gradient problem. The vessel enhancement-based filter has been used to emphasize the objects in tubular structures. The refined Aethra- Net has been obtained by including all significant features from both feature maps (i.e., estimated transmission map and vessel enhancement-based map). The airlight has been estimated using mean filter. The processed frames have been restored in the same sequence as the original video. The performance analysis demonstrates that the proposed Aethra-Net has outperformed the existing techniques in terms of BRISQUE by 15.29%. Moreover, in the proposed system, improved results for SS/M and PSNR-based performance metrics have been obtained. The average dehazing time of 0.11 seconds has been achieved along with the updated airlight for each frame. vi In another work, a novel bounding function for gray-world kernel prior (BFGKP) has been proposed to obtain the clear images from real-time foggy images. The grayworld kernel prior has been used to estimate transmission map, and 2D order statistic filter has been used to estimate airlight. The flickering effects and edge distortion have been minimized using the proposed algorithm. The proposed BFGKP has been found superior than existing algorithms in 92.59% images when FADE is calculated. Further, BFGKP has been used in a real-time dehazing framework comprising a GoPro Hero 7 camera to capture the images, a display screen or a system to view the output, a Camlink to transfer the frames captured by GoPro in 4k resolution, and a bluerigger chord to setup connectivity. In addition to that, the proposed algorithm has been used in a cloud computing-based real-time defogging process, using MATLAB mobile application. This application is accessible through the Google play store for android-powered devices. The computation time of the proposed model is 7.68 times less than the existing techniques. Further, a new algorithm to overcome the effect of smog (fog + smoke) on the roadside images has been proposed. In the proposed algorithm, to estimate the transmission map for single image desmogging, a residual regression network (RRNet) has been designed. In RRNet, the shallow features have been extracted using convolution function, which is followed by reduction in the thickness of smog to find the depth of image. To make the network stable and faster, batch-normalization function has been used. The halo effect has also been reduced in the proposed algorithm. The transmission map has been refined by using morphological erosion operation. The atmospheric light has been estimated using a 2D order statistic filter. A comparative analysis has been drawn that reveals the supremacy of proposed RRNet over existing desmogging techniques in terms of PIQE, BRISQUE, and FADE, by 8.05%, 2.64%, and 2.02%, respectively.