Please use this identifier to cite or link to this item:
Title: Tunneling Effect Mitigation through Artificial Neural Network based Head Up Display Switching System
Authors: Karar, Vinod
Supervisor: Ghosh, Smarajit
Keywords: Attention Tunelling;Head up display;Situation awareness;ANN
Issue Date: 29-Aug-2014
Abstract: Traditional aircraft cockpit contains a host of display systems with vital flight information like airspeed, artificial horizon, navigation, radar display, altitude, angle of attack, etc. displayed in different formats on separate instruments panels in the cockpit display suite. Such kind of cockpit arrangement requires the pilot to split his attention between outside world and different instrument panels. In fast-moving aircraft, flying close to the ground, operational environment changes so rapidly that pilot has little time to look down at head-down displays to obtain aircraft flight status information. This degrades his situation awareness. Pilot has to cope with continual eye adjustments (focus, luminance etc.) required in changing his line of sight between various displays and the outside world. This results in longer reaction times, pilot fatigue and decreased efficiency. In order to facilitate the view of all these displays without having to divert attention, display systems like head-up display (HUD) and helmet-mounted display (HMD) have been developed. The primary role of HUD is to provide flight, navigation, and guidance information to the pilot in forward field of view on a transparent screen known as beam combiner (hereafter referred in thesis as combiner). Its use avoids need of splitting pilot’s attention between aircraft and outside world events which facilitates instant decision making. Main advantages of HUD as compared to head-down displays (HDD) include reduced scanning distance between instrument panels/gauges and outside world, improved situation awareness (SA) of outside world due to more visual attention to outside world, less head-down and look around time as well as less visual misaccommodation due to collimation principle of HUD. Therefore, HUD theoretically allows for optimal control of an aircraft through simultaneous scanning of both instrument data as well as out-of-the-window (OTW) scene. Although HUD improves flight performance, there are perceptual and cognitive issues that need to be addressed. There are number of issues related to distribution of pilot’s near and far domain attentional resources because of the compellingness of symbology elements on HUD. The phenomenon is regarded as attention or cognitive capture. HUDs may decrease pilot’s SA in tasks that require continuous monitoring of information in the environment. In extreme cases, HUD lowers SA to an extent that pilot may fail to detect potentially critical discrete events in the environment. As a practice, pilot views aircraft information, flight information and outside world as per situation requirement and not in a sequential manner. Data displayed on HUD may tunnel pilot's attention which may result in failure to notice events and objects other than those presented on HUD display. Thus, HUD results in formation of an attentional trap drawing pilot’s information processing resources to HUD and slowing down processing of external events. Attention tunneling is caused by various parameters like clutter, information and work overload, misaccommodation, misconvergence, symbol format and location, symbol salience and clutter, limited field of view (FOV) and few others. Among other factors which may significantly affect attention tunneling are: relative HUD symbology luminance (SL), ambient luminance (AL) and symbology luminance non-uniformity (NU). Various attention optimization measures have been proposed over the years like superimposed and scene-linked symbology, use of peripheral symbology, synthetic vision, use of few prevention technologies and more practice of using HUD and actual flight scenario, NASA’s runway incursion prevention system (RIPS), synthetic vision system (SVS) , etc. However, such studies address only a single source of attention tunneling at any given time. Exclusive studies have been performed in this work to understand the effect of key factors viz. limiting FOV and luminance factors in contributing attention capture, also known as tunneling. The individual effect of these parameters along with their interaction effect has been studied using statistical tools. The methods used for the purpose basically relate to inferential statistics domain. Application of paired t-test over experimental data, spanning luminance range from 50 cd/m² to 30,000 cd/m², established that luminance affects level of event detection on HUD symbology as well as outside scene significantly. Further, p-value found through ANOVA showed that percentage of event detection gets significantly affected due to AL and SL both. It could also be inferred from the results that non-uniformity of HUD display causes differential luminance across the HUD display area. Experimental studies revealed that (i) at higher AL, NU causes more degradation in HUD event detection as compared to outside events and (ii) at low ambient lighting conditions, degradation in both the events is significant with HUD event detection getting affected more adversely. It was also observed that at lower AL, prominence of SL variation forces pilot to get engaged in HUD events and in the process he also loses focus on outside events. Another set of experiments conducted to understand effect of limiting FOV due to combiner structure showed that combiner frame provide obscuration in front view of the pilot in total field of view (TFOV) as well as in instantaneous field of view (IFOV). The angle of combiner frame structure and its width present different degrees of obscuration to the pilot within head motion box (HMB). These limitations make pilot compromise simultaneous attention on outside events and aircraft events as he has to adjust his head position to view the obscured part of the outside world. The net result is obscuration of pilot’s forward view of outside world suggesting that optimized frame thickness and inclination angle is essential for minimizing tunneled vision through HUD. This work reports a new approach to detect and mitigate attention tunneling taking place while use of HUD in aircrafts. The detection mechanism developed is based on fuzzy decision making using texture features of image extracted from HUD charge coupled device (CCD) camera video. Attention tunneling mitigation is achieved through development of Assistive Attention Tunneling Mitigation system. Texture analysis employed for detection of attention tunneling utilized composite image comprising forward view and symbology captured by HUD camera. Texture analysis could reveal discriminating features necessary to classify tunneled and the normal HUD display. The GLCM features like contrast, homogeneity and correlation of image were used for HUD symbology classification. Extracted texture features were utilized for developing a fuzzy inference system based detection of attention tunneling. Each of the input was divided into three membership functions each. Sugeno type fuzzy model was chosen for the purpose. For attention tunneling mitigation two approaches were worked upon – adaptive neuro fuzzy (ANFIS) based mitigation and artificial neural network (ANN) based mitigation. System primarily works in as assistive mode currently and gives the pilot freedom to take inputs from AATMS for SL adjustment. Initially, ANFIS based mitigation system approach was adopted which helped in automatic luminance adjustment of symbology according to ambient lighting conditions at time of flight operation. The implementation results showed improved balance for event detection on HUD as well as outside world during medium AL range. However, still there existed imbalance for high and low AL conditions. This required further improvement in system design. The drawback of misbalance in high and low AL operation observed during ANFIS implementation motivated for developing individual models for both day and night mode operations. From experimental studies, optimum contrast ratio (CR) for different ranges of AL was identified. Another data set was then generated with parameters: current AL, current SL, desired SL (derived keeping in mind optimum CR) for both day and night luminance conditions. The whole data set for each case was then divided into training data, validation data and testing data to train ANN. Two ANN models for day mode and night mode operation were developed and further integrated to form a complete attention tunneling mitigation package – ‘AATMS’ which runs in two modes: offline mode and online mode. Offline mode was developed to check the functionality of AATMS and make any improvements required. Online mode operation when selected by the user takes HUD camera input feed and check for attention tunneling condition. In case of attention tunneling taking place, AATMS generates an alert for the user and on user choice may predict SL value as well. The novelty of the proposed approach lies in the fact that this system is adaptive to day and night mode flying operations. It is a unique attempt in the direction of online attention tunneling detection & mitigation.
Description: PHD, EIED
Appears in Collections:Doctoral Theses@EIED

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.