Design of Robust and Real Time Eye-Gaze Communication Technique

Abstract

Eye tracking is a pivotal technology in gaze analysis that enhances accessibility and communication for individuals with special needs, enabling them to interact with computers using eye movements. The research presented in this thesis ex plores various eye-tracking techniques, emphasizing modern methods like machine learning, which have significantly advanced the field over the past two decades. A key focus is on developing cost-effective, real-time eye-gaze communication sys tems that do not require extensive hardware, leveraging techniques such as user specific calibration algorithms, the MediaPipe framework for feature extraction, and convolutional neural networks (CNNs) for gaze prediction. These systems are designed to be robust and user-friendly, enhancing accessibility for disabled individuals by allowing them to control computers through eye movements and blinks. Additionally, the thesis addresses challenges such as handling natural head movements and varying lighting conditions, proposing strategies to overcome these limitations. Onesignificant proposed framework of this thesis is introducing a cost-effective, real-time eye-gaze communication system utilizing a standard webcam specifically designed for disabled persons. The system employs a Video-Oculography (VOG) approach and a user-specific calibration algorithm to enable disabled individu als to control a computer through eye movements and blinks. Rigorous testing with disabled and non-disabled individuals has demonstrated the system’s robust performance and high accuracy. Further advancing the accessibility provided by eye-tracking technology, this thesis also proposes a calibration-free, eye-controlled system, which features a two-phase process: the first phase involves feature extrac tion using the MediaPipe framework, and the second phase focuses on coordinate mapping. Furthermore, this thesis introduces an affordable and dependable video-based gaze-tracking system utilizing machine learning techniques. One proposed frame work employs the MediaPipe face mesh model for extracting facial features from real-time video sequences. A user-specific calibration process and multiple regres sion techniques are used to predict gaze points accurately. The system effectively manages changes in body position and minor head movements through real-time re-calibration using z-index tracking. This system has demonstrated high sen sitivity to various environmental factors and achieved commendable visual an gle accuracy during testing with multiple participants. An additional approach presented in this thesis is an appearance-based eye-gaze estimation system that utilizes a convolutional neural network (CNN). The system processes low-quality eye images captured by standard webcams, making it widely applicable without requiring specialized hardware. The methodology includes collecting a labeled dataset of eye images, training a CNN model, and applying calibration and trans fer learning techniques to adapt the model for new users. Collectively, these systems distinguish themselves from existing approaches by effectively handling natural head movements, adapting to varying distances, and delivering robust performance without reliance on expensive hardware. These innovations underline the technical advancements and significant real-world appli cability of the proposed research.