Data Modeling
Mathematical and Computational Data Modeling | Online ISSN 3143-9217
2
Citations
14.6k
Views
48
Articles
Volume 3 Number 1 2022
RESEARCH ARTICLE (Open Access)
Real-Time Driver Drowsiness Detection Using a Multi-Task Recurrent Convolutional Neural Network: A Facial-Landmark-Based Approach
Fahad Ahmed 1*, Shaid Hasan 2, Khandaker Ataur Rahman 2
Data Modeling 3 (1) 1-8 https://doi.org/10.25163/data.3110934
Submitted: 16 September 2022 Revised: 08 November 2022 Accepted: 17 November 2022 Published: 19 November 2022
Abstract
Background: Drowsy driving remains an underappreciated but persistent contributor to road traffic fatalities, and the sensor-heavy or vehicle-embedded systems currently available to counter it are largely confined to premium vehicles, leaving most drivers unprotected. Methods: We propose a behavior-based, camera-only framework that infers drowsiness from a sequence of facial video frames rather than from physiological or vehicle-motion sensors. Faces were localized with the Dlib library, and 68 facial landmarks were used to compute three interpretable indices — the eye aspect ratio (EAR), the mouth opening ratio (MOR), and a nose-length ratio (NLR) capturing head bending — together with a per-driver calibration phase and a temporal voting rule evaluated across a rolling window of frames. These features fed a compact multi-task convolutional neural network, trained with stochastic gradient descent (batch size 64, momentum 0.6, weight decay 0.0005), on 62,910 frames from the NTHU-DDD dataset. Results: Of two candidate architectures, the narrower network generalized better, achieving 98.96% validation and 98.49% test accuracy (98.72% overall), with a corresponding average accuracy of 92.81% across behavioral conditions. Applying calibrated EAR (<0.24) and MOR (>0.16) thresholds yielded a three-tier drowsiness classification (not tired, less tired, very tired) consistent with observed eye-closure and yawning patterns. Conclusion: A single, comparatively small multi-task network can approach the accuracy of heavier ensemble or physiological approaches while remaining light enough for real-time, in-vehicle or mobile deployment, suggesting a practical route toward drowsiness monitoring that does not depend on luxury-vehicle hardware.
Keywords: Driver drowsiness detection; Convolutional neural network; Facial landmark detection; Eye aspect ratio; Mouth opening ratio; Deep learning; Road safety.
References
Bhardwaj, R., Natrajan, P., & Balasubramanian, V. (2018). Study to determine the effectiveness of deep learning classifiers for ECG based driver fatigue classification. In 2018 IEEE 13th International Conference on Industrial and Information Systems (ICIIS) (pp. 98–102). IEEE. https://doi.org/10.1109/ICIINFS.2018.8721418
Computer Vision Lab, National Tsing Hua University. (2016). Driver drowsiness detection dataset. Retrieved March 1, 2020, from http://cv.cs.nthu.edu.tw/php/callforpaper/datasets/DDD/
Dua, M., Shakshi, Singla, R., Raj, S., & Jangra, A. (2021). Deep CNN models-based ensemble approach to driver drowsiness detection. Neural Computing and Applications, 33(8), 3155–3168. https://doi.org/10.1007/s00521-020-05209-7
Ed-Doughmi, Y., Idrissi, N., & Hbali, Y. (2020). Real-time system for driver fatigue detection based on a recurrent neuronal network. Journal of Imaging, 6(3), Article 8. https://doi.org/10.3390/jimaging6030008
Hashemi, M., Mirrashid, A., & Beheshti Shirazi, A. (2020). Driver safety development: Real-time driver drowsiness detection system based on convolutional neural network. SN Computer Science, 1(5), Article 289. https://doi.org/10.1007/s42979-020-00306-9
Jabbar, R., Al-Khalifa, K., Kharbeche, M., Alhajyaseen, W., Jafari, M., & Jiang, S. (2018). Real-time driver drowsiness detection for Android application using deep neural networks techniques. Procedia Computer Science, 130, 400–407. https://doi.org/10.1016/j.procs.2018.04.060
Jamshidi, S., Azmi, R., Sharghi, M., & Soryani, M. (2021). Hierarchical deep neural networks to detect driver drowsiness. Multimedia Tools and Applications, 80, 16045–16058. https://doi.org/10.1007/s11042-021-10542-7
Krajewski, J., Sommer, D., Trutschel, U., Edwards, D., & Golz, M. (2009). Steering wheel behavior based estimation of fatigue. In Proceedings of the Fifth International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design (pp. 118–124). University of Iowa.
Mardi, Z., Ashtiani, S. N. M., & Mikaili, M. (2011). EEG-based drowsiness detection for safe driving using chaotic features and statistical tests. Journal of Medical Signals and Sensors, 1(2), 130–137.
McDonald, A. D., Lee, J. D., Schwarz, C., & Brown, T. L. (2018). A contextual and temporal algorithm for driver drowsiness detection. Accident Analysis & Prevention, 113, 25–37. https://doi.org/10.1016/j.aap.2018.01.005
Recommended articles
Reproducibility, Algorithmic Bias, and Clinical Readiness in Medical Imaging AI: A Lifecycle-Based Synthesis and Multi-Institutional Validation Protocol
Why Artificial Intelligence Models for Antimicrobial Resistance Still Fail at the Bedside: A Review of Validation, Explainability, and Equity Gaps
Article metrics
View details
0
Downloads
0
Citations
44
Views
0
Save
Save
0
Citation
Citation
44
View
View
0
Share
Share