Ahmed Hosny Abdel-Gawad Ahmed <br/><br/>
Verification of Machine Learning/Deep Learning Based Pedestrian Detectors 
/Ahmed Hosny Abdel-Gawad Ahmed 
 -  2024 
 - 177 p.   ill.   21 cm.  
<br/><br/>Supervisor: <br/>Ahmed G. Radwan 
<br/><br/>Thesis (M.A.)—Nile University, Egypt, 2024 . 
<br/><br/>"Includes bibliographical references" 
<br/><br/>Contents:<br/>Chapters:<br/>1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1<br/>1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6<br/>1.2 Importance of Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . 7<br/>1.3 Current Challenges and Limitations . . . . . . . . . . . . . . . . . . . . . 9<br/>1.4 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10<br/>1.5 Research Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11<br/>1.6 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12<br/>1.7 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14<br/>1.8 Thesis Outline and Summary of Contributions . . . . . . . . . . . . . . . . 15<br/>2. Literature Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18<br/>2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18<br/>2.2 Sensors for Environment Perception . . . . . . . . . . . . . . . . . . . . . 18<br/>2.2.1 Visible-light Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . 24<br/>2.2.2 Thermal Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24<br/>2.3 Benchmark Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26<br/>2.4 Pedestrian Detection Techniques: A Brief History . . . . . . . . . . . . . . 32<br/>2.5 Typical Detection Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . 35<br/>2.5.1 Regions of Interest . . . . . . . . . . . . . . . . . . . . . . . . . . . 35<br/>2.5.2 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . 36<br/>2.5.3 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37<br/>2.6 Deep Learning for Pedestrians Detection . . . . . . . . . . . . . . . . . . . 37<br/>2.7 Deep Learning Architectures for pedestrians Detection . . . . . . . . . . . 41<br/>2.7.1 Two-Stage Detectors . . . . . . . . . . . . . . . . . . . . . . . . . . 42<br/>2.7.2 Single Stage Detectors . . . . . . . . . . . . . . . . . . . . . . . . . 45<br/>2.8 Pedestrians Detection Challenges . . . . . . . . . . . . . . . . . . . . . . . 46<br/>2.8.1 Image Corruption . . . . . . . . . . . . . . . . . . . . . . . . . . . 47<br/>2.8.2 Pedestrian Occlusion . . . . . . . . . . . . . . . . . . . . . . . . . . 51<br/>3. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53<br/>3.1 Baseline Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55<br/>ii<br/><br/>3.1.1 Dataset Selection and Preparation . . . . . . . . . . . . . . . . . . 56<br/>3.1.2 Model Selection and Training . . . . . . . . . . . . . . . . . . . . . 59<br/>3.2 Baseline Corruption Robustness . . . . . . . . . . . . . . . . . . . . . . . . 63<br/>3.2.1 Selection of Image Corruptions . . . . . . . . . . . . . . . . . . . . 64<br/>3.2.2 Image Quality Assessment . . . . . . . . . . . . . . . . . . . . . . . 65<br/>3.2.3 Artificial Generation of Corrupted Data . . . . . . . . . . . . . . . 68<br/>3.2.4 Noise corruptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69<br/>3.2.5 Blur corruptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70<br/>3.2.6 Digital corruptions . . . . . . . . . . . . . . . . . . . . . . . . . . . 74<br/>3.2.7 Weather corruptions . . . . . . . . . . . . . . . . . . . . . . . . . . 75<br/>3.2.8 Baseline Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 78<br/>3.3 Enhancing Robustness Against Image Corruptions . . . . . . . . . . . . . 79<br/>3.3.1 Robust Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79<br/>3.3.2 Stylized Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81<br/>3.4 Baseline Occlusion Robustness . . . . . . . . . . . . . . . . . . . . . . . . 84<br/>3.4.1 Naturally Occluded JAAD Dataset . . . . . . . . . . . . . . . . . . 84<br/>3.4.2 Baseline Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 84<br/>3.5 Enhancing Robustness Against Pedestrian Occlusion . . . . . . . . . . . . 85<br/>3.5.1 Artificial Generation of Occluded Datasets . . . . . . . . . . . . . . 86<br/>3.5.2 Occluded Training . . . . . . . . . . . . . . . . . . . . . . . . . . . 87<br/>4. Results and Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89<br/>4.1 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89<br/>4.1.1 Detection Performance Metrics . . . . . . . . . . . . . . . . . . . . 89<br/>4.1.2 Corruption Robustness Evaluation Metrics . . . . . . . . . . . . . . 91<br/>4.1.3 Occlusion Robustness Evaluation Metrics . . . . . . . . . . . . . . 92<br/>4.1.4 Computational Efficiency: Inference Time . . . . . . . . . . . . . . 94<br/>4.2 Baseline Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95<br/>4.2.1 Performance on Corrupted Dataset . . . . . . . . . . . . . . . . . . 96<br/>4.2.2 Performance on Occluded Dataset . . . . . . . . . . . . . . . . . . 106<br/>4.3 Enhancing Corruption Robustness through Robust Training . . . . . . . . 110<br/>4.3.1 Training with All Corruptions as Augmentations . . . . . . . . . . 111<br/>4.3.2 Training on Each Category Alone . . . . . . . . . . . . . . . . . . . 120<br/>4.4 Enhancing Corruption Robustness through Stylized Training . . . . . . . . 131<br/>4.4.1 Training on Stylized data only . . . . . . . . . . . . . . . . . . . . 131<br/>4.4.2 Training on both Clean and Stylized data combined . . . . . . . . 139<br/>4.5 Enhancing Occlusion Robustness through Occluded Training . . . . . . . . 148<br/>5. Conclusion and Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152<br/>5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152<br/>5.2 Future directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154<br/>Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 
<br/><br/> Abstract:<br/>In the context of assisted and automated driving vehicles and advanced driver-assistance<br/>systems underscores the critical need for accurate pedestrian detection to ensure road safety.<br/>However, deploying pedestrian detection models in real-world scenarios poses a significant<br/>challenge due to image corruptions and partial occlusions. This thesis addresses these challenges through the development and evaluation of robust, stylized, and occluded training<br/>techniques.<br/>Robust Training incorporates intentionally generated corrupted examples during model<br/>training to simulate real-world challenges. Results demonstrate significant improvements<br/>in model robustness to various corruption types. Further refinement through categorized<br/>training data emphasizes the potential benefits of tailored augmentation to specific corruption<br/>profiles.<br/>Stylized Training leverages the Adaptive Instance Normalization (AdaIN) method to introduce variations in texture and style, enriching the training dataset. Empirical results showcase a substantial improvement in pedestrian detection accuracy, particularly in mean Average Precision (mAP50) on clean data. Stylized training significantly enhances the model’s<br/>robustness in adverse conditions, demonstrating proficiency in handling noise, blur, digital,<br/>and weather-related distortions.<br/>Occluded Training introduces a novel methodology for generating occluded datasets, simulating scenarios with varying levels of occlusion. The augmented training approach leads to<br/>a substantial enhancement in performance when faced with occluded scenarios. Specifically,<br/>the model exhibits a ∼2% increase in mean Average Precision (mAP50) on both partially and<br/>fully occluded samples compared to standard training. This underscores the effectiveness of<br/>occlusion training in fortifying the model’s performance in occluded real-world scenarios.<br/>The utilization of robust, stylized, and occluded training techniques significantly advances the<br/>adaptability and performance of pedestrian detection models in diverse real-world conditions<br/>and corruption scenarios, with an improvement of ∼2-4%. These findings establish a robust<br/>foundation for deploying pedestrian detection models in complex environments, ensuring<br/>safer and more reliable assisted and automated driving systems. This thesis contributes<br/>to the forefront of pedestrian detection, underscoring the efficacy of tailored augmentation<br/>strategies for model robustness in dynamic real-world environments.<br/>Keywords— Assisted and Automated Vehicles, Pedestrian Detection, Image Corruptions, Robust<br/>Training, Stylized Training, Occluded Training, Model Robustness<br/> 
<br/><br/><br/>Text in English, abstracts in English and Arabic 
<br/><br/><label>Standard No.: </label>0000-0003-0549-3441  ORCID 
<br/><br/><label>Subjects--Topical Terms: </label><br/>MSD
<br/><br/><label>Index Terms--Genre/Form: </label><br/>Dissertation, Academic
<br/><br/><label>Dewey Class. No.: </label>621