A Hybrid Deep CNN-Reinforcement Learning Model for Autonomous Driving /
Karim Mansour
- 2018
- 108 p. ill. 21 cm.
Supervisor: Mohamed A. El-Helw
Thesis (M.A.)—Nile University, Egypt, 2018 .
"Includes bibliographical references"
Contents: 1 CHAPTER 1: INTRODUCTION ................................................................................ 1 1.1 Motivation ............................................................................................................ 1 1.2 Problem Definition ............................................................................................... 1 1.3 Thesis Contributions ............................................................................................ 2 1.4 Autonomous Driving and Artificial Intelligence ................................................. 2 1.5 Organization of Thesis ......................................................................................... 2 2 CHAPTER 2: LITERATURE REVIEW ..................................................................... 4 2.1 Introduction .......................................................................................................... 4 2.2 Autonomous Driving ............................................................................................ 4 2.2.1 5 Levels of Autonomy .................................................................................. 5 2.2.1.1 Level 1: One Control Automated .............................................................. 5 2.2.1.2 Level 2: Two Controls Automated ............................................................ 6 2.2.1.3 Level 3: City to City Automation .............................................................. 7 2.2.1.4 Level 4: Full Autonomous......................................................................... 9 2.2.1.5 Level 5: No Driver .................................................................................. 10 2.2.2 Autonomous Driving Block Diagram ......................................................... 10 2.3 Artificial Intelligence ......................................................................................... 11 2.3.1 Machine Learning ....................................................................................... 11 2.3.2 Deep Learning ............................................................................................. 11 2.3.2.1 Convolutional Neural Networks.............................................................. 13 2.3.2.2 Reinforcement Learning .......................................................................... 14 2.3.3 Recurrent Neural Networks ........................................................................ 14 2.4 Rule-Based Control Systems .............................................................................. 15 2.4.1 PID Controller ............................................................................................. 15 2.4.2 Automotive Rule-Based Systems ............................................................... 16 2.4.3 Advantages and Disadvantages ................................................................... 17 2.5 End-to-End Deep Learning ................................................................................ 17 2.5.1 Advantages and Disadvantages ................................................................... 18 2.5.2 Existing End-to-End Deep Learning Networks .......................................... 18 2.5.2.1 CNN Only Steering: NVIDIA Model ..................................................... 19 2.5.2.2 CNN + LSTM Steering: VALEO Model ................................................ 21 2.5.2.3 Reinforcement Learning Steering: STANFORD Model ......................... 23 2.6 Conclusions ....................................................................................................... 25 3 CHAPTER 3: NOVEL CNN-REINFORCEMENT LEARNING STEERING MODEL 26 ix 3.1 Introduction ........................................................................................................ 26 3.2 Sensors Setup ..................................................................................................... 27 3.2.1 Camera ........................................................................................................ 27 3.2.2 Lidar ............................................................................................................ 28 3.2.3 Steering Angles ........................................................................................... 29 3.3 Deep Network Architecture to Predict Future Steering Angles ......................... 30 3.3.1 Model Architecture ..................................................................................... 30 3.3.1.1 CNN-Only Model Limitations ................................................................ 34 3.3.2 Predicting Travel Path Instead of Steering ................................................. 34 3.4 Adding Vehicle Speed ........................................................................................ 38 3.5 Hybrid Model ..................................................................................................... 40 3.5.1 Free Space Acquisition ............................................................................... 41 3.5.1.1 Cameras ................................................................................................... 41 3.5.1.2 Lidar ........................................................................................................ 41 3.5.2 Real-time Reward Integration ..................................................................... 42 3.6 Conclusion .......................................................................................................... 48 4 CHAPTER 4: EXPERIMENTS AND RESULTS .................................................... 50 4.1 Introduction ........................................................................................................ 50 4.2 Country Road Driving Simulator ....................................................................... 50 4.3 Datasets .............................................................................................................. 54 4.4 Data Augmentation ............................................................................................ 56 4.4.1 Data Processing ........................................................................................... 56 4.4.1.1 Data Flipping ........................................................................................... 56 4.4.1.2 Contrast Modifications ............................................................................ 57 4.4.1.3 Shadow Addition ..................................................................................... 57 4.4.2 Extra Sensors for Data Augmentation ........................................................ 58 4.4.2.1 Angled Cameras ...................................................................................... 58 4.4.2.2 Distanced Cameras .................................................................................. 58 4.5 Results ................................................................................................................ 59 4.5.1 Test Track ................................................................................................... 59 4.5.2 Metrics Used ............................................................................................... 59 4.5.2.1 Mean Square Error .................................................................................. 59 4.5.2.2 Validation Loss ....................................................................................... 60 4.5.2.3 Learning Rate .......................................................................................... 60 4.5.3 NVIDIA Model Results .............................................................................. 60 4.5.4 Predicting Future Steering Angles .............................................................. 62 4.5.4.1 One Second into Future ........................................................................... 62 4.5.4.2 Two Seconds into Future......................................................................... 63 4.5.5 Speed Integration ........................................................................................ 65 x 4.5.6 Free Space Only Results ............................................................................. 67 4.5.7 Autonomous Driving Hybrid Model ........................................................... 69 4.6 Conclusion .......................................................................................................... 74 5 CHAPTER 5: CONCLUSIONS AND FUTURE WORK ........................................ 76 5.1 Conclusions ........................................................................................................ 76 5.2 Avoiding Objects................................................................................................ 77 5.3 Different Vehicles Data ...................................................................................... 77 5.4 Human Brain Comparison .................................................................................. 77 5.5 End-to-End Deep Learning Limitations Overcome ........................................... 78 5.6 Safety and Reliability Considerations ................................................................ 78 6 APPENDIX – DESCRIPTION OF USED LIBRARIES .......................................... 79 6.1 Python................................................................................................................. 79 6.2 TensorFlow......................................................................................................... 79 6.3 Keras................................................................................................................... 80 6.4 Unity ................................................................................................................... 80 7 REFERENCES ..........................................................................................................
Abstract: Nowadays, to reduce the number of accidents and to increase cars safety levels, all car manufacturers are racing to reach high level of autonomy to achieve driverless cars as soon as possible. The conventional ways to achieve the current Advanced Driving Assistance Systems (ADAS) of controlling the vehicle using traditional control theory proved to be not enough to handle the unlimited scenarios of autonomous driving in a city environment. It became clear that the usage of Artificial Intelligence (AI) in terms of deep learning is needed to aid the current ADAS systems (e.g. Adaptive Cruise Control and Lane Keeping) to reach high level of autonomy using big data. Currently, the automotive industry is using deep learning excessively in classifying objects captured by on-board sensors (e.g. cameras), but lately there are many researches being done to use deep learning to control the steering using Convolutional Neural Networks (CNN) for raw camera images, or Reinforcement Learning (RL) to learn from mistakes using huge amount of simulated data. The limitation of the CNN-only solution is that it lacks live-feedback to correct it-self, and in CNNs using Long-Short-Term-Memory (LSTM) layers to reduce the noise in steering, they suffer from the need of high computational power on-board. On the other hand, the RL-only solution is relying heavily on simulations due to the nature of RL of learning from doing mistakes; therefore, RL cannot be used easily with real-life data. On top of that, both solutions are not considering the speed of the vehicle as an input but relying only on raw data from the frontcamera, which leads to a system that only works on constant velocities. This thesis proposes a novel CNN-Reinforcement hybrid solution where both CNN and RL are being used at the same time in a supervised mode to provide a live feedback that can be used in real-life data not only in simulations. It also proposes a major modification in the traditional CNN architecture to let the neural network predict the path of driving instead of steering angle to reduce the noise on the output and to avoid using LSTMs which require huge amount of computational power. Besides that, the architecture has been altered for considering the velocity of the vehicle to ensure smooth driving at different speeds. The different approaches available in the research community are reviewed in detail. Experiments and tests are done using computer simulations on both Hybrid and CNN-only solution and the results are examined in details. Finally, future work is discussed to enhance the system for better and safer driving.