A Robust Feature-Based Visual Tracker For Micro-UAVs With Video Stabilization Enhancement / Abdelrahman Mohammed Elsayed Ahmed Eldesokey

Supervisor: Mohamed A. El-Helw

Thesis (M.A.)—Nile University, Egypt, 2016 .

"Includes bibliographical references"

Contents:
1 Chapter 1: INTRODUCTION ..............................................................................1
1.1 Motivation and Objectives ............................................................................. 1
1.2 Problem Definition ......................................................................................... 2
1.3 Proposed Algorithm ....................................................................................... 2
1.3.1 Algorithm Overview. .............................................................................. 2
1.3.2 Tracking Enhancement using Video Stabilization .................................. 3
1.3.3 Feature-Based Tracking with Context .................................................... 4
1.3.4 Object Re-Detection after Full Occlusion ............................................... 4
1.4 Thesis Organization ........................................................................................ 5
2 Chapter 2: BACKGROUND ................................................................................6
2.1 Unmanned Aerial Vehicles (UAVs) and their Evolution ............................... 6
2.1.1 Evolution of UAVs ................................................................................. 6
2.1.2 UAVs Types............................................................................................ 6
2.1.3 UAVs Applications ................................................................................. 8
2.2 Motion Compensation for Video Stabilization .............................................. 8
2.3 Objects Detection and Tracking in Aerial Imagery ..................................... 10
2.3.1 Object Detection in Aerial Imagery ...................................................... 10
2.3.2 Object Tracking .................................................................................... 13
2.3.3 State-of-Art Object Trackers ................................................................. 18
2.4 Overview on Hand-Crafted Features ............................................................ 24
2.4.1 Features from Accelerated Segment Test (FAST) ................................ 25
2.4.2 Good Features To Track (GFTT) .......................................................... 26
2.4.3 Scale-Invariant Feature Transform (SIFT) ........................................... 27
2.4.4 Binary Robust Independent Elementary Features (BRIEF).................. 29
2.4.5 Oriented FAST and Rotated BRIEF (ORB) ......................................... 30
2.4.6 Binary Robust Invariant Scalable Keypoints (BRISK) ........................ 30
vii
3 Chapter 3: ROBUST FEATURE-BASED OBJECT TRACKING IN AERIAL
IMAGERY WITH VIDEO STABILIZATION ...............................32
3.1 Algorithm Architecture ................................................................................ 32
3.2 The Choice of Video Stabilization and Object Tracking Techniques .......... 32
3.3 Video Stabilization ....................................................................................... 33
3.3.1 Pyramidal Lucas-Kanade Optical Flow ................................................ 34
3.3.2 Homography Estimation using RANSAC ............................................ 35
3.4 Object Tracking ............................................................................................ 37
3.4.1 Choosing Feature Type ......................................................................... 38
3.4.2 Object Model Initialization ................................................................... 38
3.4.3 Non-Conflicting Features Matching ..................................................... 40
3.4.4 Structure Consistency Check using RANSAC ..................................... 40
3.4.5 Finding Object using Rigid Transformation Estimation ....................... 41
3.4.6 Occlusion Detection and Pools Update ................................................. 41
3.4.7 Feature Forgetting ................................................................................. 42
3.4.8 Target Reacquisition After Occlusion .................................................. 42
4 Chapter 4: RESULTS AND ANALYSIS ..........................................................45
4.1 Datasets ........................................................................................................ 45
4.2 Evaluation Metrics ....................................................................................... 47
4.3 Qualitative Analysis ..................................................................................... 48
4.4 Quantitative Analysis ................................................................................... 53
5 Chapter 5: CONCLUSION AND FUTURE WORK .........................................62
5.1 Conclusion Remarks .................................................................................... 62
5.2 Summary of Contributions ........................................................................... 63
5.3 Future Work ................................................................................................. 63
REFERENCES ............................

Abstract:
In this thesis, we propose an integrated framework for tracking targets of interest
in aerial imagery acquired from micro unmanned aerial vehicles (MAVs). A tracker is
very important for MAV target-oriented applications, such as search and rescue and
traffic monitoring, as well as in cases where it is used as the main component for visual
navigation. The task of visual tracking itself has been exhaustively investigated in the
last few decades. Nevertheless, this task is significantly complicated in aerial imagery
as several challenges arise. MAVs are very light and under-equipped, and hence
vulnerable to vibrations due to air turbulence. The ‘trembling effects’ would lead to
abrupt motion in the sequences captured by the on-board camera causing severe
discrepancies in the video footage. Motion blur, focus changes and illumination
variations are all examples of such discrepancies. The task of visual object tracking in
MAV aerial video is further complicated by the conventional object tracking challenges
such as scale changes, partial/full occlusions, similar-appearance objects, background
clutter and pose alterations.
We propose a visual tracker that is able to address the above challenges and
perform the task of visual object tracking in aerial imagery efficiently. The problem of
abrupt camera motion is addressed using video stabilization enhancement that
estimates the global motion between consecutive frames and uses it to adjust the tracker
search region. Problems due to background clutter and alike objects are alleviated using
a structured object model based on robust features to discriminate between the target
and background according to appearance and structure of the former. Target pose
variations are addressed by making this structure adaptive to target appearance changes
between video frames. Target tracking failures due to partial or full occlusions are
alleviated using an improved target reacquisition mechanism that performs constrained
search.
The proposed tracker is evaluated against the state-of-art visual trackers on longduration
real-life datasets using established evaluation metrics. The first sequence is
taken from a quad copter and comprises most of the above-mentioned tracking
challenges. The second sequence is a stabilized footage captured from a police
helicopter and commonly used to evaluate visual trackers. The proposed tracker
performs flawlessly in the first sequence using video stabilization enhancement
whereas its robustness is demonstrated in the second sequence. Such claim is supported
by quantitative analysis showing that our tracker outperforms existing trackers while
sustaining real-time performance. Further qualitative analysis provides insights in the
strengths of the proposed tracker versus the limitations of other trackers.

Text in English, abstracts in English.