Robust Target Detection snf Tracking in Unmanned Aerial Vehicles (UAVs) Imagery Mennatullah Mohamed Siam

Supervisor: Mohamed A. El-Helw

Thesis (M.A.)—Nile University, Egypt, 2013 .

"Includes bibliographical references"

Contents:
Chapter1: Introduction ........................................................................................................ 1
1.1 Overview and Motivation ............................................................................... 1
1.2 Problem Definition ......................................................................................... 3
1.3 Contributions .................................................................................................. 3
1.3.1 Real-time Onboard Target Detection and Tracking ................................ 3
1.3.2 Robust Tracking using P-N Learning ..................................................... 4
1.4 Publications .................................................................................................... 5
1.5 Organization of Thesis ................................................................................... 6
Chapter 2: Review of Detection and Tracking in UAV Imagery ....................................... 7
2.1 Unmanned Aerial Vehicles (UAVs) .............................................................. 7
2.1.1 UAVs Types............................................................................................ 7
2.1.2 Existing UAV Models............................................................................. 8
2.1.3 UAVs Applications ............................................................................... 10
2.2 Target Detection and Tracking ..................................................................... 11
2.2.1 Target Detection Taxonomy ................................................................. 11
2.2.2 Target Tracking Taxonomy .................................................................. 13
2.3 Target Detection in UAV Imagery ............................................................... 15
2.3.1 Interest Point Detectors ......................................................................... 15
2.3.2 Motion Detection .................................................................................. 16
2.3.3 Segmentation......................................................................................... 16
2.3.4 Supervised Learning Detection ............................................................. 17
vii
2.3.5 Summary ............................................................................................... 18
2.4 Target Tracking in UAV Imagery ................................................................ 19
2.4.1 Point Tracking ....................................................................................... 19
2.4.2 Kernel Tracking .................................................................................... 20
2.5 State of the Art Trackers .............................................................................. 21
Chapter 3: Integrated Framework for Target Detection and Tracking ............................. 23
3.1 Framework Architecture .............................................................................. 23
3.2 Multiple Target Detection ............................................................................ 24
3.2.1 Feature Extractor ................................................................................... 24
3.2.2 Motion Estimation ................................................................................ 25
3.2.3 Outliers Detection and DBScan Clustering .......................................... 30
3.3. Target Tracking ............................................................................................ 34
3.3.1. Overlap-Rate Based Data Association .................................................. 34
3.3.2. Kalman Prediction ................................................................................ 37
3.3.3. BRIEF Descriptors Search .................................................................... 39
Chapter 4: Enhanced Target Tracking using PN Constraints ........................................... 42
4.1 Enhanced Version Framework Architecture ................................................ 42
4.2 Updated Data Association Algorithm .......................................................... 43
4.3 P-N (Positive-Negative) Learning ................................................................ 44
4.3.1 Semi-supervised Learning Overview .................................................... 44
4.3.2 P-N Learning Approach ........................................................................ 45
4.3.3 P-N Learning in Object Tracking ......................................................... 46
4.4 Novel P-N Experts ....................................................................................... 49
Chapter 5: Results & Discussion ...................................................................................... 51
5.1 Datasets ........................................................................................................ 51
5.1.1. Quad-rotor ............................................................................................. 52
5.1.2. PandaBoard and Middleware ................................................................ 53
5.1.3. Dataset Collection ................................................................................. 54
5.2 Evaluation Test-bed and Metrics .................................................................. 55
5.2.1 Evaluation Test-bed .............................................................................. 55
5.2.2 Evaluation Metrics ................................................................................ 56
viii
5.3 Qualitative Analysis ..................................................................................... 57
5.3.1 Recapture Targets Out/In Camera Field of View: ................................ 61
5.4 Quantitative Evaluation ................................................................................ 63
Chapter 6: Conclusions and Future Work ......................................................................... 69
6.1. Summary of Contributions ........................................................................... 69
6.2. Future Work .................................................................................................

Abstract:
In this dissertation we address the problem of automatic target detection and
tracking in UAV imagery. Even though target detection and tracking are well
explored problems in the field of computer vision, several challenges arise in case of
UAV imagery due to the modus operandi of the latter in terms of operational and
processing constraints. UAVs are airborne platforms that are typically in continuous
and sometimes turbulent rapid motions. The payloads are minimal with limited
processing boards and low-resolution image sensors capturing imagery from high
altitudes. Consequently, on-board vision-based algorithms have to decouple scene
and UAV motions to detect targets of interest, shrewdly exploit low-resolution
imagery to discriminate targets based on appearance, and execute efficiently to meet
the necessary demand of real-time performance. Additional challenges include the
fact that tracking of detected targets should be adaptive and learns the appearance of
the targets to maintain their tracking case of significant illumination changes and
partial occlusions, re-acquire targets as they move out of and subsequently in the
field-of-view, and to discriminate between similar targets moving within close spatial
proximity of each other.
This thesis presents an integrated framework comprising novel set of algorithms
to alleviate the above described problems in order to facilitate on-board robust realtime
target detection and tracking in UAV imagery. The separation of scene and
camera motions from the targets' motion is achieved by using an algorithm based on
image feature processing and projective geometry. Detected targets are tracked with
Kalman filtering while an overlap-rate-based data association mechanism followed by
tracking persistency check are used to discriminate between true moving targets and
false detections. The proposed framework doesn’t involve explicit application of
image transformations to detect potential targets resulting in enhanced computational
time and reduction of registration errors. The tracking performance is further
enhanced by utilizing P-N learning to learn the targets' appearances online. To this
end, novel P-N constraints are introduced based on data association to control the
xiii
positive and negative samples while a cascaded classifier is employed to detect the
targets in case of association failure after learning targets' appearance. Experimental
results obtained with publicly available DARPA aerial datasets demonstrate that the
proposed tracker with automatic detection feedback achieves better recall and average
overlap than existing manually-initialized trackers. Further qualitative analysis proves
the framework is able to solve most of the above challenges including target recapture
and discriminating between targets with similar appearance while alleviating tracker
drift problems.

Text in English, abstracts in English.