Human Action Recognition in Videos Employing 2-DPCA on 2DHHOF and Radon Transform / Fadwa Fawzy Fouad

Supervisor: Mohamed A. El-Helw

Thesis (M.A.)—Nile University, Egypt, 2013.

"Includes bibliographical references"

Contents:
\1 Introduction 1
1.1 Challenges and characteristics of the domain . . . . . . . . . . . . . . . . . . 2
1.1.1 inter- and Intra-class variations . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 Environmental Variations and Capturing conditions . . . . . . . . . . 2
1.1.3 Temporal variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Proposed Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Overview 5
2.1 Input videos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Human detection/segmentation . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.1 Background subtraction . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.2 Optical Flow (OF) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.3 Histogram of Oriented Gradients (HOG) . . . . . . . . . . . . . . . . 10
2.3 Feature Extraction and Modeling . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3.1 Non-Parametric Approaches . . . . . . . . . . . . . . . . . . . . . . . 11
2.3.2 Volumetric Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.3 Parametric Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Action Classication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4.1 Direct classication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4.2 Temporal state-space models . . . . . . . . . . . . . . . . . . . . . . . 16
3 2DHOOF/2DPCA Contour Based Optical Flow Algorithm 18
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
vi
3.2 What 2DHOOF Features are . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2.1 Optical Flow Techniques . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2.2 Alignment issues with OF . . . . . . . . . . . . . . . . . . . . . . . . 22
3.2.3 The Calculation of 2D Histogram of Optical Flow(2DHOOF) . . . . . 23
3.3 Overall System Description . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.3.1 Training Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.3.2 Testing Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.4 Used Data-sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.1 Weizmann Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.2 IXMAS Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.5 Experimental Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . 35
3.5.1 Weizmann Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.5.2 IXMAS Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4 Human Gesture Recognition Employing Radon Transform/2DPCA 48
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.2 Radon Transform (RT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.3 Overall System Description . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.3.1 Training Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.3.2 Testing Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.4 Used Data-sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.4.1 ChaLearn Gesture Data . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.5 Experimental Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . 55
4.5.1 Input videos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.5.2 Segmentation and Blob extraction . . . . . . . . . . . . . . . . . . . . 55
4.5.3 Alignment using RT of the First Frame . . . . . . . . . . . . . . . . . 60
4.5.4 Video Chopping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.5.5 Calculate the MEI and MHI . . . . . . . . . . . . . . . . . . . . . . . 66
4.5.6 Get Radon Transform for MEI and MHI . . . . . . . . . . . . . . . . 68
4.5.7 2DPCA and Feature Extraction . . . . . . . . . . . . . . . . . . . . . 69
vii
5 Conclusion 73
5.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.2 Future Work . .

Abstract:
Human action recognition is a very important trend of research with a great interest in
computer vision area. It has applications in various elds such as surveillance, content-based
video retrieval, gesture recognition and sports analysis. The speed, storage requirements,
and accuracy of the human action recognition system are highly dependent on the extracted
features, and modeling approaches. All dierent recognition techniques aim to extract the
most representative and accurate features with low storage requirements. In this thesis, two
algorithms are proposed for human action recognition which target the high accuracy and
the low computation complexity.
For the rst algorithm the novel Two Dimensions Histogram of Oriented Optical
ow
(2DHOOF) features where extracted using the actor's contour. Two Dimensions Principal
Component Analysis (2DPCA) was used for feature extraction. The Euclidean Distance
Classier was used for the nal classication decision. Experimental results applied on
Weizmann and the IXMAS datasets achieved the highest reported recognition accuracy and
the fastest runtime compared to recent methods.
In the second algorithm Radon Transform (RT) was extracted from the moving parts of
the actor. Using RT enables us to easily align the actor's silhouette with the center of the
frame without the need of further pre-processing. 2DPCA was used for feature extraction.
The Euclidean Distance Classier was used for the nal classication decision. Experimental
results applied on one shot learning ChaLearn Gesture Data-set achieved high recognition
accuracy and the fast runtime.
Experimental results promote our algorithms for real time applications.

Text in English, abstracts in English.