Fusion of Motion History of Skeletal Volumes and Temporal Bounding Volumes for View-Invariant Human Action Recognition / abubakrelsedik alsebai karali

Supervisor: Mohamed A. El-Helw

Thesis (M.A.)—Nile University, Egypt, 2013 .

"Includes bibliographical references"

Contents:
Chapter 1 Introduction ............................................................................................................... 10
1.1 Introduction and Problem Statement ............................................................................ 10
1.2 Objectives and Approach .............................................................................................. 11
1.3 Dissertation Overview .................................................................................................. 11
1.4 Publications and Awards............................................................................................... 12
1.5 Conclusion ................................................................................................................... 12
Chapter 2 Background ............................................................................................................... 13
2.1 Human Action Recognition .......................................................................................... 13
2.2 Human Action Recognition System Block Diagram .................................................... 14
2.3 Related Work ................................................................................................................ 16
2.3.1 Work Utilizes 3D Data.............................................................................................. 16
2.4 3d Shape Features ......................................................................................................... 18
2.5 Skeletons and Skeletonization ...................................................................................... 20
2.6 Skeleton Properties ....................................................................................................... 21
2.6.1 Blum Skeleton Properties ......................................................................................... 21
2.6.2 Curve Skeleton Properties......................................................................................... 25
2.7 Pattern Recognition ....................................................................................................... 27
2
2.7.1 Naïve Bayesian ......................................................................................................... 28
2.7.2 K-Star ........................................................................................................................ 28
2.7.3 Multilayer Perceptron ............................................................................................... 29
2.7.4 Logistic Model Trees ................................................................................................ 29
2.7.5 C4.5 Decision Trees .................................................................................................. 30
2.7.6 Multiple Classifier Systems ...................................................................................... 30
2.8 Conclusion .................................................................................................................... 32
Chapter 3 Framework for Activity Classification ..................................................................... 33
3.1 Feature extraction.......................................................................................................... 34
3.1.1 Motion History of Skeletal Volumes for Human Action Recognition ..................... 34
3.1.2 Motion History of Skeletal Volumes (MHSV) ......................................................... 35
3.1.3 Motion Temporal Change in Bounding Volume (TCBV) ........................................ 36
3.1.4 Temporal Changes in Bounding Volumes (TCBVS) ............................................... 39
3.2 Processing And Classification ...................................................................................... 40
3.2.1 Motion History Of Skeletal Volumes Classification ................................................ 40
3.2.2 Temporal Changes in Bounding Volumes Classification ......................................... 42
3.2.3 MHSV And TCBV Decision Fusion Using Multiple Classifier Systems ................ 42
3.3 Conclusion .................................................................................................................... 43
Chapter 4 Experiments and Evaluation ..................................................................................... 45
4.1 Datasets ......................................................................................................................... 45
4.1.1 The IXMAS Dataset ................................................................................................. 45
4.1.2 The I3DPOST Dataset .............................................................................................. 47
4.2 Motion History of Skeletal Volume Results ................................................................. 47
3
4.3 Temporal Changes in Bounding Volumes Results ....................................................... 52
4.4 Fusion ............................................................................................................................ 54
4.5 Conclusion .................................................................................................................... 57
Chapter 5 Conclusion and Future work .....................................................................................

Abstract:
Human Activity Recognition is an active area of research in computer vision with wide range of
applications in video surveillance, motion analysis, virtual reality interfaces, robot navigation
and recognition, video indexing, browsing, HCI, choreography, ports video analysis, to name a
few. It consists of analyzing the characteristic features of various human actions and classifying
them. A human activity recognition system typically consists of the following stages:
background subtraction, tracking, feature extraction and classification.
Several approaches have been developed to solve this problem in 2D and 3D spaces. Due
to the enhanced robustness of 3D vision-based recognition techniques, we focus in this thesis on
human action recognition from 3D data. We present a framework for reliable action recognition
systems designed to achieve the objective of automatic classification of ongoing activities in
unlabeled 3D sequences. The major challenges within this framework are the extraction of
representative motion descriptors and the usage of suitable classification methods to build a
model for each motion. to this end, the thesis introduces a novel view-invariant features (a) the
motion history of skeletal volumes and (b) the temporal change in bounding volumes, where
both are used to describe specific actions. Furthermore, we propose a number of classification
techniques for action recognition as well as decision fusion rules to combine decisions when the
human action recognition system deals with multiple features using state-of-art multiple
classifier systems. Obtained results demonstrate that skeletons produce better results than
volumes and that the use of bounding volumes along with the skeletons further improves the
recognition accuracy and can hence be used to recognize human actions independent of
viewpoint and scale.

Text in English, abstracts in English.