Augmented Reality and Gamification in an Integrated Adaptive System Dalia Marzouk Refaie

Supervisor: Nashwa Abd El-Baki

Thesis (M.A.)—Nile University, Egypt, 2015 .

"Includes bibliographical references"

Contents:
1 Traditional Education System Challenges 1
1.1 Traditional Teaching Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Study with Memorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 Passive Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.3 Low Educational Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.4 Lack of Practical Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.5 Generalization of School Systems and Restrictions . . . . . . . . . . . . . . . 3
1.1.6 The Big Number of Students . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Proposed Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.1 Serious Games (SG) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.2 Augmented Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.3 Adaptive Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Education and Technologies Integration 7
2.1 Serious Games in Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Augmented Reality for Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 Augmented Reality System Components . . . . . . . . . . . . . . . . . . . . . 10
2.2.2 AR for Education Purposes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3 Adaptive Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.1 Learning Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.2 Cognitive and Learning Styles . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.3 Adaptive System for Students with Learning Disabilities . . . . . . . . . . . . 19
2.3.4 Learner Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
xi
3 Proposed Educational Games Adaptive Engine 29
3.1 The Adaptive Engine Conceptual Design . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.1.1 The Pedagogical Unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.1.2 Gaming Unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.1.3 User Interface Unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.2 Game Engine Work Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4 Augmented Biology Lab 39
4.1 Augmented Reality and Gaming Tool . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2 General Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.3 Mix and Match Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.4 Organs Shower Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.4.1 Game Scenarios and Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.4.2 Game Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.4.3 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5 Aective Status Measurement Using Facial Expressions 53
5.1 Facial Region Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.1.1 Features Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.1.2 Facial Regions Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.2 Point Distribution Model (PDM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.3 Action Points Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.3.1 Image Cropping and Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . 58
5.4 Features Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.4.1 Local Binary Pattern (LBP) . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.4.2 Scale Invariant Feature Transform (SIFT) . . . . . . . . . . . . . . . . . . . . 63
5.4.3 Geometric Features (GEO) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.5 Features Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.5.1 Principal Component Analysis (PCA) . . . . . . . . . . . . . . . . . . . . . . 65
5.6 Emotion Classication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.7 Facial Expressions Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.8 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6 Conclusion and Future Work

Abstract:
Learning is one of the most important needs in the human life. With the help of learning humans
could face life with its various challenges. Dierent sources of education are oered in ones life.
Moreover, the rst sixteen years of the citizens life the school is the main source of learning. So, all
of us should have an equivalent opportunity in the education system whatever our capabilities and
skills are.
A lot of pedagogical specialists argued that every student could be educated but on her/his own
pace. There is no meaning for social classes, or learning weakness. Understanding every student and
his capabilities, weakness, needs and learning style provides a proper way of teaching and education
that encourages her/him to learn.
This study concentrates on dening and understanding the educational challenges and its reasons.
Our objective is to introduce an assisting technological system that contributes in enhancing the
educational level.
The proposed system in this thesis is based mainly on three concepts. The rst concept is to make
the educational process an interesting and interactive experience via using the concept of Serious
Games. The second one is helping students to interact with the educational concepts and to imagine
them by using the concept of Augmented Reality in visualizing the educational content. Finally the
proposed system addresses the personality variations per student via what is called learner model
in order to provide every student with a Personalized Adaptive level of education.

Text in English, abstracts in English .