000 10703nam a22002537a 4500
008 210831s2021 ||||f mb|| 00| 0 eng d
040 _aEG-CaNU
_cEG-CaNU
041 0 _aeng
_beng
082 _a610
100 0 _aAbeer Youssef Mohamed
_9574
245 1 _aA Multi-Embeddings Approach Coupled with Deep Learning for Arabic Named Entity Recognition /
_cAbeer Youssef Mohamed
260 _c2021
300 _a89 p.
_bill.
_c21 cm.
500 _aSupervisor: Samhaa El-Beltagy
502 _aThesis (M.A.)—Nile University, Egypt, 2021 .
504 _a"Includes bibliographical references"
505 0 _aContents: ABSTRACT ................................................................................................................................ I DEDICATION ........................................................................................................................... II ACKNOWLEDGEMENTS..................................................................................................... III PUBLICATION RELATED TO THIS WORK .................................................................... IV LIST OF TABLES ................................................................................................................. VII LIST OF FIGURES .............................................................................................................. VIII LIST OF ABBREVIATIONS .................................................................................................. X CHAPTER 1 INTRODUCTION .............................................................................................. 1 1.1 PROBLEM DEFINITION ......................................................................................................... 1 1.2 OBJECTIVES ......................................................................................................................... 2 1.3 MOTIVATION ....................................................................................................................... 3 1.4 CONTRIBUTION .................................................................................................................... 3 1.5 THESIS ORGANIZATION ....................................................................................................... 3 CHAPTER 2 BACKGROUND ................................................................................................. 5 2.1 ARABIC DATASETS .............................................................................................................. 5 2.2 ARTIFICIAL NEURAL NETWORKS (ANN) ............................................................................ 6 2.2.1 Recurrent Neural Networks (RNN) ............................................................................. 7 2.3 CONDITIONAL RANDOM FIELD (CRF) ................................................................................. 9 2.4 WORD EMBEDDINGS ......................................................................................................... 10 2.4.1 Static Embeddings ..................................................................................................... 11 2.4.2 Contextual Embeddings ............................................................................................ 12 CHAPTER 3 LITERATURE REVIEW ON NAMED ENTITY RECOGNITION .......... 14 3.1 PREAMBLE ......................................................................................................................... 14 3.2 RULE-BASED APPROACHES FOR NER USING GAZEETTER AND POS .................................. 14 3.3 MACHINE LEARNING APPROACHES FOR NER ................................................................... 18 3.3.1 A Machine Learning Approach using Brown Clustering Technique ........................ 18 3.3.2 Machine Learning Approach using Bayesian Classifier Combination .................... 19 3.4 HYBRID APPROACHES FOR NER ....................................................................................... 20 3.4.1 A Hybrid Approach using Naïve Bayes Classifier and Dictionary ........................... 20 3.4.2 A Hybrid Approach using Decision Tree and POS ................................................... 21 3.5 DEEP LEARNING APPROACHES FOR NER .......................................................................... 22 3.5.1 A Deep Learning Approaches using Conditional Random Field.............................. 23 3.5.2 A Deep Learning Approaches using Attention Mechanism ...................................... 28 3.5.3 A Deep Learning Approach using Pooled Contextualized Embeddings ................... 32 vi 3.5.4 A Deep Learning Approach using Semi-Supervised Co-Training (SVM and BiLSTM-CRF) .................................................................................................................................. 33 3.6 CHAPTER SUMMARY ......................................................................................................... 36 CHAPTER 4 RESEARCH METHODOLOGY .................................................................... 37 4.1 PREAMBLE ......................................................................................................................... 37 4.2 DATASET ........................................................................................................................... 37 4.3 PROPOSED MODEL ............................................................................................................ 39 4.3.1 Multi-Embeddings ..................................................................................................... 41 4.3.2 Encoding ................................................................................................................... 44 4.3.3 Decoding ................................................................................................................... 44 4.4 TESTING OF OTHER MULTI-EMBEDDINGS MODELS ........................................................... 45 4.4.1 Pooled Contextual Embedding with Word2Vec ........................................................ 45 4.4.2 Pooled Contextual Embedding with fastText ............................................................ 47 4.4.3 Pooled Contextual Embedding with Multilingual BERT .......................................... 49 4.4.4 Pooled Contextual Embedding with Arabic BERT ................................................... 50 4.4.5 Pooled Contextual Embedding with fastText and Word2Vec ................................... 51 4.4.6 Pooled Contextual Embedding with Arabic BERT, and Word2Vec ......................... 53 4.4.7 Pooled Contextual Embedding with XLNet, and fastText ......................................... 54 4.4.8 Pooled Contextual Embedding with XLM-RoBERTa, and fastText .......................... 55 4.4.9 Pooled Contextual Embedding with Arabic BERT, and fastText .............................. 56 CHAPTER 5 EVALUATION ................................................................................................. 58 5.1 PREAMBLE ...................................................................................................................... 58 5.2 EXPERIMENTATION SETUP AND IMPLEMENTATION............................................................ 58 5.3 QUALITY METRICS ............................................................................................................ 58 5.4 RESULTS AND DISCUSSION ................................................................................................ 59 5.5 ERROR ANALYSIS .............................................................................................................. 64 5.6 CHAPTER SUMMARY ......................................................................................................... 65 CHAPTER 6 CONCLUSION AND FUTURE WORK ........................................................ 67 6.1 CONCLUSION ..................................................................................................................... 67 6.2 FUTURE WORK .................................................................................................................. 67 REFERENCES .........................................................................................................................
520 3 _aAbstract: Named Entity Recognition (NER) is an essential task in many natural language processing applications. Extracting crucial information using NER is a primary phase in most NLP downstream tasks, it is used to identify entities into predefined classes. There are several studies that have focused on NER for the English language. However, there are some limitations when applying the current methodologies directly to the Arabic language text. Recent studies have shown the effectiveness of contextual embedding representations and their significant improvements that were shown in English NER tasks. The aim of this thesis is to examine the hypotheses, that a multi-embeddings approach coupled with a deep learning network is capable of extracting named entities from Arabic sentences and thereby reduces the manual costs. Also, to examine the hypothesis, that the end-to-end deep learning NER approach is capable to outperform the traditional models based on rule-based or machine learning approaches. To evaluate these hypotheses, an end-to-end deep learning model has been presented which was able to extract Arabic Named Entities and eventually reduce human efforts such as preprocessing work and feature engineering. To reach the best model, different types of classical and contextual embeddings have been combined and evaluated. Thus, the presented model utilized a combination of several types of embeddings (multi-embeddings) such as traditional word embeddings as well as contextual embeddings to gain the advantage of both. For encoding and decoding, we chose to use the models that are heavily used in sequential data modeling. The results of the proposed model achieved a competitive advantage, outperforming all previously published results of deep learning, and non-deep learning models working on the same dataset. The presented results also surpassed those of the winning systems for the same task on the same data in the Topcoder.com competition. Since this approach does not rely on any external resources or handcrafted features, the model can be easily extended to other Arabic NER domains.
546 _aText in English, abstracts in English.
650 4 _aInformatics-IFM
_9266
655 7 _2NULIB
_aDissertation, Academic
_9187
690 _aInformatics-IFM
_9266
942 _2ddc
_cTH
999 _c9080
_d9080