000			06952nam a22002657a 4500
008			201210b2025 a|||f bm|| 00| 0 eng d
024	7		_a0009-0000-0316-2724 _2ORCID
040			_aEG-CaNU _cEG-CaNU
041	0		_aeng _beng _bara
082			_a610
100	0		_aMina Atef Yousef Wahba _93688
245	1		_aAutomated Code Assessment: _bFrom Dataset Construction To Distributed LLM-Powered Grading _c/Mina Atef Yousef Wahba
260			_c2025
300			_a93 p. _bill. _c21 cm.
500			_3Supervisor: Ghada Khoriba Tamer Arafa
502			_aThesis (M.A.)—Nile University, Egypt, 2025 .
504			_a"Includes bibliographical references"
505	0		_aContents: Contents LIST OF FIGURES xi LIST OF TABLES xi 1 Introduction 1 1.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Evolution of Educational Technology . . . . . . . . . . . . . . . . . . 2 1.1.2 Limitations of Traditional Grading in Programming . . . . . . . . . . . 3 1.1.3 Emergence and Potential of LLMs and Big Data . . . . . . . . . . . . 4 1.2 Research Contributions and Significance . . . . . . . . . . . . . . . . . . . . . 5 1.2.1 Research Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2.2 Significance of the Study . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Literature Review 7 2.1 Background and Taxonomy of LLMs . . . . . . . . . . . . . . . . . . . . . . . 7 2.1.1 General Purpose LLMs . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1.2 Domain Specific LLMs . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1.3 MoE Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 LLMs in Education Applications . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 LLM in Grading and Feedback Systems . . . . . . . . . . . . . . . . . . . . . 13 2.4 Infrastructure for Scalable LLM Deployment . . . . . . . . . . . . . . . . . . 15 2.5 Identified Research Gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 vii CONTENTS 3 Methodology 19 3.1 Dataset Collection and Pre-processing . . . . . . . . . . . . . . . . . . . . . . 19 3.1.1 Real-Time Data Processing . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1.2 Data Augmentation and Annotation . . . . . . . . . . . . . . . . . . . 23 3.1.3 Annotation of Real Campus Data . . . . . . . . . . . . . . . . . . . . 23 3.1.4 Synthetic Data Generation from Programming Books . . . . . . . . . . 25 3.2 BeGrading: Fine-Tuned Model for Automated Programming Grading . . . . . 26 4 Results & Discussion 28 4.1 Experimental Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.1.1 Hardware Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.1.2 Dataset Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.1.3 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.1.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.2 Dataset Analysis and Characteristics . . . . . . . . . . . . . . . . . . . . . . . 33 4.2.1 Real-Time Processing Performance . . . . . . . . . . . . . . . . . . . 33 4.2.2 Dataset Augmentation and Annotation . . . . . . . . . . . . . . . . . 36 4.3 Begrading: Fine-tuned Model Results . . . . . . . . . . . . . . . . . . . . . . 44 4.3.1 Effect of Dataset Grade Correction on High-Variance Records . . . . . 45 4.4 Ablation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.4.1 Overall Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.4.2 Sensitivity to Grading Guidance . . . . . . . . . . . . . . . . . . . . . 49 4.4.3 Effectiveness of Fine-Tuning . . . . . . . . . . . . . . . . . . . . . . . 50 4.5 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5 Conclusions and Future Work 56 5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 A Appendix 73
520	3		_aAbstract: rading programming assignments in educational settings presents several challenges, including computational resource constraints, inconsistent evaluation criteria, subjectivity, and delayed feedback delivery, all of which can hinder student learning progress. Although large language models (LLMs) demonstrate promising capabilities for automated code evaluation, existing approaches face limitations in scalability, processing efficiency, and deployment feasibility in resource-constrained educational environments. This research presents a comprehensive solution that addresses these challenges through an integrated architecture that combines real student-annotated data, synthetic data generation, specialized model fine-tuning, and a distributed computing infrastructure. We develop a framework for creating high-quality training datasets through both authentic student submissions and a synthetic data generation architecture that produces realistic programming assignments and student responses, while addressing concerns regarding privacy and data insufficiency. Based on this enhanced dataset, we fine-tune BeGrading, a specialized LLM optimized for comprehensive code evaluation across multiple dimensions, including correctness, efficiency, and coding style. BeGrading achieves superior performance with an absolute difference rate of 19% (± 0.95 of 5) compared to the Codestral-22B model of reference, demonstrating effective optimization for educational settings. To overcome computational limitations, we implement a scalable, distributed infrastructure that utilizes Apache Spark Streaming with GPU-accelerated worker nodes, enabling the real-time processing of high-volume grading tasks. Our experimental evaluation demonstrates significant performance improvements, with inference time reductions of up to 50% using dual workers and 70-80% with three workers, while maintaining grading accuracy. The system includes robust failover mechanisms that ensure continuous operation and a reliable distribution of tasks between nodes. This work presents a practical and scalable solution for automated programming assessment that significantly reduces instructor workload while providing students with timely, consistent, and objective feedback. This approach ultimately enhances educational outcomes by improving efficiency and reliability in code evaluation processes. Keywords Large language model, grade, spark, cluster, programming education, synthetic data generation, fine-tuning architecture
546			_aText in English, abstracts in English and Arabic
650		4	_aInformaticsIFM
655		7	_2NULIB _aDissertation, Academic _9187
690			_aInformaticsIFM
942			_2ddc _cTH
999			_c11029 _d11029