Mina Atef Yousef Wahba <br/><br/>
Automated Code Assessment:   From Dataset Construction To Distributed LLM-Powered Grading 
/Mina Atef Yousef Wahba 
 -  2025 
 - 93 p.   ill.   21 cm.  
<br/><br/>Supervisor: <br/>Ghada Khoriba<br/>Tamer Arafa<br/> 
<br/><br/>Thesis (M.A.)—Nile University, Egypt, 2025 . 
<br/><br/>"Includes bibliographical references" 
<br/><br/>Contents:<br/>Contents<br/>LIST OF FIGURES xi<br/>LIST OF TABLES xi<br/>1 Introduction 1<br/>1.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1<br/>1.1.1 Evolution of Educational Technology . . . . . . . . . . . . . . . . . . 2<br/>1.1.2 Limitations of Traditional Grading in Programming . . . . . . . . . . . 3<br/>1.1.3 Emergence and Potential of LLMs and Big Data . . . . . . . . . . . . 4<br/>1.2 Research Contributions and Significance . . . . . . . . . . . . . . . . . . . . . 5<br/>1.2.1 Research Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 5<br/>1.2.2 Significance of the Study . . . . . . . . . . . . . . . . . . . . . . . . . 5<br/>1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6<br/>2 Literature Review 7<br/>2.1 Background and Taxonomy of LLMs . . . . . . . . . . . . . . . . . . . . . . . 7<br/>2.1.1 General Purpose LLMs . . . . . . . . . . . . . . . . . . . . . . . . . . 7<br/>2.1.2 Domain Specific LLMs . . . . . . . . . . . . . . . . . . . . . . . . . . 9<br/>2.1.3 MoE Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10<br/>2.2 LLMs in Education Applications . . . . . . . . . . . . . . . . . . . . . . . . . 10<br/>2.3 LLM in Grading and Feedback Systems . . . . . . . . . . . . . . . . . . . . . 13<br/>2.4 Infrastructure for Scalable LLM Deployment . . . . . . . . . . . . . . . . . . 15<br/>2.5 Identified Research Gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17<br/>vii<br/>CONTENTS<br/>3 Methodology 19<br/>3.1 Dataset Collection and Pre-processing . . . . . . . . . . . . . . . . . . . . . . 19<br/>3.1.1 Real-Time Data Processing . . . . . . . . . . . . . . . . . . . . . . . . 19<br/>3.1.2 Data Augmentation and Annotation . . . . . . . . . . . . . . . . . . . 23<br/>3.1.3 Annotation of Real Campus Data . . . . . . . . . . . . . . . . . . . . 23<br/>3.1.4 Synthetic Data Generation from Programming Books . . . . . . . . . . 25<br/>3.2 BeGrading: Fine-Tuned Model for Automated Programming Grading . . . . . 26<br/>4 Results & Discussion 28<br/>4.1 Experimental Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28<br/>4.1.1 Hardware Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 28<br/>4.1.2 Dataset Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . 29<br/>4.1.3 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30<br/>4.1.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 31<br/>4.2 Dataset Analysis and Characteristics . . . . . . . . . . . . . . . . . . . . . . . 33<br/>4.2.1 Real-Time Processing Performance . . . . . . . . . . . . . . . . . . . 33<br/>4.2.2 Dataset Augmentation and Annotation . . . . . . . . . . . . . . . . . 36<br/>4.3 Begrading: Fine-tuned Model Results . . . . . . . . . . . . . . . . . . . . . . 44<br/>4.3.1 Effect of Dataset Grade Correction on High-Variance Records . . . . . 45<br/>4.4 Ablation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48<br/>4.4.1 Overall Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . 48<br/>4.4.2 Sensitivity to Grading Guidance . . . . . . . . . . . . . . . . . . . . . 49<br/>4.4.3 Effectiveness of Fine-Tuning . . . . . . . . . . . . . . . . . . . . . . . 50<br/>4.5 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51<br/>5 Conclusions and Future Work 56<br/>5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56<br/>5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57<br/>A Appendix 73 
<br/><br/> Abstract:<br/>rading programming assignments in educational settings presents several challenges, including computational resource constraints, inconsistent evaluation criteria, subjectivity, and delayed feedback delivery, all of which can hinder student learning progress. Although large<br/>language models (LLMs) demonstrate promising capabilities for automated code evaluation,<br/>existing approaches face limitations in scalability, processing efficiency, and deployment feasibility in resource-constrained educational environments. This research presents a comprehensive solution that addresses these challenges through an integrated architecture that combines<br/>real student-annotated data, synthetic data generation, specialized model fine-tuning, and a distributed computing infrastructure.<br/>We develop a framework for creating high-quality training datasets through both authentic student submissions and a synthetic data generation architecture that produces realistic programming assignments and student responses, while addressing concerns regarding privacy and data<br/>insufficiency. Based on this enhanced dataset, we fine-tune BeGrading, a specialized LLM optimized for comprehensive code evaluation across multiple dimensions, including correctness,<br/>efficiency, and coding style. BeGrading achieves superior performance with an absolute difference rate of 19% (± 0.95 of 5) compared to the Codestral-22B model of reference, demonstrating effective optimization for educational settings.<br/>To overcome computational limitations, we implement a scalable, distributed infrastructure that<br/>utilizes Apache Spark Streaming with GPU-accelerated worker nodes, enabling the real-time<br/>processing of high-volume grading tasks. Our experimental evaluation demonstrates significant<br/>performance improvements, with inference time reductions of up to 50% using dual workers<br/>and 70-80% with three workers, while maintaining grading accuracy. The system includes<br/>robust failover mechanisms that ensure continuous operation and a reliable distribution of tasks<br/>between nodes. This work presents a practical and scalable solution for automated programming<br/>assessment that significantly reduces instructor workload while providing students with timely,<br/>consistent, and objective feedback. This approach ultimately enhances educational outcomes by<br/>improving efficiency and reliability in code evaluation processes.<br/>Keywords<br/>Large language model, grade, spark, cluster, programming education, synthetic data generation,<br/>fine-tuning architecture 
<br/><br/><br/>Text in English, abstracts in English and Arabic 
<br/><br/><label>Standard No.: </label>0009-0000-0316-2724  ORCID 
<br/><br/><label>Subjects--Topical Terms: </label><br/>InformaticsIFM
<br/><br/><label>Index Terms--Genre/Form: </label><br/>Dissertation, Academic
<br/><br/><label>Dewey Class. No.: </label>610