Balancing Performance and Explainability in Deep Learning for Lung Disease Detection Using Unimodal and Multimodal Approaches /Ghadir Moemen Helmy Ali

Supervisor:
Dr. Mustafa Elattar

Thesis (M.A.)—Nile University, Egypt, 2025 .

"Includes bibliographical references"

Contents:
Contents
Page
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
List of Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
Chapters:
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Summary of Contributions . . . . . . . . . . . . . . . . . . . . 5
1.5 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 Inherent Deep Learning Trade-offs . . . . . . . . . . . . . . . 7
2.2 Explainable AI and Clinical Applicability . . . . . . . . . . . 9
2.3 Lung Diseases Clinical Background . . . . . . . . . . . . . . . 15
2.3.1 Non-Small Cell Lung Carcinoma (NSCLC) . . . . . . . 16
2.3.2 Pneumonia . . . . . . . . . . . . . . . . . . . . . . . . 18
3. Literature Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.1 Explainable AI and Clinical Reasoning . . . . . . . . . . . . . 22
3.1.1 Explainable AI Approaches . . . . . . . . . . . . . . . 25
3.1.2 Evaluation of Explainable AI Methods . . . . . . . . . 34
3.1.3 Limitations and Future Directions . . . . . . . . . . . 42
xi
3.2 Attention Mechanisms in Deep Learning . . . . . . . . . . . . 44
3.2.1 Main Approaches . . . . . . . . . . . . . . . . . . . . . 49
3.2.2 Attention for Explainable AI . . . . . . . . . . . . . . 61
3.3 Pneumonia Classification from Chest X-Ray . . . . . . . . . . 66
3.4 Non-Small Cell Lung Carcinoma . . . . . . . . . . . . . . . . 70
3.4.1 Data Types . . . . . . . . . . . . . . . . . . . . . . . . 73
3.4.2 AI Contribution . . . . . . . . . . . . . . . . . . . . . 83
3.5 Literature Survey Summary . . . . . . . . . . . . . . . . . . . 102
4. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
4.1 Pneumonia . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.1.1 Dataset and Preprocessing . . . . . . . . . . . . . . . . 107
4.1.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.2 Non-Small Cell Lung Carcinoma . . . . . . . . . . . . . . . . 113
4.2.1 Dataset Description and Inclusion Criteria . . . . . . . 114
4.2.2 Preprocessing and Featurization . . . . . . . . . . . . . 116
4.2.3 Models . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.1 Pneumonia . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.2 Non-Small Cell Lung Carcinoma . . . . . . . . . . . . . . . . 136
5.2.1 Molecular Features & Model . . . . . . . . . . . . . . . 136
5.2.2 Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . 136
6. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
6.1 Pneumonia . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
6.2 Non-Small Cell Carcinoma . . . . . . . . . . . . . . . . . . . . 145
7. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.1 Key Findings and Contributions . . . . . . . . . . . . . . . . 147
7.1.1 Pneumonia Classification with Attention Mechanisms . 147
7.1.2 Non-Small Cell Carcinoma Metastasis Detection using
Radiogenomics . . . . . . . . . . . . . . . . . . . . . . 148
7.1.3 Explainability and Clinical Usability . . . . . . . . . . 148
7.2 Implications for AI in Healthcare . . . . . . . . . . . . . . . . 149
7.3 Limitations and Future Directions . . . . . . . . . . . . . . . 149
7.4 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . 150
8. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
8.1 Multi-Modal Imaging Expansion . . . . . . . . . . . . . . . . 151
8.2 Advanced Multi-Information Fusion Strategies . . . . . . . . . 152
8.3 Dataset Development and Clinical Validation . . . . . . . . . 152
xii
8.4 Toward Human-Centric Explainable AI . . . . . . . . . . . . . 153
Appendices:
A. Publications and Competitions . . . . . . . . . . . . . . . . . . . . 155
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
Arabic Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Abstract:
The adoption of deep learning in healthcare applications has highlighted a
critical trade-off between model performance and explainability, particularly in
clinical decision-making contexts. This thesis explores methods to address this
trade-off through two key studies focusing on lung disease detection: pneumonia
classification using chest X-rays and metastatic classification for non-small-cell
lung cancer using CT imaging and genomics.
In our work with pneumonia, attention mechanisms were integrated into
DL models, serving as a balance between post-hoc and inherently explainable
approaches. The use of attention improved classification performance while
enhancing model focus on clinically relevant regions, as demonstrated through
Grad-CAM visualizations. Attention proved to be an effective explainability
choice when chest X-ray was the sole data source.
For the non-small-cell lung cancer study, we adopted a multimodal approach
combining CT imaging and RNA-sequencing data. Ablation studies revealed
a novel clinical insight: surrounding tissue regions were more predictive of
metastasis than tumor regions, aligning with Grad-CAM visualizations that
emphasized tissue edges. This finding correlates with medical knowledge where
tissue involvement signals metastasis progression. Genomic features alone
exhibited limited predictive power but improved model performance when fused
with imaging data. In contrast, handcrafted features, such as shape and size
metrics, showed poor predictive capability. Given access to multimodal data,
vii
this study opens the door for exploring more clinically meaningful explainability
techniques to further align AI model outputs with clinical reasoning.
These results underscore the importance of context-specific explainability
approaches in healthcare AI. While attention mechanisms are effective for
unimodal data like chest X-ray, complex tasks with rich multimodal inputs,
such as non-small-cell lung cancer classification, can benefit from advanced
explainability methods. This thesis contributes to developing clinically explainable AI systems that strike a balance between performance and interpretability,
ensuring their utility in real-world medical applications.
Keywords
Deep Learning, Convolutional Neural Networks, Attention Mechanisms, Explainable Artificial Intelligence, Lung Diseases, Medical Imagining, Bioinformatics,
Radiogenomics, Clinical Applicability

Text in English, abstracts in English and Arabic