{"id":68436,"date":"2025-10-21T08:25:11","date_gmt":"2025-10-21T08:25:11","guid":{"rendered":"https:\/\/biomedpharmajournal.org\/?p=68436"},"modified":"2025-10-22T06:23:12","modified_gmt":"2025-10-22T06:23:12","slug":"an-explainable-ai-based-ensemble-framework-for-brain-tumor-mri-classification-and-automated-clinical-reporting-using-deep-learning-and-llm-integration","status":"publish","type":"post","link":"https:\/\/biomedpharmajournal.org\/staging\/vol18octoberspledition\/an-explainable-ai-based-ensemble-framework-for-brain-tumor-mri-classification-and-automated-clinical-reporting-using-deep-learning-and-llm-integration\/","title":{"rendered":"An Explainable AI-Based Ensemble Framework for Brain Tumor MRI Classification and Automated Clinical Reporting Using Deep Learning and LLM Integration"},"content":{"rendered":"

Introduction<\/strong><\/p>\n

Cerebral neoplasms are among the most critical and life-threatening of the neurological illnesses, and accurate diagnosis of such tumors is vital for prompt intervention and treatment planning. Magnetic Resonance Imaging (MRI) has been widely regarded as the gold standard for brain tumor diagnosis due to its superior resolution for imaging soft tissue.1<\/sup> However, interpretation of the MRI scan remains plagued by serious issues heavily dependent on the expertise of radiologists, leading to widespread inconsistency and possible delay in diagnosis.<\/p>\n

Over the last several years, the integration of Artificial Intelligence (AI), machine learning,2,3,<\/sup> and particularly deep learning into medical imaging has seen impressive success in the automation of image-based diagnosis procedures and enhanced clinical effectiveness.4,5<\/sup> Despite the success, one of the major problems that afflicts AI systems within the health care industry is the black box nature of deep learning models, which may reduce the confidence of health care professionals. Explainable AI (XAI) techniques have emerged as major solutions to address the issue, providing both visual and quantitative explanations of model predictions and enhanced transparency.6<\/sup><\/p>\n

To overcome such limitations, we introduce a better MRI Analysis AI Application that utilizes an ensemble of state-of-the-art Convolutional Neural Networks, including DenseNet121, ResNet50, EfficientNet-B0, and MobileNetV3-Small, to efficiently classify brain MRI scans into four clinically relevant classes: Glioma, Meningioma, No Tumor, and Pituitary Tumor. The ensemble process increases the consistency of the classification process, and preprocessing methods ensure consistency over diverse image inputs.<\/p>\n

A distinguishing feature of this study is the use of different explainable artificial intelligence (XAI) techniques like Grad-CAM,7<\/sup> LIME,8<\/sup> SHAP,9,<\/sup> and a specially designed Saliency-based Neural Activation Pattern (SNAP) to facilitate multi-faceted interpretability. Furthermore, integration with a Large Language Model (LLM) utilizing the assistance of LangChain-Groq facilitates the creation of interpretable diagnostic reports, thus bridging the communication gap between artificial intelligence systems and medical professionals. The novelty lies in this end-to-end, interpretable, and deployable system, therefore rendering it suitable for real-time, clinician-led decision support in radiology clinics.<\/p>\n

The novelty of this research lies in the development of a comprehensive end-to-end pipeline that not only delivers high classification accuracy through a CNN ensemble but also integrates multiple Explainable AI methods like Grad-CAM, LIME, SHAP, and Edge Detection to ensure clinical transparency. Unlike prior studies that emphasize either accuracy or interpretability, our framework offers both simultaneously, making predictions reliable as well as understandable to radiologists. In addition, the system uniquely incorporates a Large Language Model (via LangChain-Groq) to generate automated, human-readable clinical reports, thereby bridging the gap between AI outputs and clinical workflows. This fusion of ensemble learning, interpretability, and automated reporting represents a distinct contribution, positioning the proposed framework as a deployable and clinician-friendly solution for real-time brain tumor diagnosis.<\/p>\n

Literature Review<\/strong><\/p>\n

Recent advances in deep learning (DL) have significantly improved the classification of brain tumors using MRI data.10,11<\/sup> Abdusalomov et al.12 <\/sup>confirmed the utility of DL for brain tumor detection using multi-sequence MRI, although the lack of interpretability remained a concern.<\/p>\n

Explainable AI (XAI) techniques have been increasingly adopted to address this. Sadeghi et al.13 <\/sup>emphasized the need for transparency in DL predictions, particularly in clinical applications. Xiao et al.14<\/sup> enhanced Grad-CAM visualizations for medical image segmentation to improve interpretability in clinical imaging tasks. Kun et al.15<\/sup> proposed an optimized LIME framework tailored for low-light medical images, enhancing local feature attribution. Sun et al16 <\/sup>utilized SHAP to uncover age-based risk factors in mammography data, supporting personalized diagnostic insights. These techniques enable visualization of decision-making processes, helping radiologists validate AI-generated results. Such tools have been applied in Alzheimer’s detection,17<\/sup> dermatology,18<\/sup> and ophthalmology,19<\/sup> proving their value in healthcare AI.<\/p>\n

Cheng et al.20 <\/sup>proposed tumor region augmentation techniques for enhancing classification robustness in MRI scans. Public datasets like Figshare, SARTAJ,21<\/sup> and Br35H 22<\/sup> have become standard benchmarks for training and evaluating brain tumor classification models. Ensemble models combining DenseNet, ResNet, and EfficientNet have shown superior accuracy over single-architecture models,23, 24<\/sup> and Mustapha et al. 25<\/sup> demonstrated improved performance using hybrid ConvNeXt frameworks for leukemia subtype detection.<\/p>\n

Recently, the integration of Large Language Models (LLMs) into medical AI pipelines has opened new dimensions for automated interpretation. LangChain-powered models like BLIP, 26<\/sup> Med-PaLM, 27<\/sup> and ClinicalT5 28<\/sup> provide human-readable diagnostic summaries from model outputs, bridging the gap between AI reasoning and clinical narratives.<\/p>\n

In summary, existing literature underscores the synergy between high-performance DL models, explainability tools, and natural language interfaces. The proposed study contributes to this growing body by offering a unified system integrating CNN ensembles, multiple XAI methods, and LLM-driven report generation.<\/p>\n

Materials and Methods<\/strong><\/p>\n

Dataset Explanation<\/strong><\/p>\n

The dataset employed in this study is an aggregation of three publicly available brain MRI datasets: Figshare, SARTAJ, and Br35H, collectively comprising a total of 7023 images. The images are divided into four categories: Glioma, Meningioma, Pituitary Tumor, and absence of a tumor, covering a broad spectrum of brain abnormalities. The inclusion of diverse data sources ensures greater generalizability across patient populations and imaging conditions. Each image underwent preprocessing, including grayscale conversion, resizing to 224\u00d7224 pixels, and intensity normalization to standardize the inputs across models. The dataset was stratified into an 80:20 ratio for training and testing, maintaining balanced representation across all classes. These datasets have been previously validated in several peer-reviewed studies and are recognized benchmarks for brain tumor classification using deep learning approaches. 20, 21, 22 <\/sup>Fig.1 shows a representative MRI image selected from the curated dataset used in this study, which includes images from Figshare, SARTAJ, and Br35H repositories. The input image undergoes preprocessing steps such as grayscale conversion, resizing, and normalization before being fed into the CNN ensemble model for classification into one of the four tumor categories: Glioma, Meningioma, Pituitary Tumor, or No Tumor. This sample image demonstrates the system\u2019s initial input format prior to analysis and classification.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 1: Sample Input Brain MRI Image from the Dataset<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Proposed System Workflow Overview<\/strong><\/p>\n

Fig. 2 illustrates the overall process flow diagram of the proposed MRI Analysis AI app. The pipeline consists of nine key modules arranged in a systematic and interpretable flow from raw image ingestion to final report generation.<\/p>\n

The workflow starts with the Dataset module, containing 7,023 brain MRI scans collected from the Figshare, SARTAJ, and Br35H datasets. These images are first subjected to a Pre-processing phase involving grayscale conversion, resizing (224\u00d7224), normalization, and data augmentation. Following preprocessing, the images are fed into the Training module, where a CNN ensemble is constructed using four well-known architectures: DenseNet121, ResNet50, EfficientNet-B0, and MobileNetV3-Small. To handle class imbalance, these networks are trained independently with stratified data splits and cyclic learning rate optimization. The optimal model is chosen by evaluating its validation accuracy and loss metrics. This model is then used for Prediction, where MRI scans are categorized into one of the four classes: Glioma, Meningioma, No Tumor, or Pituitary Tumor.<\/p>\n

To enhance model transparency, the framework includes an Explainability Analysis unit employing multiple techniques: LIME for local explanations, SHAP for feature attributions (not shown in the diagram but included in the manuscript), SNAP for saliency mapping, Grad-CAM for class-specific localization, and Edge Detection for structural enhancement. The insights derived from explainability modules are passed into the LLM Integration component, powered by LangChain-Groq, which translates predictions and visual explanations into human-readable medical reports.<\/p>\n

Subsequently, the Result Visualization module organizes outputs, including class labels and saliency maps, in a structured format. Finally, the system executes Report Generation using the FPDF library, embedding clinical summaries and annotated images into a downloadable PDF file.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 2:\u00a0 Process Flow Diagram of the Proposed MRI Analysis AI System<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Data Preprocessing<\/strong><\/p>\n

An end-to-end preprocessing pipeline was employed to provide uniform data to train and test the MRI scans, allowing the model to generalize more effectively. First, all the MRI images were normalized from RGB to grayscale to match the single-channel input layers of the modified convolutional neural networks (CNNs). The images were then resized to the size 224\u00d7224 pixels, the universal input size for deep learning models. To make it more robust, the dataset was exposed to different data augmentation methods, including random flip of the image horizontally, rotation of \u00b115\u00b0, affine transformations (with minor shifts and zoomed-in images), and random cropping of 80%\u2013100 % of the image space. Additionally, color jittering was done to simulate changes in brightness and contrast, even though the images were grayscale. After these augmentation steps, the images were converted to tensors and normalized with a mean of 0.5 and a standard deviation of 0.5 in order to stabilize training dynamics and accelerate convergence. Table 1 shows the order of preprocessing methodologies applied to brain MRI images, including grayscale conversion, resizing, augmentation, and normalization, along with their corresponding technical parameters to provide uniform input to the CNN ensemble model.<\/p>\n

Table 1: MRI Image Preprocessing Pipeline and Technical Specifications<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n
\n

Step<\/strong><\/p>\n<\/td>\n

\n

Purpose<\/strong><\/p>\n<\/td>\n

\n

Technical Detail<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

Grayscale Conversion<\/p>\n<\/td>\n

\n

Convert RGB images to single-channel grayscale<\/p>\n<\/td>\n

\n

1 output channel (L-mode image)<\/p>\n<\/td>\n<\/tr>\n

\n

Resizing<\/p>\n<\/td>\n

\n

Standardize input size to 224\u00d7224 pixels<\/p>\n<\/td>\n

\n

Resize to (224, 224)<\/p>\n<\/td>\n<\/tr>\n

\n

Random Horizontal Flip<\/p>\n<\/td>\n

\n

Introduce left-right variability in training data<\/p>\n<\/td>\n

\n

Flip probability = 0.5<\/p>\n<\/td>\n<\/tr>\n

\n

Random Rotation<\/p>\n<\/td>\n

\n

Add rotational invariance<\/p>\n<\/td>\n

\n

Max rotation = \u00b115\u00b0<\/p>\n<\/td>\n<\/tr>\n

\n

Affine Transformation<\/p>\n<\/td>\n

\n

Simulate minor shifts and zooms<\/p>\n<\/td>\n

\n

Translate = \u00b110%, Scale = 90\u2013110%<\/p>\n<\/td>\n<\/tr>\n

\n

Random Resized Crop<\/p>\n<\/td>\n

\n

Enhance generalization by random cropping<\/p>\n<\/td>\n

\n

Crop scale = 80%\u2013100% of image area<\/p>\n<\/td>\n<\/tr>\n

\n

Color Jitter<\/p>\n<\/td>\n

\n

Improve robustness to brightness\/contrast changes<\/p>\n<\/td>\n

\n

Brightness & contrast \u00b120%<\/p>\n<\/td>\n<\/tr>\n

\n

Tensor Conversion<\/p>\n<\/td>\n

\n

Convert PIL image to PyTorch tensor format<\/p>\n<\/td>\n

\n

Image converted to float tensor<\/p>\n<\/td>\n<\/tr>\n

\n

Normalization<\/p>\n<\/td>\n

\n

Scale pixel values to [-1, 1] for stable training<\/p>\n<\/td>\n

\n

Mean = 0.5, Std = 0.5<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Model Training<\/strong><\/p>\n

The training of the model is designed to efficiently process the classification of brain MRI images by utilizing a convolutional neural network (CNN) ensemble method. Initially, a custom MRIDataset class is created to support the loading of grayscale images and mapping integer labels for four tumor types: Glioma (0), Meningioma (1), No Tumor (2), and Pituitary Tumor (3). The CNN ensemble is comprised of four pre-trained models: DenseNet121, ResNet50, EfficientNet-B0, and MobileNetV3-Small. Each model is adapted to support single-channel input by altering their first convolutional layer.<\/p>\n

Model training uses the Adam optimizer along with a cyclic learning rate schedule to enhance convergence. A weighted CrossEntropyLoss function is applied to handle class imbalance. The dataset is divided into training and validation sets in an 80:20 ratio, ensuring balanced class representation through stratified sampling. Performance is assessed using accuracy, precision, recall, and F1-score, while early stopping with a patience of 5 epochs helps reduce the risk of overfitting. The model achieving either the highest validation accuracy or the lowest loss is saved for subsequent evaluation and testing. The complete technical breakdown of each training step is presented in the accompanying Table 2.<\/p>\n

Table 2: MRI Model Training Process Technical Summary<\/strong><\/p>\n\n\n\n\n\n\n\n\n
\n

Step<\/strong><\/p>\n<\/td>\n

\n

Description<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

Dataset Loading<\/p>\n<\/td>\n

\n

Custom MRIDataset class loads grayscale images and assigns class labels: Glioma (0), Meningioma (1), No Tumor (2), Pituitary Tumor (3).<\/p>\n<\/td>\n<\/tr>\n

\n

Model Ensemble<\/p>\n<\/td>\n

\n

Train DenseNet121, ResNet50, EfficientNet-B0, and MobileNetV3-Small with modified first layers for 1-channel inputs.<\/p>\n<\/td>\n<\/tr>\n

\n

Optimization<\/p>\n<\/td>\n

\n

Uses Adam optimizer with CyclicLR scheduler and weighted CrossEntropyLoss to handle class imbalance.<\/p>\n<\/td>\n<\/tr>\n

\n

Validation<\/p>\n<\/td>\n

\n

Splits dataset 80:20 for training and validation; evaluates using accuracy, precision, recall, and F1 score with early stopping (patience=5).<\/p>\n<\/td>\n<\/tr>\n

\n

Best Model Selection<\/p>\n<\/td>\n

\n

Saves the model with either the highest validation accuracy or the lowest loss during training.<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Mathematical Modelling of Core Components<\/strong><\/p>\n

Although the present work emphasizes an applied framework for brain tumor classification and automated reporting, the mathematical underpinnings of the main components are outlined here to provide theoretical clarity.<\/p>\n

Each convolutional layer applies a set of filters to the input MRI image. The operation is expressed as:<\/p>\n

\"\"<\/p>\n

Where x<\/em> denotes the input image patch, w(k)<\/sup><\/em>\u00a0is the kernel of the kth<\/sup><\/em> filter, b(k)<\/sup><\/em>is the bias term, and \u03c3(.) is the non-linear activation function (ReLU here).<\/p>\n

To optimize multi-class tumor classification, we employed a weighted cross-entropy loss:<\/p>\n

\"\"<\/p>\n

where C<\/em> is the total number of classes, \u200byc<\/sub><\/em> is the ground-truth label, yc<\/sub><\/em> is the predicted probability, and wc<\/sub><\/em>\u200b is the class-specific weight to mitigate class imbalance.<\/p>\n

The model parameters are updated iteratively using the Adam optimizer, defined as:<\/p>\n

\"\"<\/p>\n

Where \u03b7 is the learning rate mt<\/sub>, \u00a0and ut<\/sub>\u00a0are the bias-corrected first and second moment estimates of the gradients, \u03f5 is a constant for numerical stability.<\/p>\n

Although a detailed description of explainability methods is provided later (Section 3.6), the formulation of Grad-CAM is briefly given here. For a target class \u2018c\u2019, the Grad-CAM heatmap is computed as:<\/p>\n

\"\"<\/p>\n

where Ak<\/sup> is the activation\u00a0<\/sup>map of the kth<\/sup><\/em> feature channel, and \u200b\u03b1k<\/sub>c<\/sup> is obtained by global average pooling of the gradients of the score for class ccc with respect to Ak<\/sup><\/em>.<\/p>\n

These formulations capture the mathematical intuition behind CNN-based feature learning, optimization during training, and the interpretability mechanism, complementing the practical implementation described in earlier sections.<\/p>\n

Explainability Techniques<\/strong><\/p>\n

To improve the clarity and trustworthiness of classification outcomes, the system integrates four prominent explainability methods: Grad-CAM, SHAP, LIME, and Edge Detection. Grad-CAM (Gradient-weighted Class Activation Mapping) generates heatmaps specific to each class by utilizing gradients from the final convolutional layers, thereby pinpointing crucial regions in the image influencing the prediction. SHAP (SHapley Additive exPlanations) measures the impact of each input feature by calculating Shapley values, drawing on principles from cooperative game theory. LIME (Local Interpretable Model-agnostic Explanations) sheds light on individual predictions by creating a simplified, interpretable model that approximates the complex model\u2019s behavior in the vicinity of the specific instance. Finally, Canny Edge Detection is used to emphasize structural boundaries within the image, aiding in the visual interpretation of tumor margins and anatomical context. These techniques collectively provide visual and quantitative insights into how the deep learning models arrive at their decisions, thereby improving clinical trust and adoption. The summary of Explainable AI techniques is given in Table 3.<\/p>\n

Table 3: Summary of Explainable AI Techniques Used for Model Interpretation<\/strong><\/p>\n\n\n\n\n\n\n\n
\n

Technique<\/strong><\/p>\n<\/td>\n

\n

Purpose<\/strong><\/p>\n<\/td>\n

\n

Technical Detail<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

Grad-CAM<\/p>\n<\/td>\n

\n

Generates class-specific heatmaps to highlight the most influential regions in the input image based on gradients.<\/p>\n<\/td>\n

\n

Uses gradient-based localization from final convolutional layers.<\/p>\n<\/td>\n<\/tr>\n

\n

SHAP<\/p>\n<\/td>\n

\n

Computes Shapley values to quantify the contribution of each input feature towards the model’s output.<\/p>\n<\/td>\n

\n

Applies Deep SHAP with backpropagation to estimate pixel importance.<\/p>\n<\/td>\n<\/tr>\n

\n

LIME<\/p>\n<\/td>\n

\n

Provides local, interpretable explanations by perturbing input and training a surrogate model.<\/p>\n<\/td>\n

\n

Uses superpixels and perturbation sampling to train an interpretable linear model.<\/p>\n<\/td>\n<\/tr>\n

\n

Edge Detection<\/p>\n<\/td>\n

\n

Enhances structural visualization by detecting prominent edges and boundaries in the image.<\/p>\n<\/td>\n

\n

Implements Canny edge detection with predefined thresholds (e.g., 100, 200).<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Large Language Model (LLM) Integration<\/strong><\/p>\n

The system utilizes a Large Language Model (LLM) via LangChain-Groq to bridge the gap between raw model predictions and clinical applications. This module is a medical report generator, converting classification outputs and explainability-related knowledge into understandable, human-readable text. Following the model prediction of the tumor type and the associated confidence score, visualizations are created using methods like Grad-CAM, SHAP, LIME, and Edge Detection. These are then conveyed to the LLM via a systematically structured prompt. LangChain is the orchestrator, managing prompt structuring, token passing, and response interpretation, while the Groq API enables high-speed, contextually aware generation of text. The generated text is a diagnostic summary, interpretative results, and medically suitable language pertaining to the context, making the application much more usable for radiologists and clinicians. Table 4 represents the technical details for LLM to generate the reports.<\/p>\n

Table 4: Technical Overview of Large Language Model (LLM) Integration for Diagnostic Report Generation<\/strong><\/p>\n\n\n\n\n\n\n\n
\n

Component<\/strong><\/p>\n<\/td>\n

\n

Purpose<\/strong><\/p>\n<\/td>\n

\n

Technical Detail<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

LangChain<\/p>\n<\/td>\n

\n

Manages LLM prompt templates and execution flow<\/p>\n<\/td>\n

\n

Uses ChatPromptTemplate from langchain_core to define structured prompts<\/p>\n<\/td>\n<\/tr>\n

\n

Groq LLM API<\/p>\n<\/td>\n

\n

Generates coherent medical narratives based on inputs<\/p>\n<\/td>\n

\n

Model used: mixtral-8x7b-32768 accessed via LangChain-Groq interface<\/p>\n<\/td>\n<\/tr>\n

\n

Input Data<\/p>\n<\/td>\n

\n

Feeds prediction class, confidence score, and explanation types (Grad-CAM, SHAP, etc.)<\/p>\n<\/td>\n

\n

Integrated via templated system + key-value formatting<\/p>\n<\/td>\n<\/tr>\n

\n

Output Report<\/p>\n<\/td>\n

\n

Produces a structured, readable diagnostic summary for clinical users<\/p>\n<\/td>\n

\n

Text formatted into PDF using FPDF for final presentation<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Visualization and Report Generation<\/strong><\/p>\n

After creating predictions and interpretability visualizations, the system integrates all relevant outputs into a comprehensive diagnostic report using the FPDF library. The report includes metadata like the date and time of generation and model details, the medical interpretation created by the LLM, and embedded explainability visualizations (e.g., Grad-CAM, SHAP, LIME, SNAP, and Edge Detection). Each visualization is succeeded by elaborate captions to ensure clinical readability. The design layout is intentionally minimalist to promote readability and printability, hence making it readily reviewable electronically and storable physically. The generated report is automatically saved as a PDF file (mri_report.pdf) that can be downloaded or stored to be retrieved later. This capability ensures that the insights generated by the AI are presented in an easily readable format that is well integrated into clinical processes.<\/p>\n

Results<\/strong><\/p>\n

Model Performance<\/strong><\/p>\n

The DenseNet121 model presented here is demonstrated to possess good diagnostic ability in classifying brain MRI images into four categories: Glioma, Meningioma, No Tumor, and Pituitary Tumor, tested after training for 12 epochs.<\/p>\n

Evaluation Metrics and Diagnostic Performance of the DenseNet121 Model<\/strong><\/p>\n

As shown in Table 5 and Fig. 3, DenseNet121 achieves high scores across all primary evaluation metrics. The precision ranges from 0.9749 to 1.0000, while recall spans 0.9712 to 0.9943, reflecting the model\u2019s effectiveness in reducing errors from both false positive and false negative predictions. The resulting F1 scores, which balance precision and recall, are uniformly high, with a macro-average of 0.9850. These results indicate a well-generalized and balanced performance across all tumor classes.\u00a0<\/strong><\/p>\n

Table 5: Classification Metrics (Precision, Recall, F1-Score) for DenseNet121 Across All Tumor Classes<\/strong><\/p>\n\n\n\n\n\n\n\n\n
\n

Class<\/strong><\/p>\n<\/td>\n

\n

Precision<\/strong><\/p>\n<\/td>\n

\n

Recall<\/strong><\/p>\n<\/td>\n

\n

F1 Score<\/strong><\/p>\n<\/td>\n

\n

Support<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

Glioma<\/p>\n<\/td>\n

\n

0.976<\/p>\n<\/td>\n

\n

0.985<\/p>\n<\/td>\n

\n

0.9805<\/p>\n<\/td>\n

\n

334<\/p>\n<\/td>\n<\/tr>\n

\n

Meningioma<\/p>\n<\/td>\n

\n

0.987<\/p>\n<\/td>\n

\n

0.9712<\/p>\n<\/td>\n

\n

0.979<\/p>\n<\/td>\n

\n

382<\/p>\n<\/td>\n<\/tr>\n

\n

No Tumor<\/p>\n<\/td>\n

\n

1<\/p>\n<\/td>\n

\n

0.9851<\/p>\n<\/td>\n

\n

0.9925<\/p>\n<\/td>\n

\n

402<\/p>\n<\/td>\n<\/tr>\n

\n

Pituitary Tumor<\/p>\n<\/td>\n

\n

0.9749<\/p>\n<\/td>\n

\n

0.9943<\/p>\n<\/td>\n

\n

0.9846<\/p>\n<\/td>\n

\n

352<\/p>\n<\/td>\n<\/tr>\n

\n

Macro Average<\/p>\n<\/td>\n

\n

0.9851<\/p>\n<\/td>\n

\n

0.985<\/p>\n<\/td>\n

\n

0.985<\/p>\n<\/td>\n

\n

1470<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n\n
\"\"<\/td>\n\n

Figure 3: Precision, Recall, and F1-Score for Each Tumor Category Using DenseNet121<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Confusion Matrix<\/strong><\/p>\n

Fig. 4 illustrates the confusion matrix for the DenseNet121 model, demonstrating strong class-wise prediction performance with clear diagonal dominance, where the majority of samples are correctly classified into their respective tumor categories. The matrix reflects minimal off-diagonal values, indicating very few misclassifications across classes. Notably, the model demonstrates strong reliability in differentiating tumors that appear visually alike, including distinguishing between Glioma and Meningioma and separating Pituitary Tumors from cases with no tumor. The balanced distribution of correct predictions across all four categories confirms the model\u2019s robustness and generalization capability, supporting its suitability for clinical deployment.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 4: Confusion Matrix Showing Class-Wise Prediction Accuracy for DenseNet121<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Accuracy Trend<\/strong><\/p>\n

Fig. 5 is the training accuracy curve for the DenseNet121 model with a steady rising trend and minimal oscillation. The accuracy shows a consistent rise along the training epochs and eventually levels off at a high value, which indicates successful learning and convergence. The gradual nature of the curve suggests that the model learns well with no signs of overfitting and instability. This steady trend of performance confirms that the selected architecture, training approach, and preprocessing of data methods improve a stable and robust classification model suitable for real diagnostic use.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 5: Training Accuracy Curve Indicating Model Stability and Convergence<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

ROC Curve Analysis<\/strong><\/p>\n

Fig. 6 demonstrates Receiver Operating Characteristic (ROC) curves of the DenseNet121 model, which indicate excellent discriminative ability for all tumor categories. The curves slope towards the top-left, indicating a high true positive rate with a low false positive rate. The Area Under the Curve (AUC) for all categories is nearly perfect, which supports the high discriminative ability of the model between different types of tumors. This high separability also suggests the robustness of the model in clinical classification tasks; hence, its application in diagnostic decision-making is recommended.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 6: ROC Curves with AUC Scores for Four Tumor Categories Predicted by DenseNet121<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Explainability Analysis<\/strong><\/p>\n

To ensure interpretability and strengthen clinician confidence in AI-based decisions, the system incorporates a suite of Explainable AI (XAI) techniques that offer both visual and analytical insights. Fig. 7 presents the original MRI input image containing a contrast-enhanced tumor region used as a test case. This image serves as the baseline for generating multiple interpretability outputs through Grad-CAM, LIME, SHAP, and Edge Detection methods. Upon uploading the image to the deployed application (https:\/\/mrianalysis.streamlit.app\/), the system executes each XAI technique to highlight decision-critical regions, allowing clinicians to visually verify and interpret the model\u2019s predictions. This multimodal approach to explainability ensures transparency, provides deeper insights into model behavior, and facilitates trust in automated clinical decisions.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 7: Input Brain MRI Image Used for Generating Explainability Visualizations<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Grad-CAM Visualization<\/strong><\/p>\n

Fig. 8 displays the Grad-CAM output for the input MRI image. This technique generates a heatmap covering the original scan to highlight sections that strongly impact the model’s prediction. In this case, red and yellow zones are prominently focused on the tumor region, signifying that the DenseNet121 model correctly attends to critical abnormal areas during classification. The visualization confirms that the model\u2019s attention aligns with radiologically relevant features, thereby validating its diagnostic focus and supporting clinical decision-making.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 8: Grad-CAM Heatmap Highlighting Tumor-Influenced Regions Used for Classification<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

LIME Interpretation<\/strong><\/p>\n

Fig. 9 is the output generated by Local Interpretable Model-agnostic Explanations (LIME) for the same MRI image. The LIME method works by decomposing the image into superpixels and altering them to check their impact on the model’s performance regarding its prediction. The highlighted regions in the figure indicate the superpixels that positively contributed to the prediction of the tumor class. These local interpretations offer a comprehensive view of which precise parts of the image had the most impact, thus offering radiologists a clear, human-readable explanation of the model’s decision.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 9: LIME-Based Superpixel Visualization Indicating Regions Positively Influencing the Tumor Prediction<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Edge Detection<\/strong><\/p>\n

Fig. 10 illustrates the outcomes using the Canny Edge Detection algorithm applied to the MRI image. This approach highlights the scan’s structural delineations, particularly the tumor’s peripheries. Separating meaningful edges allows for the tumor’s morphology and spatial measurements information that is critical for preoperative planning and diagnostic evaluation. This enhancement serves as a secondary interpretative layer, providing a more delineated anatomical skeleton in addition to those predictions generated using artificial intelligence.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 10: Canny Edge Detection Output Emphasizing Tumor Boundaries and Anatomical Structures<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

SHAP Explanations<\/strong><\/p>\n

Fig. 11 shows the SHAP visualization that assesses the contribution of a particular pixel region to the model’s prediction outcome. SHAP assigns positive (red) and negative (blue) values to the areas based on their contribution to the prediction, and it is a class-specific feature importance explanation. In this case, the highlighted regions in the Meningioma class are the most significant regions that impact the final decision. The pixel-level transparency enables clinicians to see not just the predictions of the model but also why the predictions were made, thus building trust and facilitating clinical validation.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 11: SHAP Visualization Showing Pixel-Level Contributions for Class-Specific Model Predictions<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

These methods combined form an end-to-end interpretability system, localizing tumor areas, measuring feature importance, and allowing clinical inspection of AI-driven decisions. This interpretability pipeline increases model transparency and simplifies clinical integration for real-world deployment.<\/p>\n

Prediction and Report Generation<\/strong><\/p>\n

Once the training and validation stages were completed, the best-performing model, DenseNet121, was utilized for real-time inference. Fig. 12 presents a test case where an input MRI image was analyzed on June 08, 2025, at 10:58 AM IST. The model predicted the presence of Meningioma with a high confidence score of 0.9986, demonstrating its robust and reliable performance under practical conditions.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 12: Sample Output of the MRI Classifier Indicating Meningioma with 0.9986 Confidence Score<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Following the prediction, the system automatically triggers the diagnostic reporting pipeline. As shown in Fig. 13, the LangChain-Groq-powered LLM generates a structured, human-readable report that synthesizes the model\u2019s output and explainability visualizations. This report includes clinical insights such as tumor type, probable location, size characteristics, and MRI signal intensity features mirroring standard radiology reporting formats. The result is formatted as a downloadable PDF, providing an interpretable and clinician-friendly summary suitable for documentation or second opinions.<\/p>\n

The report is divided into clinically relevant sections such as Primary Findings, Recommendations, and Clinical Notes, and is formatted as a downloadable PDF using the FPDF library.<\/p>\n\n\n\n
\"\"<\/td>\n\n

Figure 13: AI-Generated Diagnostic Report Summarizing Prediction, Tumor Characteristics, and Clinical Insights<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Discussion<\/strong><\/p>\n

This section presents a detailed interpretation of the experimental outcomes by aligning the observed performance with clinical expectations, evaluating model robustness, and considering the relevance of interpretability features for medical applications.<\/p>\n

Interpretation of Classification Results<\/strong><\/p>\n

The high values observed in precision, recall, and macro-averaged F1-score reflect the model\u2019s effectiveness in accurately identifying brain tumor types while minimizing both false positives and false negatives. This balance is crucial in clinical diagnostics, where misclassification may lead to delayed or inappropriate treatment. The macro-F1 score close to 0.99 confirms that the classifier maintains consistent accuracy across all tumor categories, including those that often present subtle visual differences on MRI scans.<\/p>\n

DenseNet121 demonstrated reliable learning behavior with smooth convergence during training, marked by steadily increasing accuracy and minimal performance fluctuation across epochs. The model\u2019s ability to generalize well on unseen data without overfitting can be attributed to both architectural efficiency and a robust training pipeline, including augmentation and balanced stratified sampling. These characteristics suggest that DenseNet121 is well-suited for real-world deployment in clinical environments, offering stable and trustworthy outputs.<\/p>\n

Clinical Relevance of Explainability<\/strong><\/p>\n

For AI-driven diagnostics to be adopted in routine healthcare practice, clinicians must be able to understand and trust the system\u2019s decisions. This work addresses this challenge through the incorporation of multiple explainable AI (XAI) methods. Grad-CAM provides class-specific heatmaps that visually highlight the regions contributing most to the prediction, helping clinicians confirm if the model\u2019s focus aligns with known tumor structures. SHAP offers numerical insights by quantifying the importance of individual pixels or regions, revealing how each feature influences the model’s output.<\/p>\n

LIME contributes by generating interpretable local approximations around each prediction, offering additional clarity into which superpixels led to a given classification. Meanwhile, edge detection supports anatomical interpretation by emphasizing tumor boundaries and spatial relationships. These visual and analytical tools work collectively to demystify the model\u2019s internal reasoning, making the decision process transparent and fostering confidence among radiologists. This transparency is essential for integrating AI tools into diagnostic routines and ensuring they augment, rather than obscure, clinical judgment.<\/p>\n

Comparative Analysis<\/strong><\/p>\n

The performance comparison, as demonstrated in Table 6, shows the effectiveness and uniqueness of the proposed brain tumor classification model over several prior works. Umarani et al. [10] used U-Net, VGG-16, and AlexNet on the BraTS dataset with ~94% accuracy, focusing more on segmentation than classification. Abdusalomov et al. [12] achieved ~93% accuracy using CNNs but lacked interpretability, while Sadeghi et al. [13] incorporated LIME with hybrid CNNs, improving explainability but only reaching ~91% accuracy on a limited dataset. Xiao et al. [14] used SegNet with Grad-CAM for visual interpretation without reporting classification accuracy. Other high-performing models like EfficientNet and ConvNeXT [23\u201325] achieved 96\u201397% accuracy but were applied to non-MRI datasets (retinal, skin, blood), limiting their relevance for brain imaging. In contrast, this work combines DenseNet121 with an ensemble approach and integrates Grad-CAM, LIME, SHAP, and edge detection, achieving 97.8% accuracy and a macro F1-score of 0.985 on a diverse dataset (Figshare, SARTAJ, Br35H), while also offering automated report generation through LLMs, demonstrating superior accuracy, interpretability, and clinical utility.<\/p>\n

Table 6: Comparative Performance of Brain Tumor Classification Models Across Different Datasets and Architectures<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n\n
\n

Ref. No.<\/strong><\/p>\n<\/td>\n

\n

Model(s) Used<\/strong><\/p>\n<\/td>\n

\n

Dataset(s) Used<\/strong><\/p>\n<\/td>\n

\n

Accuracy \/ F1-Score<\/strong><\/p>\n<\/td>\n

\n

Remarks<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n[10]\n<\/td>\n\n

U-Net, VGG-16, AlexNet<\/p>\n<\/td>\n

\n

BraTS<\/p>\n<\/td>\n

\n

~94% Accuracy<\/p>\n<\/td>\n

\n

Focused on segmentation, not classification<\/p>\n<\/td>\n<\/tr>\n

\n[12]\n<\/td>\n\n

CNN<\/p>\n<\/td>\n

\n

Custom MRI dataset<\/p>\n<\/td>\n

\n

~93% Accuracy<\/p>\n<\/td>\n

\n

Lacks interpretability<\/p>\n<\/td>\n<\/tr>\n

\n[13]\n<\/td>\n\n

Hybrid CNN + LIME<\/p>\n<\/td>\n

\n

Private MRI Data<\/p>\n<\/td>\n

\n

~91% Accuracy<\/p>\n<\/td>\n

\n

Emphasis on interpretability<\/p>\n<\/td>\n<\/tr>\n

\n[14]\n<\/td>\n\n

SegNet + Grad-CAM<\/p>\n<\/td>\n

\n

Medical segmentation images<\/p>\n<\/td>\n

\n

Not reported<\/p>\n<\/td>\n

\n

Grad-CAM used for visualization only<\/p>\n<\/td>\n<\/tr>\n

\n[23]\n<\/td>\n\n

EfficientNet + Ensemble DL<\/p>\n<\/td>\n

\n

Retinal Fundus Images<\/p>\n<\/td>\n

\n

96.5% Accuracy<\/p>\n<\/td>\n

\n

Used in diabetic retinopathy, different modality<\/p>\n<\/td>\n<\/tr>\n

\n[24]\n<\/td>\n\n

EfficientNet + ResNet<\/p>\n<\/td>\n

\n

Skin disease dataset<\/p>\n<\/td>\n

\n

97.1% Accuracy<\/p>\n<\/td>\n

\n

Not focused on MRI<\/p>\n<\/td>\n<\/tr>\n

\n[25]\n<\/td>\n\n

ConvNeXT<\/p>\n<\/td>\n

\n

AML Blood Smears<\/p>\n<\/td>\n

\n

95.8% Accuracy<\/p>\n<\/td>\n

\n

Non-MRI application<\/p>\n<\/td>\n<\/tr>\n

\n

This Work<\/p>\n<\/td>\n

\n

DenseNet121 + Ensemble + LLM + XAI<\/p>\n<\/td>\n

\n

Figshare, SARTAJ, Br35H<\/p>\n<\/td>\n

\n

97.8% Accuracy \/ 0.985 F1-score<\/p>\n<\/td>\n

\n

High performance + Explainability + Clinical Report<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n\n
\"\"<\/td>\n\n

Figure 14: Comparative Accuracy of the Proposed Model Against Existing Brain Tumor and Medical Image Classification Approaches<\/strong><\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

This performance edge is visually represented in Fig. 14, which compares classification accuracies across several existing methods. The proposed system outperforms traditional and non-interpretable models, offering a more deployable and clinician-friendly pipeline through automated report generation and integrated XAI outputs.<\/p>\n

Conclusion<\/strong><\/p>\n

This study presents an AI-powered MRI Analysis App designed to support brain tumor classification and diagnosis through a robust and interpretable deep learning pipeline. By integrating a CNN ensemble of DenseNet121, ResNet50, EfficientNet-B0, and MobileNetV3-Small, trained on a large and diverse dataset of 7023 MRI images, the system achieves high diagnostic performance across four tumor classes. The best-performing model, DenseNet121, demonstrated impressive classification metrics, with a macro F1-score of 0.9850 and AUC values nearing 0.99 across all classes. To overcome the black-box limitations of deep learning, the system incorporates multiple explainable AI techniques\u2014Grad-CAM, SHAP, LIME, and Edge Detection\u2014offering meaningful visual and analytical justifications for each prediction. Furthermore, the integration of a Large Language Model (LLM) via LangChain-Groq enables the automated generation of clinically formatted diagnostic reports, effectively bridging the gap between AI outputs and radiologist workflows. The final system output, including prediction, explanation, and textual analysis, is packaged into a downloadable PDF, facilitating ease of use in practical settings.<\/p>\n

Future work will explore the inclusion of additional imaging modalities, such as CT or PET scans, to support multimodal tumor analysis. The system may also be adapted for mobile or edge deployment to assist diagnosis in resource-constrained environments. Further, the integration of federated learning will be considered to enable privacy-preserving model training across institutions. Finally, clinical trials and collaborations with healthcare institutions will be pursued to validate the system\u2019s real-world utility and improve its generalizability across populations and imaging conditions.<\/p>\n

Acknowledgement<\/strong><\/p>\n

The authors would like to thank the School of Computer Engineering, KIIT Deemed to be University, Bhubaneswar, for providing the necessary academic support and research infrastructure for carrying out this work. The department\u2019s guidance and resources are sincerely appreciated for facilitating the successful execution of this study.<\/p>\n

Funding Source<\/strong><\/p>\n

The author(s) received no financial support for the research, authorship, and\/or publication of this article.<\/p>\n

Conflict of Interest<\/strong><\/p>\n

The author(s) do not have any conflict of interest.<\/p>\n

Data Availability Statement<\/strong><\/p>\n

This statement does not apply to this article.<\/p>\n

Ethics Statement<\/strong><\/p>\n

This research did not involve human participants, animal subjects, or any material that requires ethical approval.<\/p>\n

Informed Consent Statement<\/strong><\/p>\n

This study did not involve human participants, and therefore, informed consent was not required.<\/p>\n

Clinical Trial Registration<\/strong><\/p>\n

This research does not involve any clinical trials.<\/p>\n

Permission to reproduce material from other sources<\/strong><\/p>\n

Not Applicable<\/p>\n

Authors\u2019 Contribution<\/strong><\/p>\n