Brain Tumor Classification from MRI Scans

The problem

A radiologist reading a brain MRI spends 15–30 minutes per scan, and two readers don't always agree — inter-reader variability is a known issue in high-volume screening. A reliable classifier that triages scans into tumor type could flag high-priority cases and shorten diagnostic turnaround, which is exactly where treatment timelines are won or lost.

Constraints

• Data scarcity. ~7,000 labeled MRI images across four classes (No Tumor, Glioma, Meningioma, Pituitary). That is tiny relative to the visual complexity, and it rules out training a deep CNN from scratch — it would memorize the training set.
• Domain mismatch. Standard ImageNet preprocessing is wrong for MRI: scans have large black borders, and contrast varies by scanner and protocol, so the same tumor looks different across machines.
• Clinical stakeholder needs. A decision-support tool has to be interpretable and calibrated — a radiologist needs "94% likely Pituitary," not a bare label. An extra 0.3% accuracy from an opaque model is a bad trade if it can't express confidence.
• Class difficulty. Glioma and Meningioma share visual features (irregular boundaries, similar intensities), so the errors that matter are concentrated in exactly the classes that are hardest to tell apart.

Approach

First attempt — fine-tuning a deep CNN end to end. The instinct was to fine-tune all layers of a pretrained network. On ~7K images it started overfitting to scanner-specific artifacts almost immediately, and validation accuracy lagged training badly. Fine-tuning every layer is too much capacity for this little data.

Second attempt — feature extraction over transfer learning. I froze a pretrained backbone and trained a lightweight classifier on the extracted features. Low-level edge and texture detectors learned on ImageNet's 1.2M images transfer cleanly to MRI; only the classifier head needs domain data. I compared two backbones:

• ResNet50 (25.6M params) — strong, well-understood residual baseline.
• EfficientNet-B3 (12M params) — compound scaling, better accuracy per FLOP.

Domain-specific preprocessing — where the real gains came from. This mattered more than the backbone choice:

1. Brain-contour cropping (OpenCV) to strip the black borders — removed ~40% of irrelevant pixels and focused the model on tissue.
2. CLAHE (Contrast Limited Adaptive Histogram Equalization, clip 2.0, 8×8 grid) to normalize local contrast across scanners so tumor boundaries are visible.
3. Standardization — resize to 224×224 and normalize to ImageNet mean/std for backbone compatibility.

The final classifier was logistic regression on frozen EfficientNet-B3 features — interpretable, fast, and calibrated, which the clinical use case demanded.

Evaluation

I evaluated against a real baseline, per class, and for stability — not just a single headline number.

Against baseline.

The +21.35% over baseline is the number I trust most, because it's measured against a real alternative on the same data — not an absolute accuracy quoted in a vacuum. Critically, it came from preprocessing plus pretrained features together, not from swapping in a bigger model: EfficientNet-B3 beat ResNet50 by only 0.7%.

Per-class — where the errors actually live.

As predicted, Glioma and Meningioma are the weak spots — and clinically they're among the more consequential to confuse, so the macro number hides the errors that matter most.

Stability. 5-fold stratified cross-validation gave a fold-to-fold variance of ±0.8%, which says the result generalizes and isn't a lucky split. Feature extraction also beat full fine-tuning here — fine-tuning added only ~0.5% accuracy for 10× the training time and more overfitting risk.

What the metric doesn't capture. 91.6% on a curated academic dataset is not clinical readiness. The data is clean and balanced in a way real hospital PACS data is not; there's no out-of-distribution class (a scan with an unlisted pathology gets forced into one of four buckets), no measure of performance across scanner vendors, and accuracy says nothing about calibration under distribution shift — the failure mode that actually endangers a patient. A confidently wrong "No Tumor" is the error this evaluation is least equipped to catch.

What I'd do differently

• Add an abstain / out-of-distribution path. The four-class softmax has no way to say "I don't know." For decision support, a confidence-thresholded abstain that routes uncertain scans to a human is more valuable than another point of accuracy — and it directly addresses the confidently-wrong failure mode.
• Evaluate across scanners, not just across folds. Cross-validation shuffles within one distribution. I'd hold out entire scanner sources to measure real generalization, since scanner shift is the gap most likely to break this in a hospital.
• Attention or Grad-CAM for the Glioma/Meningioma confusion. The hardest classes need the model to focus on discriminative regions, and clinicians need to see why a prediction was made. Attention-based methods plus saliency maps would attack both the accuracy gap and the interpretability requirement at once.