Back to Projects Projects / Computer Vision
Handwritten Bangla Name Recognition & XAI Preview
Computer Vision 2025 · Deep Learning & XAI

Handwritten Bangla Name Recognition & XAI

An interpretable transfer learning pipeline on custom self-curated handwritten Bangla script, featuring ResNet50 (83.89% test accuracy), Grad-CAM heatmaps, and local LIME explanations.

Project Overview

Bangla handwriting exhibits rich cursive geometry, intricate vowel-sign attachments (kar/fola), and high intra-writer variation, making automated transcription a non-trivial optical character and word recognition challenge.

This research engineered an end-to-end recognition architecture utilizing transfer learning across deep convolutional representations. In addition to achieving competitive test accuracy (83.89%), the project places explainability at the center: utilizing Grad-CAM to visualize neural attention maps and LIME to generate interpretable visual attributions that reveal which structural strokes governed the classifier's decisions.

Key Features

ResNet50 Deep Backbone

Leverages deep residual bottlenecks with specialized fine-tuning layers and adaptive learning rate schedulers to capture multi-scale stroke patterns.

Grad-CAM Spatial Heatmaps

Extracts gradients entering the final convolutional layer to project dense heatmaps over cursive characters, verifying focal attention on phonemic roots.

Local LIME Explanations

Partitions handwritten samples into superpixels to inspect which positive and negative segments drove category logit probabilities.

Curated Dataset & Augmentation

Constructed with an extensive custom dataset featuring multi-writer samples subjected to affine shear, morphological dilation, and elastic deformations.

Technical Challenges Faced

High Inter-Class Stroke Similarity: Distinct Bangla characters frequently differ only by subtle upper-matra marks or delicate loops. Implemented center-loss penalization to pull intra-class representations tightly together while forcing larger inter-class margins.
Superpixel Granularity in LIME: Standard QuickShift segmentation yielded superpixels too coarse to reflect delicate pen strokes. Refined kernel size and distance metrics to align segmentation boundaries tightly with ink traces.
Generalization Across Diverse Pen Pressures: Variations in ballpoint versus gel pens created differing stroke thicknesses. Applied contrast-limited adaptive histogram equalization (CLAHE) and Otsu binarization during image preprocessing.

Improvements & Future Roadmap

Vision Transformer (ViT) Backbone: Benchmarking patch-based self-attention mechanisms against CNN backbones to evaluate global stroke context modeling.
Real-Time Canvas Interface: Building an in-browser pen digitizer where users can write characters live on touch screens and observe instantaneous Grad-CAM attributions.