Mudra-VLM

Overview

Mudra-VLM is a research project that adapts Vision-Language Models for fine-grained recognition of Bharatanatyam mudras (hand gestures). This work bridges computer vision, natural language processing, and classical Indian dance, demonstrating how AI can be applied to preserve and understand cultural heritage.

Motivation

Bharatanatyam is one of India’s oldest classical dance forms, with a rich vocabulary of hand gestures (mudras) that convey meaning and emotion. Each mudra has specific characteristics and significance, making their recognition and classification a challenging problem in computer vision.

Technical Approach

Vision-Language Model Adaptation

• Fine-grained Recognition: Specialized model training for subtle gesture differences
• Multimodal Learning: Combining visual features with textual descriptions
• Transfer Learning: Leveraging pre-trained VLMs and adapting them for cultural domain

Challenges Addressed

• Limited Datasets: Working with specialized dance gesture data
• Fine-grained Distinctions: Recognizing subtle differences between similar mudras
• Cultural Context: Incorporating traditional knowledge into AI systems

Research Impact

This project demonstrates:

• Cultural AI: Applying modern AI techniques to traditional art forms
• Heritage Preservation: Using technology to document and preserve cultural knowledge
• Interdisciplinary Research: Bridging computer science and performing arts

Publication

Accepted at ICCV 2025 Workshop on Computer Vision for Developing Countries (Non-Archival)

Authors: Samarth P and Sakshi Rajani

Broader Applications

The techniques developed can be extended to:

• Other classical dance forms (Kathak, Odissi, etc.)
• Sign language recognition
• Gesture-based human-computer interaction
• Cultural education and training tools

This work showcases how AI can serve as a tool for cultural preservation and education, making traditional art forms more accessible to learners worldwide.