Portfolio
SignaVerse: Audio & Text to American Sign Language Using 3D Avatars
An AI system that extracts audio from video or accepts text, transcribes speech, generates sign-language motion, and renders it on a realistic 3D avatar—delivered through a modern web interface.
Project details
What we delivered
Overview
SignaVerse converts audio, video (including YouTube links), and plain text into American Sign Language output using a realistic 3D avatar.
It targets scalable accessibility for deaf and hard-of-hearing audiences without requiring manual interpretation for every piece of content.
High-Level Pipeline
Input processing (extract audio from video when needed)
Speech-to-text transcription (time-aligned text)
Sign language motion generation (avatar motion data)
3D avatar rendering (full-body + hands + facial articulation)
Web delivery through a responsive interface
Core Technologies
Whisper-based transcription
SignAvatars dataset for motion representation
SMPL-X parametric human model
FastAPI backend (Python) for inference APIs
React frontend for visualization and interaction
Use Cases
YouTube/media accessibility overlays
E-learning and lectures
Corporate training and compliance
Public information and government media
Assistive technology products (kiosks/displays)