EmotionML

Overview

EmotionML is a comprehensive real-time emotion detection system powered by 9 pre-trained deep learning models. The application combines a React frontend with a FastAPI backend to deliver accurate emotion recognition through webcam or image upload.

Trained on multiple datasets (FER2013, RAF-DB, CK+48) with models achieving up to 98.99% accuracy on controlled datasets and 80.28% on real-world scenarios.

Key Features

9 Pre-Trained Models

Choose from different architectures based on your accuracy/speed needs:

• MobileNetV2: 3.5M parameters, optimized for mobile (55.68% - 97.98% accuracy)
• ResNet50: 25.6M parameters, balanced performance (56.60% - 98.99% accuracy)
• VGG19: 143.7M parameters, highest accuracy (58.75% - 80.28% accuracy)

Real-Time Detection

Live emotion recognition through webcam with instant results. Optimized to process frames at ~10-15 FPS on CPU, higher with GPU.

Image Upload Mode

Upload static images for batch emotion analysis with detailed confidence scores for all 7 emotions (Angry, Disgust, Fear, Happy, Sad, Surprise, Neutral).

Interactive UI

Clean, responsive interface built with React + Vite + Tailwind CSS with dark/light mode support.

Model Performance by Dataset

| Model | FER2013 | RAF-DB | CK+48 | |-------|---------|--------|--------| | MobileNetV2 | 55.68% | 73.57% | 97.98% | | ResNet50 | 56.60% | 76.27% | 98.99% | | VGG19 | 58.75% | 80.28% | 98.99% |

Tech Stack

Frontend

• React 18.3.1 with TypeScript
• Vite for fast builds
• Tailwind CSS for styling
• Lucide React for icons

Backend

• FastAPI high-performance API
• PyTorch 2.1.0 deep learning framework
• Torchvision computer vision utilities
• Uvicorn ASGI server
• Pillow image processing

Deployment

• Frontend: Netlify
• Backend: Railway / Render / Local Docker
• Models: Stored as .pt files (PyTorch)

Architecture

┌─────────────────┐
│   React Frontend │
│  (Vite + TSX)   │
└────────┬────────┘
         │ HTTP Requests
         ↓
┌─────────────────┐     ┌──────────────┐
│  FastAPI Backend │────▶│  PyTorch     │
│   (Port 8000)    │     │  Models      │
└────────┬────────┘     └──────────────┘
         │
         ↓
┌─────────────────┐
│   Model Cache   │
│  (GPU if avail) │
└─────────────────┘

Model Training

Datasets Used

• FER2013: 35,887 grayscale, 7 emotions
• RAF-DB: 15,339 real-world images, 7 emotions
• CK+48: 981 lab-controlled images, 7 emotions

Optimization Achievements

• MobileNetV2 optimized: Reduced model size from 23MB → 9MB (60% reduction)
• Inference time improved: From 250ms → 150ms (40% faster)
• Throughput: 1,000+ images/hour with GPU acceleration
• User satisfaction: 95% from 30+ beta testers

API Endpoints

• GET /api/v1/models — List available models and configurations
• POST /api/v1/predict — Upload image and get emotion predictions
• GET /api/v1/health — Health check for monitoring

Challenges & Solutions

Challenge 1: Real-time Performance

Inference speed too slow for real-time webcam use. Solution: Optimized models with TorchScript, implemented frame skipping, and leveraged GPU acceleration (CUDA).

Challenge 2: Model Size

Large models (VGG19: 143MB) not suitable for edge deployment. Solution: Used knowledge distillation and quantization to compress models; MobileNetV2 reduced to 9MB.

Challenge 3: Cross-Platform Camera Access

Browser requires HTTPS for camera in production. Solution: Configured Netlify with SSL, implemented graceful fallback to image upload mode.

Testing & Validation

• Tested with 100+ real-world images
• Compared predictions across all 9 models for consensus
• User testing with 30+ beta testers
• Performance benchmarking on CPU vs GPU

Future Plans

• Add more emotions (fine-grained: joy, sadness, anger, fear, surprise, disgust, neutral)
• Multi-person detection and tracking
• Emotion trend analysis over time
• Mobile app version
• Integration with emotion-aware content recommendation

Live Demo

• Frontend: https://lively-cannoli-82572a.netlify.app
• Backend: Deploy your own using Railway or Render (see README)