Skip to content
←Return to Projects
Case Study

AR Smart Mirror

Real-time interactive AR virtual try-on system designed for retail fashion.

The Problem

In-store clothing fitting is slow, creates long queues in fitting rooms, and increases garment wear. Retailers need an instant, hygienic, and engaging digital mirror that lets shoppers visualize outfits without undressing.

My Role

Built solo with AI-assisted development — computer vision pipeline design, MediaPipe pose detection integration, Canvas AR overlay engine, FastAPI backend services, Docker multi-container topology, and Cloudflare Zero Trust ingress.

Official Website ↗Mobile Controller ↗Architecture Docs ↗Try-On Pipeline ↗

Highlights

  • Sub-30ms real-time body tracking and landmark extraction using MediaPipe Pose
  • Dynamic Canvas AR garment warping and affine transformation for realistic clothing drape
  • FastAPI backend with PostgreSQL catalog and Redis session pub/sub synchronization
  • Mobile controller pairing allowing customers to browse catalogs and trigger try-ons from their smartphones
  • Production-grade Docker Compose architecture with Nginx reverse proxy and Cloudflare Tunnel

Stack

Python 3.11FastAPIMediaPipeHTML5 CanvasPostgreSQL 15Redis 7DockerNginx

Constraints

  • Hard real-time latency target (<33ms per frame / 30 FPS) to avoid motion sickness and visual lag.
  • Zero customer biometric persistence — video feeds must process in memory without saving raw footage to disk.
  • Compatibility with standard consumer hardware (e.g. RTX 3060) without requiring specialized depth cameras.

System Architecture

Capture & Client
Webcam Feed
MediaPipe Pose (Client A)
Mobile Controller (Client B)
Edge & Ingress
Cloudflare Tunnel
Nginx Reverse Proxy
SSL Termination
Application Services
FastAPI Gateway (:8000)
Garment Overlay Service
WebSocket Pub/Sub
Data & Storage
PostgreSQL 15 (Catalog/Users)
Redis 7 (Session Sync)
Local Volume Storage
Telemetry
Prometheus
Grafana Monitoring (:3000)

Key Trade-offs

The decisions worth defending — what I chose, what I turned down, and why.

Pose tracking execution model

Chose

Client-side MediaPipe Pose on Canvas API with GPU acceleration

Rejected

Server-side video streaming and frame-by-frame inference

Server inference introduces network jitter, high bandwidth overhead, and privacy concerns streaming raw camera feeds over the network.

Garment deformation approach

Chose

Real-time 2D affine mesh deformation keyed to skeletal anchor landmarks

Rejected

Full 3D physics-based volumetric cloth simulation

3D cloth simulation cannot sustain 30+ FPS on edge kiosk hardware; 2D mesh triangulation provides fluid response with low computational cost.

Remote store interconnect

Chose

Cloudflare Tunnel over outbound HTTPS

Rejected

Direct port forwarding and dynamic DNS on store routers

Retail store Wi-Fi and ISP firewalls prevent inbound port forwarding; Cloudflare Tunnels punch through NAT securely without public IPs.

What I'd Do Differently

An honest retrospective — the stuff I'd change with more time, more users, or a second pass.

  1. 1Implement WebGPU shaders for cloth lighting and shadow blending to improve realism under dynamic store lighting.
  2. 2Add an offline fallback queue for store analytics so telemetry buffers locally during internet interruptions.
  3. 3Optimize landmark smoothing filters (OneEuro filter) to eliminate micro-jitter when shoppers stand at edge distances.

Computer Vision & Pose Tracking Shell

Benchmark the real-time AR try-on pipeline (status, camera init, pose detect, mesh align -shirt, latency bench).

icelegends-interactive-shell ~ [armirror]
History: ↑ / ↓
cv-engine@ar-mirror:~/pipeline$

Technical Deep-Dive

Architecture, specifications, and implementation details.

11 — Performance Benchmarks

#Hardware Baseline: RTX 3060 12GB, i5 14th Gen, 16GB RAM

#Target SLAs

MetricMVP TargetProduction
Pose detection FPS30fps60fps
Garment render latency<100ms<50ms
API response p95<200ms<100ms
Session start time<2s<1s
Docker cold start<5s<3s
Redis cache hit rate>90%>95%
GPU VRAM usage<8GB<6GB
Concurrent sessions48
System CPU usage<60%<40%

#Optimization Strategies

##Frontend

  • Run MediaPipe in WebWorker (off main thread)
  • Use OffscreenCanvas for garment rendering
  • Preload top 10 garment PNGs on session start
  • Debounce garment selection to avoid rapid redraws

##Backend

  • Redis caches garment catalog (TTL: 5 min)
  • PostgreSQL connection pooling via PgBouncer
  • Async FastAPI endpoints — no blocking I/O

##GPU

  • MediaPipe uses WebGL backend automatically
  • RTX 3060 handles 30fps easily — room to scale
  • ONNX Runtime can replace MediaPipe for 2x speed

##Docker

  • Multi-stage builds reduce image size
  • Health checks prevent traffic to unhealthy containers
  • Resource limits prevent one container from starving others

#Monitoring

  • Grafana dashboard: http://localhost:3000
  • Prometheus metrics: http://localhost:8000/admin/metrics
  • Key metrics to watch:
    • ar_pose_fps — frames processed per second
    • ar_api_latency_p95 — API response time
    • ar_active_sessions — current live sessions
    • ar_garment_try_ons_total — usage counter
~ End of Document ~