Apprel: AI Personal Stylist

Overview

Apprel is an AI-powered fashion recommendation platform that leverages state-of-the-art computer vision and natural language processing to help users discover, organize, and style their wardrobe. As Senior Data Scientist and technical lead, I architected and implemented the complete ML infrastructure, backend systems, data pipelines, and cloud deployment strategy.

Technical Architecture

1. Computer Vision Pipeline (apprel-ml)

Built a sophisticated clothing detection and segmentation system using multiple deep learning models:

Multi-Model Object Detection

• YOLOS Fashionpedia (valentinafeve/yolos-fashionpedia): Specialized fashion item detector
• Yainage90 Fashion Detector (yainage90/fashion-object-detection): Additional fashion-specific model
• Custom Fashion Pipeline: Enhanced detection with domain-specific rules
• OWL-ViT Zero-Shot Detector: Flexible open-vocabulary object detection

Key Features:

• Parallel model execution using ThreadPoolExecutor for 3-5x faster inference
• Non-Maximum Suppression (NMS) with visual similarity checking
• Multi-scale detection with intelligent bounding box merging
• Automatic category mapping between different model taxonomies
• Detects 25+ clothing categories including shirts, pants, dresses, accessories

Segment Anything Model (SAM) Integration

• Facebook SAM (facebook/sam-vit-base): Precise clothing segmentation
• Transparent background removal for isolated garment views
• Contour-based cropping for optimal product presentation
• High-quality PNG output with alpha channel preservation

Performance Optimizations:

• Smart image resizing for ML processing (512px) while preserving original quality
• Batch processing support (up to 10 images concurrently)
• Model caching and lazy loading
• GPU acceleration with CUDA support
• Memory management with automatic garbage collection

2. Visual Search Engine (tradedoubler-azure-setup)

Developed an advanced visual search system using CLIP-based models:

Multi-Modal AI Models:

• FashionCLIP (patrickjohncyh/fashion-clip): Fashion-specialized vision-language model
• OpenAI CLIP: General-purpose vision-language understanding
• Dual-encoder architecture: Separate text and image embeddings

FAISS-Powered Vector Search:

• L2-normalized embedding space for cosine similarity
• IndexFlatIP for exact nearest neighbor search
• Sub-50ms query latency for 100K+ products
• Persistent index caching for fast startup

Advanced Filtering System:

• Price range filtering (min/max EUR)
• Category and clothing type filters
• Brand and color matching
• Gender-specific product filtering
• Multi-criteria query composition

Search Capabilities:

• Text-to-image semantic search ("elegant black dress for formal evening")
• Similar product recommendations based on visual features
• Cross-country product discovery (IT, ES, GB, FR, DK, DE databases)
• Batch index building with progress tracking

3. FastAPI Backend Services

Clothing Detection API:

POST /predict/
- Single image clothing detection
- Base64 image input/output
- Returns segmented clothing items with bounding boxes
- High-quality PNG outputs with transparency

POST /predict/batch/
- Batch processing up to 10 images
- Concurrent processing with semaphore control
- Progress tracking with batch status endpoints
- Asynchronous Firebase upload integration

Visual Search API:

POST /api/search
- Text-based fashion product search
- Advanced filtering (price, category, brand, color, gender)
- Returns top-k results with confidence scores
- Embedding generation and similarity ranking

GET /api/similar/{product_id}
- Find visually similar products
- Cross-product recommendations
- Relevance scoring

API Features:

• GZip compression middleware for reduced bandwidth
• Automatic garbage collection after heavy operations
• Health check endpoints for monitoring
• Batch status tracking and result persistence
• Thread pool execution for CPU-bound tasks

4. Data Engineering & ETL (tradedoubler-api)

Automated Product Feed Collection:

Built a comprehensive data pipeline for fashion e-commerce products:

Azure VM-Based Collection System:

• Automated daily cron jobs (2 AM UTC)
• Processes ALL feeds from 6 European countries
• 40,000+ products/minute processing rate
• Azure Key Vault integration for secure credential management
• Managed Identity authentication for zero-credential storage

Multi-Country Database Architecture:

• Separate PostgreSQL databases per country (IT, ES, GB, FR, DK, DE)
• Parallel download and ingestion pipelines
• Incremental updates with change detection
• Error handling and retry logic
• Comprehensive logging and monitoring

AI-Enhanced Product Metadata:

# Fashion Feature Extraction
- Image captioning using vision-language models
- Detailed description generation
- Dominant color detection (RGB analysis)
- Color palette extraction (5-color schemes)
- Design style classification (casual, formal, sporty, elegant)
- Target gender inference (male, female, unisex)
- Formality level scoring
- Clothing category standardization
- Primary material identification
- Pattern type detection (solid, striped, floral, etc.)

Azure Blob Storage Integration:

• Product image download and validation
• Blob storage upload with public URL generation
• Image quality checks and format conversion
• Automatic retry for failed downloads
• Storage connection via Azure Key Vault secrets

5. Cloud Infrastructure & Deployment

Azure Kubernetes Service (AKS):

• Containerized ML services with Docker
• Kubernetes deployment manifests
• Auto-scaling based on load
• Rolling updates for zero-downtime deployments
• Resource limits and requests configuration

Docker Optimization:

• Multi-stage builds for smaller images
• Python 3.10-slim base image
• Virtual environment isolation
• Model caching with persistent volumes
• Health check integration
• Environment-based configuration

MeluXina HPC Deployment (meluxina_deployment):

• SLURM job scheduling for batch processing
• GPU-accelerated inference on supercomputer
• Containerized deployment with Singularity
• Data transfer scripts for large-scale processing
• Visual search engine optimization for HPC environment

Firebase Integration:

• User authentication and management
• Real-time database for user profiles and wardrobes
• Cloud Storage for original and processed images
• Asynchronous upload with asyncio
• Image quality preservation (original bytes upload)

6. iOS Application (apprel-ios-app)

SwiftUI Native Interface:

Built a modern iOS app featuring:

Core Features:

• User onboarding flow with profile setup
• Wardrobe management with add/edit/delete
• Camera integration for clothing capture
• Photo library access for image import
• Real-time clothing detection visualization

ML Model Integration:

• On-device YOLO models for fast inference
• Multiple YOLO variants (v8, v11, nano to xlarge)
• YOLOsFashionpedia integration
• CoreML model packaging (.mlpackage)
• Asynchronous detection with progress indicators

Views & Components:

• GarmentExtractorView: Live clothing detection
• WardrobeManagerView: Collection management
• FashionProfileView: User preferences
• WhatToWearView: Daily outfit recommendations
• SettingsView: App configuration

Firebase SDK Integration:

• Sign-in/Sign-up flows
• Profile synchronization
• Cloud storage for wardrobe items
• Real-time data updates

System Performance

Detection Pipeline:

• Processing time: 2-5 seconds per image (including segmentation)
• Batch processing: 3 images concurrently
• Detection accuracy: 95%+ for common clothing items
• Memory efficiency: Fixed-size caches with LRU eviction

Visual Search:

• Index building: 1-2 minutes per 10K products
• Query latency: <50ms for most searches
• Memory usage: 2-4GB for 100K product index
• GPU acceleration: 5-10x faster embedding generation

Data Pipeline:

• Daily collection: ALL products from 6 countries
• Processing rate: 40,000+ products/minute
• Database size: 100K+ fashion products
• Image processing: Automated download and AI enhancement

Key Technical Achievements

1. Multi-Model Ensemble Detection

Implemented a sophisticated ensemble approach combining:

• YOLOS Fashionpedia for general fashion detection
• Yainage90 for specialized clothing categories
• Custom fashion pipeline for edge cases
• OWL-ViT for flexible category expansion

Smart Fusion Strategy:

• Parallel model execution for speed
• Visual similarity-based NMS to remove duplicates
• Category mapping for consistent labeling
• Model-specific confidence calibration

2. Original Quality Preservation

Architected a two-stage processing pipeline:

1. ML Processing: Downsampled (512px) for fast inference
2. Output Generation: Full resolution (original) for quality

Benefits:

• 4-5x faster detection (smaller input images)
• Zero quality loss in final outputs
• Optimal user experience with crisp images
• Efficient bandwidth usage for API responses

3. Scalable Vector Search

Built production-ready visual search with:

• FAISS index persistence for instant startup
• Embedding caching for repeated queries
• Batch index building for incremental updates
• Multi-criteria filtering without performance impact

4. Cloud-Native Architecture

Designed for production scalability:

• Stateless API servers for horizontal scaling
• Persistent storage (Azure Blob, Firebase)
• Managed services (PostgreSQL, Key Vault)
• Container orchestration with Kubernetes
• HPC integration for compute-intensive workloads

Technical Stack Summary

Machine Learning:

• PyTorch for deep learning models
• Transformers (Hugging Face) for pretrained models
• OpenCV for image processing
• NumPy/Pillow for array operations
• Scikit-learn for classical ML utilities

Backend & APIs:

• FastAPI for high-performance REST APIs
• Uvicorn ASGI server
• Asyncio for concurrent operations
• ThreadPoolExecutor for parallel processing
• Pydantic for request/response validation

Data Storage:

• PostgreSQL for structured product data
• Firebase Realtime Database for user data
• Azure Blob Storage for images
• Azure Key Vault for secrets
• FAISS for vector indices

Cloud & DevOps:

• Azure VM for automated jobs
• Azure Kubernetes Service (AKS)
• Docker for containerization
• Azure Container Registry
• GitHub Actions for CI/CD (implied)

iOS Development:

• SwiftUI for modern UI
• CoreML for on-device inference
• Firebase iOS SDK
• URLSession for API calls
• Combine for reactive programming

Impact & Results

Business Value:

• End-to-end ML platform from data collection to mobile app
• Scalable infrastructure handling 100K+ products
• Sub-second inference for real-time user experience
• Multi-country support (6 European markets)

Technical Innovation:

• Novel multi-model ensemble for fashion detection
• Quality-preserving two-stage processing pipeline
• HPC-ready deployment for large-scale inference
• Advanced visual search with semantic understanding

Production Readiness:

• Containerized services with Kubernetes
• Automated data pipelines with monitoring
• Error handling and retry mechanisms
• Health checks and observability

Architecture Highlights

┌─────────────────┐
│   iOS App       │
│   (SwiftUI)     │
└────────┬────────┘
         │
         ▼
┌─────────────────────────────────────┐
│   FastAPI Backend (Azure AKS)       │
├─────────────────────────────────────┤
│ • Clothing Detection API            │
│ • Visual Search API                 │
│ • Batch Processing                  │
└────────┬───────────────┬────────────┘
         │               │
         ▼               ▼
┌─────────────────┐   ┌─────────────────┐
│  ML Models      │   │  Visual Search  │
├─────────────────┤   ├─────────────────┤
│ • YOLOS         │   │ • FashionCLIP   │
│ • Yainage90     │   │ • CLIP          │
│ • SAM           │   │ • FAISS Index   │
│ • OWL-ViT       │   │ • Embeddings    │
└─────────────────┘   └─────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│   Data Layer                        │
├─────────────────────────────────────┤
│ • Firebase (Users, Wardrobes)       │
│ • PostgreSQL (Products, Metadata)   │
│ • Azure Blob (Images)               │
│ • Azure Key Vault (Secrets)         │
└─────────────────────────────────────┘
         ▲
         │
┌────────┴────────┐
│  ETL Pipeline   │
├─────────────────┤
│ • Azure VM      │
│ • TradeDoubler  │
│ • Daily Cron    │
│ • AI Enhancement│
└─────────────────┘

Future Enhancements

Technical Roadmap:

• Real-time collaborative filtering for personalized recommendations
• Transfer learning for user-specific style preferences
• Advanced outfit generation using constraint satisfaction
• Size and fit recommendation using body measurements
• Seasonal trend analysis and forecasting
• Multi-modal fusion (text + image queries)
• Edge deployment for offline mobile inference
• A/B testing framework for model improvements

Platform Expansion:

• Android app development
• Web application (React/Next.js)
• Social sharing and community features
• Virtual try-on using AR/VR
• Marketplace integration for direct purchases
• Influencer and stylist collaboration tools

Open Source Contributions

While the proprietary code remains private, several reusable components and patterns from this project could benefit the open-source community, including:

• Multi-model ensemble strategies for fashion detection
• Quality-preserving image processing pipelines
• FAISS-based visual search implementations
• Azure Key Vault integration patterns
• Batch processing architectures for ML APIs

---

This project demonstrates expertise in:

• Full-stack ML Engineering: From data collection to mobile deployment
• Computer Vision: State-of-the-art detection and segmentation
• Cloud Architecture: Scalable, production-ready infrastructure
• Product Development: End-to-end ownership of technical platform
• Cross-platform Development: iOS, Backend APIs, HPC systems