AI MAKERSPACE CERTIFICATION CHALLENGE WALKTHROUGH
AI-powered design-to-code component generation that transforms Figma designs and screenshots into production-ready, accessible React components using shadcn/ui patterns.
Transform design assets into high-quality TypeScript components in seconds, not hours.
- π· Screenshot Processing: Extract design tokens from any UI screenshot using GPT-4V
- π― Figma Integration: Direct token extraction from Figma files (colors, typography, spacing)
- π€ Multi-Agent Pipeline: Custom 6-agent system with parallel execution via asyncio
- π Pattern Matching: Intelligent retrieval of shadcn/ui component patterns (hybrid BM25 + semantic search)
- β¨ Code Generation: Production-ready TypeScript + Storybook components
- β‘ Modern Frontend: Next.js 15.5.4 + React 19 + shadcn/ui + Tailwind CSS v4
- π Powerful Backend: FastAPI + OpenAI SDK + Custom Multi-Agent System + LangSmith observability
- βΏ Accessibility First: Built-in axe-core testing for WCAG compliance
- π State Management: Zustand (client) + TanStack Query (server state)
- ποΈ Vector Database: Qdrant for semantic search and pattern retrieval
- π³ Containerized: PostgreSQL + Redis + Qdrant via Docker Compose
- Node.js 18+
- Python 3.11+
- Docker Desktop (for services)
- OpenAI API Key (for AI features)
make installThis will:
- Install npm packages (shadcn/ui, TanStack Query, Zustand, axe-core)
- Install Playwright browsers for E2E testing
- Create Python virtual environment
- Install AI dependencies (OpenAI SDK, LangSmith for tracing, Pillow)
- Copy environment templates (
.envand.env.local.example)
make devOr manually in separate terminals:
# Terminal 1: Start Docker services
docker-compose up -d
# Terminal 2: Start backend
cd backend && source venv/bin/activate && uvicorn src.main:app --reload
# Terminal 3: Start frontend
cd app && npm run devAfter starting Docker services, seed the Qdrant vector database with component pattern embeddings:
make seed-patternsOr manually:
cd backend
source venv/bin/activate
python scripts/seed_patterns.pyExpected output:
INFO: Loading pattern library...
INFO: Loaded 10 patterns from library
INFO: Creating Qdrant collection 'patterns'...
INFO: Generating embeddings for 10 patterns...
INFO: Pattern seeding complete! (10 vectors)
Why this is required:
- Enables semantic search (70% of retrieval accuracy)
- Without seeding, system falls back to BM25-only mode (keyword search)
Verify seeding succeeded:
curl http://localhost:6333/collections/patterns | jq '.result.vectors_count'
# Should return: 10Copy and configure your environment files:
# Backend configuration
cp backend/.env.example backend/.env
# Add your OPENAI_API_KEY and other secrets
# Frontend configuration
cp app/.env.local.example app/.env.local
# Add your AUTH_SECRET and API URLs- ComponentForge UI: http://localhost:3000
- Backend API Docs: http://localhost:8000/docs
- Health Check: http://localhost:8000/health
- Qdrant Dashboard: http://localhost:6333/dashboard
- Storybook: http://localhost:6006 (see below for setup)
Check that semantic search is working (not just BM25 fallback):
# Test retrieval endpoint
curl -X POST http://localhost:8000/api/v1/retrieval/search \
-H "Content-Type: application/json" \
-d '{"requirements": {"component_type": "Button"}}' \
| jq '.retrieval_metadata'Expected output (SUCCESS):
{
"methods_used": ["bm25", "semantic"],
"weights": { "bm25": 0.3, "semantic": 0.7 }
}Failure output (degraded mode):
{
"methods_used": ["bm25"],
"weights": { "bm25": 1.0, "semantic": 0.0 }
}If you see BM25-only mode:
- Verify Qdrant is running:
curl http://localhost:6333/health - Check pattern collection exists:
curl http://localhost:6333/collections/patterns - Re-run seeding:
make seed-patterns - Restart backend: Kill and restart
uvicorn src.main:app --reload
Comprehensive documentation is available in the docs/ directory:
- Getting Started - Installation, FAQ, and contributing guide
- Architecture - System design and technical decisions
- API Reference - Endpoints, authentication, and error codes
- Features - Token extraction, Figma integration, observability
- Testing - Integration tests, manual testing, and test reference
- Deployment - Production deployment and security guidelines
- Development - Setup guides and best practices
- Backend Docs - Backend-specific documentation
- Backend Analysis - Caching strategy, guardrails assessment, and technical analysis
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β ComponentForge AI Pipeline β
βββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββ¬ββββββββββββββββββββββββββ€
β π· Input β π€ Multi-Agent System (6 Agents)β π Retrieval β β¨ Generation β
β β β β β
β β’ Screenshots β 1. Token Extractor (GPT-4V) β β’ BM25 Keyword β β’ TypeScript Component β
β β’ Figma Files β 2. Component Classifier β Search β β’ Storybook Stories β
β β’ Design Specs β ββββββββββββββββββββββββββββββ β β’ Semantic β β’ Accessibility Tests β
β β Orchestrator β Parallel (4): β Similarity β β’ Design Tokens JSON β
β β 3. Props Proposer β β β’ Weighted β β
β β 4. Events Proposer β Async β Fusion β β
β β 5. States Proposer β Parallel β β’ Explainabilityβ β
β β 6. A11y Proposer β β β β
βββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββ΄ββββββββββββββββββββββββββ
βββββββββββββββββββ βββββββββββββββββββ βββββββββββββββββββ
β Frontend β β Backend β β Services β
β (Next.js) βββββΊβ (FastAPI) βββββΊβ (Docker) β
β β β β β β
β β’ Next.js 15 β β β’ OpenAI SDK β β β’ PostgreSQL β
β β’ shadcn/ui β β β’ Custom Agents β β β’ Qdrant Vector β
β β’ Zustand β β β’ LangSmith β β β’ Redis Cache β
β β’ TanStack β β β’ GPT-4V β β β
βββββββββββββββββββ βββββββββββββββββββ βββββββββββββββββββ
Frontend (/app)
- Next.js 15.5.4 with App Router and React 19
- shadcn/ui + Radix UI for accessible component library
- Tailwind CSS v4 with CSS variables for theming
- Zustand for client state management
- TanStack Query for server state and caching
- TypeScript 5.9.3 for strict type safety
- axe-core for accessibility testing
- Playwright for E2E testing
Backend (/backend)
- FastAPI for high-performance async API
- OpenAI SDK (
AsyncOpenAI) for direct GPT-4V and GPT-4o integration - Custom Multi-Agent System with 6 specialized agents (token extraction, classification, requirements)
- LangSmith for optional AI observability and tracing (gracefully degrades if unavailable)
- GPT-4V for vision and screenshot processing
- Pillow for image preprocessing
- SQLAlchemy with async PostgreSQL
- Pydantic for data validation
- asyncio for parallel agent orchestration
Services (docker-compose.yml)
- PostgreSQL 16 - Primary database (Port 5432)
- Qdrant - Vector database for AI (Ports 6333/6334)
- Redis 7 - Cache and sessions (Port 6379)
# Install all dependencies
make install
# Start development environment
make dev
# Run all tests
make test
# Prepare demo environment
make demo
# Clean up containers and dependencies
make clean
# Show help
make help# Start Storybook development server
cd app && npm run storybook
# Build static Storybook for deployment
cd app && npm run build-storybookStorybook runs on http://localhost:6006 and provides:
- Interactive component development - Build and test components in isolation
- Visual documentation - Auto-generated docs for all component variants
- Accessibility testing - Built-in a11y addon for WCAG compliance checks
- Component testing - Integrated Vitest for component unit tests
component-forge/
βββ app/ # Next.js 15 Frontend (React 19)
β βββ src/
β β βββ app/ # App Router pages and routes
β β β βββ demo/ # Demo page for testing
β β β βββ extract/ # Token extraction flow
β β β βββ patterns/ # Pattern library browsing
β β β βββ preview/ # Component preview page
β β β βββ requirements/ # Requirements management
β β β βββ layout.tsx # Root layout with providers
β β β βββ page.tsx # Home page
β β β βββ error.tsx # Error boundary
β β β βββ providers.tsx # React Query, Zustand providers
β β β βββ globals.css # Global styles and CSS variables
β β βββ components/
β β β βββ ui/ # shadcn/ui base components (Button, Card, etc.)
β β β βββ composite/ # Composed business components
β β β βββ extract/ # Token extraction components
β β β βββ patterns/ # Pattern display components
β β β βββ preview/ # Code preview and editor
β β β βββ requirements/ # Requirements form components
β β β βββ tokens/ # Design token components
β β β βββ layout/ # Layout components (Header, Footer)
β β β βββ onboarding/ # User onboarding flow
β β βββ hooks/ # Custom React hooks
β β βββ lib/ # Utilities and helpers
β β βββ services/ # API client services
β β βββ store/ # Zustand store (global state)
β β βββ stores/ # Individual feature stores
β β βββ stories/ # Storybook stories for components
β β βββ types/ # TypeScript type definitions
β βββ e2e/ # Playwright E2E tests
β βββ public/ # Static assets (images, fonts)
β βββ components.json # shadcn/ui configuration
β βββ eslint.config.mjs # ESLint configuration
β βββ next.config.ts # Next.js configuration
β βββ playwright.config.ts # Playwright test configuration
β βββ postcss.config.mjs # PostCSS configuration
β βββ tsconfig.json # TypeScript configuration
β βββ vitest.config.ts # Vitest test configuration
β βββ .env.local.example # Frontend environment template
β βββ package.json # Dependencies (React 19, Next.js 15.5.4)
β βββ README.md # Frontend documentation
β
βββ backend/ # FastAPI Backend
β βββ src/
β β βββ agents/ # 6 AI agents (custom system)
β β β βββ token_extractor.py # GPT-4V token extraction
β β β βββ component_classifier.py # Component type classification
β β β βββ props_proposer.py # Props inference
β β β βββ events_proposer.py # Event handlers inference
β β β βββ states_proposer.py # State management inference
β β β βββ a11y_proposer.py # Accessibility requirements
β β β βββ requirement_orchestrator.py # asyncio-based parallel orchestration
β β βββ api/ # API routes and endpoints
β β βββ cache/ # Redis caching layer
β β βββ core/ # Core utilities and database
β β βββ generation/ # Code generation and validation
β β β βββ generator_service.py # TypeScript generation
β β β βββ code_validator.py # ESLint, TypeScript validation
β β β βββ storybook_generator.py # Storybook story generation
β β βββ monitoring/ # LangSmith observability and metrics
β β βββ prompts/ # AI prompt templates
β β βββ retrieval/ # Pattern retrieval system
β β β βββ bm25_retriever.py # Keyword-based search
β β β βββ semantic_retriever.py # Vector similarity search
β β β βββ weighted_fusion.py # Hybrid retrieval (0.3/0.7)
β β β βββ explainer.py # Confidence scoring
β β βββ services/ # Business logic services
β β βββ types/ # Pydantic models and schemas
β β βββ validation/ # Input validation and sanitization
β β βββ main.py # FastAPI application entry point
β βββ docs/ # Backend technical documentation
β βββ tests/ # Unit and integration tests
β β βββ unit/ # Unit tests for individual modules
β β βββ integration/ # Integration tests for workflows
β βββ scripts/ # Utility scripts (seed data, etc.)
β βββ alembic/ # Database migrations
β βββ .env.example # Backend environment template
β βββ requirements.txt # Python dependencies (LangGraph, etc.)
β βββ pyproject.toml # Python project configuration
β βββ venv/ # Python virtual environment
β
βββ docs/ # Comprehensive Documentation
β βββ getting-started/ # Installation, setup, FAQ
β βββ architecture/ # System design and architecture decisions
β βββ api/ # API reference and examples
β βββ features/ # Feature documentation
β βββ testing/ # Testing guides and strategies
β βββ deployment/ # Production deployment guides
β βββ development/ # Development workflow and guides
β βββ project-history/ # Epic completion reports
β βββ coursework/ # Academic coursework documentation
β βββ adr/ # Architecture Decision Records
β βββ backend/ # Backend-specific documentation
β βββ screenshots/ # Documentation screenshots
β βββ slides/ # Presentation materials
β βββ README.md # Documentation index
β
βββ scripts/ # Utility Scripts
β βββ seed_patterns.py # Seed pattern library to Qdrant
β βββ setup_dev.sh # Development environment setup
β
βββ notebooks/ # Jupyter Notebooks
β βββ (research and experimentation)
β
βββ .claude/ # Claude Code Configuration
β βββ BASE-COMPONENTS.md # Component library specification
β
βββ docker-compose.yml # Services (PostgreSQL, Qdrant, Redis)
βββ Makefile # Development commands (install, dev, test)
βββ CLAUDE.md # Claude Code project instructions
βββ LICENSE # MIT License
βββ RAG_Fusion.ipynb # RAG-Fusion evaluation notebook
βββ pyproject.toml # Python project metadata
βββ README.md # This file
Frontend (.env.local)
# Authentication (Auth.js v5)
AUTH_SECRET=your-32-char-secret-key
NEXTAUTH_URL=http://localhost:3000
# API Connection
NEXT_PUBLIC_API_URL=http://localhost:8000
API_URL=http://localhost:8000
# AI Configuration
NEXT_PUBLIC_OPENAI_MODEL=gpt-4o
NEXT_PUBLIC_VISION_MODEL=gpt-4o
# Feature Flags
NEXT_PUBLIC_ENABLE_FIGMA_INTEGRATION=true
NEXT_PUBLIC_ENABLE_SCREENSHOT_UPLOAD=true
NEXT_PUBLIC_ENABLE_ACCESSIBILITY_TESTING=trueBackend (.env)
# Database Configuration
DATABASE_URL=postgresql+asyncpg://demo_user:demo_pass@localhost:5432/demo_db
# Vector Database (Qdrant)
QDRANT_URL=http://localhost:6333
QDRANT_API_KEY=your-qdrant-api-key
# Cache (Redis)
REDIS_URL=redis://localhost:6379
# AI Services - Required for ComponentForge
OPENAI_API_KEY=your-openai-api-key
# Optional: LangSmith Tracing (for AI observability)
LANGCHAIN_API_KEY=your-langsmith-api-key # Optional: Only if using LangSmith
LANGCHAIN_TRACING_V2=true # Optional: Enable LangSmith tracing
LANGCHAIN_ENDPOINT=https://api.smith.langchain.com
LANGCHAIN_PROJECT=component-forge
# Authentication
AUTH_SECRET=your-auth-secret-key-here# Backend tests (AI agents, API endpoints)
cd backend && source venv/bin/activate && pytest tests/ -v
# Frontend unit tests (components, utilities)
cd app && npm test
# Component tests with Storybook + Vitest
cd app && npx vitest
# Accessibility testing (axe-core)
cd app && npm run test:a11y
# E2E tests with Playwright (full component generation flow)
cd app && npm run test:e2eComponentForge includes a comprehensive end-to-end evaluation system that validates the complete screenshot-to-code pipeline with quantified metrics.
15 component screenshots with ground truth data:
- 8 component types: Button (3), Card (2), Badge (3), Input (2), Checkbox, Alert (2), Select, Switch
- Expected tokens, pattern IDs, and code properties
- Located in
backend/data/golden_dataset/
cd backend
export OPENAI_API_KEY='your-key-here'
python scripts/run_e2e_evaluation.pyDisplays formatted metrics and saves JSON report to backend/logs/.
cd backend
pytest tests/evaluation/test_e2e_pipeline.py -vValidates metrics against thresholds. Fails if pipeline success < 80%.
curl http://localhost:8000/api/v1/evaluation/metricsReturns comprehensive JSON with E2E and retrieval-only metrics.
Navigate to: http://localhost:3000/evaluation
Features:
- Overall pipeline metrics (success rate, latency)
- Stage-by-stage performance breakdown
- Retrieval comparison (E2E vs isolated)
- Per-screenshot results with detailed logs
- Visual log viewer for debugging pipeline failures
- Export JSON functionality
| Metric | Target | Description |
|---|---|---|
| Pipeline Success Rate | β₯ 80% | % producing valid code end-to-end |
| Token Extraction | β₯ 85% | % of tokens correctly extracted |
| Retrieval MRR | β₯ 0.90 | Context precision (mean reciprocal rank) |
| Retrieval Hit@3 | β₯ 90% | Context recall (correct pattern in top-3) |
| Code Compilation | β₯ 90% | % of generated code that compiles |
| Quality Score | β₯ 0.85 | Average code quality from validator |
| E2E Latency | < 20s | Time from screenshot to valid code |
All metrics align with industry-standard RAGAS framework.
- Full docs:
backend/src/evaluation/README.md - Demo materials:
docs/coursework/DEMO_METRICS.md - Dataset format:
backend/data/golden_dataset/README.md - Evaluation fixes:
backend/src/evaluation/EVALUATION_FIXES.md - Pipeline gaps analysis:
backend/src/evaluation/EVALUATION_GAPS.md
- ComponentForge Health: http://localhost:8000/health
- API Documentation: http://localhost:8000/docs (FastAPI Swagger)
- Metrics: http://localhost:8000/metrics (Prometheus format)
- Storybook: http://localhost:6006 (Component library & testing)
- LangSmith Traces: Monitor agent performance and costs
- Token Extraction Confidence: Track vision model accuracy
- Pattern Retrieval Scores: Semantic search effectiveness
- Generation Quality: TypeScript compilation and accessibility scores
- Qdrant Dashboard: http://localhost:6333/dashboard (Vector operations)
- PostgreSQL: Database performance and query logs
- Redis: Cache hit rates and performance
The project includes three essential services:
# PostgreSQL Database
postgres:5432
- User: demo_user
- Password: demo_pass
- Database: demo_db
# Qdrant Vector Database
qdrant:6333/6334
- Dashboard: :6333/dashboard
- API: :6333
# Redis Cache
redis:6379
- Memory limit: 256MB
- Policy: allkeys-lruDocker not starting:
# Ensure Docker Desktop is running
open -a Docker
# Check Docker status
docker --version
docker-compose psPython environment issues:
# Recreate virtual environment
rm -rf backend/venv
cd backend && python -m venv venv
source venv/bin/activate && pip install -r requirements.txtNode.js dependency issues:
# Clean install
cd app && rm -rf node_modules package-lock.json
npm installPort conflicts:
- Frontend (3000), Backend (8000), PostgreSQL (5432), Qdrant (6333), Redis (6379)
- Check for other services using these ports:
lsof -i :3000
Qdrant/Semantic Search Issues:
Symptom: "Semantic retriever unavailable" in backend logs
This means the system is running in BM25-only fallback mode (degraded accuracy).
Solution:
# 1. Verify Qdrant is running
curl http://localhost:6333/health
# 2. Check if patterns collection exists
curl http://localhost:6333/collections/patterns
# 3. If collection missing, seed it
cd backend
source venv/bin/activate
python scripts/seed_patterns.py
# 4. Restart backend to reinitialize semantic retriever
# (Kill uvicorn and restart)Symptom: "Architecture mismatch (arm64 vs x86_64)" when seeding
Your Python venv was created with wrong architecture.
Solution:
# Recreate venv with correct architecture
cd backend
deactivate
rm -rf venv
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# Retry seeding
python scripts/seed_patterns.pySymptom: OpenAI API errors during seeding
Seeding requires OpenAI API to generate embeddings.
Solution:
# Check API key is set
echo $OPENAI_API_KEY
# If empty, add to backend/.env
echo "OPENAI_API_KEY=your-key-here" >> backend/.env
# Export it
export OPENAI_API_KEY="your-key-here"
# Retry seeding
cd backend && source venv/bin/activate
python scripts/seed_patterns.py- Screenshot Upload: Drag & drop any UI design screenshot
- Figma Integration: Connect with Personal Access Token
- Design Analysis: GPT-4V extracts visual design patterns
- Token Extraction: Colors, typography, spacing with confidence scores
- Requirement Proposal: Inferred props, states, behaviors, accessibility needs
- Pattern Retrieval: Semantic search through shadcn/ui component patterns
- Quality Validation: TypeScript, ESLint, axe-core accessibility checks
- TypeScript Component: Production-ready React component with proper types
- Storybook Stories: Interactive documentation and testing
- Accessibility: WCAG-compliant with proper ARIA attributes
- Design Tokens: JSON file with extracted design system values
- AI-First Architecture: Direct OpenAI SDK integration with optional LangSmith tracing
- Custom Agent System: 6 specialized agents with manual asyncio orchestration
- Hot Reloading: Both frontend and backend support instant updates
- Type Safety: Strict TypeScript across the entire stack
- Accessibility: Built-in axe-core testing prevents WCAG violations
- Production Ready: Docker containerization for easy deployment
ComponentForge is designed to scale from atomic components to complex compositions and full page layouts. Current release focuses on establishing quality foundations; future versions will tackle increasing complexity.
Focus: Single-component generation with quality validation
- Basic shadcn/ui components (Button, Card, Input, Badge, etc.)
- Design token extraction with 85%+ accuracy
- Pattern retrieval from vector database
- Validation pipeline (TypeScript, ESLint, axe-core)
- Success metrics: 90% compilation rate, 80% pipeline success
Focus: Multi-component generation with compositional reasoning
- Composite components: ProductCard, DataTable, MultiStepForm, NavigationMenu
- Multi-pattern retrieval: AI selects and composes 5-10 sub-components
- Prop threading: Automatically connects parent/child component data
- State scaffolding: Generates Zustand stores for complex state
- Examples:
- ProductCard = Card + AspectRatio + Image + Badge + Typography + Button
- DataTable = Table + Select (filters) + Pagination + Dropdown (actions)
- MultiStepForm = Tabs + FormFields[] + ValidationLogic + ProgressIndicator
Focus: Complete page generation with architectural decisions
- Full dashboard pages: Sidebar + Header + MetricCards + DataTables + Charts
- Landing pages: Hero + Features + Testimonials + CTA sections
- Application pages: Settings (Tabs + Forms), Profile (Layout + Cards)
- Architectural AI:
- Layout detection (Grid, Flex, responsive breakpoints)
- Routing decisions (Next.js App Router patterns)
- State management strategy (global vs local state)
- API mocking (fake endpoints for development)
- Multi-file generation:
app/page.tsx- Main page componentcomponents/*.tsx- Extracted componentsstores/*.ts- State managementapi/*.ts- Mock API routes
- Design System Import - Upload Figma Design System β Auto-populate vector DB
- Learning System (Phase 4) - AI learns from user edits and preferences
- Custom Component Libraries - Support Material-UI, Chakra UI, Ant Design
- Multi-framework Support - Vue, Svelte, Angular code generation
- Advanced Testing - Auto-generate unit, integration, and E2E tests
- Real-time Collaboration - WebSocket support for team editing
- Version Control - Component versioning and diff tracking
- Performance Optimization - Bundle analysis and optimization suggestions
- Quality first: Atomic components establish 90% quality baseline before scaling
- Infrastructure reuse: Multi-agent system, vector DB, validation pipeline built for all phases
- Learning foundation: Phase 1 builds pattern library that Phase 2 and 3 depend on
- Risk management: If complex generation fails, atomic generation still provides value
- Technical scaling: Compositional reasoning requires validated atomic patterns first
Current state: Phase 1 complete with production-ready infrastructure. Vision: Full design-to-code automation from screenshots to complete applications.
- Fork the repository
- Create your feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add some amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.
Transform designs into code with AI! π§©β¨
Built with β€οΈ for developers who want to focus on logic, not repetitive UI coding.