Spaces:

Hardik-25
/

VisionSeq

Sleeping

App Files Files Community

VisionSeq / docs /ARCHITECTURE.md

Antigravity Agent

Update docs, readme, and add project context

dc0d7c0 13 days ago

preview code

Raw

History Blame Contribute Delete

6.05 kB

System Architecture

VisionSeq is built as a decoupled, service-oriented system: a Next.js frontend, a FastAPI backend, and an in-process PyTorch inference service. This document describes each component, their responsibilities, and how a request flows end-to-end.

Live system: huggingface.co/spaces/Hardik-25/VisionSeq

Architecture Diagram

graph TD
    User([User Client]) -->|Web UI| FE[Next.js Frontend]
    FE -->|REST API Requests| BE[FastAPI Backend]
    BE -->|Preprocessed Tensor| INF[PyTorch Inference Service]
    INF -->|Model weights| Model[ResNet-18 Multi-Head Classifier]
    Model -->|Logits| Dec[Character Decoder]
    Dec -->|Prediction & Confidence| BE
    BE -->|JSON Response| FE

Design Principles

Separation of concerns: the frontend never touches the model; it only speaks REST/JSON to the backend. The backend never renders UI; it only validates, orchestrates, and serves predictions.
Single model instance: the model is loaded exactly once per process, not per request, eliminating repeated disk I/O and weight-loading latency from the hot path.
Non-blocking request handling: CPU/GPU-bound work never runs directly on FastAPI's async event loop — it's offloaded to a threadpool so I/O-bound requests (like concurrent uploads) are never starved.
Fail fast, fail loud: if model weights are missing or corrupted, the server refuses to start rather than serving broken predictions.

Component Details

1. Frontend — Next.js + Tailwind CSS

Presentation: Responsive dashboard with glassmorphism styling, Framer Motion micro-interactions, and a live canvas preview of the uploaded image.
API integration: Async client with support for both single and batch prediction, rendering per-character confidence breakdowns alongside the decoded sequence.
Config: Backend base URL is injected via NEXT_PUBLIC_API_URL, keeping the frontend environment-agnostic across local, staging, and production deployments.

2. Backend — FastAPI

API surface: JSON endpoints for single prediction, batch prediction, model metadata, and health monitoring (see API.md).
Middleware: A boundary-level Content-Length check rejects oversized uploads before the body is read, backed by a token-bucket rate limiter to protect compute resources from abuse.
Threadpool scheduling: Preprocessing and inference are wrapped in run_in_threadpool(), keeping the event loop free to accept and parse concurrent requests while CPU/GPU work happens on background threads.

3. ML Inference Service — PyTorch & ResNet-18

Lifecycle: Instantiated once at startup as a singleton (InferenceService), avoiding per-request model-loading overhead.
Execution mode: Runs in eval() mode; a dummy [1, 1, 100, 200] tensor is pushed through the model during boot to trigger kernel compilation and CUDA context initialization ahead of the first real request.
Device selection: Automatically targets cuda when available, with a transparent fallback to CPU — no configuration required to run in either environment.
Multi-position head: CAPTCHA decoding is framed as 6 concurrent per-position classification tasks. A ResNet-18 backbone (modified for 1-channel input) is pooled to [B, 512, 6] and classified into a shared 31-class vocabulary per position — see the Modeling Approach section of the README for the full rationale.

Request Data Pipeline

Step	Stage	Detail
1	Upload	Client submits a distorted CAPTCHA image via `multipart/form-data`.
2	Defensive check	Middleware validates request size against `Content-Length` and checks the caller's rate-limit bucket.
3	MIME verification	API confirms the declared content type starts with `image/`.
4	Streaming read	Image bytes are read in 64KB chunks, aborting if the stream exceeds the 5MB limit — protects against missing/spoofed `Content-Length` headers.
5	Structural integrity	`PIL.Image.verify()` confirms the file is a physically valid image, not a disguised payload.
6	Preprocessing (threadpool)	Grayscale conversion, resize to `100×200`, normalization, tensor conversion to `[1, 1, 100, 200]`.
7	Inference (threadpool)	ResNet-18 forward pass produces raw logits of shape `[1, 6, 31]`.
8	Decoding	Softmax per position; the argmax index at each of the 6 positions maps to a character in the 31-symbol vocabulary.
9	Telemetry	Uptime and request counters are updated; structured logs capture request ID, latency, and prediction.

Why This Design

Per-position classification over CTC. Because every label in this dataset is exactly 6 characters, there's no variable-length alignment problem for CTC to solve — a shared linear head over 6 pooled feature vectors directly optimizes the actual objective with less architectural and training complexity.

Threadpool over async-native inference. PyTorch's synchronous CPU/GPU execution doesn't natively play well with asyncio; rather than rewriting the inference stack around async primitives, offloading to a threadpool is the minimal-risk way to keep the event loop responsive under concurrent load.

Singleton over per-request instantiation. Model loading (deserializing ~44MB of weights, initializing CUDA context) is orders of magnitude slower than a single forward pass — doing it once at startup, rather than per request, is what makes sub-200ms latency achievable at all.