openspec/specs/deployment/spec.md

deployment Specification

Purpose

Deploy Lingo Bridge to serverless GPU infrastructure with strict cost controls suitable for a hackathon dev budget, while supporting the GPU features the target models need.

Requirements

Requirement: Modal serverless deployment

The system SHALL deploy on Modal.com as app lingo-bridge (file modal_app.py), serving the FastAPI app as an ASGI app, with model weights stored in a Modal Volume named lingua-models.

Scenario: Deploy and serve

• WHEN the app is deployed to Modal
• THEN the FastAPI app is served as an ASGI web endpoint backed by the lingua-models Volume
• AND it is reachable at the live URL https://uiharu-kazari--lingo-bridge-web.modal.run

Requirement: Cost guards

The deployment SHALL enforce cost guards: scale-to-zero (min_containers=0), max_containers=1, and scaledown_window=120. The dev budget cap is $50.

Scenario: Idle cost is zero

• WHEN there are no requests for longer than the scaledown window
• THEN the container stops and idle cost is zero, never fanning out beyond a single container

Requirement: GPU tier supports target models

The deployment SHALL run on a GPU capable of the target workload. It currently runs on T4 and is moving to L4, which is required for Qwen3-TTS / FlashAttention-2 and also speeds up the LLM.

Scenario: GPU upgrade for Qwen3-TTS

• WHEN Qwen3-TTS (requiring FlashAttention-2) is enabled
• THEN the deployment runs on an L4 GPU

Requirement: Prebuilt CUDA wheel for llama.cpp

llama-cpp-python SHALL be installed from a prebuilt CUDA wheel (cu125 index) on a CUDA runtime base image, because compiling from source on a GPU-less builder fails.

Scenario: Image build uses prebuilt wheel

• WHEN the Modal image is built
• THEN llama-cpp-python is installed from the cu125 prebuilt wheel index on a CUDA runtime base image (no source compilation)