README.md

metadata

title: Kalpanā Plain Model Baselines
emoji: 🧪
colorFrom: blue
colorTo: purple
sdk: docker
app_port: 7860
pinned: false

Kalpanā Plain Model Baselines API

This Hugging Face Space hosts a stateless multi-model API serving baseline models without RIF, RAG, search, or conversation memory. It provides a clean comparison point for the Kalpanā benchmark.

Environment Variables

• HF_TOKEN: Hugging Face gated model authentication token (required to download Llama models).
• INFERENCE_PROFILE: gpu-transformers (uses transformers + bitsandbytes 4-bit) or cpu-gguf (uses llama.cpp GGUF quants).
• DEFAULT_MODEL_ID: Default loaded model on startup (default: plain-qwen-0.5b).
• MODEL_CONTEXT_LIMIT_LLAMA_3_8B: Token context limit for Llama 3 8B.
• MODEL_CONTEXT_LIMIT_LLAMA_3_2_3B: Token context limit for Llama 3.2 3B.
• MODEL_CONTEXT_LIMIT_QWEN_0_5B: Token context limit for Qwen 1.5 0.5B.
• MODEL_THREADS: Threads allocated for local GGUF execution.
• MODEL_BATCH_SIZE: Batch size for evaluation.
• BENCHMARK_API_KEY: API Key to protect endpoints (optional).

Endpoint Overview

• GET /health: Health metrics, active model status, and device metadata.
• GET /v1/models: Returns list of plain models.
• GET /v1/config: Returns configurations, quantization specs, and hashes.
• POST /v1/models/load: Selects and loads one model, unloading other active models.
• POST /v1/tokenize: Counts tokens using the model's actual tokenizer.
• POST /v1/chat/completions: OpenAI-compatible completion endpoint.
• POST /v1/benchmark/query: Context-bounded query endpoint applying deterministic truncation policies.

Local Development and Build Commands

Build Docker Container

docker build -t plain-models-api .

Run Docker Container

docker run -p 7860:7860 -e INFERENCE_PROFILE=cpu-gguf -e MOCK_INFERENCE=true plain-models-api

Run Tests

pytest tests/