Spaces:
Running
Running
Commit History
-
Update NVIDIA NIM models - keep top 5, add minimax-m2.7 Running
ef22b95
-
Fix Silicon Flow model IDs with correct API names
24b9325 -
Keep only 5 best Silicon Flow models for free tier
5bba595 -
Add high-rate-limit Silicon Flow models for free tier usage
c9c8b95 -
Better handle rate limit errors that come as 413 or BadRequestError.
Also detect rate limits from response body type. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>c66ebaa -
Add detailed error mapping debug logs.
Will show exact exception type and mapping path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>860ed88 -
Add error details to APIError messages.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>2492112 -
Add routing debug log to trace provider resolution.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>65739aa -
Add detailed error info to auth failure messages.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>979bf9b -
Add credential logging at provider creation.
Shows which API key is being used for each provider. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>a1a14b2 -
Add detailed error logging to trace API failures.
Will show the exact exception type and response on HuggingFace. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>ce65a74 -
Add Cerebras, Silicon Flow, and Groq providers with debug logging.
- Added Groq provider with model ID mapping - Added debug logging to trace API key and model issues - Silicon Flow base URL changed from .cn to .com - Cerebras rate limits increased to 300/80 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>db83b53
-
$(cat <<EOF
Fix Groq and Cerebras model IDs and rate limits. Groq: Model IDs need inner prefixes (qwen/qwen3-32b not qwen3-32b) Cerebras: Removed inaccessible models, set higher rate limits Silicon: Increased rate limits to 300/min All providers now have appropriate concurrency and rate limit settings. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> EOF )332dd16 -
$(cat <<EOF
Add Groq provider support with ultra-fast inference models. Groq API key validated and working. Models added: - groq/llama-3.3-70b-versatile (fast, 70B reasoning) - groq/llama-3.1-8b-instant (fastest, 8B general) - groq/qwen3-32b (fast, reasoning capable) Files modified: - config/provider_catalog.py: Added GROQ_DEFAULT_BASE and groq provider descriptor - providers/defaults.py: Re-export GROQ_DEFAULT_BASE - config/settings.py: Added groq_api_key and provider_is_configured check - providers/registry.py: Added _create_groq factory - providers/groq/client.py: New provider client - providers/groq/__init__.py: New provider init - core/model_capabilities.py: Registered groq models - api/routes.py: Added groq models to REQUESTED_PROVIDER_MODELS - .env: Added GROQ_API_KEY Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> EOF )98fdd46 -
$(cat <<EOF
Fix Silicon Flow base URL from .cn to .com domain. Silicon Flow API key only works with api.siliconflow.com, not .cn. Also added silicon_api_key to .env for testing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> EOF )55f294b -
$(cat <<EOF
Update Cerebras model list based on API test results. Cerebras key only has access to llama3.1-8b currently. qwen-3-235b-a22b-instruct-2507 exists but is rate-limited. zai-glm-4.7 and gpt-oss-120b are not accessible with this key. Also added all models to CEREBRAS_MODEL_MAP for flexibility. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> EOF )0ba585f -
$(cat <<EOF
Fix model ID format for Cerebras and Silicon Flow providers. Cerebras API expects bare model IDs (e.g. "qwen-3-235b-a22b-instruct-2507") not the full "provider/model" format. Strip cerebras/ prefix before sending. Silicon Flow API similarly expects bare model IDs. Strip silicon/ prefix. Also updated REQUESTED_PROVIDER_MODELS and MODEL_CAPABILITIES to use correct ref format (cerebras/qwen-3-235b-a22b-instruct-2507, etc.). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> EOF )58a3721 -
Add Cerebras and Silicon Flow models to REQUESTED_PROVIDER_MODELS
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>0223890 -
Add Z.ai GLM 4.7 model to Cerebras provider
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>6993cbc -
Add Cerebras and Silicon Flow provider support
- Cerebras: qwen/qwen-3-235b-a22b-instruct-2507 (Qwen 3 235B) - Silicon Flow: Qwen3.6-35B/27B, Qwen3.5-35B/27B (VLM multimodal), Gemma-4-26B/31B reasoning models - Both use OpenAI-compatible /chat/completions via OpenAIChatTransport - API keys: CEREBRAS_API_KEY, SILICON_API_KEY Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>43ea069 -
Extend session visibility in admin dashboard with configurable retention
- Add retention_seconds param to SessionTracker (default 30min) - Sessions now stay visible 30min after last request instead of 2min - Add SESSION_RETENTION_MINUTES env var for configuration - Show Active/Idle status in admin UI (<5min vs >5min idle) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>6339a53 -
Implement image support in proxy with vision-aware routing
- Add Anthropic→OpenAI image block conversion with size validation - Update ModelCapabilities with granular vision support fields - Support base64-encoded images with memory optimization - Add 20MB image size limit guard in conversion - Clear base64 data from request blocks to reduce memory footprint - Update mistral-large-3-675b model entry with vision capabilities - Enable vision-aware model routing for image requests Closes: image upload support in Claude Code proxy Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>574e4e7 -
NIM speed optimization — adaptive rate limiting and increased throughput
- Add AdaptiveRateLimiter with auto-backoff on 429s (starts at 100 req/min, backs off to 10, recovers gradually) - Increase NIM rate limit to 100 req/min with 40 max concurrency (configurable via NIM_RATE_LIMIT, NIM_MAX_CONCURRENCY) - Tighten NIM timeouts: connect=8s, first_chunk=20s, fallback_first_chunk=12s - Lock-free fast path in StrictSlidingWindowLimiter (reduces contention under load) - Faster retry delays: base=0.3s, max=20s, jitter=0.1s - Max out HTTP connection pool: 100 keepalive, 500 connections, 5s expiry - Aggressive HTTP pool warmup on startup (forces TCP+TLS connection establishment) - Record 429s to ModelHealthTracker for adaptive recovery - Update CLAUDE.md with per-provider health params and session cleanup notes Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>aa9c0b0
-
Fix stale sessions in admin dashboard and improve auto-routing health
- Add session cleanup background task (runs every 60s) to remove inactive sessions - Wire cleanup loop into AppRuntime startup/shutdown lifecycle - Per-provider health parameters: NIM gets tighter TTL (15s/2 failures), Zen looser (60s/5) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>188ffa9 -
docs: update CLAUDE.md with auto-routing optimizations
- Zen unlimited rate limit (9999 req/min) - Silent blocked-skip for NIM providers - Load-based candidate ordering - Session tracking via X-Session-ID header - Fix AUTO_MODEL_PRIORITY -> AUTO_MODEL_ORDER Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>84a115b -
Optimize auto routing: Zen unlimited, smarter fallback skipping
- Zen provider gets 9999 req/min scoped limiter (no rate limit wait) - Silent skip for blocked NIM providers (no failure penalty) - Zen blocked check uses debug instead of warning level - Minor ruff fixes (quoted types, ternary style) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>9358a6f -
Track sessions via X-Session-ID header for accurate admin dashboard
Claude Code --session-id <uuid> sends X-Session-ID on every request. Use this header as the session identifier instead of inferring from client IP (gateway_<ip>), which is unreliable when multiple sessions share an IP. Fall back to gateway IP for non-session-ID requests. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>948c8f9 -
docs: update CLAUDE.md with model list and health tracking
- Add Zen free models (big-pickle, ring-2.6-1t-free, nemotron-3-super-free) - Add NVIDIA NIM model descriptions - Document auto-routing with health tracking - Simplify key modules section - Update environment configuration Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>d64f2a2 -
Add free Zen models: big-pickle, ring-2.6-1t-free, nemotron-3-super-free
Only free tier models that are Claude Code capable per opencode.ai Zen API. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>04fcbd7 -
Speed up NIM provider with failure tracking and faster timeouts
- Add ModelHealthTracker for per-model failure tracking (30s TTL) - Faster timeouts: connect=10s, first_chunk=30s, fallback=20s - Auto model pre-checks health before attempting requests - Record failures on timeout/rate-limit for smart fallback - Skip unhealthy models in both router and services Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>a5ea640
-
docs: complete README refactor with cloud deploy guide
- Restructured to lead with the problem/solution approach - Added HuggingFace Spaces as primary deployment method - Documented all deployment options (HF, Railway, Render, Fly.io, Docker) - Added architecture overview diagram - Simplified troubleshooting section - Updated model list with speed ratings - Added visual ASCII diagram for auto-routing flow Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>fcc5278 -
Speed optimizations and enhanced auto-model fallback routing
- Connection pool tuning: increased max_connections (100→200), max_keepalive (20→50), reduced keepalive_expiry (30s→15s) - NIM concurrency: increased max_concurrency for better throughput - Faster timeouts: connect (30→15s), first_chunk (60→45s), fallback (60→30s) - Retry backoff: reduced base_delay (1→0.5s), max_delay (60→30s) - New trivial request fast-path for hi/ok/status checks - Auto model now routes to all 7 NIM models by default - Pre-flight rate limit check before trying each candidate - Timeout/transient errors trigger automatic fallback to next model - Import cleanup for httpx/asyncio at module level Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>ebba9d6 -
Performance optimizations for faster proxy routing
- Remove cleanup from hot path in session tracking - Skip _cleanup_old_sessions on every track_request call - Add track_request_async for contexts needing async guarantees - Add provider warmup on startup to eliminate cold-start penalty - Pre-establish HTTP connections before first request Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>49813da -
Display admin dashboard on root page
- Root "/" now shows admin dashboard instead of JSON - Uses same template as /admin endpoint - Auto-refresh every 10 seconds Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>1985e64 -
Upgrade admin dashboard to terminal-style dark theme
- GitHub-inspired dark theme with monospace fonts - Compact summary bar: Active Sessions | Total Requests | Providers - Sessions table with Session ID, Requests, Providers, Last Active, Req/Min - Provider Load table with Active, Sessions, Req/Min, Status - Green checkmark for healthy, warning icon for rate limited - Auto-refresh every 10 seconds Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>a47c2e0 -
Track sessions by gateway client IP
- Add _get_client_ip() to detect gateway/proxy requests - Track sessions only when X-Forwarded-For, X-Real-IP, X-Client-IP, or Via headers present - Each unique client IP behind a gateway = one session - Direct connections are not tracked Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>f56589d -
Fix: use sync version of track_request to avoid SyntaxError
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>b5bd2a8 -
Fix: sessions tracking now works correctly in dashboard and JSON
- Added `track_request` calls in services.py for both single-model and fallback paths - Sessions now show in `/admin` and `/api/admin/sessions` - Added `/` root endpoint to show active sessions count Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>ef123a8 -
Add templates directory to Dockerfile for admin dashboard
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>89ba257 -
Update CLAUDE.md with admin dashboard docs
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>0015069 -
Add jinja2 explicit dependency for admin dashboard templates
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>f69902b -
Add admin dashboard for session monitoring
- Create api/admin.py with /admin and /api/admin/sessions endpoints - Add Bootstrap-styled templates/admin.html dashboard - Auto-refresh every 10 seconds for real-time monitoring - Shows session info, provider load, and summary stats Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>f3220aa -
Add git origins table to README
28e4b90 -
Add smart task-aware routing (Phase 1)
- core/model_capabilities.py: Model capability registry - core/task_detector.py: Detect task requirements from requests - core/chain_engine.py: Multi-model pipeline engine (placeholder) - api/model_router.py: Add resolve_with_task_awareness method Now 'auto' model can detect coding/reasoning and route to best model.4974012 -
Fix httpx keepalive_expiry parameter name
8238c16 -
Performance optimizations for proxy speed and shared sessions
- Per-provider rate limiting: Zen gets unlimited, NVIDIA NIM gets 40/min - Higher Zen concurrency: 4x max_concurrency for fast minimax model - Connection pool tuning: Keepalive connections for faster reuse - Session tracker: Fair resource sharing across Claude Code instances - Smart auto-routing: Prioritize Zen (no limits), then by load Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>cc3287d -
Remove groq/cerebras, add zen model
d6a1875 -
Remove groq from provider catalog
af4fba5 -
Remove readme reference from pyproject.toml
981da17 -
Add HF Space metadata
7745218