- Fix Day display: server sends phase in datetime_str, index.html uses state
directly instead of fragile regex parse that defaulted to Day 1
- Fix DB growth: snapshots DELETE-then-INSERT by name, prune old events/convos
- Fix console errors: pointer-lock→pointerlock, demo mode stops WS reconnect
and REST polling, mat() guards against undefined color
- FPV NPC chat: E key starts conversation, NPC freezes and faces player,
chat dialog with text input and Web Speech API microphone, contextual
NPC responses (name, age, occupation, mood), ESC to end chat
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Weather system, day/night cycle, agent behaviors, city expansion
- Sun movement east→zenith→west with dynamic shadows
- Weather effects: rain (4000 particles), snow (2000 particles), thunderstorms (lightning flashes), fog, cloudy skies
- Sun brightness/size adjusts per weather (bright sunny → dim stormy)
- Storm clouds layer (18 dark clouds) for bad weather
- Night: residential buildings become transparent, agents sleep in beds
- Furniture inside buildings: tables at restaurants/bars, desks in offices/schools, seats in cinema/church
- Agent poses: walking, sitting (at tables/desks), sleeping (in beds)
- Bad weather: 70% agents stay indoors; night: all go home to sleep
- Demo mode cycles through full day/night + weather types automatically
- Moon enlarged (10), stars visible from 19:00, brighter after 21:00
- Trees halved in size for better proportion with buildings
- City expanded: WORLD_SIZE 100→130, shadow camera enlarged
- 10 new public buildings (bank, court, gallery, daycare, vet, yoga, pub, etc.)
- 4 new streets for expanded area
- Ollama defa…
Add 3D city visualization: buildings, humanoid agents, sky environment
- Three.js isometric 3D city with 70 locations: houses, apartments, shops,
offices, factories, public buildings (police, fire, museum, mall, townhall,
market), parks, and entertainment venues
- Detailed humanoid figures with arms, legs, clothing, walking animation
- Sky system: sun/moon, stars, clouds, mountains, day/night lighting
- 75 demo agents with collision avoidance around buildings
- Grid of streets connecting all city blocks
- /3d route added to FastAPI server
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Major simulation features:
- Age progression (1 year per 365 sim days) with lifecycle stages
- Death system for 80+ with exponential probability curve
- Divorce mechanic for unhappy marriages (low trust/sentiment/mood)
- Married couples move in together (cohabitation)
- Church wedding ceremonies, hospital births after 9 sim-day pregnancy
- Baby agents born as real citizens, living with parents, autonomous at age 4
- Annual mayor elections with weighted voting (18+ citizens)
- Short-term (weekly) and long-term goals with auto-refresh
- Memory query API: ask any agent about their memories via LLM
- Community score tracking for election candidacy
- UI: lifecycle badges, deceased agents, mayor display, split goals,
memory query input, parent info, new event icons
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NN training: realistic persona data, graphs, hot-reload endpoint
- Update all 20 personas in nn_train.py and nn_selfimprove.py to match
enhanced personas.yaml (corrected names, ages, traits, occupations)
- Add persona-specific behavioral patterns: night shift schedule (George),
nightly bar routine (Frank), crush proximity (Lila→Elena), taxi wandering
(Omar), overworked doctor (Priya), morning exercise (Marcus/Yuki), etc.
- Add persona-aware needs generation, mood calculation, starting locations
- Add training graph plots (loss + accuracy curves) after training
- Add --graph flag to display graphs from last training run
- Decouple --push from training (push existing model without retraining)
- Add POST /api/nn/reload endpoint for hot-reloading ONNX model from HF Hub
- NNClient.reload() deletes cache, re-downloads, swaps ONNX session in-place
- Both scripts auto-trigger /api/nn/reload after pushing to HF Hub
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix StockEx order endpoint: use shared database helper
Was creating a new Database() instance which crashes. Now uses
_get_player_from_token() like all other player endpoints.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add StockEx trading integration for player agents
- New API endpoints: /stockex/market, /stockex/leaderboard,
/stockex/portfolio/{id}, /stockex/order
- Player agents can browse market data, view portfolios, and place
buy/sell orders on StockEx clearing house
- Orders recorded as life events and memories for the agent
- Web UI: StockEx button in player panel opens trading modal with
live market data, portfolio view, and order form
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add life events, goals, pregnancy system; fix profile editor bugs
- Life events: append-only timeline seeded from persona, extended by
LLM reflections and romance milestones (dating/engaged/married/birth)
- Goals: personal aspirations with progress tracking, managed by LLM
reflections (add/complete/progress)
- Pregnancy: married couples can conceive, birth after ~7 sim-days,
baby names, both parents get memories and life events
- Agent context: LLM now sees life story, goals, children, pregnancy
- Fix profile editor: fetch full agent detail instead of WS summary
(occupation, background, personality sliders now load correctly)
- Fix Go dropdown: only rebuild when empty, preserves user selection
- Fix WS state summary: include age, occupation, daily_plan, is_player
- NN self-improve: track accuracy in training_stats.json, compare with
previous model on push
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Gemini uses "quota" in ALL 429 responses, even per-minute rate limits.
The circuit breaker was triggering on the first per-minute 429, declaring
daily quota exhausted and blocking all calls until midnight.
Now only circuit-breaks on "per-day"/"daily" keywords or retry-after > 120s.
Per-minute throttles wait up to 30s and retry normally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The circuit breaker was arming on any retry-after > 15s, but Groq sends
retry-after: 30-60s for normal per-minute rate limits. This caused
immediate "quota exhausted" on first use with many agents.
Now only circuit-breaks on retry-after > 120s or body containing
"daily"/"limit" keywords. Per-minute throttles wait and retry normally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Detect actual quota exhaustion via llm_status (429 errors), not just
in-memory counter which resets on server restart
- NN auto-sets probability to 100% when selected
- Remove "local" from NN label
- Popup probability slider initializes from current probability, not hardcoded 20%
- Fix pqForCalc to include rpm field for accurate runtime estimates
- Quota badge shows "exhausted" when API is actually rate-limited
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix quota runtime estimates: use RPM as bottleneck, sync top probability slider
- Runtime estimate now based on RPM (Gemini 4 RPM = 240 calls/h, ~6.25h for 1500 RPD)
not probability, which barely affects duration with 50 agents
- Remove misleading per-probability runtime table from popup
- Show RPM and single accurate runtime estimate instead
- Fix ID collision between top slider and popup slider (use class selectors)
- Sync top probability slider when switching provider via popup
- Expose RPM and max_calls_per_hour in /api/llm/quota per-provider data
- Update nn_selfimprove budget calculations to use RPM-based math
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix quota endpoint: per-provider tick rates for accurate runtime estimates
- Add GET /api/llm/quota endpoint exposing per-provider daily quota (Gemini 1500 RPD, Groq 14400 RPD)
- Add personality/home/work fields to agent detail API for NN training data collection
- Fix nn_selfimprove: /api/agents returns dict not list, correct probability endpoint path
- Fix Gemini scheduled reset to 10:00 AM Athens instead of midnight
- Add 'budget' CLI mode to calculate and apply optimal probability for target duration
- Scheduled mode now auto-calculates probability from remaining quota
- Frontend: rate-limited providers (Gemini/Groq) show quota badge and expand a panel
with runtime estimates table, probability slider, and confirm button before switching
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Diversify agent activities, add local training script, scheduled Gemini cycle, architecture diagram
- routine.py: Add morning/post-work exercise, park lunch, sports_field for more
personality types, varied activity descriptions, park as exercise destination
- nn_train.py: Standalone local training script (equivalent to Kaggle notebook)
with synthetic data gen, ONNX export, HF push, CUDA support
- nn_selfimprove.py: Add 'scheduled' mode for nightly Gemini collection + retrain,
oversample LLM-sourced samples 3x during training
- routes.py: Fix valid providers list (nn instead of hf)
- docs/architecture.html: SVG diagram of SociAgentTransformer architecture
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add NN provider as default, remove HF inference (7B/0.5B)
- New NNClient: downloads ONNX model from RayMelius/soci-agent-nn on
first use, runs inference via onnxruntime on CPU (~1ms for 50 agents).
Zero API calls, zero cost, works offline.
- NN is now the default provider — always available, no API key needed.
LLM probability defaults to 100% for NN (no rate limiting).
- Removed HFInferenceClient and all 7B/0.5B model references — these
required HF Pro credits and never worked on free tier.
- Updated provider list in UI, fallback order, and provider icons.
- Added training notebook (notebooks/soci_agent_nn.ipynb).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Stop persisting LLM probability to DB and default endpoint to 0.10
The DB-saved probability was overriding the provider default on restarts.
Also changed the endpoint default from 1.0 to 0.10 to match the intended
behavior for cloud providers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Default LLM probability to 10% for cloud providers, ignore stale DB value
The DB-saved slider value (often 1.0 from local Ollama sessions) was
overriding the provider default on HF deploys, causing Gemini to burn
through its daily quota in minutes. Now always starts with the provider
default (10% for cloud, 100% for Ollama) — users can still adjust via
the UI slider during the session.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix Gemini conversations: bump LLM budget to 2 and remove double probability gate
With budget=1, action decisions consumed the entire per-tick allowance,
leaving zero budget for conversations. Also, active conversations were
gated by llm_call_probability on every turn — now they always continue
once started.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reset Gemini circuit breaker after failed startup probe
A single 429 on the probe was circuit-breaking for the entire day,
leaving the simulation in routine-only mode. Now the breaker resets
so the sim loop can retry gracefully per-tick.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add logging config so server output is visible in HF container logs
HFInferenceClient.__init__ no longer raises ValueError when token is absent.
Logs a warning instead; complete() already returns "" gracefully.
Allows switching to HF models from the UI even without a token configured.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Show soci-agent HF models unconditionally (don't require HF_TOKEN)
soci-agent-7b and soci-agent-q4 are public repos — always show them
regardless of whether HF_TOKEN is set (e.g. on Render).
Other HF models still gated behind HF_TOKEN.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Clean up LLM selector: only deployed HF models, remove Ollama soci variants
- HF soci entries: keep only deployed models (7b, q4); drop 8b/3b/1b5 (not on HF)
- Ollama: remove all soci-agent variants (Ollama unreachable on HF Space)
- Keep generic Ollama (local) entry for local deployments
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add all soci-agent model variants to LLM selector
- HF provider now defaults to RayMelius/soci-agent-q4 (was SmolLM3)
- Ollama provider now defaults to soci-agent (was llama3.1:8b)
- Startup priority: Claude > HF/soci-agent > Groq > Gemini > Ollama
- auto-detect order in create_llm_client matches new priority
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add Soci fine-tuned model and training data pipeline
- Add RayMelius/soci-agent-q4 (fine-tuned Qwen2.5-0.5B) to HF provider list
- Add soci-agent as named Ollama option for GGUF-loaded model
- Add MODEL_HF_SOCI and MODEL_OLLAMA_SOCI constants to llm.py
- Persist conversation_history in simulation snapshots (to_dict/from_dict)
- scripts/collect_training_data.py: poll Render API, save raw JSONL
- scripts/convert_to_training_jsonl.py: convert to SFT chat format
- scripts/finetune_local.py: local Unsloth fine-tune (RTX 4050, Windows-safe)
Fixes: transformers 4.56 list_repo_templates patch, TORCHINDUCTOR_DISABLE
Round 1: 116 examples, 3 epochs, loss 1.9222, LoRA -> RayMelius/soci-agent-q4
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix NameError: import PROVIDER_OLLAMA and PROVIDER_CLAUDE in server.py
Prevents hitting daily quotas quickly. Users can raise via the slider
or SOCI_LLM_PROB env var. At 10%: ~15 calls/h on Gemini free tier.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Auto-fallback provider on startup if configured provider is quota-exhausted
On startup, probe the chosen provider with a minimal LLM call. If it fails
(quota exhausted, model unavailable, etc.), try Groq → Gemini → Ollama in
order until one works. This prevents the sim from starting in broken state
when e.g. SOCI_PROVIDER=gemini but the daily quota was already used up.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix Gemini default: use gemini-2.0-flash-lite (confirmed free-tier working)
Live testing showed:
- gemini-1.5-flash* models → 404 on current API keys (deprecated/unavailable)
- gemini-2.0-flash-lite → reaches API, only fails when daily quota exhausted
- gemini-2.0-flash-001 → same (valid model, quota-limited)
Update default to gemini-2.0-flash-lite, fallback chain to 2.x models only,
and fix provider label in UI. Switch live Space to Groq (working now).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add _last_error tracking to GeminiClient and GroqClient for diagnosis
Both clients previously returned empty string silently on all error paths,
making the /llm/test endpoint show error:"" even when calls were failing.
Now every silent-return path sets _last_error so the test endpoint can
surface the real reason.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix HF provider: require token, hide from menu if absent, fix stuck provider
- HFInferenceClient.__init__ now raises ValueError if no token found
(checks HF_TOKEN, HUGGINGFACE_TOKEN, HF_API_TOKEN); prevents silently
creating a broken client that gets stuck as active provider
- /llm/providers: HF models only shown when token is available
- /llm/provider: ValueError from missing credentials returns 400 not 500
- Fix Gemini label: "1.5 Flash" (matches actual default model)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix Gemini model: default to gemini-1.5-flash, robust fallback detection
- Change default from gemini-2.0-flash to gemini-1.5-flash (reliably
available on the OpenAI-compatible endpoint; 2.0 opt-in via GEMINI_MODEL)
- Model-unavailable detection now triggers on ANY non-429 status code
where the body contains an unavailability keyword (not just 400/404),
covering 403/422/etc. that Gemini may return for unsupported models
- Extract keywords into _GEMINI_MODEL_UNAVAILABLE_KWS constant
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix Gemini fallback: add 'serverless' to model-not-found keyword list
The Gemini API returns 'serverless' in the error body when a model isn't
available on the OpenAI-compatible endpoint. The fallback detection was
missing this keyword so gemini-1.5-flash was never tried.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Persist LLM probability across restarts; fix Gemini model fallback
- Add `settings` table to SQLite DB for key-value config persistence
- On startup: load saved llm_call_probability from DB (env var still wins);
env var value is also written back so other workstations inherit it
- On slider change: persist new value to DB immediately
- GeminiClient: auto-fallback to gemini-1.5-flash when configured model
returns 400/404 "not found" on the serverless endpoint; tracks unavailable
models per-instance so future calls skip straight to fallback
- Add _GEMINI_FALLBACK_CHAIN and MODEL_GEMINI_FLASH_FALLBACK constants
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add LLM probability slider: controls AI usage per call site
Adds a 0-100% slider (brain emoji, in the toolbar between speed and
zoom controls) that gates every LLM call in the simulation:
- Action decisions, conversation turns, reflections, social starts,
daily plan generation — each rolls random() < llm_call_probability.
Provider defaults (tuned for free-tier daily quotas):
Gemini → 45% (4 RPM + budget=1 → ~150 calls/h → ~10h at 1500 RPD)
Groq → 70% (conserves daily token budget)
HF → 45%
Ollama/Claude → 100% (no quota)
Also reduces Gemini's _max_llm_calls_this_tick from 4 → 1 so the
rate limiter (15s/call) naturally paces the simulation without
needing to burn all 4 budget slots every tick.
Override default via SOCI_LLM_PROB env var (0.0–1.0).
API: GET /api/controls → llm_call_probability field
POST /api/controls/llm_probability?value=0.45
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix Gemini circuit-breaker: expire at midnight Pacific, not fixed 8h
The fixed 28800s (8h) window could block calls even after the daily
quota reset at midnight Pacific. Now computes exact seconds until
midnight Pacific so the circuit-breaker lifts the moment Gemini's
quota refreshes.
Also improves quota warning to show remaining requests and hours
until reset: "750 remaining, resets in 4.2h (midnight Pacific)".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix Gemini RPM (4 not 14) and add daily quota warnings at 50/70/90/99%
Free tier is 5 RPM (not 15) — using max_rpm=14 caused every call to 429.
Now uses max_rpm=4 to stay safely under 5 RPM.
Adds _track_daily_request() called on every successful response:
- Warns at 50%, 70%, 90%, 99% of GEMINI_DAILY_LIMIT (default 1500 RPD)
- Resets counter at midnight Pacific Time
- Daily limit overridable via GEMINI_DAILY_LIMIT env var
Free tier facts: 5 RPM, ~1500 RPD, 250k TPM; resets midnight Pacific.
Cost on paid tier: ~$0.000133/request (~$0.20/day at 1500 RPD).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix Gemini quota circuit-breaker: detect 'quota' in 429 body
Gemini sends retry-after:5 even for daily quota exhaustion, so the
old 'wait > 30' check never fired. Now detect 'quota' in the body
and circuit-break for 8 hours, stopping the retry loop immediately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix Gemini 429 log truncation: escape braces in response body
HF Space log system parses { as JSON object start, truncating the
error message. Escape { and } in all Gemini 429/error log lines so
the full response body is visible in logs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- HFInferenceClient: drop hf_soci_token / soci_token / HW_WR_TOKEN —
HF_TOKEN is the only supported var (auto-injected by HF Spaces)
- create_llm_client: same — only HF_TOKEN triggers HF auto-detection
- _choose_provider: show Groq + Gemini (both free-tier) before HF and Ollama;
remove has_claude (Claude never appeared in the menu anyway)
- HF now labelled "requires HF PRO / credits" to set correct expectations
- SOCI_PROVIDER / LLM_PROVIDER env var still overrides everything (can force
any provider including claude)
Tokens safe to delete from HF settings: soci_token, soci_read, hf_soci_token
SOCI_PROVIDER is not a secret — safe to use as a public Space variable
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove paid providers from menu; circuit-break on 402 (no credits)
- _choose_provider: offer only free providers (HF SmolLM3, Groq, Ollama);
Claude/Gemini can still be forced via SOCI_PROVIDER env var
- HFInferenceClient: add 402 to circuit-breaker (same as 401/403/410) so
the sim stops retrying when the token lacks Inference Providers permission
- Fix: add hf_soci_token Space secret with a personal HF token that has
Inference Providers (Write) permission to enable SmolLM3 for free
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix HF LLM: use free SmolLM3 as default, strip thinking tokens
The Qwen/7B model was causing 402 (credit balance depleted) because it
auto-routes to a paid third-party provider via the HF router.
SmolLM3-3B with :hf-inference runs on HF's own CPU cluster for free.
- Default model: HuggingFaceTB/SmolLM3-3B:hf-inference (no credits needed)
- Prepend /no_think to system prompts to disable SmolLM3 reasoning mode
- Strip <think>...</think> blocks from all HF responses as a safety net
- Add HW_WR_TOKEN to recognized env var names (Space write token)
- HW_WR_TOKEN / hf_soci_token now take priority over HF_TOKEN so a
personal token with credits overrides the Space's limited auto-token
- create_llm_client: detect HW_WR_TOKEN for HF auto-selection
- _choose_provider: detect HW_WR_TOKEN to offer HF in the menu
- Tested: complete() and complete_json() both work correctly with SmolLM3
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix HF Inference: migrate to router.huggingface.co/v1, use Qwen as default
- Update HFInferenceClient base_url to https://router.huggingface.co/v1/
(old hf-inference/ path returns 404 since the API was restructured in 2025)
- Change POST path from models/{id}/v1/chat/completions to chat/completions
- Replace defunct Zephyr-7b default with Qwen/Qwen2.5-7B-Instruct
(auto-routed by the HF router, produces coherent responses)
- Add MODEL_HF_SMOL as a no-credits fallback (hf-inference CPU provider)
- _map_model: fall back to model itself rather than default_model for unknown IDs
- _choose_provider: detect HF_TOKEN/hf_soci_token/soci_token and offer HF
as the first option (before Groq) so HF Spaces auto-selects it
- create_llm_client: also detect hf_soci_token/soci_token for auto-selection
- Tested: complete() returns 'Hello there!', complete_json() returns correct JSON
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
