23

difference.md

@@ -1,290 +0,0 @@
-# Valhalla vs. Stanford's "Generative Agents" — What's Different, and What We Built on Top
-
-> A plain-language report. No coding background needed.
-> It compares our project (Valhalla) against the 2023 Stanford research paper
-> "Generative Agents: Interactive Simulacra of Human Behavior" (Park et al.),
-> whose demo world was called *Smallville*.
-
----
-
-## 1. What both projects are (30 seconds)
-
-Both projects create **virtual people** — AI characters that live in a simulated
-world, plan their day, move around, bump into each other, chat, and remember
-what happened. They are built on the same 2023 idea: instead of programming
-characters by hand, you give each one a description of who they are and let a
-large language model (a chatbot-style AI) make their decisions.
-
-- **Stanford (the original):** a research paper plus a demo. 25 characters in a
-  fictional small town called Smallville, drawn as cartoon sprites in a game.
-  It ran for two in-game days, was watched by 100 human evaluators, and proved
-  the idea worked.
-- **Valhalla (ours):** the same idea, rebuilt as a real product — a running
-  web simulation of **IIT Ropar campus**, with students who have energy and
-  moods, a real campus map, real event fests, admin controls, and the ability
-  to run for days and pick back up from a saved checkpoint.
-
-The short version: **Stanford built the experiment. We built the machine that
-can keep running it.**
-
----
-
-## 2. Quick comparison table
-
-| | Stanford (2023) | Valhalla (ours) |
-|---|---|---|
-| World | A fictional town (cafe, bar, park, houses) | Real IIT Ropar campus (departments, hostels, library, mess, sports complex) |
-| Characters | 25, described in one paragraph each | 12, described with structured student profiles (branch, hostel, hobbies, goals) |
-| AI brain | ChatGPT (GPT-3.5) | Google Gemini, with forced structured answers |
-| Speed of a day | ~4 real minutes | ~96 real minutes (adjustable, 0.25×–32×) |
-| Memory | One ever-growing diary file per character | Per-day diary files + a searchable long-term memory database |
-| "Reflection" (deep thinking about past events) | Yes — a headline feature | Not built yet (honest gap, see §8) |
-| Day planning | 3-level plan, no checking | 3-level plan with built-in quality checks and retries |
-| Can you talk to the characters? | Yes — as yourself, or as a character | Not yet — you watch them; chat is read-only |
-| Safety nets | None | Checkpoints, budget limits, health monitoring, admin login |
-| Energy/mood | None | Full energy + emotion system with circadian rhythm |
-
----
-
-## 3. The seven big architectural differences (the reasoning, not the code)
-
-### Difference 1 — Time: a day should feel like a day
-
-**Stanford** compressed time aggressively. One simulated hour was roughly ten
-real seconds, so a whole in-game day passed in about four real minutes. That
-was fine for a demo — you could watch a Valentine's Day party unfold in an
-evening — but characters never really "lived" a day.
-
-**Valhalla** runs one real minute of simulation per tick, 1,440 ticks per day
-(the same 1,440 minutes every day has). At normal speed a full simulated day
-takes about 96 real minutes, and you can slow it down or speed it up live
-from the browser. Reasoning: believability at the *scale of a day* — sleeping,
-attending classes, mealtimes — only shows up if the day actually lasts.
-
-### Difference 2 — The world: fictional town vs. real campus
-
-**Stanford** authored a toy town: a cafe, a bar, a park, a school, a few houses
-and stores. Everything in it was invented for the demo.
-
-**Valhalla** models **IIT Ropar for real**: 21 locations including the
-departments, lecture-hall complex (LHC), Super Academic Block (SAB), library,
-six hostels (Brahmaputra, Chenab, Beas, Satluj, Ravi, Jhelum), mess, SAC club
-rooms, and sports complex. Each place carries real information a student would
-care about: opening hours, sub-areas (e.g. "programming labs" inside the CS
-department), capacity, and notes that are explicitly marked **verified** vs.
-**assumed** from public sources. Reasoning: a believable campus simulation
-should obey campus facts — you can't walk into the mess at 3 a.m. just because
-a character feels like it.
-
-### Difference 3 — Characters: one paragraph vs. a student profile
-
-**Stanford** gave each character a single paragraph of prose as their
-backstory, and let the AI fill in the rest as it went.
-
-**Valhalla** gives each character a **structured student profile**: age, branch,
-home city, hostel, a "daily plan request" ("skip morning classes, spontaneous
-plans"), personality layers (innate / learned / lifestyle), hobbies, goals, and
-interests. The tone is deliberately **casual student voice** — the paper itself
-complained its AI characters came out "overly polite," so we pushed the other
-direction. Reasoning: a profile with structure gives the planner clear
-handles ("this person is an Electrical branch student who lives in Ravi
-hostel"), and the casual voice makes two students gossiping believable.
-
-### Difference 4 — Memory: one big diary vs. a filing cabinet + a search engine
-
-**Stanford** kept one giant natural-language diary per character (the "memory
-stream"), and every memory carried an AI-assigned importance score. When the
-character needed to remember something, the system scored all memories by
-importance + recency + relevance and picked the best ones. It also had
-**reflection**: periodically, the character would "think" about recent events
-and write down higher-level insights ("Klaus is dedicated to his research").
-
-**Valhalla** splits memory in two:
-
-1. **Short-term**: a per-day diary file for each character — their plan, what
-   they did, conversations, an AI-written end-of-day summary.
-2. **Long-term**: a real search engine (a vector database called Qdrant). When
-   a day ends, its summary and key moments are indexed so future plans can
-   search for them by meaning. The score used is 65% meaning + 20% importance +
-   15% recency (Stanford used equal weights, and decayed recency every tick;
-   we decay by real days).
-
-Two practical differences Stanford never had to worry about: **storage limits**
-(we prune old, unimportant memories when the database grows too big) and
-**diversity** (a conversation from a past day can't crowd out everything else).
-
-Honest gap: the "reflection" step is **not implemented**. That is the single
-biggest piece of the Stanford architecture we deliberately left out of the
-current build (it's a documented future step, not a hidden bug).
-
-### Difference 5 — Planning: hope-for-the-best vs. check-and-retry
-
-**Stanford** planned in three levels — day in broad strokes, then hour-sized
-chunks, then 5–15 minute actions — and accepted whatever the AI produced. If
-the AI wrote a plan with overlapping times or impossible locations, that's
-what the character did.
-
-**Valhalla** wraps the same three-level decomposition in a **quality-control
-loop**:
-
-1. The AI writes the coarse plan → checked (does it cover the whole day?
-   does it end at midnight?).
-2. Hourly chunks → checked (do they tile the day with no gaps or overlaps?
-   do they stay inside their parent block?).
-3. Fine actions → checked (times, locations must exist on campus, the
-   branch-venue policy — e.g. AIDE lectures belong at the CS department, not
-   in random buildings — and content safety).
-
-Any failed check sends the plan back for a rewrite (up to 3 tries). If the AI
-still fails, the system installs a small, safe fallback schedule (sleep, then
-quiet downtime at a valid location) so a character is never left without a
-plan. This is pure engineering the paper never needed for a 2-day demo, and
-it's the difference between a simulation that survives day 3 and one that
-glitches on day 2.
-
-### Difference 6 — "Should I change my plan?" — ask the AI sparingly, not every tick
-
-**Stanford** consulted the AI at every time step and let it decide whether to
-keep going or react. That's the most expensive habit a simulation can have.
-
-**Valhalla** only asks the AI to reconsider when something actually changed
-(an observation is new), and even then only if the character has enough
-energy and mood, isn't on cooldown, and the hourly AI budget allows it. The
-"react" decision in between is a cheap, no-AI rule. There's also a
-deterministic backstop: if a character is ever stuck on the fallback
-"Unscheduled downtime" plan, the engine force-triggers a replan of their
-remaining day instead of hoping the AI notices. Reasoning: an LLM costs money
-and time; a 10,000-tick simulation needs cognition to be an ingredient, not
-the whole meal.
-
-### Difference 7 — Conversations: one shot vs. word-by-word
-
-**Stanford** generated dialogue word by word — each sentence was a separate AI
-call, and the conversation bounced back and forth for many calls.
-
-**Valhalla** generates a **whole conversation in a single call**: 4–12
-messages with a duration, a sentiment, and explicit effects — how much the
-relationship changed (stored in a relationship matrix with scores and tags
-like "close-friends" or "friendly-rivalry"), and how each participant's
-energy and mood shifted. Reasoning: one call instead of a dozen is ~10×
-cheaper, and the effects (relationship, mood) become structured data the
-engine can act on instead of text buried in a diary.
-
-### Bonus difference — Energy and emotion don't exist in Stanford at all
-
-The paper's characters had no body. Ours do:
-
-- **Energy** (0–100%) drains during the day and recovers during sleep.
-- **Emotion** (0–100%) sits on a personality-derived baseline and shifts with
-  events.
-- The planner is told the time-of-day rhythm (energy high in the morning,
-  low near midnight) and writes an **energy target** into every plan action;
-  the engine gently glides each character toward that target. This is the
-  mechanism behind believable behavior like "skips morning class because she
-  stayed up chatting."
-
----
-
-## 4. Novel stuff we added — a checklist of things Stanford doesn't have
-
-| # | Feature | What it is, plainly |
-|---|---|---|
-| 1 | **Energy & emotion system** | Characters get tired, moods shift; personality baselines; circadian energy targets in every plan (§3, bonus). |
-| 2 | **Plan validation + retry + safe fallback** | Every generated plan is checked (times, locations, venue rules, safety) and rewritten up to 3 times before a safe fallback is installed (§3, diff 5). |
-| 3 | **LLM cost budget governor** | A rolling one-hour meter on AI calls; when the ceiling is hit, cognition gracefully degrades and the sim keeps running without AI. Protects free-tier API keys. |
-| 4 | **Multi-key rotation + circuit breaker** | Several Gemini keys are rotated automatically; if all fail, the sim stops cleanly with instructions to resume from checkpoint instead of crashing mid-tick. |
-| 5 | **Structured AI outputs** | The AI must answer in enforced JSON shapes (plan blocks, decisions, conversation schemas), so its output can't corrupt the simulation. |
-| 6 | **Checkpoint / resume / rewind** | The whole world is saved every tick. You can stop, resume tomorrow, or rewind hours and replay. Stanford's demo: run it and hope. |
-| 7 | **Event calendar** | Real campus fests (Aadvitya open-mic, inter-hostel volleyball, exam-week library) with deterministic attendance — the engine decides attendees by interest + social score, splices the event into their plans, and applies mood/relationship effects afterwards. |
-| 8 | **Relationship matrix** | An explicit, directed scorecard between every pair of characters, seeded with curated backstory and updated by every conversation. |
-| 9 | **Daily flavor** | Each day starts with a random theme + emotion ("Today's vibe: Motivated, leaning into Sports") so no two days feel scripted. |
-| 10 | **Health monitor + debug panel** | The engine watches itself every tick (stalled travel, overdue actions, desyncs) and shows anomalies in the UI. |
-| 11 | **Admin login (auth)** | Sim controls (rewind, speed, stop/start, roster edits) are protected by a login; viewers can watch without control. |
-| 12 | **Roster management** | Add a new character from a natural-language description (AI generates the profile), rename, or retire them — while stopped. |
-| 13 | **Real campus grounding** | Verified place facts (open hours, sub-areas, capacity) and a branch-specific venue policy so classes land in the right buildings. |
-| 14 | **Headless CLI + sidecar monitor** | Run the sim without the web UI for automated overnight runs; a separate read-only tool can watch it and write reports. |
-| 15 | **Day/night visuals + timeline controls** | The map fades to night by clock time; fast-forward up to 32×; live speed changes. |
-
----
-
-## 5. What Stanford has that we don't (yet)
-
-Be honest — the original paper has a few things this codebase hasn't built:
-
-1. **Reflection** — the periodic "think about your life and write insights"
-   step. We have end-of-day summaries and a rolling 3-day recap instead, but
-   the deep reflective layer is a documented future step, not implemented.
-2. **Talking to the characters** — in Stanford's demo you could type to a
-   character (as a reporter, or as their "inner voice") and they'd answer,
-   remember you, and obey commands. Our UI is read-only: you watch their
-   conversations live, but there's no chat input. (Several behind-the-scenes
-   hooks are stubbed for it.)
-3. **World objects** — in Stanford's world you could edit an object's state
-   ("the stove is burning") and characters would react. Our world has no
-   interactive objects like stoves or beds.
-4. **Embodied user avatar** — you could join Stanford's town as a walking
-   character. No equivalent here.
-5. **Formal human evaluation** — the paper ran a 100-evaluator believability
-   study. We haven't run any formal evaluation, so "how believable is it
-   really?" is an open question.
-
-Also small: Stanford showed action emoji above characters' heads (like a
-"✍️" while journaling); our map shows a colored dot and the action text for
-the character you've clicked.
-
----
-
-## 6. The design philosophy difference in one paragraph
-
-Stanford's paper is a **science experiment**: control the variables, run two
-days, measure believability. Everything about it is optimized for that —
-small invented world, LLM consulted constantly, no need to survive a crash.
-
-Valhalla is built for the **opposite problem**: keep a simulation running
-for a long time, on a tight API budget, without breaking. That's why we spent
-engineering effort on things the paper never considered — quality-checked
-plans, cost budgets, checkpoints, health monitoring, structured outputs,
-deterministic fallbacks. The paper proved *the idea*; our codebase is
-*the attempt to industrialize it*.
-
----
-
-## 7. Where the project stands (rough status)
-
-**Working today:** full day loop (plan → act → move → perceive → chat →
-replan), energy/mood simulation, real campus map with pathfinding, live
-dashboard, events, relationships, memory (short + long term), checkpoints,
-auth, roster tools, headless runs.
-
-**Known rough edges / notes:**
-
-- The event calendar has a data bug — one event's date is written as month
-  "18," so it can never fire — and dates are formatted inconsistently.
-- The default AI budget ceiling is off unless configured (protection exists
-  but is opt-in).
-- Long-term memory needs a Qdrant cloud account; without it, memory falls
-  back to a simpler mode.
-- One old planning pipeline (`tick_graph.py`) is a documented leftover — the
-  live engine doesn't use it. Same for a few disabled features (e.g.
-  post-conversation replanning).
-- Semantic long-term memory is the weakest-link dependency: if it's down,
-  memory quality drops.
-
----
-
-## 8. Sources
-
-This report is grounded in the actual code, not the README's claims.
-Everything above traces to files in `backend/` (engine: `src/core/world_engine.py`;
-planner: `src/agents/day_planner.py`; memory: `src/agents/Short_term.py`,
-`src/agents/vector_memory.py`; events: `src/core/world_events.py`; budget:
-`src/core/budget.py`; checkpoints: `src/core/checkpoint_manager.py`) and the
-React UI in `frontend/`, plus the attached paper. Key facts were verified
-against file contents (e.g. "no reflection module," "no chat endpoint,"
-"no emoji rendering").
-
-*Scope note: this is a whole-codebase comparison. Every file in the project
-was read (55 source files, ~10k lines); the most deeply traced areas are the
-simulation engine, the day planner, memory, and the frontend. Confidence on
-the "what's different" section is high; the "what's left" section reflects
-explicit code evidence (TODOs, disabled flags, dead code) plus judgment.*