Upload 190 files

Code-agent-main/.env.example

@@ -0,0 +1,9 @@
+# YCode needs no API key: it answers with its own built-in model (YCode-LM).
+# Copy this file to .env (or ~/.ycode/.env) only to change settings. Never commit .env.
+
+# Which built-in model to use: v2, v3, v4, v5 (default) or a model directory.
+YCODE_LOCAL_MODEL=
+
+# Optional, only for Claude (`ycode --provider anthropic`, after `pip install -e ".[claude]"`):
+# ANTHROPIC_API_KEY=
+# YCODE_MODEL=claude-opus-5-5

Code-agent-main/.gitignore

@@ -0,0 +1,40 @@
+# Secrets
+.env
+.env.*
+!.env.example
+
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+
+# Virtual environments
+.venv/
+venv/
+env/
+
+# Editors / OS
+.idea/
+.vscode/
+.DS_Store
+
+# Local YCode state
+.ycode/history
+
+# YCode-LM datasets and trained models (large binaries)
+/data/
+/models/*
+*.bin
+*.pt
+# ...except the published, exported YCode-LM models
+!/models/ycode-lm-*/
+!/models/ycode-lm-*/model.pt
+!/models/ycode-lm-*/model-*-of-*.pt

Code-agent-main/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 yazhiniyanMS
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

Code-agent-main/README.md

@@ -0,0 +1,535 @@
+# YCode
+
+```
+██╗   ██╗ ██████╗ ██████╗ ██████╗ ███████╗
+╚██╗ ██╔╝██╔════╝██╔═══██╗██╔══██╗██╔════╝
+ ╚████╔╝ ██║     ██║   ██║██║  ██║█████╗
+  ╚██╔╝  ██║     ██║   ██║██║  ██║██╔══╝
+   ██║   ╚██████╗╚██████╔╝██████╔╝███████╗
+   ╚═╝    ╚═════╝ ╚═════╝ ╚═════╝ ╚══════╝
+```
+
+YCode is an AI coding agent that runs in your terminal. Start it inside a project, describe a task in plain English, and it works through it: it looks around the repository, makes a short plan, edits files, runs your tests and build, reads any failures, fixes them, and then tells you what changed.
+
+**YCode works out of the box with no API key and no training.** By default it answers with **YCode-LM**, its own coding language model, trained from scratch and shipped in this repository (`models/`). Clone, install, and run `ycode`: nothing to sign up for, no key to paste, nothing sent over the network. See [Answering without an API key](#answering-without-an-api-key).
+
+YCode-LM is the only model YCode needs. If you also want full autonomous coding (reading, editing and running files in your project, as in the example below), you can optionally plug in Claude: see [Using Claude (optional)](#using-claude-optional). Everything else always runs locally: the tool system, the permission checks and the terminal UI. The model sits behind a small provider interface, so you can add other LLM providers without rewriting the agent.
+
+With Claude (`--provider anthropic`), YCode works through whole tasks:
+
+```text
+ycode> Fix the failing add() test
+
+  ◇ Plan
+    ▸ 1. Read calc.py
+    ○ 2. Fix add
+    ○ 3. Run tests
+
+  ◇ Reading calc.py
+  ✓ Read calc.py (2 of 2 lines)
+
+  ◇ Editing calc.py
+  ✓ Edited calc.py (+1 -1)
+      @@ -1,2 +1,2 @@
+       def add(a, b):
+      -    return a - b
+      +    return a + b
+
+  ◇ Running `python -m pytest -q`
+  ✓ Succeeded (exit code 0, 0.3s)
+
+  Fixed `add` in calc.py; pytest passes (1 test).
+
+  Changes:
+   M calc.py
+
+  Commands:
+   ✓ python -m pytest -q
+
+  ✓ YCode completed the task.  (5 steps)
+```
+
+## Features
+
+- **Terminal first.** Uses Rich for output: spinners, streamed Markdown, syntax-highlighted diffs. Uses prompt_toolkit for input: persistent history, multiline editing, and tab completion for `/commands`.
+- **Autonomous agent loop.** The model can make many tool calls per task, parallel calls included, until it is done. A configurable step limit (default 50) stops runaway loops.
+- **Planning.** For non-trivial work the model writes a short plan through an `update_plan` tool and keeps it current. For trivial requests it skips the plan.
+- **Verification.** The model picks verification commands from your project type (`npm test`, `pytest`, `cargo test`, ...), reads failures and iterates. It is told never to claim success it did not verify.
+- **Tools:** `read_file`, `write_file`, `edit_file`, `list_directory`, `search_files`, `find_files`, `run_command`, `git_status`, `git_diff`, `git_log`, `git_branch`, `update_plan`.
+- **Security.** Commands are classified as SAFE, REQUIRES_APPROVAL or BLOCKED. Risky ones need an explicit `y`. File access is confined to the workspace, and API keys are stripped from the environment that commands run in.
+- **Repository awareness.** Detects Node.js, TypeScript, JavaScript, React, Next.js, Vue, Python, Java, Kotlin, C/C++, Rust, Go, Flutter/Dart, Ruby and PHP projects, plus the package manager and the likely test and build commands. Large generated directories are ignored.
+- **Diff awareness.** After each task YCode summarizes the changed files and the commands it ran. `/diff` shows the full diff.
+- **Project instructions.** A `YCODE.md` file in the repository is loaded into the agent's instructions.
+- **Layered configuration:** global and per-project TOML files, environment variables and CLI flags.
+- **No API key needed.** The default provider is YCode's own bundled model. It answers Python documentation questions exactly from Python's built-in docs, and writes functions that are checked to compile and define what you asked for. Everything runs offline.
+- **Model-agnostic core.** The agent depends only on `LLMProvider`. There are two implementations: your own local model (default) and Claude (optional).
+- **Your own LLM.** `ycode-lm` builds a byte-level BPE tokenizer and a GPT-style transformer from scratch, pretrains it on code, and instruction-tunes it to answer programming questions. Four trained versions ship with the repository, up to the 1.5B-parameter v5.
+
+## Architecture
+
+```text
+                ┌──────────────┐
+                │   YCode CLI  │  cli.py, commands.py, ui/
+                └──────┬───────┘
+                       │
+                ┌──────▼───────┐
+                │ Agent Engine │  agent/loop.py, planner.py, context.py, prompts.py
+                └──────┬───────┘
+                       │
+              ┌────────▼────────┐
+              │   LLM Provider  │  llm/base.py (interface), llm/anthropic.py
+              └────────┬────────┘
+                       │
+                  Claude API
+                       │
+              ┌────────▼────────┐
+              │  Tool Execution │  tools/ (+ security/permissions.py)
+              └────────┬────────┘
+          ┌────────────┼────────────┐
+          ▼            ▼            ▼
+       Filesystem   Terminal       Git
+```
+
+```text
+ycode/
+├── __main__.py          # python -m ycode
+├── cli.py               # argument parsing, REPL, graceful shutdown
+├── app.py               # wires config, context, permissions, tools, provider, agent, UI
+├── commands.py          # /help /clear /status /diff /files /model /plan /approval /exit
+├── config.py            # layered TOML + env + CLI configuration
+├── errors.py            # user-facing error hierarchy (no raw tracebacks)
+├── agent/
+│   ├── loop.py          # the agent loop, step limit, interruption handling
+│   ├── planner.py       # Plan model + update_plan tool
+│   ├── context.py       # project detection, YCODE.md loading
+│   └── prompts.py       # system prompt
+├── llm/
+│   ├── base.py          # provider-neutral messages, ToolSpec, LLMProvider
+│   ├── anthropic.py     # Claude via the official anthropic SDK (streaming)
+│   └── local.py         # your own YCode-LM model (answer-only, no API key)
+├── lm/                  # YCode-LM: the from-scratch model; assist.py answers docs questions and checks code
+│   ├── tokenizer.py     # byte-level BPE tokenizer for code
+│   ├── model.py         # GPT transformer: RoPE, RMSNorm, SwiGLU, KV-cache sampling
+│   ├── data.py          # corpus collection, instruction-pair extraction, encoding
+│   ├── train.py         # pretraining + instruction tuning, checkpoints, resume
+│   ├── optim.py         # Muon optimizer (v3) + AdamW wiring
+│   ├── evaluate.py      # bits/byte on held-out code, pass@k on executed problems
+│   ├── generate.py      # inference (chat + completion, streaming)
+│   └── cli.py           # the `ycode-lm` command
+├── tools/
+│   ├── base.py          # Tool, ToolRegistry (validation + dispatch), ToolResult
+│   ├── filesystem.py    # read/write/edit/list
+│   ├── search.py        # regex content search, glob file search
+│   ├── terminal.py      # run_command (timeouts, output capture, Ctrl+C)
+│   └── git.py           # git helpers + read-only git tools
+├── security/
+│   └── permissions.py   # command classifier, PathGuard, PermissionManager
+└── ui/
+    ├── banner.py        # ASCII banner
+    ├── console.py       # Rich renderer (implements AgentEvents)
+    └── prompts.py       # prompt_toolkit input with history/multiline
+```
+
+The dependencies point one way. The agent knows nothing about Rich or Anthropic: it talks to an `LLMProvider` and reports progress through `AgentEvents` callbacks. The UI implements those callbacks. Tools don't know about the model.
+
+**Adding a provider:** subclass `LLMProvider` in `ycode/llm/`, translate the neutral `UserMessage`/`AssistantMessage`/`ToolResultsMessage` types to your API's format, and register it in `ycode/llm/__init__.py:create_provider`. Assistant messages carry an opaque `provider_data` field. Your provider can use it to replay its own native content exactly, which Claude needs for its thinking blocks.
+
+## Requirements
+
+- Python 3.10+
+- About 4 GB of free RAM for the default model, YCode-LM v5 (1.5B parameters). With less, YCode uses the smaller v4 (about 1 GB) automatically.
+- Git, to clone the repository (it contains the trained models) and for the git features
+
+That's all. **No API key, no account and no internet connection are needed** to run YCode.
+
+## Installation
+
+```bash
+git clone https://github.com/yazhiniyanMS/Code-agent.git
+cd Code-agent
+python -m venv .venv
+source .venv/bin/activate        # Windows: .venv\Scripts\activate
+pip install -e .                 # includes PyTorch for the built-in model
+ycode                            # start asking questions: no API key, no training
+```
+
+This installs a `ycode` command. `python -m ycode` works too. Install from the clone (as above) rather than from a bare package: the trained models live in the repository's `models/` folder. To keep them elsewhere, set `YCODE_MODELS_DIR`.
+
+Optional extras:
+
+```bash
+pip install -e ".[hf]"       # Hugging Face: transformers, huggingface_hub, safetensors, accelerate
+pip install -e ".[claude]"   # Claude support (only if you want --provider anthropic)
+pip install -e ".[dev]"      # tests
+```
+
+## Answering without an API key
+
+`ycode` uses YCode's own model by default. Every answer goes through three steps:
+
+1. **Documentation questions are answered exactly.** "What does `len()` do?", "How do I use `json.loads`?", "What is `yield`?", "Explain `os.path.join`", "What's the difference between a list and a tuple?": YCode looks the names up in Python's own built-in documentation, which ships with every Python install, and shows the signature and the official description. Only built-in names and standard-library modules are looked up. Your project's code is never imported.
+2. **"Write a function `name(...)`" requests are checked.** The model writes up to four candidates. YCode keeps the first one that compiles and defines the function you asked for. Candidates are only parsed, never run. The answer says that the name and syntax were checked, not the logic.
+3. **Everything else** goes to the model as a normal chat turn.
+
+```text
+ycode> What does the zip() function do?
+
+  zip(*iterables, strict=False) (class)
+    Yield tuples until an input is exhausted.
+    >>> list(zip('abcdefg', range(3), range(4)))
+    [('a', 0, 0), ('b', 1, 1), ('c', 2, 2)]
+    ...
+  (From Python's built-in documentation.)
+```
+
+**Choosing a model.** YCode picks a model in this order: one you trained yourself (`~/.ycode/models/ycode-lm`), otherwise the bundled **v5**. If less than 4 GB of memory is free, it uses v4 instead. Pick one explicitly with `--local`:
+
+```bash
+ycode                      # default: YCode-LM v5 (1.5B parameters), loaded in 16-bit: about 3 GB of RAM
+ycode --local v4           # YCode-LM v4 (100M parameters): much faster on a CPU
+ycode --local path/to/dir  # any model you trained
+```
+
+You can also set `local_model = "v4"` in `~/.ycode/config.toml` or `YCODE_LOCAL_MODEL=v4`. v5 models code best of all versions (see [the results](#your-own-llm-ycode-lm-no-api-key)), but a 1.5B model is slow on a CPU: expect a few tokens per second, so a long answer can take a minute or more. On a slow machine, v4 is the practical choice.
+
+**What to expect.** The bundled models were trained from scratch on a CPU, with a tiny fraction of the data and compute behind commercial assistants. They write well-formed Python, but their own code is often wrong, so test it. The local model also runs in **answer-only mode**: it answers in its reply but cannot read, edit or run files in your project. For autonomous work on a codebase, use Claude.
+
+## Hugging Face (optional)
+
+`pip install -e ".[hf]"` installs the Hugging Face libraries: `transformers`, `huggingface_hub`, `safetensors` and `accelerate`. With them, `ycode-lm push-hf` publishes a model to your Hugging Face account, and `ycode-lm autotrain` trains on GPUs while saving checkpoints there.
+
+These commands need a Hugging Face **access token** with Write access, created at https://huggingface.co/settings/tokens. Give it to YCode in either of these ways:
+
+```bash
+export HF_TOKEN=hf_...      # for this shell (or put HF_TOKEN=... in ~/.ycode/.env)
+hf auth login               # or save it once with the Hugging Face CLI
+```
+
+Then, for example:
+
+```bash
+ycode-lm push-hf --model models/ycode-lm-v5 --repo your-name/ycode-lm-v5 --public
+```
+
+The token is only used to upload models. Running YCode never needs it. Keep it secret: never commit it, and don't paste it into a chat.
+
+## Using Claude (optional)
+
+YCode never needs Claude. If you want it for autonomous work on a codebase, install the extra with `pip install -e ".[claude]"`, then run `ycode --provider anthropic` or set `provider = "anthropic"` in your config. YCode then reads `ANTHROPIC_API_KEY`. In order of precedence, the places it looks are:
+
+1. Your shell environment: `export ANTHROPIC_API_KEY=sk-ant-...`
+2. A `.env` file in the project you run YCode in
+3. `~/.ycode/.env`, which applies to every project
+4. The `.env` in the YCode checkout itself, which is handy with `pip install -e .`
+
+Values that are already set are never overridden. Any of these work too: `ANTHROPIC_AUTH_TOKEN`, an `ant auth login` profile, or workload identity federation. If you ask for Claude and YCode finds no credentials, it shows the banner, prints a clear error and exits with status 1. Without `--provider anthropic`, YCode never asks for a key.
+
+YCode never prints your key. It also removes `ANTHROPIC_API_KEY` and `ANTHROPIC_AUTH_TOKEN` from the environment of commands it runs, so a stray `env` can't leak the key to the model. **Never commit `.env`.** The included `.gitignore` excludes it.
+
+## Your own LLM (YCode-LM, no API key)
+
+YCode-LM is a programming language model you train from scratch on your own machine. Nothing is downloaded: the tokenizer, the model architecture and the training loop are all in this repository (`ycode/lm/`), and the data is code you already have.
+
+**What it is:**
+- **Tokenizer:** byte-level BPE (`ycode/lm/tokenizer.py`) trained on your corpus. A code-aware pre-tokenizer turns indentation runs, identifiers and operators into compact tokens.
+- **Model:** a decoder-only transformer (`ycode/lm/model.py`) with rotary position embeddings, RMSNorm, SwiGLU MLPs, tied embeddings and a KV cache for fast sampling. **Version 2** (the default) adds grouped-query attention, which shares key/value heads for a 4× smaller KV cache and faster generation, and QK-norm, which keeps attention stable at higher learning rates. v1 checkpoints still load.
+- **Training data:** any code directories you point it at. With no `--source`, it uses the Python standard library, which every Python install has.
+- **Two training stages:**
+  1. *Pretraining:* next-token prediction on code.
+  2. *Instruction tuning (SFT):* teaches the model to answer. The data is generated automatically from the real code in your corpus, so every answer is real code:
+     - write a function from its description
+     - explain a function
+     - complete a function from its signature and docstring
+     - write a docstring
+     - explain a class
+     - **find and fix a bug**: YCode-LM injects realistic one-token bugs (`<` vs `<=`, `+` vs `-`, off-by-one, `and`/`or`, `True`/`False`). The original code is the fix, so these examples are correct by construction.
+
+     You can add your own pairs as JSONL (`{"prompt": ..., "response": ...}`) with `--sft-data`.
+
+**What v2 improves:**
+
+| | v1 | v2 |
+| --- | --- | --- |
+| Attention | multi-head | grouped-query (smaller cache) + QK-norm |
+| LR schedule | cosine over steps; cut short by `--minutes` | warmup-stable-decay. Progress is the larger of steps done and time used, so a time-limited run always finishes its decay |
+| End of pretraining | code only | anneals with 20% instruction data during the decay phase, so the model already knows the chat format before SFT |
+| SFT batches | one example per row, padded | several examples packed per row, so no compute is wasted on padding |
+| Data | write/explain pairs | + bug fixing, completion, docstrings, classes; de-duplicated prompts; licence headers stripped |
+| Data prep | single process | tokenization on all CPU cores |
+| Measurement | loss only | `ycode-lm eval`: bits per byte on held-out code + pass@k on 30 executed coding problems |
+
+**The included models (no training needed).** The repository ships four trained models. `ycode` uses v5 by default (v4 when memory is short):
+
+| Model | Params | Size | Card |
+| --- | --- | --- | --- |
+| **YCode-LM v5** (default) | **1,523M** | 2.9 GB in 103 shards | [`models/ycode-lm-v5`](models/ycode-lm-v5/README.md) |
+| YCode-LM v4 | 100M | 200 MB in 5 shards | [`models/ycode-lm-v4`](models/ycode-lm-v4/README.md) |
+| YCode-LM v3 | 40.8M | 81.8 MB | [`models/ycode-lm-v3`](models/ycode-lm-v3/README.md) |
+| YCode-LM v2 | 7.7M | 15.6 MB | [`models/ycode-lm-v2`](models/ycode-lm-v2/README.md) |
+
+```bash
+ycode --local v4                          # or: ycode-lm chat --model models/ycode-lm-v4
+```
+
+**Set up and train:**
+
+```bash
+pip install -e .                   # PyTorch + NumPy are included
+
+# 1. Build the dataset: collect code, train the tokenizer, create instruction pairs.
+#    --exclude keeps code out of training so you can evaluate on it later.
+ycode-lm prepare --out data/ --source ~/my-projects --source /usr/lib/python3.11 --exclude '*/holdout/*'
+
+# 2. Pretrain on code (bf16 on CPUs with AMX/AVX512-BF16, CUDA or Apple MPS when available)
+ycode-lm train --data data/ --out models/base --preset v2-small --minutes 180
+
+# 3. Instruction-tune it so it answers requests
+ycode-lm sft --data data/ --init-from models/base --out ~/.ycode/models/ycode-lm --minutes 40
+
+# 4. Measure it (compare several models by repeating --model)
+ycode-lm eval --model ~/.ycode/models/ycode-lm --heldout path/to/unseen/code --samples 5
+
+# 5. Use it
+ycode-lm chat                                   # quick Q&A in the terminal
+ycode                                           # inside YCode: your model in ~/.ycode/models/ycode-lm is used automatically
+ycode --local models/chat                       # or point to any trained model
+```
+
+Other commands: `ycode-lm sample --model models/base --prompt "def quicksort("` (raw code completion), `ycode-lm info`, `ycode-lm export --model DIR --out DIR` (slim bf16 copy for sharing, about 4× smaller than a training checkpoint), and `ycode-lm train --resume` (continue an interrupted run). Ctrl+C during training keeps the last checkpoint. Training options include `--schedule wsd|cosine`, `--anneal-mix`, `--no-pack`, `--compile` (torch.compile) and `--grad-accum`.
+
+**Version 3: the 40M-parameter model.** `--preset v3-40m` uses the v2 architecture scaled to 40.8M parameters with a 1024-token context. It trains with the **Muon optimizer** by default: transformer weight matrices are updated with orthogonalized momentum (Newton–Schulz iterations), while embeddings and norms stay on AdamW. On small transformers, Muon reaches a given loss with fewer tokens, which matters most when compute is the bottleneck. Use `--optimizer adamw` to switch back.
+
+```bash
+ycode-lm train --data data/ --out models/v3-base --preset v3-40m --compile --minutes 480
+ycode-lm sft   --data data/ --init-from models/v3-base --out ~/.ycode/models/ycode-lm --compile --minutes 60
+```
+
+In an A/B test (same model, data, seed and 600 steps), Muon reached a held-out loss of **4.13** against AdamW's **4.26**, for 14% more wall-clock per step at that tiny size; the overhead shrinks as models grow.
+
+A 40M model wants far more data and compute than a CPU can supply. The compute-optimal budget is about 800M training tokens, while 8 hours on a 4-core CPU covers about 50M. On a single consumer GPU, the same run takes hours instead of days.
+
+**Measured results (v1 → v5).** All five models were evaluated the same way. Bits per byte is measured on 11 packages none of them saw in training. The code tasks are *executed* against unit tests:
+
+| | v1 | v2 | v3 | v4 | v5 |
+| --- | --- | --- | --- | --- | --- |
+| Parameters | 6.9M | 7.7M | 40.8M | 100M | 1,523M |
+| Training | ~41M tokens | ~82M tokens | ~60M tokens (Muon) | grown from v3 + ~24.6M tokens | grown from v4 + ~2.5M tokens (Lion, CPU) |
+| Bits per byte on held-out code | 1.095 | 0.966 | 0.857 | 0.814 | **0.758** |
+| Write a function, pass@1 (30 problems) | 0 / 30 | 0 / 30 | 0 / 30 | **1 / 30** | 0 / 30 |
+| Fix an injected bug, fix@1 (15 functions) | 0 / 15 | **5 / 15** | 4 / 15 | **5 / 15** | 4 / 15 |
+
+Each version models real code better than the last; v5's 7% gain over v4 is the largest step so far, but it has not yet turned into solving more tasks (it is badly under-trained for its size). v2–v4 write short, well-formed answers and can find and fix simple bugs ("The bug is in `result = 1`. It should be `result = 0`."), while v1 falls into repetition loops. v4 is the first to write a correct function from a description (`square` → `return x * x`). Writing functions reliably needs much more training data and compute than a CPU provides.
+
+**Growing models.** `ycode-lm grow --model v3 --out v4-init --layers 34` deepens a trained model by inserting copies of existing blocks with zeroed output projections. The result computes exactly the same function, so `train --init-from v4-init` continues from everything the small model learned. v4 was built this way. For files over GitHub's 100 MB limit, `export --max-shard-mb 45` splits the weights into shards that load transparently.
+
+For long runs on machines that may be interrupted, train in segments: `--until-step N` stops and saves, `--resume` continues with the same LR schedule, and `--save-interval N` saves cheaply between evaluations.
+
+**Version 5: 1.5 billion parameters, trained on a CPU.** v5 (`--preset v5-1.5b`, 34 layers × 2048 wide, 32 query / 8 KV heads, 1,523M parameters) is v4 grown 4× in width with `ycode-lm grow --width 2048`, so it starts out computing exactly what v4 computed and keeps everything v4 learned. A 1.5B model normally needs ~18 GB just for fp32 weights, gradients and optimizer state. v5 trains in **15 GB of RAM** with a low-memory mode (`--optimizer lion`):
+
+- **Lion optimizer:** the update is the sign of a momentum, so it keeps one state per weight instead of AdamW's two, stored in bf16 (3 GB).
+- **Updates inside backward:** each weight is updated the moment its gradient is ready, and the gradient is freed at once. A full set of gradients (6 GB) never exists.
+- Gradient checkpointing, bf16 autocast without the cast-weight cache (which would be another 3 GB copy), and memory-mapped checkpoint loading.
+
+```bash
+ycode-lm grow  --model models/ycode-lm-v4 --out models/v5-base --width 2048 --version 5
+ycode-lm train --data data/ --out models/v5-base --resume --steps 600 --batch-size 4 \
+               --optimizer lion --lr 3e-5 --grad-checkpoint --save-interval 20 --until-step 80
+```
+
+On 4 CPU cores this runs at about 55 tokens per second, so a day of training covers only a few million tokens. A 1.5B model would ideally see tens of billions. v5 therefore benefits most from GPUs. With several GPUs, `--shard` splits the weights, gradients and optimizer state across them (FSDP) so models too large for one GPU still train. Muon works on the sharded weights. `ycode-lm autotrain` drives a whole run across time-limited GPU sessions, saving checkpoints to Hugging Face. [`notebooks/kaggle_train_v5.ipynb`](notebooks/kaggle_train_v5.ipynb) does this on free Kaggle GPUs. To publish any model directory, use `ycode-lm push-hf --model DIR --repo you/name`.
+
+**Evaluation.** `ycode-lm eval` reports:
+- **Bits per byte** on code the model never saw. Lower is better. It is comparable across tokenizers, so v1, v2 and v3 compare fairly.
+- **pass@1 / pass@k** on 30 small programming problems, plus **fix@1** on 15 functions with one injected bug each. YCode-LM writes each solution or fix, and it is *executed* against unit tests in an isolated subprocess with a timeout. Only evaluate models you trained yourself, because generated code is run.
+
+**Model sizes** (`--preset`; parameter counts are for an 8k vocabulary):
+
+| Preset | Layers × width | Heads (KV) | Context | Params | Where to train |
+| --- | --- | --- | --- | --- | --- |
+| **`v5-1.5b`** | 34 × 2048 | 32 (8) | 1024 | **1,523M** | grow from v4; CPU with `--optimizer lion` (slow), or GPUs (`--shard` for several) |
+| `v5-385m` | 34 × 1024 | 16 (4) | 1024 | ~385M | grow from v4; a 16 GB GPU |
+| **`v4-100m`** | 34 × 512 | 8 (2) | 1024 | **100.0M** | grow from v3 (`ycode-lm grow`); GPU recommended for training |
+| **`v3-40m`** | 13 × 512 | 8 (2) | 1024 | **40.8M** | GPU ideal; CPU works but slowly (measured ~2.1k tok/s on 4 Xeon cores with bf16 + compile) |
+| `v3-tiny` | 3 × 64 | 4 (2) | 128 | ~0.7M | tests |
+| `v2-tiny` | 2 × 64 | 4 (2) | 128 | ~0.6M | tests / smoke runs |
+| **`v2-small`** (default) | 8 × 256 | 8 (2) | 512 | ~7.7M | laptop CPU, 2–4 h |
+| `v2-base` | 12 × 512 | 8 (2) | 1024 | ~38M | GPU |
+| `v2-medium` | 16 × 768 | 12 (4) | 1024 | ~107M | GPU, many hours |
+| `v2-large` | 24 × 1024 | 16 (4) | 2048 | ~274M | multi-hour GPU runs |
+
+The v1 presets (`tiny`, `small`, `base`, `medium`, `large`) are still available.
+
+Use `--layers/--heads/--embd/--context` to customize a preset.
+
+**Using it in YCode.** The local model is YCode's default provider. Related settings: `local_model` (`v2`–`v5` or a path), `local_max_tokens` and `local_temperature`. Nothing is sent over the network.
+
+**Be realistic about quality.** Frontier coding models are trained on trillions of tokens with thousands of GPUs. A few-million-parameter model trained for an hour or two on a CPU learns real Python syntax, idioms and naming conventions, and can write small functions and short explanations. It will also often be wrong, repetitive or confused. Two consequences:
+
+- **Answer-only mode.** In YCode, the local model runs in answer-only mode. It hasn't learned the tool-calling protocol, so it cannot read, edit or run files in your project. YCode shows a notice when it starts. Use Claude (`--provider anthropic`) for autonomous coding tasks.
+- **How to make it better:** more and better data (your own repositories), a bigger preset, a GPU and longer training. The loss figures in `train_log.json` show whether it is still improving.
+
+## Usage
+
+```bash
+cd my-project
+ycode                                  # interactive session
+ycode "add a --verbose flag to the CLI" # one-shot: run the task, then exit
+ycode -p "explain src/auth.ts"         # same, with -p
+ycode --local v5                       # pick a bundled model (v2-v5) or a model directory
+ycode --provider anthropic             # optional: use Claude (needs the [claude] extra and ANTHROPIC_API_KEY)
+ycode --provider anthropic -m claude-sonnet-5-5   # pick a Claude model
+ycode --approval-mode strict           # approve every write and non-read-only command
+ycode --yes                            # auto-approve risky commands (blocked ones stay blocked)
+ycode -C ../other-project              # run against another directory
+```
+
+In the interactive session:
+
+| Key / command | Action |
+| --- | --- |
+| Enter | Send the request |
+| Esc+Enter (or a line ending in `\`) | Insert a newline |
+| Up / Down | History (stored in `~/.ycode/history`) |
+| Ctrl+C | Interrupt the running task; YCode kills any running command and returns to the prompt |
+| Ctrl+C twice / Ctrl+D | Exit |
+| `/help` | List commands |
+| `/status` | Project, branch, modified files, model, steps used, token usage |
+| `/diff [full] [path...]` | Show the git diff, with a stat summary if it is very large |
+| `/files` | Changed files: working tree plus files written this session |
+| `/model [id]` | Show or switch the model for this session |
+| `/plan` | Show the current plan |
+| `/approval [strict\|normal\|auto]` | Show or change the approval mode |
+| `/clear` | Forget the conversation and clear the screen |
+| `/exit`, `/quit` | Quit |
+
+Slash commands are handled locally and are never sent to the model. Each request continues the same conversation, so a follow-up like "now add tests for that" works.
+
+## Examples
+
+```text
+ycode> Add dark mode to the settings page
+ycode> Why does `npm run build` fail? Fix it.
+ycode> Write unit tests for src/utils/date.ts and make sure they pass
+ycode> Rename the `User.fullname` field to `full_name` everywhere
+ycode> Explain how requests are authenticated in this project
+ycode> Commit these changes with a good message      # git commit asks for approval
+```
+
+## Security model
+
+YCode gives a language model the ability to edit files and run commands, so it treats both as risky.
+
+**Command classification** (`ycode/security/permissions.py`). Every command line is split into segments (`&&`, `||`, `;`, `|`), and command substitutions (`$(...)` and backticks) and `bash -c "..."` strings are classified recursively. The most severe result wins.
+
+| Level | Examples | Behavior |
+| --- | --- | --- |
+| SAFE | `npm test`, `pytest`, `git status`, `ls`, `cargo build` | Runs immediately |
+| REQUIRES_APPROVAL | `rm`, `mv`, `sudo`, `chmod`, `chown`, `kill`, `git push`, `git reset --hard`, `git clean`, `git commit`, `git checkout -- .`, `npm publish`, `curl … \| sh`, `find -delete`, writes redirected outside the project, reading `.env`/key files | Shows `⚠ YCode wants to run: … Allow? [y/N/a]` |
+| BLOCKED | `rm -rf /`, `rm -rf ~`, fork bombs, `mkfs`, `dd of=/dev/…`, `shutdown`, writes to raw devices | Never runs, even with `--yes` |
+
+At the approval prompt, `y` allows the command once and `a` allows that exact command for the rest of the session. Anything else denies it, and the model is told not to retry.
+
+**Approval modes:**
+- `normal` (default): SAFE commands run, risky commands ask, blocked commands are refused.
+- `strict`: also asks before every file write and every command that isn't read-only.
+- `auto`: risky commands run without asking. Blocked commands are still refused.
+
+**Filesystem confinement.** Every path is resolved, symlinks included, and has to stay inside the project directory. `../`, absolute paths and symlinks that point outside the project are rejected. Writes into `.git/` are always rejected. Reading files that look like secrets (`.env*`, `*.pem`, `id_rsa`, ...) asks first. `allow_outside_workspace = true` in your *global* config lifts the confinement.
+
+**Untrusted repositories.** A project's `.ycode/config.toml` can only make settings *stricter*. It cannot switch on `auto` approvals or access outside the workspace, so cloning a repository can't silently loosen your safety settings.
+
+**Git.** YCode never commits or pushes on its own. Those commands always need your approval.
+
+**Limits.** This is a guardrail, not a sandbox. When the agent runs code it wrote (`python script.py`, `npm test`), that code can do anything your user account can. For untrusted code, run YCode inside a container or VM.
+
+## Configuration
+
+Settings are resolved in this order, with later sources winning:
+
+1. Built-in defaults
+2. Global config: `~/.ycode/config.toml` (move this directory with `YCODE_HOME`)
+3. Project config: `<project>/.ycode/config.toml` (can only tighten security settings)
+4. Environment: `YCODE_PROVIDER`, `YCODE_LOCAL_MODEL`, `YCODE_MODELS_DIR`, `YCODE_MODEL`, `YCODE_MAX_STEPS`, `YCODE_MAX_TOKENS`, `YCODE_EFFORT`, `YCODE_APPROVAL_MODE`, `YCODE_THEME`, `YCODE_COMMAND_TIMEOUT`, `YCODE_IGNORE_DIRS`
+5. Command-line flags
+
+```toml
+# ~/.ycode/config.toml
+provider = "local"            # local (default: YCode's own model, no key) | anthropic (Claude)
+local_model = "v5"            # v2 | v3 | v4 | v5 | a model directory (default: v5, or v4 when memory is short)
+model = "claude-opus-5-5"     # any Claude model ID (provider = "anthropic")
+max_steps = 50                # model calls per task before YCode stops
+max_tokens = 64000            # output token cap per model call
+effort = "high"               # low | medium | high | xhigh | max (models that support it)
+approval_mode = "normal"      # strict | normal | auto
+theme = "default"             # default | mono (no colors)
+command_timeout = 120         # seconds, per command
+max_output_chars = 30000      # command/tool output sent back to the model
+refusal_fallback = true       # retry declined requests on Anthropic's recommended fallback model
+# ignore_dirs = [...]         # replaces the default ignore list
+extra_ignore_dirs = ["generated", "vendor"]   # adds to it
+```
+
+The default model is `claude-opus-5-5`. You can set any model ID, for example `claude-sonnet-5-5`, `claude-fable-5-1`, `claude-haiku-4-5` or `claude-sonnet-4-5`. YCode adapts the request to the model. It turns on adaptive thinking and `effort` only on models that support them, and server-side refusal fallbacks only on Claude Fable 5.1, Opus 5.5, Opus 5 and Sonnet 5.5. If you route through a gateway with `ANTHROPIC_BASE_URL`, YCode sends a plainer request shape.
+
+By default these directories are ignored: `node_modules .git dist build .next .nuxt .cache venv .venv env __pycache__ .pytest_cache .mypy_cache .ruff_cache target coverage .gradle .idea .dart_tool .tox .eggs`.
+
+## YCODE.md instructions
+
+Put a `YCODE.md` at the root of your repository (or at `.ycode/YCODE.md`) and YCode adds it to the agent's instructions for every task in that project:
+
+```markdown
+# YCode Project Instructions
+
+Use TypeScript.
+
+Run npm test before finishing.
+
+Do not modify generated files.
+
+Use functional React components.
+```
+
+The banner and `/status` show when instructions are loaded. Files longer than 20,000 characters are truncated.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+python -m pytest
+```
+
+Set `YCODE_DEBUG=1` to see full tracebacks for unexpected internal errors. Normal user errors are always shown as short messages, never as tracebacks.
+
+Some guidelines:
+- Keep the agent free of UI and provider imports. Communicate through `AgentEvents` and `LLMProvider`.
+- Tools return a `ToolResult` and never raise for expected failures. The registry turns exceptions into error results the model can react to.
+- New command patterns belong in `security/permissions.py`, with tests in `tests/test_security.py`.
+
+## Testing
+
+The test suite needs no API key and makes no network calls:
+
+- `tests/test_anthropic_provider.py` mocks the Anthropic client: request shape, stream parsing, error mapping and missing credentials.
+- `tests/test_anthropic_wire.py` runs the real `anthropic` SDK against a mock HTTP transport that returns canned server-sent events. It checks the exact JSON YCode sends across a full tool-use round trip.
+- `tests/test_lm.py` covers YCode-LM: tokenizer round-trips (including Unicode), special tokens, a KV-cache check against full recomputation, answer-only loss masking, dataset preparation, pretraining that measurably lowers loss, SFT, resume, checkpoint round trips and the `ycode-lm` CLI end to end. These tests are skipped if PyTorch isn't installed.
+- `tests/test_local_provider.py` checks the local provider: no API key, no tools sent, `--local` from the CLI, and how bundled models are found.
+- `tests/test_assist.py` checks answering without an API: documentation lookups and checked function writing.
+- `tests/test_agent_loop.py` drives the agent with a scripted fake provider: multi-step tasks, parallel tool calls, the step limit, interruption, truncated tool calls and refusals.
+- The other test files cover configuration, the banner, file tools, path security, command classification and approvals, command execution (timeouts, exit codes, secret stripping), git, project detection and YCODE.md, slash commands, and the CLI and REPL.
+
+```bash
+python -m pytest -q
+```
+
+## Roadmap
+
+- YCode-LM: tool-calling training data, so the local model can drive the agent
+- YCode-LM: train v5 much longer on GPUs (`ycode-lm autotrain`, `--shard` are ready), then publish it on Hugging Face
+- YCode-LM: faster CPU inference for v5 (8-bit weights)
+- More providers: OpenAI-compatible, local models through Ollama
+- Session persistence and resume (`ycode --continue`)
+- Context compaction for very long sessions
+- Optional sandboxed command execution (containers)
+- `/undo` that restores files to the state before the last task
+- MCP server support for extra tools
+- A richer TUI mode (Textual) as an optional front-end

Code-agent-main/YCODE.md

@@ -0,0 +1,9 @@
+# YCode Project Instructions
+
+This repository is YCode itself (Python 3.10+).
+
+- Run `python -m pytest -q` before finishing; all tests must pass without an API key or network.
+- Keep layers separate: `agent/` must not import from `ui/` or a specific provider in `llm/`.
+- Tools return `ToolResult` objects; don't let expected failures raise out of a tool.
+- Any change to command classification in `security/permissions.py` needs a test in `tests/test_security.py`.
+- Never print or log API keys.

Code-agent-main/models/ycode-lm-v2/README.md

@@ -0,0 +1,44 @@
+# YCode-LM v2 (7.7M parameters)
+
+See also the larger [YCode-LM v3 (40.8M)](../ycode-lm-v3/README.md).
+
+A small programming language model trained **from scratch** with this repository's `ycode-lm`
+pipeline: its own BPE tokenizer, its own transformer and training loop, no pretrained weights.
+It is published here so `ycode --local` works right after cloning, with no API key and no training.
+
+| | |
+| --- | --- |
+| Architecture | decoder-only transformer, 8 layers × 256 wide, 8 query / 2 key-value heads (GQA), QK-norm, RoPE, SwiGLU, RMSNorm |
+| Parameters | 7,738,112 |
+| Context | 512 tokens |
+| Tokenizer | byte-level BPE, 8,192 tokens (embedded in `model.pt`) |
+| Weights | bfloat16 (15.6 MB), converted to float32 on load |
+| Pretraining | 10,000 steps, ~82M tokens of Python (176 MB: the standard library plus installed packages), warmup-stable-decay LR, annealing with 20% instruction data |
+| Instruction tuning | 2,400 packed steps on 109k pairs generated from real code: write and explain functions, complete stubs, write docstrings, explain classes, fix injected bugs |
+| Hardware | 4-core Intel Xeon CPU (bf16, `torch.compile`) |
+
+## Results
+
+Measured with `ycode-lm eval` on 11 packages the model never saw (click, anyio, jsonschema,
+markdown_it, starlette, uvicorn, h11, attr, pluggy, filelock, idna):
+
+| | v1 (6.9M) | **v2 (this model)** |
+| --- | --- | --- |
+| Bits per byte on held-out code (lower is better) | 1.095 | **0.966** |
+| pass@1 on 30 executed coding problems | 0 / 30 | 0 / 30 |
+| fix@1 on 15 buggy functions (fixes executed against tests) | 0 / 15 | 5 / 15 |
+
+It writes fluent, well-formed Python and follows the instruction formats, but its code is usually
+wrong. It is a learning/research model, not a coding assistant. In YCode it runs in answer-only
+mode (no tools).
+
+## Use
+
+```bash
+pip install -e ".[local]"
+ycode --local models/ycode-lm-v2
+ycode-lm chat --model models/ycode-lm-v2
+ycode-lm eval --model models/ycode-lm-v2 --heldout path/to/code/it/never/saw
+```
+
+To train your own (or continue from this one), see "Your own LLM" in the main README.

Code-agent-main/models/ycode-lm-v2/info.json

@@ -0,0 +1,19 @@
+{
+  "step": 2400,
+  "stage": "sft",
+  "val_loss": 0.9569745600223541,
+  "version": 2,
+  "params": 7738112,
+  "dtype": "bf16",
+  "config": {
+    "vocab_size": 8192,
+    "block_size": 512,
+    "n_layer": 8,
+    "n_head": 8,
+    "n_embd": 256,
+    "dropout": 0.0,
+    "n_kv_head": 2,
+    "qk_norm": true,
+    "arch_version": 2
+  }
+}

Code-agent-main/models/ycode-lm-v2/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c41151d3759a4d8e6cea992d9e1285aa61728c7220f73d6523639f0a9321a541
+size 15585317

Code-agent-main/models/ycode-lm-v3/README.md

@@ -0,0 +1,49 @@
+# YCode-LM v3 (40.8M parameters)
+
+The largest YCode-LM, trained **from scratch** with this repository's `ycode-lm` pipeline: its own
+BPE tokenizer, its own transformer and training loop, no pretrained weights.
+
+| | |
+| --- | --- |
+| Architecture | decoder-only transformer, 13 layers × 512 wide, 8 query / 2 key-value heads (GQA), QK-norm, RoPE, SwiGLU, RMSNorm |
+| Parameters | 40,836,608 |
+| Context | 1,024 tokens |
+| Tokenizer | byte-level BPE, 8,192 tokens (shared with v2, embedded in `model.pt`) |
+| Weights | bfloat16 (81.8 MB), converted to float32 on load |
+| Pretraining | 7,300 steps × 8,192 tokens ≈ 60M tokens of Python (same 176 MB corpus as v2), **Muon** optimizer for the transformer matrices + AdamW for embeddings/norms, warmup-stable-decay LR, annealing with 20% instruction data |
+| Instruction tuning | 1,400 packed steps on 87k pairs generated from real code. Answers use compact one-line docstrings and only standalone functions for write/complete tasks |
+| Hardware | 4-core Intel Xeon CPU (bf16, `torch.compile`), about 10 hours in total |
+
+## Results
+
+All three models were evaluated the same way with `ycode-lm eval`. Bits per byte is measured on 11
+packages none of them saw in training (click, anyio, jsonschema, markdown_it, starlette, uvicorn, h11,
+attr, pluggy, filelock, idna). The code tasks are executed against unit tests.
+
+| | v1 (6.9M) | v2 (7.7M) | **v3 (40.8M)** |
+| --- | --- | --- | --- |
+| Bits per byte on held-out code (lower is better) | 1.095 | 0.966 | **0.857** |
+| Base model before instruction tuning (pretraining held-out loss) | 2.07 nats/token* | 2.063 | **1.805** |
+| Write a function from scratch, pass@1 (30 problems) | 0 / 30 | 0 / 30 | 0 / 30 |
+| Fix an injected bug, fix@1 (15 functions) | 0 / 15 | 5 / 15 | 4 / 15 |
+
+\*v1 used a different tokenizer and held-out split, so its per-token loss is not directly comparable.
+
+**What this means:**
+- v3 models real code clearly best: it predicts unseen code 11% better than v2 and 22% better than v1.
+- It writes short, well-formed answers and can find and fix simple bugs. For example, given
+  `result = 1` in a summing loop it answers "The bug is in `result = 1`. It should be `result = 0`."
+  and returns the corrected function.
+- It does **not** yet write correct functions from a description. On bug fixing, v2 and v3 are
+  within noise of each other on 15 problems.
+- More data and compute (a GPU) are the way forward. The pipeline supports both unchanged.
+
+In YCode it runs in answer-only mode (no tools).
+
+## Use
+
+```bash
+pip install -e ".[local]"
+ycode --local models/ycode-lm-v3
+ycode-lm chat --model models/ycode-lm-v3
+```

Code-agent-main/models/ycode-lm-v3/info.json

@@ -0,0 +1,19 @@
+{
+  "step": 1400,
+  "stage": "sft",
+  "val_loss": 0.5056432619690895,
+  "version": 3,
+  "params": 40844416,
+  "dtype": "bf16",
+  "config": {
+    "vocab_size": 8192,
+    "block_size": 1024,
+    "n_layer": 13,
+    "n_head": 8,
+    "n_embd": 512,
+    "dropout": 0.0,
+    "n_kv_head": 2,
+    "qk_norm": true,
+    "arch_version": 3
+  }
+}

Code-agent-main/models/ycode-lm-v3/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:add7e319e66c14eb1ef05d26a1b49b4b7bf591e3031ecc6f5e0f712c806eb8e2
+size 81808535

Code-agent-main/models/ycode-lm-v4/README.md

@@ -0,0 +1,50 @@
+# YCode-LM v4 (100M parameters)
+
+The 100M-parameter YCode-LM, **grown from YCode-LM v3** rather than trained from scratch:
+v3's 13 trained layers were kept and 21 new layers were inserted between them, each a copy of its
+neighbour with zeroed output projections. The grown model computed exactly what v3 computed, and
+continued pretraining then trained the new layers.
+
+| | |
+| --- | --- |
+| Architecture | decoder-only transformer, 34 layers × 512 wide, 8 query / 2 key-value heads (GQA), QK-norm, RoPE, SwiGLU, RMSNorm |
+| Parameters | 100,047,616 |
+| Context | 1,024 tokens |
+| Tokenizer | byte-level BPE, 8,192 tokens (shared with v2/v3) |
+| Weights | bfloat16, **sharded** into `model-0000N-of-00005.pt` files (each < 45 MB, under GitHub's file limit); `model.pt` holds the config, the tokenizer and the shard index |
+| Continued pretraining | 3,000 steps × 8,192 tokens ≈ 24.6M tokens on top of v3, Muon (lr 0.015) + AdamW (lr 1e-3), warmup-stable-decay with annealing |
+| Instruction tuning | 700 packed steps on the corrected 87k-pair instruction set |
+| Hardware | 4-core Intel Xeon CPU, ~760 tokens/s, about 13 hours |
+
+## Results
+
+All four models were evaluated the same way with `ycode-lm eval`. Bits per byte is measured on 11
+packages none of them saw in training. The code tasks are executed against unit tests.
+
+| | v1 (6.9M) | v2 (7.7M) | v3 (40.8M) | **v4 (100M)** |
+| --- | --- | --- | --- | --- |
+| Bits per byte on held-out code (lower is better) | 1.095 | 0.966 | 0.857 | **0.814** |
+| Write a function from scratch, pass@1 (30 problems) | 0 / 30 | 0 / 30 | 0 / 30 | **1 / 30** |
+| Fix an injected bug, fix@1 (15 functions) | 0 / 15 | 5 / 15 | 4 / 15 | **5 / 15** |
+
+**What this means:**
+- v4 models unseen code best of all versions: 5% better than v3 and 26% better than v1.
+- It is the first YCode-LM to write a correct function from a description:
+  `square(x)` → `return x * x`. That answer also passes extra tests (0, 5, 1.5, -7). One problem
+  out of 30 is a small step, not a reliable skill.
+- It fixed 5 of 15 injected bugs (`add`, `is_even`, `filter_even`, `celsius_to_fahrenheit`,
+  `is_palindrome`).
+- **The trade-off:** on the training corpus's own validation split, v4's pretraining loss ended
+  slightly *worse* than v3's (1.832 vs 1.805 nats/token). Raising the learning rate again after
+  growing disturbed v3's tuned weights, and 24.6M tokens is far too little to fully train 60M new
+  parameters. It still generalises better to unseen packages (bits per byte above). More training,
+  ideally on a GPU, is the clear next step.
+
+## Use
+
+```bash
+pip install -e ".[local]"
+ycode --local models/ycode-lm-v4
+```
+
+Keep all six `model*.pt` files together. Loading reassembles the shards automatically.

Code-agent-main/models/ycode-lm-v4/info.json

@@ -0,0 +1,27 @@
+{
+  "step": 700,
+  "stage": "sft",
+  "val_loss": 0.5377802371978759,
+  "version": 4,
+  "params": 100047616,
+  "dtype": "bf16",
+  "config": {
+    "vocab_size": 8192,
+    "block_size": 1024,
+    "n_layer": 34,
+    "n_head": 8,
+    "n_embd": 512,
+    "dropout": 0.0,
+    "n_kv_head": 2,
+    "qk_norm": true,
+    "arch_version": 4
+  },
+  "files": [
+    "model.pt",
+    "model-00001-of-00005.pt",
+    "model-00002-of-00005.pt",
+    "model-00003-of-00005.pt",
+    "model-00004-of-00005.pt",
+    "model-00005-of-00005.pt"
+  ]
+}

Code-agent-main/models/ycode-lm-v4/model-00001-of-00005.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:961f55773b4f423f96d22b20d5712637108465ad70471d59767174abc236164d
+size 44993885

Code-agent-main/models/ycode-lm-v4/model-00002-of-00005.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64a3035fea4b8f3c67d6157effb4fecbfb611513973becc244ce9e782134f085
+size 43688311

Code-agent-main/models/ycode-lm-v4/model-00003-of-00005.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:068af5d12a43b770056c9e811eb1cf29419184a2053262cc0a145a2bc50915f4
+size 44603673

Code-agent-main/models/ycode-lm-v4/model-00004-of-00005.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2187ef1f0121ec9b240674edd729fb7b2d383d8df78a68795bef0618c5828f0b
+size 44343735

Code-agent-main/models/ycode-lm-v4/model-00005-of-00005.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5853cebc4e0c4022378a80727ab6765f449307fc1f3af12e936d3ec62986a59c
+size 22565529

Code-agent-main/models/ycode-lm-v4/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f150539f8c69c16374cfe712220f49221fc2bbfc3182b1ca24d54b75dc97216
+size 91937

Code-agent-main/models/ycode-lm-v5/README.md

@@ -0,0 +1,62 @@
+# YCode-LM v5 (1.5B parameters)
+
+The 1.5-billion-parameter YCode-LM, **grown from YCode-LM v4** and trained entirely on a CPU.
+v4's 34 layers × 512 channels were widened 4× to 2048 channels: new channels and attention heads
+started with zeroed outputs, so the grown model computed exactly what v4 computed (checked: identical
+loss on the same batches before training). Continued pretraining then trained the new capacity.
+
+| | |
+| --- | --- |
+| Architecture | decoder-only transformer, 34 layers × 2048 wide, 32 query / 8 key-value heads (GQA), QK-norm, RoPE, SwiGLU, RMSNorm |
+| Parameters | 1,523,202,304 |
+| Context | 1,024 tokens |
+| Tokenizer | byte-level BPE, 8,192 tokens (shared with v2–v4) |
+| Weights | bfloat16, **sharded** into 103 `model-*-of-00103.pt` files (each < 45 MB, under GitHub's file limit); `model.pt` holds the config, the tokenizer and the shard index |
+| Continued pretraining | 600 steps × 4,096 tokens ≈ 2.5M tokens, Lion (lr 3e-5), warmup-stable-decay with 20% instruction data during the decay |
+| Instruction tuning | 60 packed steps, Lion (lr 1.5e-5) |
+| Hardware | 4-core Intel Xeon CPU, 15 GB of RAM. 55–75 tokens/s on a CPU with AMX bf16 units, 8–9 tokens/s on one without |
+
+## Training a 1.5B model in 15 GB of RAM
+
+Weights, gradients and AdamW state for 1.5B parameters take ~24 GB in fp32. v5 used a low-memory
+mode (`ycode-lm train --optimizer lion`):
+
+- **Lion** keeps one momentum value per weight (AdamW keeps two), stored in bf16: 3 GB.
+- **Updates inside backward:** each weight is updated as soon as its gradient is ready, and the
+  gradient is freed. A full set of gradients (6 GB) never exists.
+- Gradient checkpointing, bf16 autocast without the cast-weight cache, memory-mapped checkpoints.
+
+Peak memory was about 12 GB. Training ran in restartable segments that saved checkpoints, and it
+survived three container restarts.
+
+## Results
+
+Evaluated with `ycode-lm eval`, the same way as every earlier version. Bits per byte is measured on
+11 Python packages none of the models saw in training. The code tasks are executed against unit tests.
+
+| | v1 (6.9M) | v2 (7.7M) | v3 (40.8M) | v4 (100M) | **v5 (1,523M)** |
+| --- | --- | --- | --- | --- | --- |
+| Bits per byte on held-out code (lower is better) | 1.095 | 0.966 | 0.857 | 0.814 | **0.758** |
+| Write a function from scratch, pass@1 (30 problems) | 0 / 30 | 0 / 30 | 0 / 30 | **1 / 30** | 0 / 30 |
+| Fix an injected bug, fix@1 (15 functions) | 0 / 15 | **5 / 15** | 4 / 15 | **5 / 15** | 4 / 15 |
+
+**What this means:**
+- v5 models unseen code best of all versions: **7% better than v4**, the largest step between any two
+  versions, after only ~2.5M tokens of extra training.
+- That has not yet turned into solving more tasks. v5 fixed 4 bugs (`total`, `filter_even`,
+  `celsius_to_fahrenheit`, `is_palindrome`) and wrote no fully correct function. With 30 and 15 tasks,
+  a difference of one is within noise. Its instruction tuning was cut to 60 steps by the slow CPU.
+- A 1.5B model would ideally see tens of billions of training tokens; v5 saw a few million on top of
+  v4. It is badly under-trained, and more training (ideally on GPUs: `ycode-lm autotrain`, `--shard`)
+  is the clear next step.
+- YCode uses v5 as its default model on machines with at least 4 GB of free memory, and v4 otherwise. v4 is just as good at the benchmark tasks and much faster on a CPU, so `ycode --local v4` is a good choice on slow machines.
+
+## Use
+
+```bash
+pip install -e .
+ycode                         # v5 is the default; loads in bfloat16: about 3 GB of RAM
+ycode-lm chat --model models/ycode-lm-v5
+```
+
+Keep all 104 `model*.pt` files together. Loading reassembles the shards automatically.

Code-agent-main/models/ycode-lm-v5/info.json

@@ -0,0 +1,125 @@
+{
+  "step": 60,
+  "stage": "sft",
+  "val_loss": 0.5093614831566811,
+  "version": 5,
+  "params": 1523202304,
+  "dtype": "bf16",
+  "config": {
+    "vocab_size": 8192,
+    "block_size": 1024,
+    "n_layer": 34,
+    "n_head": 32,
+    "n_embd": 2048,
+    "dropout": 0.0,
+    "n_kv_head": 8,
+    "qk_norm": true,
+    "arch_version": 5
+  },
+  "files": [
+    "model.pt",
+    "model-00001-of-00103.pt",
+    "model-00002-of-00103.pt",
+    "model-00003-of-00103.pt",
+    "model-00004-of-00103.pt",
+    "model-00005-of-00103.pt",
+    "model-00006-of-00103.pt",
+    "model-00007-of-00103.pt",
+    "model-00008-of-00103.pt",
+    "model-00009-of-00103.pt",
+    "model-00010-of-00103.pt",
+    "model-00011-of-00103.pt",
+    "model-00012-of-00103.pt",
+    "model-00013-of-00103.pt",
+    "model-00014-of-00103.pt",
+    "model-00015-of-00103.pt",
+    "model-00016-of-00103.pt",
+    "model-00017-of-00103.pt",
+    "model-00018-of-00103.pt",
+    "model-00019-of-00103.pt",
+    "model-00020-of-00103.pt",
+    "model-00021-of-00103.pt",
+    "model-00022-of-00103.pt",
+    "model-00023-of-00103.pt",
+    "model-00024-of-00103.pt",
+    "model-00025-of-00103.pt",
+    "model-00026-of-00103.pt",
+    "model-00027-of-00103.pt",
+    "model-00028-of-00103.pt",
+    "model-00029-of-00103.pt",
+    "model-00030-of-00103.pt",
+    "model-00031-of-00103.pt",
+    "model-00032-of-00103.pt",
+    "model-00033-of-00103.pt",
+    "model-00034-of-00103.pt",
+    "model-00035-of-00103.pt",
+    "model-00036-of-00103.pt",
+    "model-00037-of-00103.pt",
+    "model-00038-of-00103.pt",
+    "model-00039-of-00103.pt",
+    "model-00040-of-00103.pt",
+    "model-00041-of-00103.pt",
+    "model-00042-of-00103.pt",
+    "model-00043-of-00103.pt",
+    "model-00044-of-00103.pt",
+    "model-00045-of-00103.pt",
+    "model-00046-of-00103.pt",
+    "model-00047-of-00103.pt",
+    "model-00048-of-00103.pt",
+    "model-00049-of-00103.pt",
+    "model-00050-of-00103.pt",
+    "model-00051-of-00103.pt",
+    "model-00052-of-00103.pt",
+    "model-00053-of-00103.pt",
+    "model-00054-of-00103.pt",
+    "model-00055-of-00103.pt",
+    "model-00056-of-00103.pt",
+    "model-00057-of-00103.pt",
+    "model-00058-of-00103.pt",
+    "model-00059-of-00103.pt",
+    "model-00060-of-00103.pt",
+    "model-00061-of-00103.pt",
+    "model-00062-of-00103.pt",
+    "model-00063-of-00103.pt",
+    "model-00064-of-00103.pt",
+    "model-00065-of-00103.pt",
+    "model-00066-of-00103.pt",
+    "model-00067-of-00103.pt",
+    "model-00068-of-00103.pt",
+    "model-00069-of-00103.pt",
+    "model-00070-of-00103.pt",
+    "model-00071-of-00103.pt",
+    "model-00072-of-00103.pt",
+    "model-00073-of-00103.pt",
+    "model-00074-of-00103.pt",
+    "model-00075-of-00103.pt",
+    "model-00076-of-00103.pt",
+    "model-00077-of-00103.pt",
+    "model-00078-of-00103.pt",
+    "model-00079-of-00103.pt",
+    "model-00080-of-00103.pt",
+    "model-00081-of-00103.pt",
+    "model-00082-of-00103.pt",
+    "model-00083-of-00103.pt",
+    "model-00084-of-00103.pt",
+    "model-00085-of-00103.pt",
+    "model-00086-of-00103.pt",
+    "model-00087-of-00103.pt",
+    "model-00088-of-00103.pt",
+    "model-00089-of-00103.pt",
+    "model-00090-of-00103.pt",
+    "model-00091-of-00103.pt",
+    "model-00092-of-00103.pt",
+    "model-00093-of-00103.pt",
+    "model-00094-of-00103.pt",
+    "model-00095-of-00103.pt",
+    "model-00096-of-00103.pt",
+    "model-00097-of-00103.pt",
+    "model-00098-of-00103.pt",
+    "model-00099-of-00103.pt",
+    "model-00100-of-00103.pt",
+    "model-00101-of-00103.pt",
+    "model-00102-of-00103.pt",
+    "model-00103-of-00103.pt"
+  ]
+}

Code-agent-main/models/ycode-lm-v5/model-00001-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afb6784fcfe746bda869bb438cf11a73b6085b06e64d3185b1bb6e67b65124f8
+size 33560525

Code-agent-main/models/ycode-lm-v5/model-00002-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4810a38f8034caed7957c5f012235af1e0a75838ce97442daea8ef7354323ccd
+size 43523441

Code-agent-main/models/ycode-lm-v5/model-00003-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ace4ad13912e6b42be6167df2501b0e86b7645c18ff13ada1fb54036927ac835
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00004-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc9693fc86b58bd3876e8744e66eaec6ae54bffe018481d0e82fabad62725739
+size 43527802

Code-agent-main/models/ycode-lm-v5/model-00005-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eda1e06ef093890df14b6ba39cd1bcf34fa58efdcfbb94e51dc099581d35a1a2
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00006-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9984c157d39719c6946b0d6bbed7406070a985ca50990a07989b1b5a8946beb1
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00007-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bab4af1b31d732ddab6ff5b9867892ee53f45eb5d1a82f23308d57a29f11f2ff
+size 43527802

Code-agent-main/models/ycode-lm-v5/model-00008-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ddec4cd6da1d5dde02cbf4aa97b8bc7d7dbe9d8693a34e4cad4c059252e7b74
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00009-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f37c8f1dc7e9fbd5e10be412f7cfb29b491bcf90ec751f4637abbd157a3351a
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00010-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1164afe87f1a4293be3a18765d9595885f7c3f4c8928c5f87dc8ebc9e9f1ca07
+size 43527802

Code-agent-main/models/ycode-lm-v5/model-00011-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c753815abcedeca1841ae02838c0d249131b71278add0790bff0a01b95b651c
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00012-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b550d0cb07c6e6af2b9f163bebb042a717dfe2c3a673ea88040707c8c775632e
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00013-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7a77ef7b3924df87a6a05e04f10e6f42d4dbdac163199d51f3d14f495f1a4f7
+size 43527802

Code-agent-main/models/ycode-lm-v5/model-00014-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a876ecc507ec7e3c6740aa754c0e85bbccf969588d3860afbdb42f801fe7e7f0
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00015-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2fd7fe47fba754513b2b57eef76b4f9a988ac190e520bec6533ea9c0ab862f0
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00016-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c6efcd1a8b4b2d8782d3db4c9bb1d26a8b4406ccb97e6ced7ddb49e6377b9f9
+size 43527802

Code-agent-main/models/ycode-lm-v5/model-00017-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e54dbc95b695c00a36e2a8d9cb873c1efdf212e818f869387ae133ef8c8ca59
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00018-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:777d6e9c09225e9861ee3fea5d6e680e8900487b5b9afdae760aecbb82f7b938
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00019-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a9c40570e3dd449056a205ff8b6b9ed857579b730e9dd13cd542910864b5845
+size 43527802

Code-agent-main/models/ycode-lm-v5/model-00020-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7781407201ec22454a9dfe1e0df55679ab7782888252bf74eeb408219b0796db
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00021-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb5d8f51e0c86aefe23951dc4b1522c01e549a3b6e7e3a310052134c85744a6c
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00022-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42cfbe77e58237ef5cff76b34561f369f696af4920c47349192fae5b5c6a5d32
+size 43527802

Code-agent-main/models/ycode-lm-v5/model-00023-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b3e9f260378cbab9af16c6b89ab4f68c1b2e81a464118e6def83ff5f70d92ea
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00024-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a7f934228bf51343e73a8d56e937812ff1dbe28c2fcf11db622525680645115
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00025-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfc8feb483b56a20953aaa674dcc7cda637666cbe74229c1331e5c61a559414d
+size 43527802

Code-agent-main/models/ycode-lm-v5/model-00026-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:056341c06293ded4b744b9c4fca012a195492c128ac9b44b83282f0fdfb67db9
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00027-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6951910520f56ceb6909987c90e095a0d4128e51b6659dbad1416ff08447dca8
+size 22546116

Code-agent-main/models/ycode-lm-v5/model-00028-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:610348ff139ac765a969eca2448778122c6f02b700cb3454091e830cbac366f5
+size 43527802

Code-agent-main/models/ycode-lm-v5/model-00029-of-00103.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:644716475242a6f01044f9461d30b992e23e0743df9b86832bdbdcb5dd871df8
+size 22546116