|
Download README.md from tetu-AI/Tetu-Code-2: direct link, hf CLI and curl.
- Browser
- Download file 6.63 kB
-
https://huggingface.co/tetu-AI/Tetu-Code-2/resolve/main/README.md
- Command line
-
hf download hf://tetu-AI/Tetu-Code-2/README.md
-
curl -L -o README.md https://huggingface.co/tetu-AI/Tetu-Code-2/resolve/main/README.md
6.63 kB
| license: apache-2.0 | |
| language: | |
| - en | |
| - fr | |
| - code | |
| pipeline_tag: text-generation | |
| tags: | |
| - code | |
| - coding-agent | |
| - small-language-model | |
| - trained-from-scratch | |
| - custom-architecture | |
| - tool-use | |
| - opencode | |
| model-index: | |
| - name: Tetu-Code-2 | |
| results: | |
| - task: | |
| type: text-generation | |
| dataset: | |
| name: HumanEval+ | |
| type: evalplus/humanevalplus | |
| metrics: | |
| - type: pass@1 | |
| name: pass@1 (greedy, chat prompt) | |
| value: 15.9 | |
| - task: | |
| type: text-generation | |
| dataset: | |
| name: MBPP+ | |
| type: evalplus/mbppplus | |
| metrics: | |
| - type: pass@1 | |
| name: pass@1 (greedy, chat prompt) | |
| value: 30.7 | |
| # Têtu Code 2 | |
| Têtu Code 2 is a small coding agent trained from scratch. It is designed to run in the terminal through | |
| [OpenCode](https://opencode.ai), where it reads and edits files, runs commands and reports what it did. | |
| The model was trained from random initialization; no pretrained weights were used. Qwen3.6-35B-A3B (Apache 2.0) | |
| served as a teacher for the agent demonstrations, and every demonstration was checked by a program before training. | |
| Training ran on a single RTX 5070 (12 GB) plus about 30 hours of cloud GPU time. | |
| ## Model details | |
| | | | | |
| |---|---| | |
| | Developer | tetu-AI | | |
| | Architecture | Decoder-only transformer (custom) | | |
| | Parameters | 1.15B total, 475M active per token (the rest is a lookup memory with negligible compute cost) | | |
| | Context length | 65,536 tokens | | |
| | Languages | Python first, plus HTML, CSS, JavaScript and others; instructions in English or French | | |
| | Vocabulary | 32,768 tokens | | |
| | Weights | bfloat16 (`model.safetensors`) | | |
| | License | Apache 2.0 | | |
| Capabilities: | |
| - tool use through OpenCode: read, write, edit, bash, glob, grep; | |
| - a short written reasoning between `<|think|>` and `<|/think|>` before each action; | |
| - a per-token confidence head that estimates the probability of an error; | |
| - an OpenAI-compatible server (`tetu_code.serve`) for the OpenCode 1.x and 2.x tool formats, which keeps the KV | |
| cache across turns and stops repeated identical tool calls. | |
| ## Training | |
| | Stage | Data | | |
| |---|---| | |
| | Pre-training | 7.08B tokens, mostly code (Python first, then web and other languages), with code-related web pages, French and English web text, and math | | |
| | Annealing | The final quarter adds teacher projects, exercises whose solutions pass their tests, agent trajectories and commits | | |
| | Agent fine-tuning | 2,093 verified teacher runs on multi-file projects (CLI tools, games, websites, APIs, bug fixes, tests), plus DAgger states in which the model acts and the teacher supplies the correct action | | |
| ## Evaluation | |
| ### Code generation (EvalPlus) | |
|  | |
|  | |
| | Model | Size | Training tokens | HumanEval+ | MBPP+ | | |
| |---|---|---|---|---| | |
| | DeepSeek-Coder-1.3B-instruct | 1.3B | 2T | 60.4 | 54.8 | | |
| | DeepSeek-Coder-1.3B-base | 1.3B | 2T | 25.6 | 47.9 | | |
| | Qwen2.5-Coder-0.5B (base) | 0.5B | 5.5T | 23.8 | 47.1 | | |
| | Têtu Code 1 (previous model) | 193M active | 7B | 18.3 | 27.5 | | |
| | **Têtu Code 2** | **475M active** | **7.08B** | **15.9** | **30.7** | | |
| | SantaCoder-1.1B | 1.1B | 236B | 14.0 | – | | |
| | StarCoderBase-1B | 1B | 1T | 12.2 | – | | |
| | InCoder-1.3B | 1.3B | ~60B | 11.0 | – | | |
| | CodeGen2-1B | 1B | ~400B | 9.1 | – | | |
| Têtu Code results use EvalPlus 0.3.1 with greedy decoding, one sample per problem and a chat prompt. A repeated run | |
| gave HumanEval+ 14.6 and MBPP+ 30.7, so differences of about 1.5 points are within noise. Other results are taken from | |
| the official EvalPlus leaderboard (23 Oct. 2026); the Qwen2.5-Coder figures are from its authors. | |
| Têtu Code 2 scores above the 2023 1B code models (SantaCoder, StarCoderBase-1B) after 30 to 140 times fewer training | |
| tokens, and below DeepSeek-Coder-1.3B and Qwen2.5-Coder-0.5B, which were trained on 280 to 780 times more tokens. | |
| ### Agent tasks | |
|  | |
| | | Têtu Code 1 | Têtu Code 2 | Teacher (Qwen3.6-35B-A3B) | | |
| |---|---|---|---| | |
| | 40 held-out multi-file projects in OpenCode 2.1.4 (hidden tests) | 0/40 | **8/40** | 29/40 | | |
| | Single-function agent benchmark (OpenCode 1.19.4) | 46.5 % | 28.5 % | 85.5 % | | |
| | Python validation loss (pre-training) | 0.821 | **0.777** | – | | |
| Throughput on an RTX 5070: about 143 tokens/s at 2k context and 71 tokens/s at 16k. | |
| ## Limitations | |
| - As a small model, it often starts from incorrect logic or uses undefined variables. | |
| - When its own tests fail, it tends to re-run commands rather than fix the code. | |
| - In multi-turn conversations it can miss a correction or a follow-up request. | |
| - It is tuned as an agent: in the plain chat script (`chat.py`) it often still tries to call tools. Use it through | |
| OpenCode. | |
| - It sometimes writes files outside the project directory; the bundled server maps such paths back into the project. | |
| - Run it in a sandbox or a disposable directory, not unattended on important code. | |
| ## Usage | |
| The architecture is not supported by `transformers`; use the bundled inference code. | |
| ```bash | |
| git clone https://huggingface.co/tetu-AI/Tetu-Code-2 && cd Tetu-Code-2 | |
| pip install -r requirements.txt | |
| python chat.py # terminal chat without tools (--cpu without a GPU) | |
| python -m tetu_code.serve --model . --port 8070 # OpenAI-compatible server for OpenCode | |
| ``` | |
| `opencode.json`: | |
| ```json | |
| "provider": {"tetu": {"npm": "@ai-sdk/openai-compatible", "name": "Têtu Code", | |
| "options": {"baseURL": "http://127.0.0.1:8070/v1", "apiKey": "x"}, | |
| "models": {"tetu-code": {"name": "Têtu Code", "tool_call": true, | |
| "limit": {"context": 65536, "output": 4096}}}}} | |
| ``` | |
| From Python: | |
| ```python | |
| import torch | |
| from tetu_code.generate import Decoder, load | |
| model, tok = load(".", device="cuda") # or device="cpu", dtype=torch.float32 | |
| dec = Decoder(model, tok, max_len=16384) | |
| ``` | |
| ## License | |
| Apache 2.0. | |
| Teachers (both Apache 2.0): Qwen3.6-35B-A3B for the agent runs, Qwen2.5-7B-Instruct for general conversation. | |
| Training data: github-code-clean (permissively licensed files only), FineWeb-Edu, FineWeb2-HQ and FineMath (ODC-By), | |
| OpenCodeInstruct, Nemotron-SWE trajectories and CommitPackFT. Check the terms of each dataset before commercial use. | |
| ## Citation | |
| ```bibtex | |
| @misc{tetucode2026, | |
| title = {Têtu Code 2: a small coding agent trained from scratch}, | |
| author = {tetu-AI}, | |
| year = {2026}, | |
| url = {https://huggingface.co/tetu-AI/Tetu-Code-2} | |
| } | |
| ``` | |