README.md

---
license: mit
language:
  - en
tags:
  - agent-traces
  - coding-agent
---

# SynthTraces

![synthtraces-banner](https://cdn-uploads.huggingface.co/production/uploads/5dd96eb166059660ed1ee413/0KEiR1V6QfFU3jzMpvqlr.png)

A minimal codebase to generate **synthetic coding agent session traces** using [Pi](https://github.com/earendil-works/pi/blob/main/packages/coding-agent).

Each session pairs **two models** working inside one of the project codebases:

- a **remotely hosted open model** (e.g. `deepseek-ai/DeepSeek-V4-Pro`, `openai/gpt-oss-120b`, `Qwen/Qwen3.6-27B`) backs the **coding agent**, equipped with the default Pi tools — `read`, `write`, `edit`, and `bash`;
- a **local model** running in llama.cpp plays the **user**, opening with one of the starting questions and driving the conversation.

The full exchange is recorded as a trace. The dataset is the cartesian product of every agent model × user model × codebase × starting question.

## Generation matrix

| Dimension                            | Count      | Examples                                                                                                            |
| ------------------------------------ | ---------- | ------------------------------------------------------------------------------------------------------------------- |
| Agent models (remotely hosted, open) | 20         | `deepseek-ai/DeepSeek-V4-Pro`, `openai/gpt-oss-120b`, `Qwen/Qwen3.6-27B`, `zai-org/GLM-5.1`, …                      |
| User models (local, llama.cpp)       | 3          | `ggml-org/Qwen3.6-27B-GGUF:Q8_0`, `ggml-org/Qwen3.6-35B-A3B-MTP-GGUF:Q8_0`, `ggml-org/gemma-4-26B-A4B-it-GGUF:Q8_0` |
| Project codebases                    | 20         | `transformers`, `diffusers`, `lerobot`, `peft`, `candle`, …                                                         |
| Starting questions                   | 20         | _"How do I run this code?"_, _"How is CI set up in this repo?"_, _"What recent changes were made and why?"_, …      |
| **Total sessions**                   | **24,000** | **20 × 3 × 20 × 20**                                                                                                |

## How a session works

```text
           1 of 20 starting questions
                        │
                        ▼
        ┌────────────────────────────────┐
        │  USER MODEL · local, llama.cpp │   plays the user
        └───────────────┬────────────────┘
                        │   prompt ↔ reply  (≤ N turns)
        ┌───────────────┴────────────────┐
        │  AGENT MODEL · remote, open    │   the coding agent
        │  ( one of 20 router models )   │
        └───────────────┬────────────────┘
                        │   reads · edits · runs
        ┌───────────────┴────────────────┐
        │  PROJECT CODEBASE  (1 of 20)   │   cloned locally
        └───────────────┬────────────────┘
                        │
                        ▼
              session trace  ──▶  dataset   (1 of 24,000)
```

## Final Statistics

> _TODO: populated after generation (success rate, total turns, token counts, traces per model/codebase, …)._

## Dataset Details

- **Generated by:** [julien-c](https://huggingface.co/julien-c)
- **Language:** English
- **License:** MIT

## Links

- 📦 Code: [`julien-c/synthtraces`](https://github.com/julien-c/synthtraces)
- 🤗 Dataset: [`julien-c/synthtraces`](https://huggingface.co/datasets/julien-c/synthtraces)