C1-Tachu / README.md
Pomoika24's picture
Upload README.md with huggingface_hub
33c2b5c verified
|
Raw
History Blame Contribute Delete
6.59 kB
---
library_name: transformers
license: apache-2.0
pipeline_tag: text-generation
base_model:
- Qwen/Qwen2.5-Coder-14B
tags:
- code
- coding
- software-engineering
- agent
- debugging
- architecture
- qwen2
- qlora
language:
- en
---
# C1-Tachu
**C1-Tachu** is a specialized coding AI fine-tuned from [Qwen2.5-Coder-14B](https://huggingface.co/Qwen/Qwen2.5-Coder-14B) with a single mission: **minimize the time from prompt to production-ready software.**
The model is not optimized for benchmark scores. It is optimized for **developer throughput** β€” fewer iterations, faster debugging, faster code navigation, faster architecture decisions, and faster project completion.
## Model Details
| | |
|---|---|
| **Base model** | Qwen2.5-Coder-14B |
| **Architecture** | Qwen2 (Causal LM) |
| **Parameters** | 14B |
| **Hidden size** | 5120 |
| **Layers** | 48 |
| **Attention heads** | 40 (8 KV heads, GQA) |
| **Context length** | 32,768 tokens |
| **Precision** | bfloat16 |
| **Format** | Qwen ChatML |
| **License** | Apache 2.0 |
## Training
C1-Tachu was trained using **QLoRA** (4-bit nf4 quantization with LoRA adapters) via the [Axolotl](https://github.com/axolotl-ai-cloud/axolotl) framework on AWS g5.2xlarge (NVIDIA A10G).
### Pipeline (8 stages)
Each stage trains a fresh LoRA adapter on the previous stage's merged model, then merges it back into the base weights. Adapters are not stacked β€” each stage builds cleanly on the merged result.
| Stage | Name | LoRA Rank | Examples | Loss |
|---|---|---|---|---|
| 1 | Software Foundation | 64 | ~10,000 | β€” |
| 2 | Fast Code Generation | 64 | ~6,000 | β€” |
| 3 | Debugging Speed | 64 | ~7,000 | β€” |
| 4 | Project Navigation | 64 | ~3,000 | β€” |
| 5 | Rapid Architecture | 64 | 1,926 | 1.129 |
| 6 | Self Verification | 32 | 1,500 | 0.668 |
| 7 | Preference Optimization (ORPO) | β€” | β€” | *Skipped* |
| 8 | Agent Workflows | 32 | 1,000 | 0.791 |
**Stage 7 (ORPO) was skipped** in this training run due to time constraints. The model retains all capabilities from stages 1–6 and 8.
### QLoRA Hyperparameters
| Parameter | Value |
|---|---|
| Quantization | nf4, double quantization |
| Compute dtype | bfloat16 |
| LoRA alpha | 128 (2Γ— rank) |
| LoRA dropout | 0.05 |
| LoRA target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Optimizer | paged_adamw_8bit |
| Learning rate | 1e-4 |
| LR scheduler | cosine |
| Warmup ratio | 0.03 |
| Gradient checkpointing | ON |
| Flash attention | 2 |
## Capabilities
C1-Tachu is trained across six skill domains:
1. **Software Foundation** β€” Broad code familiarity across languages and patterns, calibrated to a terse, correct response style
2. **Fast Code Generation** β€” Produces correct implementations with minimal tokens; no unnecessary comments or padding
3. **Debugging Speed** β€” Jumps to root cause instead of enumerating possibilities; trained on real GitHub commit-fix pairs
4. **Project Navigation** β€” Answers "where" and "how" questions about unfamiliar repos without reading every file
5. **Rapid Architecture** — Compresses the planning→coding loop: requirements → architecture → plan → code skeleton
6. **Self Verification** β€” Agent loop with tool use (run_tests, compile, read_file, write_file, run_command) to reach working solutions with minimal human intervention
7. **Agent Workflows** β€” Complex multi-step task decomposition with parallel tool calls
## Quickstart
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Pomoika24/C1-Tachu"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto",
)
messages = [
{"role": "system", "content": "You are Tachu, a fast software engineer."},
{"role": "user", "content": "Implement a retry wrapper with exponential backoff in Python."},
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(
**model_inputs,
max_new_tokens=2048,
)
response = tokenizer.decode(generated_ids[0][len(model_inputs.input_ids[0]):], skip_special_tokens=True)
print(response)
```
### Deployment with vLLM
```bash
pip install vllm
vllm serve "Pomoika24/C1-Tachu"
```
### Deployment with Ollama
```bash
ollama run hf.co/Pomoika24/C1-Tachu
```
## Security Posture
**Stance: passive (do no harm).**
- **Never generates** known-vulnerable patterns: hardcoded secrets, SQL injection, command injection, path traversal, XSS, insecure deserialization, disabled auth checks
- **Defaults to safe patterns**: parameterized queries, input validation, proper error handling without leaking stack traces
- **Does not actively scan** code for vulnerabilities unless explicitly asked for a security review
- **Does not refuse** to work on codebases with existing vulnerabilities β€” just doesn't make them worse
- **When uncertain**, prefers the safer pattern even if more verbose
## Limitations
- **Stage 7 (ORPO) not trained** β€” the model has not undergone preference optimization, so it may produce verbose solutions where a shorter one would suffice
- **Training data not published** β€” the dataset was generated on cloud infrastructure and is not publicly available
- **Single-annotator evaluation** β€” eval sets were scored by one developer with LLM-as-judge assistance; no multi-annotator agreement metrics
- **Context limit** β€” while the base model supports 32K context, training stages used 2048–4096 token sequences; performance may degrade on very long contexts
- **English-focused** β€” training data is predominantly English
## Intended Use
C1-Tachu is designed for:
- Software engineers looking to accelerate their workflow
- Coding agents and assistants that need fast, correct code generation
- Automated debugging and code navigation tasks
- Architecture planning and code scaffolding
**Not intended for:**
- Automated security auditing (passive stance only)
- Code generation in safety-critical systems without human review
- Production deployment without human oversight
## Citation
```bibtex
@misc{c1-tachu,
title={C1-Tachu: A Speed-Optimized Coding AI},
author={Vladislav Kondratyev},
year={2026},
url={https://huggingface.co/Pomoika24/C1-Tachu}
}
```