Upload README.md

README.md

@@ -0,0 +1,105 @@
+# Tiny Coder Prompt Completion 0.5B
+
+A tiny model (~494M parameters, ~942MB weights) for **prompt autocomplete** in agentic coding editors like Claude Code, Codex CLI, aider, etc.
+
+## What it does
+
+When you're typing a prompt to an AI coding agent, this model suggests how to **complete your thought**. For example:
+
+- `fix the bug in src/utils` → `.py where the auth token isn't refreshing`
+- `refactor the database` → `connection logic to use connection pooling`
+- `implement caching for the` → `user profile endpoint using Redis with a 5-minute TTL`
+
+## Base Model
+
+This is a fine-tuned version of [Qwen2.5-Coder-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct), which was purpose-built for code completion.
+
+## On-Device Requirements
+
+- **RAM**: ~1GB for fp16 inference (easily fits in 16GB MacBooks)
+- **Disk**: ~1GB for model weights
+- **CPU/GPU**: Works great on CPU (M1/M2/M3 MacBooks), even better with GPU
+
+## Quick Start
+
+### Python (transformers)
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+model_id = "finnvoorhees/tiny-coder-prompt-completion-0.5B"
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    trust_remote_code=True,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+def complete_prompt(prompt: str) -> str:
+    messages = [
+        {"role": "system", "content": "You are a helpful coding assistant. Complete the user's partial prompt concisely."},
+        {"role": "user", "content": prompt},
+    ]
+    inputs = tokenizer.apply_chat_template(
+        messages,
+        tokenize=True,
+        return_tensors="pt",
+        return_dict=True,
+        add_generation_prompt=True,
+    ).to(model.device)
+    
+    outputs = model.generate(
+        **inputs,
+        max_new_tokens=32,
+        temperature=0.3,
+        top_p=0.9,
+        do_sample=True,
+    )
+    
+    new_tokens = outputs[0][inputs["input_ids"].shape[1]:]
+    return tokenizer.decode(new_tokens, skip_special_tokens=True).strip()
+
+print(complete_prompt("refactor the database"))
+# → "connection logic to use connection pooling instead of creating new connections each time"
+```
+
+### Even smaller: GGUF / llama.cpp
+
+For ultra-fast CPU inference, convert to GGUF (Q4_K_M quantization → ~300MB):
+
+```bash
+# Install llama.cpp
+brew install llama.cpp
+
+# Convert to GGUF (or download from the GGUF tag on this repo)
+python convert_hf_to_gguf.py finnvoorhees/tiny-coder-prompt-completion-0.5B --outfile tiny-coder-0.5b-Q4_K_M.gguf
+
+# Run inference
+llama-cli -m tiny-coder-0.5b-Q4_K_M.gguf -p "fix the bug in src/utils"
+```
+
+## Integration with Agentic Editors
+
+### Claude Code
+Create a shell script that calls this model and pipe completions into your prompt:
+
+```bash
+# ~/.claude-complete.sh
+python -c "from transformers import pipeline; print(pipeline('text-generation', model='finnvoorhees/tiny-coder-prompt-completion-0.5B')('$1'))"
+```
+
+### Codex CLI / aider
+Set up a local API endpoint using `transformers` or `llama.cpp` server mode:
+
+```bash
+# llama.cpp server (fast!)
+llama-server -m tiny-coder-0.5b-Q4_K_M.gguf --port 8080
+
+# Then use http://localhost:8080/v1/completions in your editor
+```
+
+## License
+
+Apache-2.0 (same as the base Qwen2.5-Coder-0.5B-Instruct model)