Instructions to use CoBeDigger/krama-coder-1.5b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use CoBeDigger/krama-coder-1.5b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf CoBeDigger/krama-coder-1.5b:Q4_K_M # Run inference directly in the terminal: llama cli -hf CoBeDigger/krama-coder-1.5b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf CoBeDigger/krama-coder-1.5b:Q4_K_M # Run inference directly in the terminal: llama cli -hf CoBeDigger/krama-coder-1.5b:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf CoBeDigger/krama-coder-1.5b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf CoBeDigger/krama-coder-1.5b:Q4_K_M
Use Docker
docker model run hf.co/CoBeDigger/krama-coder-1.5b:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use CoBeDigger/krama-coder-1.5b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "CoBeDigger/krama-coder-1.5b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CoBeDigger/krama-coder-1.5b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/CoBeDigger/krama-coder-1.5b:Q4_K_M
- Ollama
How to use CoBeDigger/krama-coder-1.5b with Ollama:
ollama run hf.co/CoBeDigger/krama-coder-1.5b:Q4_K_M
- Unsloth Desktop
- Pi
How to use CoBeDigger/krama-coder-1.5b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf CoBeDigger/krama-coder-1.5b:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "CoBeDigger/krama-coder-1.5b:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use CoBeDigger/krama-coder-1.5b with Docker Model Runner:
docker model run hf.co/CoBeDigger/krama-coder-1.5b:Q4_K_M
- Lemonade
How to use CoBeDigger/krama-coder-1.5b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull CoBeDigger/krama-coder-1.5b:Q4_K_M
Run and chat with the model
lemonade run user.krama-coder-1.5b-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use CoBeDigger/krama-coder-1.5b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf CoBeDigger/krama-coder-1.5b:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default CoBeDigger/krama-coder-1.5b:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use CoBeDigger/krama-coder-1.5b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf CoBeDigger/krama-coder-1.5b:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "CoBeDigger/krama-coder-1.5b:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Krama Coder 1.5B (v0.1 preview)
Krama is a small, offline coding tutor for Python and FastAPI beginners. For a coding task it answers in three short parts: Plan (numbered steps), Code, and Why (plain-language reasons). For a quick question it gives a short, direct answer. It runs on an ordinary laptop CPU with no internet.
"Krama" is Sanskrit for order or sequence.
Preview. This is an early release for trying out and giving feedback. It is not a general-purpose coding model and it is not production-ready. Always run the code it gives you.
Krama is a fine-tune of Qwen2.5-Coder-1.5B-Instruct by the Qwen team (Apache 2.0). All of its coding ability comes from that model. Krama only changes how it answers.
Run it
With Ollama:
ollama run hf.co/CoBeDigger/krama-coder-1.5b
The file is 986 MB (4-bit Q4_K_M). On a laptop CPU (Intel Core Ultra 7 155H) the base model runs at about 33-36 tokens per second, and Krama is the same size.
You can also use it from code through Ollama's local API (http://localhost:11434), or in VS Code through an extension that supports Ollama, such as Continue.
To build it yourself from the downloaded file, use the Modelfile in this repo:
ollama create krama -f Modelfile
What an answer looks like
These are real answers from the release file, at temperature 0.
"Write a Python function safe_divide(a, b) that returns a divided by b. If b is zero, return None instead of crashing."
Plan:
1. Define the function `safe_divide`.
2. Check if `b` is zero.
3. Return `None` if `b` is zero, otherwise return the division result.
Code:
```python
def safe_divide(a, b):
if b == 0:
return None
return a / b
```
Why:
- The function checks if `b` is zero before performing the division to avoid division by zero errors.
- If `b` is zero, the function returns `None`, which can be handled gracefully in the calling code.
"What is the difference between == and is in Python?"
`==` checks if two values are equal, while `is` checks if both variables point to the same object. For example:
```python
x = [1, 2]
y = [1, 2]
print(x == y) # True because they have the same elements
print(x is y) # False because they are two different lists
```
How it was tested
All three systems below are the same 986 MB Q4_K_M file size, run in Ollama at temperature 0. "Base + instruction" is the base model given a written instruction to answer like Krama.
Correctness. 30 tasks of our own (12 Python, 12 FastAPI, 6 quick questions), where the code is run against tests. The 164-task HumanEval benchmark is the standard check.
| System | Python (12) | FastAPI (12) | Quick answers, average length | HumanEval (164) |
|---|---|---|---|---|
| Base model | 9 | 6 | 273 tokens | 120 (73.2%) |
| Base + instruction | 9 | 5 | 218 tokens | 98 (59.8%) |
| Krama v0.1 | 9 | 6 | 121 tokens | 112 (68.3%) |
Runs on a CPU are not exactly repeatable, even at temperature 0. Krama passed 10 Python tasks in its first run and 9 in each of four reruns, so we report 9. The base model passed 9 Python and 6 FastAPI tasks in all three of its runs. HumanEval was run once per system.
Teaching quality. Each answer was scored blind by a different model (Llama 3.1 8B), which never saw which system wrote it. Scoring rules were written before training. Averages are out of 2 per rule.
| System | Plan | Why | Clarity | Length | Coding total (of 8) | Quick answers (of 4) |
|---|---|---|---|---|---|---|
| Base model | 1.08 | 0.67 | 1.04 | 1.58 | 4.38 | 1.33 |
| Base + instruction | 1.83 | 1.17 | 1.17 | 1.79 | 5.96 | 1.33 |
| Krama v0.1 | 1.92 | 1.17 | 1.21 | 1.96 | 6.25 | 2.00 |
What this means, honestly:
- Krama matches the base model's correctness on our tasks, and adds a plan and an explanation.
- On HumanEval it is 8 tasks below the plain base model. Asking the base model to plan and explain costs it far more (22 tasks), so the fine-tune is the better way to get this answer style, but it is not free.
- Against the instructed base, Krama wins 7, ties 15 and loses 2 of the 24 coding tasks on teaching quality, and wins 3 and ties 3 of the 6 quick questions. Its short answers are about half as long.
- Our 30 tasks share topics with the training data. Exact copies were removed, but those scores may flatter Krama a little. HumanEval is the fairer measure.
- The blind grader is a small model and was lenient. It sometimes gave the base model credit for a plan it did not have. That error favours the base, not Krama.
Known weaknesses
- FastAPI: 6 of 12 tasks still fail. Krama sometimes adds a Pydantic model where none is needed, and once it read a JSON body as a plain string parameter.
- Explanations are short and sometimes generic, and they are not always fully accurate. Treat the "Why" as a starting point.
- English only. Python and FastAPI only.
- 1.5B parameters: it makes logic mistakes that larger models would not.
How it was made
- Data: 1,578 examples: 776 Python, 471 FastAPI, 316 short answers, and 15 about who made it. Most answers were written by Qwen2.5-Coder-7B-Instruct (Apache 2.0) and kept only if their code passed their own tests. 92 examples were written by hand. Some questions were inspired by short pieces of the author's own code. Any example whose code passed one of our test tasks, or which asked one of our test questions, was removed. The dataset is not released in this preview.
- Training: LoRA on all linear layers (rank 16, alpha 32, dropout 0.05), learning rate 2e-4 with cosine decay, 2 epochs, max length 1,024, effective batch 16, fp16, on one free Kaggle T4 GPU (11 minutes). Held-out loss went from 0.522 to 0.474. The adapter was merged and converted to Q4_K_M GGUF with llama.cpp.
- Instruction it was trained with: "You are Krama, a Python and FastAPI coding assistant for beginners, fine-tuned by CoBeDigger from Qwen2.5-Coder-1.5B-Instruct." The
systemfile in this repo sets it for you.
What comes next
- A
kramacommand-line tool that runs Krama's code against tests on your own machine before showing it. - Version 0.2, trained on the mistakes real beginners find.
- Later, a larger version.
Feedback is very welcome in the Community tab, especially questions where Krama got it wrong.
License
Apache 2.0, the same as the base model. Built on Qwen2.5-Coder-1.5B-Instruct by the Qwen team.
- Downloads last month
- 34
4-bit
Model tree for CoBeDigger/krama-coder-1.5b
Base model
Qwen/Qwen2.5-1.5B