v11: updated README with working harmony Modelfile template
Browse files
README.md
CHANGED
|
@@ -18,7 +18,7 @@ datasets:
|
|
| 18 |
pipeline_tag: text-generation
|
| 19 |
---
|
| 20 |
|
| 21 |
-
# ChatTLA-20b (
|
| 22 |
|
| 23 |
ChatTLA is a fine-tuned version of [openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) specialised in generating **TLA+ formal specifications** — the language used by AWS, Microsoft, and Intel to mathematically verify distributed systems.
|
| 24 |
|
|
@@ -26,7 +26,7 @@ Given a plain-English description of a concurrent or distributed system, ChatTLA
|
|
| 26 |
|
| 27 |
---
|
| 28 |
|
| 29 |
-
## Benchmark Results (
|
| 30 |
|
| 31 |
Evaluated on a handcrafted 20-problem suite covering distributed algorithms, concurrency primitives, and protocol specs. Three tiers:
|
| 32 |
|
|
@@ -72,7 +72,8 @@ Evaluated on a handcrafted 20-problem suite covering distributed algorithms, con
|
|
| 72 |
| v8 | 8/20 (40%) | 1/20 (5%) | 0.72 |
|
| 73 |
| v9 | 6/20 (30%) | 3/20 (15%) | 0.86 |
|
| 74 |
| v9 best-of-5 + self-correct | 16/20 (80%) | 5/20 (25%) | 0.88 |
|
| 75 |
-
|
|
|
|
|
| 76 |
|
| 77 |
> Single-shot scores are conservative. With `--attempts 5 --self-correct` v9 reaches 80% SANY / 25% TLC.
|
| 78 |
|
|
@@ -116,11 +117,11 @@ print(result[0]["generated_text"])
|
|
| 116 |
```bash
|
| 117 |
# Download GGUF
|
| 118 |
huggingface-cli download EricSpencer00/chattla-20b \
|
| 119 |
-
gguf/chattla-20b-
|
| 120 |
--local-dir ./chattla
|
| 121 |
|
| 122 |
# Run with llama.cpp
|
| 123 |
-
./llama-cli -m chattla/gguf/chattla-20b-
|
| 124 |
-n 1024 --temp 0.4 \
|
| 125 |
-p "Write a TLA+ spec for mutual exclusion with N processes."
|
| 126 |
```
|
|
@@ -186,7 +187,7 @@ EricSpencer00/chattla-20b
|
|
| 186 |
├── pytorch_model.bin # Full BF16 weights (39 GB)
|
| 187 |
├── generation_config.json
|
| 188 |
└── gguf/
|
| 189 |
-
├── chattla-20b-
|
| 190 |
└── Modelfile # Ollama Modelfile
|
| 191 |
```
|
| 192 |
|
|
|
|
| 18 |
pipeline_tag: text-generation
|
| 19 |
---
|
| 20 |
|
| 21 |
+
# ChatTLA-20b (v11)
|
| 22 |
|
| 23 |
ChatTLA is a fine-tuned version of [openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) specialised in generating **TLA+ formal specifications** — the language used by AWS, Microsoft, and Intel to mathematically verify distributed systems.
|
| 24 |
|
|
|
|
| 26 |
|
| 27 |
---
|
| 28 |
|
| 29 |
+
## Benchmark Results (v11, single-shot)
|
| 30 |
|
| 31 |
Evaluated on a handcrafted 20-problem suite covering distributed algorithms, concurrency primitives, and protocol specs. Three tiers:
|
| 32 |
|
|
|
|
| 72 |
| v8 | 8/20 (40%) | 1/20 (5%) | 0.72 |
|
| 73 |
| v9 | 6/20 (30%) | 3/20 (15%) | 0.86 |
|
| 74 |
| v9 best-of-5 + self-correct | 16/20 (80%) | 5/20 (25%) | 0.88 |
|
| 75 |
+
| v10 | 6/20 (30%) | 2/20 (10%) | 0.87 |
|
| 76 |
+
| **v11** | **6/20 (30%)** | **2/20 (10%)** | **0.87** |
|
| 77 |
|
| 78 |
> Single-shot scores are conservative. With `--attempts 5 --self-correct` v9 reaches 80% SANY / 25% TLC.
|
| 79 |
|
|
|
|
| 117 |
```bash
|
| 118 |
# Download GGUF
|
| 119 |
huggingface-cli download EricSpencer00/chattla-20b \
|
| 120 |
+
gguf/chattla-20b-v11-Q8_0.gguf \
|
| 121 |
--local-dir ./chattla
|
| 122 |
|
| 123 |
# Run with llama.cpp
|
| 124 |
+
./llama-cli -m chattla/gguf/chattla-20b-v11-Q8_0.gguf \
|
| 125 |
-n 1024 --temp 0.4 \
|
| 126 |
-p "Write a TLA+ spec for mutual exclusion with N processes."
|
| 127 |
```
|
|
|
|
| 187 |
├── pytorch_model.bin # Full BF16 weights (39 GB)
|
| 188 |
├── generation_config.json
|
| 189 |
└── gguf/
|
| 190 |
+
├── chattla-20b-v11-Q8_0.gguf # Quantised GGUF for Ollama / llama.cpp
|
| 191 |
└── Modelfile # Ollama Modelfile
|
| 192 |
```
|
| 193 |
|