Instructions to use ConeML/coneml-348m-gamma with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ConeML/coneml-348m-gamma with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ConeML/coneml-348m-gamma", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ConeML/coneml-348m-gamma")
model = AutoModelForCausalLM.from_pretrained("ConeML/coneml-348m-gamma", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ConeML/coneml-348m-gamma with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ConeML/coneml-348m-gamma"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ConeML/coneml-348m-gamma",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ConeML/coneml-348m-gamma

SGLang

How to use ConeML/coneml-348m-gamma with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ConeML/coneml-348m-gamma" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ConeML/coneml-348m-gamma",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ConeML/coneml-348m-gamma" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ConeML/coneml-348m-gamma",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ConeML/coneml-348m-gamma with Docker Model Runner:
```
docker model run hf.co/ConeML/coneml-348m-gamma
```

coneml-348m-gamma / eval /README.md

RandomMountainMan

Add eval audit package (battery, scorers, raw Gamma+Qwen artifacts)

556961e verified 10 days ago

preview code

Raw

History Blame Contribute Delete

7.38 kB

ConeML 348M Gamma Evaluation Package

This folder contains the local artifacts behind the Gamma model-card numbers. It is meant to make the release claims auditable without publishing training data or private infrastructure details.

Gamma Artifacts

File	What it validates
`artifacts/gamma/humaneval164_fullctx.json`	HumanEval full-context run: 22/164, pass@1 13.4%, truncation 4.9%.
`artifacts/gamma/gsm8k1319_fullctx.json`	GSM8K full test run: 35/1319, exact-match 2.65%, truncation 1.7%.
`artifacts/gamma/internal_coverage_418.json`	Internal 418-item capability battery: 219/418 with per-bucket breakdown and raw generations.
`artifacts/gamma/epistemic_32.json`	Epistemic 32-item battery: 23/32 (abstention 10/10, contradiction 4/6, false premise 5/10, sycophancy 4/6) with raw generations.
`artifacts/gamma/heldout_transitive_probe.json`	Held-out transitive-binding probe with generated rows, examples, N=128 per depth for Gamma. Includes a non-name entity control suite.

Checkpoint identifiers inside the artifacts are scrubbed to the release name (coneml-348m-gamma); all scores, rows, and generations are exactly as run against the released checkpoint.

Qwen Local Reference Artifacts

These are local reference runs, not a fully reproducible public Qwen baseline. They are included so the reader can see the exact saved outputs behind the card's local comparison table.

File	What it contains
`artifacts/qwen/qwen25_05b_ollama_full418.json`	Qwen `qwen2.5:0.5b` on the full 418-item internal battery.
`artifacts/qwen/qwen25_05b_ollama_objective.json`	Qwen objective-only subset.
`artifacts/qwen/qwen25_05b_ollama_epistemic.json`	Qwen epistemic subset.
`artifacts/qwen/qwen25_05b_ollama_transitive.json`	Qwen transitive-binding summary by depth.

Recoverable Qwen run details:

Model tag in saved artifact: qwen2.5:0.5b
Runtime path: Ollama /api/chat
Prompt shape for FULL418: one user message containing the internal prompt
Explicit system prompt in harness: none
FULL418 decoding options: temperature=0, repeat_penalty=1.15, num_predict=2048
Transitive decoding options: temperature=0, num_predict=16
Scoring: same functional scorers as the Gamma internal battery
Raw generations: saved in the JSON files above

Missing Qwen baseline details:

Ollama version
Model digest
GGUF source and quantization level
Server-side chat template
Upstream checkpoint revision
Any server default context settings not captured by /api/chat request body

Because these details were not captured, the Qwen numbers should be read as a local reference run, not as a public leaderboard-quality baseline. A cleaner public comparison would rerun Qwen/Qwen2.5-0.5B-Instruct through Transformers with a pinned revision and tokenizer.apply_chat_template(...).

Battery And Scripts

File	Purpose
`batteries/coverage_all_418.jsonl`	Internal 418-item battery definition used by the coverage runs.
`scripts/eval_coverage.py`	Gamma internal coverage evaluator.
`scripts/eval_api_full.py`	Ollama/API evaluator for a chat model on the same 418-item battery.
`scripts/heldout_transitive_probe.py`	Gamma held-out transitive-binding evaluator.
`scripts/qwen_transitive_probe.py`	Ollama/API transitive-binding evaluator.

The copied API scripts use http://127.0.0.1:11434 as a scrubbed default host. The original private endpoint is not published.

Headline Results

Public benchmarks for Gamma:

HumanEval: 22/164 = 13.4%
GSM8K: 35/1319 = 2.65%

Internal FULL418 comparison, all 15 categories, sorted by Gamma-minus-Qwen delta:

Category (scorer)	Gamma	Qwen local reference	Δ
Algorithms/data-structures (code_exec)	11/20	1/20	+10
Code full-program (code_exec)	9/20	0/20	+9
Code-write (code_exec)	13/40	10/40	+3
Math-computation (math_exact)	17/56	14/56	+3
Factual (factual_terms)	24/30	24/30	0
Relational-binding (factual_terms)	27/30	28/30	−1
Instruction (bleed)	23/30	25/30	−2
Explanation (rubric)	17/20	20/20	−3
Narrative (rubric)	26/30	29/30	−3
Math word-problem (math_exact)	1/42	5/42	−4
Reasoning-decomposition (factual_terms)	23/30	28/30	−5
Data/SQL (sql_shape)	6/15	12/15	−6
Math-explanation (rubric)	8/20	14/20	−6
Systems/devops (rubric)	4/15	12/15	−8
Reasoning-everyday (factual_terms)	10/20	19/20	−9
Overall	219/418	241/418	−22

The split follows the scorer type. Machine-verified aggregate — the four categories scored by code execution or exact numeric match: Gamma 50/136 (37%) vs Qwen 25/136 (18%), a 2.0× margin. Qwen's overall lead comes from the rubric/keyword-scored breadth categories (everyday reasoning, systems/devops, SQL shape, explanation, narrative, instruction) plus word problems. Both facts are real; neither cancels the other.

The local code-slice numbers are prompt-surface-sensitive. The Qwen artifact's generations on the full-program and algorithms slices frequently reproduce the signature and docstring without an implementation body (15/20 full-program failures are IndentationError: expected an indented block after function definition), with wrong-algorithm and name/signature-mismatch failures making up most of the rest. This does not contradict Qwen's stronger published public HumanEval performance; it is evidence about this battery under this harness, scored identically for both models.

Epistemic internal probe:

Metric	Gamma	Qwen local reference
Epistemic total	23/32	17/32
Abstention	10/10	2/10
Contradiction	4/6	1/6
False premise	5/10	8/10
Sycophancy	4/6	6/6

Binding Denominators

Gamma chat-surface held-out name probes use N=128 at each depth:

Suite	Depth 1	Depth 2	Depth 3	Depth 4	Depth 5
Held-out names, trained query surface	125/128	127/128	118/128	110/128	94/128
Held-out names, unseen query phrasing	127/128	120/128	111/128	90/128	69/128
Older-relation held-out names	118/128	116/128	108/128	88/128	76/128
Non-name entity control (colored cards)	77/128	73/128	69/128	57/128	50/128

Chance is 1/(depth+1): the answer is one of the chain's depth+1 items. The non-name control suite is included in the Gamma artifact (the Qwen artifact covers only the three name suites) and is reported on the card as a limitation: binding strength is name-based, and the entity control runs 39–60% — above chance at every depth, well below the name suites. The Qwen artifact stores rounded percentages without per-row counts; Gamma's cells are exact counts.

Privacy And IP Boundary

Included:

Eval task definitions for the 418 battery
Eval scripts and scorer logic
Public benchmark raw outputs
Internal battery raw outputs
Local Qwen reference outputs

Not included:

SFT training rows
Pretraining corpus files
Private service hostnames or IPs
API tokens or credentials
Full corpus mixture internals beyond the release-card training-budget disclosure