Instructions to use IvmeLabs/Ivme-Conversate-v2-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use IvmeLabs/Ivme-Conversate-v2-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="IvmeLabs/Ivme-Conversate-v2-Base", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("IvmeLabs/Ivme-Conversate-v2-Base", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use IvmeLabs/Ivme-Conversate-v2-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IvmeLabs/Ivme-Conversate-v2-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IvmeLabs/Ivme-Conversate-v2-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/IvmeLabs/Ivme-Conversate-v2-Base

SGLang

How to use IvmeLabs/Ivme-Conversate-v2-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "IvmeLabs/Ivme-Conversate-v2-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IvmeLabs/Ivme-Conversate-v2-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "IvmeLabs/Ivme-Conversate-v2-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IvmeLabs/Ivme-Conversate-v2-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use IvmeLabs/Ivme-Conversate-v2-Base with Docker Model Runner:
```
docker model run hf.co/IvmeLabs/Ivme-Conversate-v2-Base
```

ereniko commited on 3 days ago

Commit

09d5620

verified ·

1 Parent(s): 8b157ab

Update README.md

Browse files

Files changed (1) hide show

README.md +500 -249

README.md CHANGED Viewed

@@ -22,276 +22,527 @@ pipeline_tag: text-generation
 library_name: transformers
 ---
-# İvme-Conversate-v2-Base (Codenamed Apple 2)
-![Conversate-v2 Logo](https://cdn-uploads.huggingface.co/production/uploads/670562d6ac129959c16f84d4/9oWY_UmFyQ297XmaZEK7G.png)
-**İvme** (Turkish: *acceleration*) is a series of stupidly small language models built to punch above their weight. This is the second release: a 24M parameter decoder-only base model trained from scratch, this time with a much heavier training diet.
-v1 produced grammatically correct sentences that did not really say anything when strung together, fluent without being about anything. v2 keeps almost the exact same architecture as v1 on purpose, and instead fixes the two things that actually mattered: far more training data, and a data mix weighted toward material that teaches a model to stay on topic across sentences.
----
-## Model Details
-| Parameter | Value |
-|---|---|
-| Architecture | Decoder-only transformer, dense (no loops, no exotic recurrence) |
-| Parameters | 23,846,784 |
-| Layers | 10 |
-| Hidden dim | 384 |
-| FFN dim | SwiGLU |
-| Attention heads | 6, full attention (no GQA) |
-| Context length | 1024 tokens |
-| Vocab size | 16,000 (custom BPE) |
-| Positional encoding | RoPE (θ=10,000) |
-| Normalization | RMSNorm (pre-norm) |
-| Embeddings | Tied input/output |
-| Biases | None |
-Nearly every setting above matches v1 on purpose. The point of v2 was to isolate the improvement to data and training, not to a bigger model.
-<a href="https://hfviewer.com/IvmeLabs/Ivme-Conversate-v2-Base?utm_source=huggingface&amp;utm_medium=embedded_model_card&amp;utm_campaign=IvmeLabs__Ivme-Conversate-v2-Base_card&amp;utm_content=embedded_card_open_viewer&amp;from=embedded-model-card" target="_blank" rel="noopener">
-  <img
-    src="https://hfviewer.com/api/card.svg?source=IvmeLabs%2FIvme-Conversate-v2-Base&amp;granularity=auto&amp;v=20260516-title-pills-card"
-    alt="Architecture graph for IvmeLabs/Ivme-Conversate-v2-Base. Open in hfviewer"
-    width="100%"
-  />
 </a>
----
-## Benchmarks
-Benchmarks were run with lm-evaluation-harness via a custom model adapter (previous numbers from an internal script have been superseded by these).
-| Benchmark | v1 | v2 |
-|---|---|---|
-| WikiText-2 (byte perplexity) ↓ | 2.96 | **2.2250** |
-| BLiMP (macro-average, 68 paradigms) ↑ | 61.40% | **75.09%** |
-| ARC-Easy (acc_norm) ↑ | 30.85% | **39.98%** |
-| ARC-Easy (acc) ↑ | — | **43.56%** |
-Every metric improved. The ARC-Easy acc_norm gain (30.85% → 39.98%) is more modest than the raw acc numbers alone suggest, but still a real improvement over v1.
-### BLiMP paradigm breakdown
-Strong on core agreement paradigms:
-| Paradigm | Accuracy |
-|---|---|
-| blimp_principle_A_case_1 | 100.00% |
-| blimp_existential_there_quantifiers_1 | 99.40% |
-| blimp_anaphor_number_agreement | 98.00% |
-| blimp_sentential_negation_npi_licensor_present | 97.30% |
-| blimp_determiner_noun_agreement_1 | 96.90% |
-| blimp_principle_A_domain_1 | 95.40% |
-Weaker on long-distance dependencies and island constraints, a known hard case for small models:
-| Paradigm | Accuracy |
-|---|---|
-| blimp_wh_vs_that_with_gap_long_distance | 17.20% |
-| blimp_existential_there_quantifiers_2 | 30.20% |
-| blimp_left_branch_island_echo_question | 31.40% |
-| blimp_superlative_quantifiers_2 | 33.20% |
-| blimp_principle_A_reconstruction | 36.10% |
-This pattern, strong local agreement paired with weaker long-distance syntax, is typical for models at this scale.
----
-## Does it actually make more sense now?
-None of the benchmarks above directly test whether the model's writing holds together as connected text, which was v1's real problem. Sample output, EMA weights, temperature 0.8, top_k 50:
-**Prompt:** "Once upon a time, there was a"
-> Once upon a time, there was a wise old turtle named Timmy who lived on the coast of South America. In this magical place, no matter how big or large, people could look up at the sea and talk to each other. Timmy asked, "What do you mean, Timmy?" The turtle replied, "Well, I think you might see people talking about ocean creatures. They are like little waves that carry their voices. Sometimes they say they're too big or small to hear." After a few moments, Timmy had an idea. "Can we go on a boat-boat tour? I can't believe all the kids in the village are doing that!" As they sailed further, they saw many beautiful islands and vibrant colors. Each island had its unique culture and traditions. When they reached the top, they saw a group of kids playing and splashing around. "Wow, lookgies!" said Timmy. "They live in a big ocean full of colorful fish and
----
-## Training
-### Data Mix (~12.85B tokens, roughly 8 to 9x more than v1's 1.57B)
-v1 was trained Chinchilla-optimal. v2 deliberately overtrains well past that point, since the model is small and cheap to run regardless.
-| Source | Share |
-|---|---|
-| HuggingFaceFW/fineweb-edu | 50% |
-| HuggingFaceTB/smollm-corpus (cosmopedia-v2) | 27% |
-| mlfoundations/dclm-baseline-1.0 | 8% |
-| SimpleStories/SimpleStories | 5% |
-| HuggingFaceTB/finemath (finemath-3plus) | 5% |
-Python-Edu was originally planned as a fifth source, but the actual code text lives behind a gated dataset with no practical way to align it against the sampled subset at this scale, so it was dropped and its share redistributed across the rest.
-### Hyperparameters
-| Setting | Value |
-|---|---|
-| Optimizer | Muon (body weights) + AdamW (embeddings, norms) |
-| Muon lr | 0.02 |
-| AdamW lr | 3e-4 |
-| LR schedule | Warmup-Stable-Decay (WSD) |
-| Weight decay | 0.1 |
-| Gradient clipping | 1.0 |
-| Batch size | 192 sequences x 1024 tokens (196,608 tokens/step) |
-| Total steps | 65,376 |
-| Precision | bfloat16 |
-| Attention | PyTorch scaled_dot_product_attention (Flash Attention backend) |
-| Compilation | torch.compile, roughly 2x throughput over eager |
-| Final weights | EMA (β=0.999) of training trajectory |
-### Hardware
-Trained on a single NVIDIA RTX PRO 6000 Blackwell (96GB) in approximately **4.75 hours**.
----
-## Tokenizer
-Custom byte-level BPE tokenizer trained from scratch on a sample of the pretraining mix. Vocab size 16,000.
----
-## Inference
-This model can now be loaded with `AutoModelForCausalLM` instead of the
-manual pickle-loading workflow, and weights are available as
-`model.safetensors`.
-```python
-import torch
-from transformers import AutoModelForCausalLM, AutoTokenizer
-model = AutoModelForCausalLM.from_pretrained(
-    "IvmeLabs/Ivme-Conversate-v2-Base", trust_remote_code=True, dtype=torch.float32,
 )
-tokenizer = AutoTokenizer.from_pretrained("IvmeLabs/Ivme-Conversate-v2-Base", trust_remote_code=True)
-model.eval()
-inputs = tokenizer("Once upon a time, there was a", return_tensors="pt")
-out = model.generate(
-    **inputs, max_new_tokens=200, do_sample=True,
-    temperature=0.8, top_k=50, pad_token_id=tokenizer.pad_token_id,
 )
-print(tokenizer.decode(out[0], skip_special_tokens=True))
-```
-`trust_remote_code=True` is required (custom architecture: RoPE + SwiGLU +
-RMSNorm dense decoder). The original `ckpt_final.pt` pickle checkpoint and
-`model/` architecture source remain in this repo unchanged for backwards
-compatibility.
-**Note on batch generation:** use left-padding
-(`tokenizer.padding_side = "left"`) the model doesn't use an explicit
-attention mask over padded positions, so right-padding within a batch will
-give incorrect results.
-## Legacy Inference
-Here's a basic inference code in case you want to work with the pickle files.
-```python
-import sys
-import torch
-from tokenizers import Tokenizer
-from huggingface_hub import hf_hub_download, snapshot_download
-repo_id = "IvmeLabs/Ivme-Conversate-v2-Base"
-# Download just the model/ folder (architecture code) into the HF cache,
-# then add it to sys.path so `from model import ...` works without the
-# user needing to manually copy any files.
-repo_local_dir = snapshot_download(repo_id, allow_patterns=["model/*"])
 sys.path.append(repo_local_dir)
-from model import IvmeConfig, IvmeConversateV2
-ckpt_path = hf_hub_download(repo_id, "ckpt_final.pt")
-tokenizer_path = hf_hub_download(repo_id, "tokenizer.json")
-tokenizer = Tokenizer.from_file(tokenizer_path)
-# IvmeConfig is a plain dataclass saved into the checkpoint. Trust this only
-# because it's our own checkpoint, produced by our own training code.
-torch.serialization.add_safe_globals([IvmeConfig])
-ckpt = torch.load(ckpt_path, map_location="cuda")
-cfg = ckpt["config"]
-model = IvmeConversateV2(cfg)
-# Use EMA weights (smoothed), not the raw training weights, for inference.
-# Strip torch.compile's "_orig_mod." prefix if the checkpoint was compiled.
-state_dict = ckpt["ema_state_dict"]
-state_dict = {k.removeprefix("_orig_mod."): v for k, v in state_dict.items()}
-model.load_state_dict(state_dict)
-model.cuda().eval()
-prompt = "Once upon a time, there was a"
-ids = tokenizer.encode(prompt).ids
-idx = torch.tensor([ids], dtype=torch.long, device="cuda")
-eot_id = tokenizer.token_to_id("<|endoftext|>")
-with torch.no_grad():
-    for _ in range(200):
-        idx_cond = idx if idx.size(1) <= cfg.context_len else idx[:, -cfg.context_len:]
         logits, _ = model(idx_cond)
-        logits = logits[:, -1, :] / 0.8  # temperature
-        v, _ = torch.topk(logits, 50)
-        logits[logits < v[:, [-1]]] = -float("inf")
-        probs = torch.softmax(logits, dim=-1)
-        next_id = torch.multinomial(probs, num_samples=1)
-        idx = torch.cat([idx, next_id], dim=1)
-        if next_id.item() == eot_id:
-            break
-print(tokenizer.decode(idx[0].tolist()))
-```
-## Limitations
-- Base model only, not instruction tuned, will not follow instructions or answer questions
-- English only
-- 1024 token context window
-- Weaker on long-distance syntactic dependencies than on local agreement, see BLiMP breakdown above
-- No code data in the training mix (Python-Edu was dropped, see above)
-- Weak knowledge. While the model has learned syntax and grammar very well, it falls short on knowledge and easily hallucinates. We are expecting to fix this in upcoming variants via distillation.
----
-## What's Next
-Still on the table for a future version: distillation from a larger teacher model, and a return to the more experimental İvmetron architecture once time allows for the kind of patient debugging a genuinely novel design needs.
-You can check our other upcoming models on our organization card!
----
-## Citation
-```bibtex
-@misc{ivme-conversate-v2-Base,
   author       = {IvmeLabs},
   title        = {İvme-Conversate-v2-Base},
-  year         = {2026},
   publisher    = {Hugging Face},
   url          = {https://huggingface.co/IvmeLabs/Ivme-Conversate-v2-Base}
-}
-```
----
-## Credits
-The apple photo by <a href="https://unsplash.com/@cenali?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Matheus Cenali</a> on <a href="https://unsplash.com/photos/bunch-of-red-apples-wXuzS9xR49M?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>
----
-*Built by IvmeLabs. Small models, deliberate choices.*

 library_name: transformers
 ---
+<style>
+@import url('https://fonts.googleapis.com/css2?family=Newsreader:ital,opsz,wght@0,6..72,380;0,6..72,500;0,6..72,600;0,6..72,700;1,6..72,400&family=JetBrains+Mono:wght@400;500;600&display=swap');
+*, html, body, div, p, table, td, th, ul, li { box-sizing: border-box; }
+body{
+margin:0;
+background:
+repeating-linear-gradient(180deg, rgba(0,0,0,0.014) 0px, rgba(0,0,0,0.014) 1px, transparent 1px, transparent 3px),
+#f7f4ec !important;
+color:#1c1a16;
+font-family:'Newsreader', Georgia, serif;
+}
+::selection{background:#ff0000; color:#f7f4ec;}
+a{color:#c40000; text-decoration:none; border-bottom:1px solid rgba(255,0,0,0.35);}
+a:hover{border-bottom-color:#ff0000;}
+.card{max-width:800px; margin:0 auto; padding:52px 40px 70px;}
+.eyebrow{
+font-family:'JetBrains Mono', monospace;
+font-size:10.5px;
+letter-spacing:0.24em;
+text-transform:uppercase;
+color:#ff0000;
+margin-bottom:16px;
+}
+h1.title{
+font-size:36px;
+font-weight:700;
+letter-spacing:-0.01em;
+margin:0 0 6px;
+line-height:1.1;
+}
+.codename-line{
+font-family:'JetBrains Mono', monospace;
+font-size:12px;
+color:#8a8474;
+letter-spacing:0.03em;
+margin-bottom:28px;
+}
+.codename-line b{color:#4a463d; font-weight:500;}
+.hero-logo{
+width:100%;
+max-width:320px;
+display:block;
+margin:0 auto 32px;
+border:1px solid #d9d2bf;
+filter:contrast(1.03);
+}
+p.dek{
+font-size:16.5px;
+line-height:1.7;
+color:#2b2822;
+margin:0 0 40px;
+padding-bottom:36px;
+border-bottom:1px solid #d9d2bf;
+}
+p.dek a{border-bottom-color:rgba(255,0,0,0.35);}
+.stat-strip{
+display:grid;
+grid-template-columns: repeat(4, 1fr);
+border-top:1px solid #1c1a16;
+border-bottom:1px solid #1c1a16;
+margin-bottom:44px;
+}
+.stat{padding:14px 12px; border-right:1px solid #d9d2bf; text-align:center;}
+.stat:last-child{border-right:none;}
+.stat-num{
+font-family:'JetBrains Mono', monospace;
+font-size:17px;
+font-weight:600;
+color:#1c1a16;
+display:block;
+}
+.stat-label{
+font-family:'JetBrains Mono', monospace;
+font-size:9px;
+letter-spacing:0.06em;
+text-transform:uppercase;
+color:#8a8474;
+margin-top:3px;
+display:block;
+}
+section{margin-bottom:48px;}
+.section-head{display:flex; align-items:baseline; gap:14px; margin-bottom:18px;}
+.section-index{font-family:'JetBrains Mono', monospace; font-size:12px; color:#ff0000;}
+h2{font-size:22px; font-weight:600; margin:0; letter-spacing:-0.005em;}
+h3{font-size:15px; font-weight:600; margin:26px 0 12px; font-family:'JetBrains Mono', monospace; letter-spacing:0.01em; color:#1c1a16;}
+p{font-size:15px; line-height:1.72; color:#2b2822; margin:0 0 14px;}
+/* spec / kv table */
+table.kv{width:100%; border-collapse:collapse; font-size:14px; margin-top:6px;}
+table.kv td{padding:9px 12px 9px 0; border-bottom:1px solid #ece7d8; vertical-align:top;}
+table.kv tr:last-child td{border-bottom:1px solid #1c1a16;}
+table.kv td:first-child{
+font-family:'JetBrains Mono', monospace;
+font-size:11px;
+letter-spacing:0.02em;
+color:#8a8474;
+width:38%;
+white-space:nowrap;
+padding-top:11px;
+}
+table.kv td:last-child{color:#2b2822;}
+/* results table */
+table.results{width:100%; border-collapse:collapse; font-size:14px; margin-top:6px;}
+table.results th{
+text-align:left;
+font-family:'JetBrains Mono', monospace;
+font-size:10px;
+letter-spacing:0.06em;
+text-transform:uppercase;
+color:#8a8474;
+font-weight:500;
+border-bottom:2px solid #1c1a16;
+padding:8px 12px 8px 0;
+}
+table.results td{padding:12px 12px 12px 0; border-bottom:1px solid #d9d2bf; font-size:14px;}
+table.results tr:last-child td{border-bottom:1px solid #1c1a16;}
+table.results td:first-child{font-weight:500;}
+table.results td.num{font-family:'JetBrains Mono', monospace; color:#4a463d;}
+table.results td.win{font-family:'JetBrains Mono', monospace; color:#a02020; font-weight:600;}
+table.results.compact{font-size:13px;}
+table.results.compact td, table.results.compact th{padding:8px 10px 8px 0;}
+.two-col{display:grid; grid-template-columns:1fr 1fr; gap:0 28px;}
+.two-col .col-label{
+font-family:'JetBrains Mono', monospace;
+font-size:10px;
+letter-spacing:0.06em;
+text-transform:uppercase;
+color:#8a8474;
+margin-bottom:8px;
+display:block;
+}
+/* sample output block */
+.sample-prompt{
+font-family:'JetBrains Mono', monospace;
+font-size:11px;
+letter-spacing:0.04em;
+text-transform:uppercase;
+color:#8a8474;
+margin-bottom:10px;
+}
+blockquote.sample{
+margin:0 0 8px;
+border-left:3px solid #ff0000;
+padding:16px 20px;
+background:rgba(255,0,0,0.03);
+font-size:14.5px;
+line-height:1.75;
+color:#2b2822;
+font-style:italic;
+}
+.sample-meta{
+font-family:'JetBrains Mono', monospace;
+font-size:10.5px;
+color:#8a8474;
+margin-top:2px;
+}
+/* incident / callout box, reused signature element */
+.callout{border:1px solid #1c1a16; position:relative; padding:26px 26px 22px; margin-bottom:8px;}
+.callout::before{
+content:attr(data-label);
+position:absolute;
+top:-9px; left:20px;
+background:#f7f4ec;
+padding:0 8px;
+font-family:'JetBrains Mono', monospace;
+font-size:9.5px;
+letter-spacing:0.12em;
+color:#ff0000;
+}
+.callout p{font-size:14.5px; margin-bottom:12px;}
+.callout p:last-child{margin-bottom:0;}
+/* code blocks, no JS dependency, hand-spanned tokens */
+pre{
+background:#1c1a16 !important;
+padding:20px 22px;
+overflow-x:auto;
+border-left:3px solid #ff0000;
+margin:0 0 14px;
+border-radius:0;
+}
+pre code{
+background:transparent !important;
+background-color:transparent !important;
+border:none !important;
+border-radius:0 !important;
+padding:0 !important;
+font-family:'JetBrains Mono', monospace !important;
+font-size:12px !important;
+line-height:1.65 !important;
+color:#f7f4ec !important;
+white-space:pre !important;
+display:block;
+}
+code{
+font-family:'JetBrains Mono', monospace !important;
+font-size:0.88em !important;
+background-color:#ece5d3 !important;
+color:#a02020 !important;
+padding:2px 6px !important;
+border-radius:2px !important;
+border:1px solid #d9d2bf !important;
+white-space:nowrap;
+}
+.tok-kw { color:#7fb8e8 !important; font-weight:600; }
+.tok-str { color:#a8d992 !important; }
+.tok-cls { color:#e8c66a !important; }
+.tok-fn { color:#8fc7ff !important; }
+.tok-num { color:#e8b56a !important; }
+.tok-com { color:#8a8474 !important; font-style:italic; }
+/* note box */
+.note-box{
+font-size:13.5px;
+color:#4a463d;
+background:rgba(255,0,0,0.03);
+border-left:3px solid #ff0000;
+padding:14px 18px;
+line-height:1.6;
+margin-bottom:14px;
+}
+/* architecture graph embed link */
+.graph-embed{display:block; margin:20px 0 8px;}
+.graph-embed img{width:100%; display:block; border:1px solid #d9d2bf;}
+.graph-caption{
+font-family:'JetBrains Mono', monospace;
+font-size:10px;
+color:#8a8474;
+margin-top:8px;
+letter-spacing:0.02em;
+}
+/* limitations list */
+ul.limits{margin:0; padding:0; list-style:none;}
+ul.limits li{
+font-size:14.5px;
+line-height:1.6;
+color:#2b2822;
+padding:11px 0 11px 22px;
+border-bottom:1px solid #ece7d8;
+position:relative;
+}
+ul.limits li:last-child{border-bottom:none;}
+ul.limits li::before{
+content:"–";
+position:absolute;
+left:0;
+color:#ff0000;
+font-weight:600;
+}
+footer{
+border-top:1px solid #d9d2bf;
+margin-top:56px;
+padding-top:20px;
+font-family:'JetBrains Mono', monospace;
+font-size:10.5px;
+color:#8a8474;
+display:flex;
+justify-content:space-between;
+}
+footer .fmark{color:#ff0000;}
+.credit-line{
+font-family:'JetBrains Mono', monospace;
+font-size:10px;
+color:#8a8474;
+margin-top:6px;
+}
+@media (max-width: 620px){
+.card{padding:36px 20px 50px;}
+.stat-strip{grid-template-columns:repeat(2,1fr);}
+.stat:nth-child(2){border-right:none;}
+.two-col{grid-template-columns:1fr;}
+table.kv td:first-child{width:auto; white-space:normal;}
+footer{flex-direction:column; gap:6px;}
+}
+</style>
+<div class="card">
+<div class="eyebrow">İvmeLabs &middot; Model Card &middot; Conversate Family</div>
+<h1 class="title">İvme-Conversate-v2-Base</h1>
+<div class="codename-line">Codename <b>Apple 2</b> &middot; dense decoder-only &middot; 23,846,784 parameters</div>
+<img class="hero-logo" src="https://cdn-uploads.huggingface.co/production/uploads/670562d6ac129959c16f84d4/9oWY_UmFyQ297XmaZEK7G.png" alt="Conversate-v2 logo">
+<p class="dek">The second release in the Conversate line: a 24M parameter decoder-only base model trained from scratch, this time with a much heavier training diet. v1 produced grammatically correct sentences that did not really say anything when strung together, fluent without being about anything. v2 keeps almost the exact same architecture as v1 on purpose, and instead fixes the two things that actually mattered: far more training data, and a data mix weighted toward material that teaches a model to stay on topic across sentences.</p>
+<div class="stat-strip">
+<div class="stat"><span class="stat-num">23.8M</span><span class="stat-label">Parameters</span></div>
+<div class="stat"><span class="stat-num">12.85B</span><span class="stat-label">Tokens Trained</span></div>
+<div class="stat"><span class="stat-num">10</span><span class="stat-label">Layers</span></div>
+<div class="stat"><span class="stat-num">4.75h</span><span class="stat-label">Train Time</span></div>
+</div>
+<section>
+<div class="section-head"><span class="section-index">01</span><h2>Model Details</h2></div>
+<table class="kv">
+<tr><td>Architecture</td><td>Decoder-only transformer, dense (no loops, no exotic recurrence)</td></tr>
+<tr><td>Parameters</td><td>23,846,784</td></tr>
+<tr><td>Layers</td><td>10</td></tr>
+<tr><td>Hidden dim</td><td>384</td></tr>
+<tr><td>FFN</td><td>SwiGLU</td></tr>
+<tr><td>Attention heads</td><td>6, full attention (no GQA)</td></tr>
+<tr><td>Context length</td><td>1024 tokens</td></tr>
+<tr><td>Vocab size</td><td>16,000 (custom BPE)</td></tr>
+<tr><td>Positional encoding</td><td>RoPE (&theta;=10,000)</td></tr>
+<tr><td>Normalization</td><td>RMSNorm (pre-norm)</td></tr>
+<tr><td>Embeddings</td><td>Tied input/output</td></tr>
+<tr><td>Biases</td><td>None</td></tr>
+</table>
+<p style="margin-top:16px; font-size:13.5px; color:#4a463d;">Nearly every setting above matches v1 on purpose. The point of v2 was to isolate the improvement to data and training, not to a bigger model.</p>
+<a class="graph-embed" href="https://hfviewer.com/IvmeLabs/Ivme-Conversate-v2-Base?utm_source=huggingface&amp;utm_medium=embedded_model_card&amp;utm_campaign=IvmeLabs__Ivme-Conversate-v2-Base_card&amp;utm_content=embedded_card_open_viewer&amp;from=embedded-model-card" target="_blank" rel="noopener">
+<img src="https://hfviewer.com/api/card.svg?source=IvmeLabs%2FIvme-Conversate-v2-Base&amp;granularity=auto&amp;v=20260516-title-pills-card" alt="Architecture graph for IvmeLabs/Ivme-Conversate-v2-Base. Open in hfviewer">
 </a>
+<div class="graph-caption">Interactive architecture graph, opens in hfviewer</div>
+</section>
+<section>
+<div class="section-head"><span class="section-index">02</span><h2>Benchmarks</h2></div>
+<p>Benchmarks were run with lm-evaluation-harness via a custom model adapter. Previous numbers from an internal script have been superseded by these.</p>
+<table class="results">
+<tr><th>Benchmark</th><th>v1</th><th>v2</th></tr>
+<tr><td>WikiText-2 (byte perplexity) &darr;</td><td class="num">2.96</td><td class="win">2.2250</td></tr>
+<tr><td>BLiMP (macro-average, 68 paradigms) &uarr;</td><td class="num">61.40%</td><td class="win">75.09%</td></tr>
+<tr><td>ARC-Easy (acc_norm) &uarr;</td><td class="num">30.85%</td><td class="win">39.98%</td></tr>
+<tr><td>ARC-Easy (acc) &uarr;</td><td class="num">&mdash;</td><td class="win">43.56%</td></tr>
+</table>
+<p style="margin-top:14px;">Every metric improved. The ARC-Easy acc_norm gain (30.85% to 39.98%) is more modest than the raw acc numbers alone suggest, but still a real improvement over v1.</p>
+<h3>BLiMP paradigm breakdown</h3>
+<div class="two-col">
+<div>
+<span class="col-label">Strong &middot; Core Agreement</span>
+<table class="results compact">
+<tr><td>principle_A_case_1</td><td class="num">100.00%</td></tr>
+<tr><td>existential_there_quantifiers_1</td><td class="num">99.40%</td></tr>
+<tr><td>anaphor_number_agreement</td><td class="num">98.00%</td></tr>
+<tr><td>sentential_negation_npi_licensor</td><td class="num">97.30%</td></tr>
+<tr><td>determiner_noun_agreement_1</td><td class="num">96.90%</td></tr>
+<tr><td>principle_A_domain_1</td><td class="num">95.40%</td></tr>
+</table>
+</div>
+<div>
+<span class="col-label">Weak &middot; Long-Distance / Islands</span>
+<table class="results compact">
+<tr><td>wh_vs_that_with_gap_long_distance</td><td class="num">17.20%</td></tr>
+<tr><td>existential_there_quantifiers_2</td><td class="num">30.20%</td></tr>
+<tr><td>left_branch_island_echo_question</td><td class="num">31.40%</td></tr>
+<tr><td>superlative_quantifiers_2</td><td class="num">33.20%</td></tr>
+<tr><td>principle_A_reconstruction</td><td class="num">36.10%</td></tr>
+</table>
+</div>
+</div>
+<p style="margin-top:16px;">This pattern, strong local agreement paired with weaker long-distance syntax, is typical for models at this scale.</p>
+</section>
+<section>
+<div class="section-head"><span class="section-index">03</span><h2>Does it actually make more sense now?</h2></div>
+<p>None of the benchmarks above directly test whether the model's writing holds together as connected text, which was v1's real problem. Sample output below, EMA weights, temperature 0.8, top_k 50.</p>
+<div class="sample-prompt">Prompt: "Once upon a time, there was a"</div>
+<blockquote class="sample">Once upon a time, there was a wise old turtle named Timmy who lived on the coast of South America. In this magical place, no matter how big or large, people could look up at the sea and talk to each other. Timmy asked, "What do you mean, Timmy?" The turtle replied, "Well, I think you might see people talking about ocean creatures. They are like little waves that carry their voices. Sometimes they say they're too big or small to hear." After a few moments, Timmy had an idea. "Can we go on a boat-boat tour? I can't believe all the kids in the village are doing that!" As they sailed further, they saw many beautiful islands and vibrant colors. Each island had its unique culture and traditions. When they reached the top, they saw a group of kids playing and splashing around. "Wow, lookgies!" said Timmy. "They live in a big ocean full of colorful fish and</blockquote>
+</section>
+<section>
+<div class="section-head"><span class="section-index">04</span><h2>Training</h2></div>
+<h3>Data mix</h3>
+<p>~12.85B tokens total, roughly 8 to 9x more than v1's 1.57B. v1 was trained Chinchilla-optimal. v2 deliberately overtrains well past that point, since the model is small and cheap to run regardless.</p>
+<table class="kv">
+<tr><td>fineweb-edu</td><td>50%</td></tr>
+<tr><td>smollm-corpus (cosmopedia-v2)</td><td>27%</td></tr>
+<tr><td>dclm-baseline-1.0</td><td>8%</td></tr>
+<tr><td>SimpleStories</td><td>5%</td></tr>
+<tr><td>finemath (finemath-3plus)</td><td>5%</td></tr>
+</table>
+<p style="margin-top:14px;">Python-Edu was originally planned as a fifth source, but the actual code text lives behind a gated dataset with no practical way to align it against the sampled subset at this scale, so it was dropped and its share redistributed across the rest.</p>
+<h3>Hyperparameters</h3>
+<table class="kv">
+<tr><td>Optimizer</td><td>Muon (body weights) + AdamW (embeddings, norms)</td></tr>
+<tr><td>Muon lr</td><td>0.02</td></tr>
+<tr><td>AdamW lr</td><td>3e-4</td></tr>
+<tr><td>LR schedule</td><td>Warmup-Stable-Decay (WSD)</td></tr>
+<tr><td>Weight decay</td><td>0.1</td></tr>
+<tr><td>Gradient clipping</td><td>1.0</td></tr>
+<tr><td>Batch size</td><td>192 sequences &times; 1024 tokens (196,608 tokens/step)</td></tr>
+<tr><td>Total steps</td><td>65,376</td></tr>
+<tr><td>Precision</td><td>bfloat16</td></tr>
+<tr><td>Attention</td><td>PyTorch scaled_dot_product_attention (Flash Attention backend)</td></tr>
+<tr><td>Compilation</td><td>torch.compile, roughly 2x throughput over eager</td></tr>
+<tr><td>Final weights</td><td>EMA (&beta;=0.999) of training trajectory</td></tr>
+</table>
+<h3>Hardware</h3>
+<p>Trained on a single NVIDIA RTX PRO 6000 Blackwell (96GB) in approximately <b>4.75 hours</b>.</p>
+</section>
+<section>
+<div class="section-head"><span class="section-index">05</span><h2>Tokenizer</h2></div>
+<p>Custom byte-level BPE tokenizer trained from scratch on a sample of the pretraining mix. Vocab size 16,000.</p>
+</section>
+<section>
+<div class="section-head"><span class="section-index">06</span><h2>Inference</h2></div>
+<p>This model can now be loaded with <code>AutoModelForCausalLM</code> instead of the manual pickle-loading workflow, and weights are available as <code>model.safetensors</code>.</p>
+<pre><code><span class="tok-kw">import</span> torch
+<span class="tok-kw">from</span> transformers <span class="tok-kw">import</span> <span class="tok-cls">AutoModelForCausalLM</span>, <span class="tok-cls">AutoTokenizer</span>
+model = <span class="tok-cls">AutoModelForCausalLM</span>.<span class="tok-fn">from_pretrained</span>(
+    <span class="tok-str">"IvmeLabs/Ivme-Conversate-v2-Base"</span>, trust_remote_code=<span class="tok-kw">True</span>, dtype=torch.float32,
 )
+tokenizer = <span class="tok-cls">AutoTokenizer</span>.<span class="tok-fn">from_pretrained</span>(<span class="tok-str">"IvmeLabs/Ivme-Conversate-v2-Base"</span>, trust_remote_code=<span class="tok-kw">True</span>)
+model.<span class="tok-fn">eval</span>()
+inputs = <span class="tok-fn">tokenizer</span>(<span class="tok-str">"Once upon a time, there was a"</span>, return_tensors=<span class="tok-str">"pt"</span>)
+out = model.<span class="tok-fn">generate</span>(
+    **inputs, max_new_tokens=<span class="tok-num">200</span>, do_sample=<span class="tok-kw">True</span>,
+    temperature=<span class="tok-num">0.8</span>, top_k=<span class="tok-num">50</span>, pad_token_id=tokenizer.pad_token_id,
 )
+<span class="tok-fn">print</span>(tokenizer.<span class="tok-fn">decode</span>(out[<span class="tok-num">0</span>], skip_special_tokens=<span class="tok-kw">True</span>))</code></pre>
+<p><code>trust_remote_code=True</code> is required (custom architecture: RoPE + SwiGLU + RMSNorm dense decoder). The original <code>ckpt_final.pt</code> pickle checkpoint and <code>model/</code> architecture source remain in this repo unchanged for backwards compatibility.</p>
+<div class="note-box">Use left-padding (<code>tokenizer.padding_side = "left"</code>) for batch generation. The model doesn't use an explicit attention mask over padded positions, so right-padding within a batch will give incorrect results.</div>
+<h3>Legacy inference</h3>
+<p>Here's a basic inference path in case you want to work with the pickle files directly.</p>
+<pre><code><span class="tok-kw">import</span> sys
+<span class="tok-kw">import</span> torch
+<span class="tok-kw">from</span> tokenizers <span class="tok-kw">import</span> <span class="tok-cls">Tokenizer</span>
+<span class="tok-kw">from</span> huggingface_hub <span class="tok-kw">import</span> hf_hub_download, snapshot_download
+repo_id = <span class="tok-str">"IvmeLabs/Ivme-Conversate-v2-Base"</span>
+<span class="tok-com"># Download just the model/ folder (architecture code) into the HF cache,</span>
+<span class="tok-com"># then add it to sys.path so `from model import ...` works without the</span>
+<span class="tok-com"># user needing to manually copy any files.</span>
+repo_local_dir = <span class="tok-fn">snapshot_download</span>(repo_id, allow_patterns=[<span class="tok-str">"model/*"</span>])
 sys.path.append(repo_local_dir)
+<span class="tok-kw">from</span> model <span class="tok-kw">import</span> <span class="tok-cls">IvmeConfig</span>, <span class="tok-cls">IvmeConversateV2</span>
+ckpt_path = <span class="tok-fn">hf_hub_download</span>(repo_id, <span class="tok-str">"ckpt_final.pt"</span>)
+tokenizer_path = <span class="tok-fn">hf_hub_download</span>(repo_id, <span class="tok-str">"tokenizer.json"</span>)
+tokenizer = <span class="tok-cls">Tokenizer</span>.<span class="tok-fn">from_file</span>(tokenizer_path)
+<span class="tok-com"># IvmeConfig is a plain dataclass saved into the checkpoint. Trust this only</span>
+<span class="tok-com"># because it's our own checkpoint, produced by our own training code.</span>
+torch.serialization.<span class="tok-fn">add_safe_globals</span>([<span class="tok-cls">IvmeConfig</span>])
+ckpt = torch.<span class="tok-fn">load</span>(ckpt_path, map_location=<span class="tok-str">"cuda"</span>)
+cfg = ckpt[<span class="tok-str">"config"</span>]
+model = <span class="tok-cls">IvmeConversateV2</span>(cfg)
+<span class="tok-com"># Use EMA weights (smoothed), not the raw training weights, for inference.</span>
+<span class="tok-com"># Strip torch.compile's "_orig_mod." prefix if the checkpoint was compiled.</span>
+state_dict = ckpt[<span class="tok-str">"ema_state_dict"</span>]
+state_dict = {k.removeprefix(<span class="tok-str">"_orig_mod."</span>): v <span class="tok-kw">for</span> k, v <span class="tok-kw">in</span> state_dict.items()}
+model.<span class="tok-fn">load_state_dict</span>(state_dict)
+model.cuda().<span class="tok-fn">eval</span>()
+prompt = <span class="tok-str">"Once upon a time, there was a"</span>
+ids = tokenizer.<span class="tok-fn">encode</span>(prompt).ids
+idx = torch.<span class="tok-fn">tensor</span>([ids], dtype=torch.long, device=<span class="tok-str">"cuda"</span>)
+eot_id = tokenizer.<span class="tok-fn">token_to_id</span>(<span class="tok-str">"&lt;|endoftext|&gt;"</span>)
+<span class="tok-kw">with</span> torch.no_grad():
+    <span class="tok-kw">for</span> _ <span class="tok-kw">in</span> range(<span class="tok-num">200</span>):
+        idx_cond = idx <span class="tok-kw">if</span> idx.size(<span class="tok-num">1</span>) &lt;= cfg.context_len <span class="tok-kw">else</span> idx[:, -cfg.context_len:]
         logits, _ = model(idx_cond)
+        logits = logits[:, -<span class="tok-num">1</span>, :] / <span class="tok-num">0.8</span>  <span class="tok-com"># temperature</span>
+        v, _ = torch.<span class="tok-fn">topk</span>(logits, <span class="tok-num">50</span>)
+        logits[logits &lt; v[:, [-<span class="tok-num">1</span>]]] = -float(<span class="tok-str">"inf"</span>)
+        probs = torch.<span class="tok-fn">softmax</span>(logits, dim=-<span class="tok-num">1</span>)
+        next_id = torch.<span class="tok-fn">multinomial</span>(probs, num_samples=<span class="tok-num">1</span>)
+        idx = torch.<span class="tok-fn">cat</span>([idx, next_id], dim=<span class="tok-num">1</span>)
+        <span class="tok-kw">if</span> next_id.item() == eot_id:
+            <span class="tok-kw">break</span>
+<span class="tok-fn">print</span>(tokenizer.<span class="tok-fn">decode</span>(idx[<span class="tok-num">0</span>].tolist()))</code></pre>
+</section>
+<section>
+<div class="section-head"><span class="section-index">07</span><h2>Limitations</h2></div>
+<ul class="limits">
+<li>Base model only, not instruction tuned, will not follow instructions or answer questions</li>
+<li>English only</li>
+<li>1024 token context window</li>
+<li>Weaker on long-distance syntactic dependencies than on local agreement, see BLiMP breakdown above</li>
+<li>No code data in the training mix (Python-Edu was dropped, see Training)</li>
+<li>Weak knowledge: the model has learned syntax and grammar well but falls short on knowledge and easily hallucinates. Expected to improve in upcoming variants via distillation.</li>
+</ul>
+</section>
+<section>
+<div class="section-head"><span class="section-index">08</span><h2>What's next</h2></div>
+<p>Still on the table for a future version: distillation from a larger teacher model, and a return to the more experimental İvmetron architecture once time allows for the kind of patient debugging a genuinely novel design needs.</p>
+<p>Check our other upcoming models on the <a href="https://huggingface.co/IvmeLabs">İvmeLabs organization page</a>.</p>
+</section>
+<section>
+<div class="section-head"><span class="section-index">09</span><h2>Citation</h2></div>
+<pre><code>@misc{<span class="tok-fn">ivme-conversate-v2-Base</span>,
   author       = {IvmeLabs},
   title        = {İvme-Conversate-v2-Base},
+  year         = {<span class="tok-num">2026</span>},
   publisher    = {Hugging Face},
   url          = {https://huggingface.co/IvmeLabs/Ivme-Conversate-v2-Base}
+}</code></pre>
+</section>
+<footer>
+<div>
+<span>İvmeLabs <span class="fmark">/</span> Conversate</span>
+<div class="credit-line">Apple photo by <a href="https://unsplash.com/@cenali">Matheus Cenali</a> on Unsplash</div>
+</div>
+<span>Hugging Face Model Card</span>
+</footer>
+</div>