README.md

---
license: apache-2.0
language:
  - en
library_name: transformers
pipeline_tag: text-generation
tags:
  - juwel
  - juwel-garnet
  - vext-base
  - retired-flagship
  - composition-architecture
  - cip
---

# JUWEL Garnet

> **JUWEL Garnet** (internal lineage: Vext-Base-v8) — the eighth-generation Vext Labs base
> model and the origin stone of the open archive. Garnet is the January birthstone: the
> beginning. Released under Apache 2.0 after being superseded internally by JUWEL Emerald
> (Vext-Base-v9).
>
> Built better. Priced fairly. And what we used to ship, the world can ship too.

Vext-Base-v8 is the first open-source release in the Vext Labs OSS archive
program. It is the BF16 base model that powered the Theron entity from
2026-04-04 through 2026-05-13, formerly named "Theron-Base v8" while it
was the live flagship. It is published here under the public "Vext-*"
name so that the Theron brand stays attached to the live flagship and
downstream forks can't muddy the name.

## At a glance

| Field | Value |
|---|---|
| Public name | JUWEL Garnet |
| HF repo | `VextLabsinc/juwel-garnet` |
| Internal lineage | Theron-Base v8 (Vext-Base-v8) |
| Architecture | Transformer (Qwen3 lineage, organically upscaled) |
| Parameters | ~78B post-merge |
| Precision | BF16 (no quantization) |
| Context window | 131K (YaRN extended) |
| Attention | GQA 64:8 + FlashAttention-3 |
| Live as flagship | 2026-04-04 — 2026-05-13 |
| Superseded by | Vext-Base-v9 (Qwen3-VL-32B + CIP additive) |
| Release license | Apache-2.0 |
| Release date (target) | 2026-06-12 |
| Hardware floor | 1× H100 80GB for inference (BF16 with offload), 2× H100 80GB recommended |

## What this is, honestly

This is a **retired** Vext flagship. The current production system uses
Vext-Base-v9 and a Composition layer of 15 cluster LoRAs that delivers
materially better numbers on every domain we benchmark (headline:
+51 GSM8K, +53 MMLU-Pro vs v8 measured on the same harness — see the
audit trail at https://vextlabs.ai/transparency).

We release v8 anyway because:

1. v8 is still a **strong** model — at its retirement it was within
   striking distance of contemporary frontier-class bases on general
   benchmarks, and remains a useful research artifact.
2. The world deserves access. Frontier labs keep their last three
   generations sealed; we keep our last one in your hands.
3. **Feedback loop.** When the open community uses, breaks, fine-tunes,
   and benchmarks v8, that signal feeds back into how we build v10+.

## Intended use

- Research baselines for transformer scaling / composition / CIP papers
- Fine-tuning starting point for domain LoRAs
- Self-hosted deployment for organizations that want the Vext lineage
  without depending on Vext-hosted inference
- Inputs to derivative open-source models (the Apache-2.0 license
  permits this with attribution)

## Out of scope (we explicitly don't recommend these)

- Replacement for the live Vext-Base-v9 + Composition stack — quality
  delta is significant, especially on reasoning and math
- Production deployment without your own evaluation harness — every
  model has corner cases the headline numbers don't capture
- Distillation as a teacher — we don't distill on principle (see
  `feedback_no_distillation.md`) and we don't recommend others do either
- Quantized inference if quality is the goal — v8 was trained and
  validated in BF16; INT4/INT8 hits the long tail hard

## Training

- **Recipe**: V6+ canonical (4× H100 + DDP + local SSD + device_map per
  rank). Documented at `scripts/theron/train_theron_base_v8.py` in the
  Vext repo.
- **No teacher distillation.** Every training example is a curated
  primary source (real documents, validated traces, verified answers).
  The raw corpus stays proprietary (we never published an LLM-generated
  Q&A pool, and we don't ship one here); a *data card* describing the
  source families + licensing is included so downstream users can
  reproduce the recipe shape.
- **Organic upscale + CIP** for capability injection: instead of
  retraining existing weights, we add +16 layers and LoRA-train only the
  new layers, then merge. v8 was built this way over v7.

## Eval

Honest delta vs the successor Vext-Base-v9:

| Benchmark | Vext-Base-v8 | Vext-Base-v9 | Δ |
|---|---|---|---|
| GSM8K | (published in audit trail) | +51 | v9 ahead |
| MMLU-Pro | (published in audit trail) | +53 | v9 ahead |
| HumanEval | (published in audit trail) | small Δ | comparable |

Full audit trail (raw responses, extracted code, test cases, errors,
timestamps, model version) lives at https://vextlabs.ai/transparency
per `feedback_benchmark_audit_trail.md`. Pass/fail only is unverifiable;
we publish the trail so external auditors can reproduce.

## Weights — download (Cloudflare R2, public, BF16)

> Weights live on Cloudflare R2, not inside this HF repo. Download them locally, then load from the local dir. (Garnet is a Qwen3-Next MoE base — the GEM LoRA adapters do NOT apply to it; it is a standalone base.)

```bash
BASE=https://pub-a6ae0476e46849f98f1746a61dc4c106.r2.dev/juwel-garnet
mkdir -p juwel-garnet && cd juwel-garnet
for f in config.json model.safetensors.index.json tokenizer.json tokenizer_config.json generation_config.json; do curl -sO $BASE/$f; done
for i in $(seq -w 1 32); do curl -O $BASE/model-000$i-of-00032.safetensors; done
```

## How to load

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "./juwel-garnet",
    torch_dtype=torch.bfloat16,   # do NOT quantize — see notes above
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("./juwel-garnet")

inputs = tokenizer("Explain how attention works in one paragraph.", return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.1)
print(tokenizer.decode(out[0], skip_special_tokens=True))
```

## Hardware requirements

| Setting | Floor | Recommended |
|---|---|---|
| BF16 inference with offload | 1× H100 80GB + system RAM | 2× H100 80GB |
| BF16 inference fully on GPU | 2× H100 80GB | 4× H100 80GB |
| LoRA fine-tune | 4× H100 80GB DDP | 8× H100 80GB DDP |
| Full fine-tune | 8× H100 80GB | 16× H100 80GB |

vLLM and SGLang both load this checkpoint out of the box. Vext Labs
serves the live successor on SGLang TurboLoRA — see
https://vextlabs.ai/technology.

## Known limitations

- v8 inherits all known limitations of its base lineage (Qwen3) plus
  any specific quirks introduced by the CIP additive training step.
- Performance below v9 on reasoning + math (see Eval table). For best
  results pair with a verifier (Vext ships Iris; OSS alternatives like
  Self-RAG, CoT-SC, MCTS-style search all compose cleanly).
- No deliberate refusal training (see `feedback_no_refusal_ever.md`).
  Users are expected to apply their own safety policy at the
  human-to-model boundary; we don't ship one baked in.

## License

[Apache License 2.0](./LICENSE). Use it. Fork it. Build on it. Attribution
appreciated; not required for the weights themselves.

## Citation

```bibtex
@misc{juwel-garnet,
  author       = {{Vext Labs}},
  title        = {{JUWEL Garnet: A Retired Flagship, Open}},
  year         = 2026,
  publisher    = {HuggingFace},
  url          = {https://huggingface.co/VextLabsinc/juwel-garnet},
  note         = {Internal lineage Theron-Base v8 / Vext-Base-v8. Apache-2.0.},
}
```

## Acknowledgements

Built on the shoulders of the Qwen3 lineage and the broader
open-research community whose papers + datasets shaped every stage of
training. The OSS archive program is a 30-day-after-supersession cadence
(see https://vextlabs.ai/open-source) and every retired Vext base, LoRA,
and Brain version follows v8 here.