Instructions to use NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline

SGLang

How to use NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline with Docker Model Runner:
```
docker model run hf.co/NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline
```

babylm26_multiling_hawk_MoP16K_baseline

A multilingual (English / Dutch / Chinese) Hawk language model trained for the BabyLM 2026 Multilingual track (EMNLP 2026). This is a baseline model in the NeTS-lab BabyLM-2026 series: it pairs a recurrent Hawk (RG-LRU) backbone with the morphologically-aware MorPiece (MoP) tokenizer, trained on the three target languages (eng / nld / zho) with a byte-premium-balanced word budget under the baseline regimen.

It is the Hawk counterpart to the Transformer baseline NeTS-lab/babylm26_multiling_gpt2_MoP16K_baseline; the two share the same tokenizer and the same training data, so the comparison isolates the architecture (linear-recurrent vs. attention).

Note on baselines. These models are released as controlled reference points for the NeTS-lab BabyLM-2026 study. Their purpose is a clean, matched comparison across architectures and tokenizers — not leaderboard maximisation.

Model details

Developed by: Cristiano Chesi and NeTS Lab, IUSS Pavia (with Claude Opus 4.8 fixes and optimizations for our HPC)
Model type: Decoder-only causal LM, recurrent (Hawk / RG-LRU) — no global self-attention
Languages: English (eng), Dutch (nld), Chinese (zho)
Tokenizer: MorPiece (MoP), ~39.7K vocabulary, shared multilingual
License: MIT
Sibling models: *_gpt2_MoP16K_baseline (Transformer), and the eMG / SSM-eMG variants

Architecture

Hawk is the gated linear-recurrent backbone from the Griffin family (De et al., 2024). Each block uses a Real-Gated Linear Recurrent Unit (RG-LRU) in place of attention, combined with a gated MLP and RMSNorm. Loading requires trust_remote_code=True because the hawk_rglru block is provided via custom modelling code.

Field	Value
Backbone	Hawk (RG-LRU recurrent) — no attention
`model_type`	`hawk_rglru`
Layers (`n_layer`)	12
Hidden size (`n_embd`)	704
Recurrent width (`rnn_width`)	768
Conv kernel	4
MLP expansion	3
RG-LRU `c`	8.0
RMSNorm eps	1e-6
Max position embeddings	1024
Tied input/output embeddings	yes
Vocabulary size	39,697
Total parameters	≈ 115.3M (115,270,528)
Precision	float32

The model is a pure Hawk recurrent stack (RG-LRU + depthwise conv + gated MLP, RMSNorm pre-norm). There is no self-attention, so the leaderboard "attention heads" field is -1. Parameter count is with tied embeddings (lm_head shares wte). Note that Hawk has no learned position embeddings — max_position_embeddings is a config field, not a hard context limit.

Tokenizer — MorPiece (MoP)

MorPiece is a split-based tokenizer that incrementally segments words into candidate morphemes by applying Yang's (2016) Tolerance Principle at every character as a word traverses a dual root/inflection trie. Splits are licensed only when the TP holds bilaterally (root trie and inflection trie). The result is a morphology-aware vocabulary motivated by developmental linguistics rather than pure frequency. For this multilingual experiment we used the --boundary-discovery option to ignore whitespaces and process zho the same way of eng and nld.

Shared across all three languages (single multilingual MoP tokenizer)
Exported in HuggingFace WordPiece format with ++ continuing-subword prefix
Actual vocabulary: 39,697 tokens (the MoP16K in the repo name is a nominal per-language label; the merged multilingual vocabulary is ~39.7K)

See the MorPiece repository for details: https://github.com/cristianochesi/morpiece

Training data

Official BabyLM 2026 Multilingual data for eng / nld / zho. Languages are drawn in a byte-premium round-robin during training, and the save-point milestones are denominated in byte-premium-adjusted English-equivalent words (BP: eng 1.000, nld 1.0516, zho 0.9360), per the multilingual track's word budget. Training regimen: baseline. Per-corpus sizes: eng 56.2M / nld 57.0M / zho 50.0M model tokens (≈34.2M English-equivalent words each ≈ 102.6M total/epoch).

No custom corpus, no synthetic augmentation, no human-annotated preference data. A cleaning procedure stripped metalinguistic information (e.g. tiers). Preprocessing routine can be found here: https://github.com/cristianochesi/babylm-2026/tree/main/01-preprocess

Training procedure

Field	Value
Optimizer	AdamW (β1=0.9, β2=0.999, weight decay 0.1, fused), no-decay on norms/biases/embeddings
LR scheduler	cosine decay with linear warmup (warmup = 1% of steps)
Max learning rate	5e-4
Min learning rate	5e-5
Epochs	~3.1 of 10 planned (run cut by a 24h cluster time limit)
Per-device batch	16 sequences × 4 grad-accum = 32,768 tokens / optimizer step
Training sequence length	512 (config `max_position_embeddings` = 1024)
Gradient clip	1.0 (with a non-finite-grad firewall)
Random seed	42
Precision	bf16 mixed precision (no GradScaler); weights stored as float32
Tokens processed	~~504M model tokens (~~317M eng-equiv words) at stop
Hardware	1 GPU + 8 CPUs, IUSS SLURM cluster (`gp02`, `gpuq`, conda `env_py3_12_torch2_91_CUDA_12_8`)
GPU-hours (training)	~24 (single GPU; throughput ~2.6k steps/h)
Training FLOPs (approx.)	~1.4 × 10¹⁸ (6·N·D over executed positions)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline"

tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True)

text = "the cats are"
inputs = tokenizer(text, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(out[0], skip_special_tokens=True))

trust_remote_code=True is required to load the custom hawk_rglru block. The repo must contain modeling_hawk.py alongside config.json (the auto_map points to it). Generation is correct but has no KV cache — the recurrent backbone recomputes the full prefix each step, so generate() is O(T) per token.

Evaluation

Evaluated with the BabyLM 2026 multilingual harness (lm-eval-style), including BLiMP, MultiBLiMP (Dutch), and SIGMORPHON 2022 morphology, alongside the official multilingual benchmarks.

Benchmark	Score
BLiMP (filtered)	`0.724`
BLiMP-nld (nld)	`0.803`
BLiMP-zho (zho)	`0.803`

Intended use & limitations

A small, sample-efficient research LM for studying cognitively-plausible language modelling under a constrained (developmentally motivated) data budget. It is not intended for production use. As a baseline trained on a limited multilingual corpus, outputs are not reliable for downstream generation and may reflect biases in the training data.

Citation

@misc{chesi2026babylm_hawk_mop,
  title  = {Multilingual Hawk + MorPiece baseline for BabyLM 2026},
  author = {Chesi, Cristiano and {NeTS Lab, IUSS Pavia}},
  year   = {2026},
  note   = {BabyLM 2026 Multilingual track baseline},
  howpublished = {\url{https://huggingface.co/NeTS-lab/babylm26_multiling_hawk_MoP16K_baseline}}
}

Contact: cristiano.chesi@iusspavia.it · NeTS Lab, IUSS Pavia

Downloads last month: 13,691

Safetensors

Model size

0.1B params

Tensor type

F32