manueldeprada HF Staff commited on
Commit
cbdaeb1
·
verified ·
1 Parent(s): 22b339c

Model card: add minimal transformers + vLLM usage snippets

Browse files
Files changed (1) hide show
  1. README.md +17 -8
README.md CHANGED
@@ -21,21 +21,30 @@ Part of the **LittleLearner** scale-up study (*pedagogically-controlled knowledg
21
  ## Model
22
  - **Architecture:** Qwen3 dense (`Qwen3ForCausalLM`) — standard `transformers`, no custom code / `trust_remote_code`.
23
  - **Size:** 5.04B params — hidden 3072, 44 layers, 24 query / 8 KV heads, FFN 9216. **Context:** 4096.
24
- - **Tokenizer:** custom 64k byte-level BPE with per-digit splitting (trained for this project; ChatML special tokens).
25
- - **Pretraining:** 88B tokens on K-5 **LittleCurriculum** (FineWeb-Edu filtered to U.S. grades K–5). WSD schedule, sharded Muon optimizer, MXFP8, Megatron-Core on 8×B200.
26
 
27
  ## Evaluation
28
  - In-domain bits-per-byte (BPB): **0.536** (vs the 2B nanochat reference 0.805).
29
 
30
-
31
  ## Usage
32
  ```python
 
33
  from transformers import AutoModelForCausalLM, AutoTokenizer
34
- import torch
35
- tok = AutoTokenizer.from_pretrained("manueldeprada/littlelearner-5b-bounded-base")
36
- model = AutoModelForCausalLM.from_pretrained("manueldeprada/littlelearner-5b-bounded-base", torch_dtype=torch.bfloat16, device_map="auto")
37
  ids = tok("The sum of 2 and 3 is", return_tensors="pt").to(model.device)
38
- print(tok.decode(model.generate(**ids, max_new_tokens=64)[0]))
39
  ```
40
 
41
- > Research artifact. The **bounded** models carry an intentional K–5 knowledge boundary (they cannot model above-grade-5 material); the **base** models are not instruction-tuned.
 
 
 
 
 
 
 
 
 
 
21
  ## Model
22
  - **Architecture:** Qwen3 dense (`Qwen3ForCausalLM`) — standard `transformers`, no custom code / `trust_remote_code`.
23
  - **Size:** 5.04B params — hidden 3072, 44 layers, 24 query / 8 KV heads, FFN 9216. **Context:** 4096.
24
+ - **Tokenizer:** custom 64k byte-level BPE with per-digit splitting (ChatML special tokens).
25
+ - **Pretraining:** 88B tokens on K-5 **LittleCurriculum** (FineWeb-Edu filtered to U.S. grades K–5). WSD schedule, sharded Muon, MXFP8, Megatron-Core on 8×B200.
26
 
27
  ## Evaluation
28
  - In-domain bits-per-byte (BPB): **0.536** (vs the 2B nanochat reference 0.805).
29
 
 
30
  ## Usage
31
  ```python
32
+ # transformers (completion)
33
  from transformers import AutoModelForCausalLM, AutoTokenizer
34
+ repo = "manueldeprada/littlelearner-5b-bounded-base"
35
+ tok = AutoTokenizer.from_pretrained(repo)
36
+ model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16", device_map="cuda")
37
  ids = tok("The sum of 2 and 3 is", return_tensors="pt").to(model.device)
38
+ print(tok.decode(model.generate(**ids, max_new_tokens=64)[0], skip_special_tokens=True))
39
  ```
40
 
41
+ ```python
42
+ # vLLM
43
+ from vllm import LLM, SamplingParams
44
+ llm = LLM("manueldeprada/littlelearner-5b-bounded-base")
45
+ print(llm.generate(["The sum of 2 and 3 is"], SamplingParams(max_tokens=64))[0].outputs[0].text)
46
+ ```
47
+
48
+ > **vLLM note:** vLLM JIT-compiles CUDA kernels, so it needs a CUDA toolkit (`nvcc`) on PATH. On an HPC cluster, `module load cuda/13.x cudnn/9.x` (match your torch CUDA build). On a box with no toolkit, pass `LLM(repo, enforce_eager=True)`. Verified on vLLM 0.15 and 0.23 (and transformers ≥ 4.51).
49
+
50
+ > Research artifact. The **bounded** models carry an intentional K–5 knowledge boundary (they cannot model above-grade-5 material). Base models are **not** instruction-tuned.