Instructions to use nazdef/1gpu-llm-medium-en-it-base-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nazdef/1gpu-llm-medium-en-it-base-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nazdef/1gpu-llm-medium-en-it-base-v2")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nazdef/1gpu-llm-medium-en-it-base-v2")
model = AutoModelForCausalLM.from_pretrained("nazdef/1gpu-llm-medium-en-it-base-v2")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nazdef/1gpu-llm-medium-en-it-base-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nazdef/1gpu-llm-medium-en-it-base-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nazdef/1gpu-llm-medium-en-it-base-v2

SGLang

How to use nazdef/1gpu-llm-medium-en-it-base-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nazdef/1gpu-llm-medium-en-it-base-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nazdef/1gpu-llm-medium-en-it-base-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use nazdef/1gpu-llm-medium-en-it-base-v2 with Docker Model Runner:
```
docker model run hf.co/nazdef/1gpu-llm-medium-en-it-base-v2
```

1gpu-llm-medium-en-it-base-v2 / decoding_grid_report.md

nazdef

Release 1gpu-llm-medium-v2 step 34200

7b5acd6 verified about 15 hours ago

preview code

Raw

History Blame Contribute Delete

10.7 kB

Comparative 1000-token decoding grid — medium checkpoint selection

Executive verdict

Recommended candidate for 1gpu-llm-medium:

checkpoint: step_34200
preset: creative
tuning score: 3.1683
holdout score: 3.8405
EOS termination: 100% on tuning and holdout
truncation at 1000 tokens: 0%
loop rate: 0% on both splits
repeated 4-gram rate: 0% tuning, 25% holdout
mean generated length: 410.6 tuning / 494.8 holdout tokens
median generated length: 392 tuning / 582.5 holdout tokens
distinct-2: 0.9531 tuning / 0.9536 holdout
language switches: 0% on both splits

This is the strongest overall combination because it is the only checkpoint/preset pair that combines the scalar champion checkpoint, a tuning winner confirmed by the holdout, long completions, zero truncation, zero loops, high diversity, and no language switching.

Conservative alternative:

checkpoint: step_34000
preset: anti_loop
tuning score: 3.4540
holdout score: 3.6036
EOS termination: 100% on both splits
truncation: 0%
loop rate: 0% on both splits
repeated 4-gram rate: 0% tuning, 25% holdout
mean length: 183.0 tuning / 265.5 holdout

It is cleaner against repetition but produces shorter, more conservative answers and remains behind step_34200 + creative on the holdout score.

The behavior champion step_34800 is not promoted. Its tuning winner is creative, but holdout selects balanced; balanced is very short (141.8 tokens mean) and therefore is not rewarded as a final choice merely for stopping early. step_34800 does not provide a robust behavior advantage over step_34200 under this 1000-token comparison.

Experimental controls

Checkpoints: step_34000, step_34200, step_34800
Presets: anti_loop_conservative, anti_loop, balanced, creative
max_new_tokens: 1000 for every preset
Tuning prompts: 7
Holdout prompts: 4, disjoint from tuning
Seed: 1337
Tokenizer: /mnt/apps/llm-nanochat/tokenizers/tokenizer_20260515_en50it50_webwiki_stratified_500M
Device/dtype: CUDA / bf16
Generation count: 7 per preset on tuning and 4 per preset on holdout, one seed
Early stopping: only the model EOS path; no artificial stop was introduced
Holdout coverage: all four presets were evaluated, not only the tuning top-k

The repo runner does not serialize an explicit terminated_with_eos boolean. Since the generator has only two exits — EOS or the max_new_tokens loop limit — this report derives EOS/truncation as follows:

num_generated_tokens < 1000: EOS termination
num_generated_tokens == 1000: truncation at the configured limit

Complete tuning table

rep is repeated-4gram rate; loop is the stricter repeated-4gram loop rate. Higher EOS, distinct-1/2 and language consistency are better; lower rep/loop and switch rates are better.

checkpoint	preset	score	EOS	trunc.	mean	median	distinct-1	distinct-2	rep	loop	lang. consistency
34000	anti_loop_conservative	2.9462	71.4%	28.6%	351.0	125.0	0.4763	0.8471	42.9%	0%	0.9848
34000	anti_loop	3.4540	100%	0%	183.0	183.0	0.5750	0.9567	0%	0%	0.9262
34000	balanced	2.0965	100%	0%	312.1	256.0	0.4833	0.8582	71.4%	14.3%	0.9286
34000	creative	2.3089	85.7%	14.3%	299.4	220.0	0.5312	0.9237	28.6%	14.3%	0.9203
34200	anti_loop_conservative	3.0936	85.7%	14.3%	265.9	66.0	0.4779	0.8545	14.3%	0%	0.7850
34200	anti_loop	3.0177	100%	0%	268.0	243.0	0.5552	0.9439	0%	0%	0.9259
34200	balanced	2.5360	100%	0%	202.7	146.0	0.5922	0.9508	28.6%	0%	0.9286
34200	creative	3.1683	100%	0%	410.6	392.0	0.5119	0.9531	0%	0%	0.9339
34800	anti_loop_conservative	2.5619	85.7%	14.3%	279.3	130.0	0.5378	0.7822	28.6%	14.3%	0.8571
34800	anti_loop	2.5392	85.7%	14.3%	258.9	133.0	0.5580	0.9134	28.6%	0%	0.9259
34800	balanced	2.6047	100%	0%	98.1	57.0	0.5764	0.9290	14.3%	0%	0.9286
34800	creative	2.8813	100%	0%	323.4	95.0	0.5315	0.9591	14.3%	0%	0.9263

Tuning winners:

step_34000: anti_loop
step_34200: creative
step_34800: creative

Complete holdout table

checkpoint	preset	score	EOS	trunc.	mean	median	distinct-1	distinct-2	rep	loop	lang. consistency
34000	anti_loop_conservative	3.4012	100%	0%	129.5	110.5	0.6005	0.9024	25%	0%	1.0000
34000	anti_loop	3.6036	100%	0%	265.5	199.5	0.6262	0.9524	25%	0%	1.0000
34000	balanced	3.1137	100%	0%	205.0	195.0	0.5928	0.9180	50%	0%	0.9886
34000	creative	3.3935	100%	0%	322.0	358.0	0.6184	0.9644	0%	0%	1.0000
34200	anti_loop_conservative	3.0267	100%	0%	142.0	127.5	0.6136	0.8899	50%	0%	1.0000
34200	anti_loop	3.5983	100%	0%	285.0	351.5	0.6158	0.9389	25%	0%	0.9931
34200	balanced	2.5511	100%	0%	157.5	112.0	0.6856	0.9550	0%	0%	1.0000
34200	creative	3.8405	100%	0%	494.8	582.5	0.5777	0.9536	25%	0%	1.0000
34800	anti_loop_conservative	3.5655	75%	25%	348.0	192.0	0.5101	0.8564	25%	0%	0.9975
34800	anti_loop	3.5889	100%	0%	224.0	184.0	0.6499	0.9696	25%	0%	1.0000
34800	balanced	3.8743	100%	0%	141.8	149.5	0.6645	0.9621	0%	0%	1.0000
34800	creative	2.8716	100%	0%	179.0	159.0	0.6790	0.9591	25%	0%	1.0000

Holdout winners by checkpoint:

step_34000: anti_loop
step_34200: creative
step_34800: balanced

The step_34200 tuning winner is confirmed by holdout. The step_34800 tuning winner is not confirmed: holdout prefers balanced, but that preset terminates at only 141.8 tokens on average. That shortness is treated as a weakness, not a bonus.

First degeneration / loop analysis

The first-loop metric is conservative: it reports the first generated word position where a 4-gram has appeared three times. When no such event occurs, the value is none; first repeated-4gram position is also tracked separately.

Tuning observations

step_34000 + anti_loop: no repeated 4-gram and no loop across all 7 samples.
step_34200 + creative: no repeated 4-gram and no loop across all 7 samples.
step_34800 + creative: one repeated 4-gram sample, but no strict loop.
step_34000 + balanced: first strict loop at token 41 in one sample; repeated-4gram rate 71.4%.
step_34000 + creative: first strict loop at mean token 142 in one sample.
step_34800 + anti_loop_conservative: one strict loop, first occurring at token 248.

Holdout observations

No holdout configuration produced a strict loop (loop_rate=0%). Repeated 4-grams remain in several cases, especially step_34200 + creative and the anti-loop variants, but they occur without reaching the stricter three-repetition threshold.

Qualitative inspection

Representative long generations were inspected for factuality, relevance, language stability and late degeneration.

step_34000 + anti_loop is the most controlled: it reaches EOS consistently and avoids strict loops, but often produces conservative encyclopedic continuations with factual drift (for example, Paris geography and historical details).
step_34200 + creative produces the longest useful continuations. It stays in the requested language and avoids strict loops, but factual/semantic drift appears in long completions. The issue is quality drift, not a decoding collapse.
step_34800 + creative is lively and diverse, but is less stable as a checkpoint choice: tuning and holdout select different presets, and the median tuning length is only 95 tokens despite a 1000-token allowance.
balanced on step_34800 wins the raw holdout score mainly with short, clean completions. It is rejected as the final preset because early EOS must not be mistaken for superior long-form behavior.

No preset systematically truncates on the selected final pair step_34200 + creative. The main remaining limitation is factual/semantic degradation in long text, not EOS handling, language switching or strict repetition loops.

Final operational recommendation

Definitive candidate

Keep and use:

checkpoint: /mnt/apps/llm-nanochat/checkpoints/20260715_resume-gpt2medium-gpt2preln-k20-optimizeronly-cpt14700-step34000-d1700-webwiki/step_34200.pt
preset: creative

Reason: tuning winner confirmed by holdout, zero truncation, zero strict loops, longest useful completions, high distinct-2, and stable EN/IT language behavior.

Behavior-oriented alternative

Keep as a conservative fallback:

checkpoint: /mnt/apps/llm-nanochat/checkpoints/20260703_continual-pretraining-gpt2medium-gpt2preln-k20-step14700-lr5e5-w500-s18500-d2000-final1e5-webwiki/step_34000.pt
preset: anti_loop

This pair is the most robust against repetition and is confirmed by holdout, at the cost of shorter and less expressive completions.

Retention and deletion candidates

Retain now:

step_34200.pt — definitive candidate with creative
step_34000.pt — parent/reference and conservative fallback with anti_loop
step_34800.pt — behavior experiment retained until the final release decision

Potentially eliminable only after explicit confirmation:

step_34800.pt, if the project keeps only the definitive candidate plus parent reference.

No checkpoint was deleted by this operation. The other 34k-tail checkpoints were not part of this three-candidate comparison and are outside this cleanup decision.

Raw artifacts

Config: configs/eval/20260716_medium_checkpoint_decoding_grid_1000.yaml
Output root: /mnt/apps/llm-nanochat/evals/20260716_medium_checkpoint_decoding_grid_1000
Parent output: /mnt/apps/llm-nanochat/evals/20260716_medium_checkpoint_decoding_grid_1000/parent_step34000
Scalar output: /mnt/apps/llm-nanochat/evals/20260716_medium_checkpoint_decoding_grid_1000/scalar_step34200
Behavior output: /mnt/apps/llm-nanochat/evals/20260716_medium_checkpoint_decoding_grid_1000/behavior_step34800
Launch log: /mnt/apps/llm-nanochat/launch_logs/20260716_135423_decoding_grid_medium_three_checkpoints_1000.log