Instructions to use nazdef/1gpu-llm-small-en-it-base-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nazdef/1gpu-llm-small-en-it-base-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nazdef/1gpu-llm-small-en-it-base-v2")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nazdef/1gpu-llm-small-en-it-base-v2")
model = AutoModelForCausalLM.from_pretrained("nazdef/1gpu-llm-small-en-it-base-v2")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nazdef/1gpu-llm-small-en-it-base-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nazdef/1gpu-llm-small-en-it-base-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-small-en-it-base-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nazdef/1gpu-llm-small-en-it-base-v2

SGLang

How to use nazdef/1gpu-llm-small-en-it-base-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nazdef/1gpu-llm-small-en-it-base-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-small-en-it-base-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nazdef/1gpu-llm-small-en-it-base-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-small-en-it-base-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use nazdef/1gpu-llm-small-en-it-base-v2 with Docker Model Runner:
```
docker model run hf.co/nazdef/1gpu-llm-small-en-it-base-v2
```

1gpu-llm-small-en-it-base-v2 / report_small_v2_decoding_grid.md

nazdef

Publish 1gpu-llm-small-en-it-base-v2 from step_18800

3ce48eb verified 12 days ago

preview code

Raw

History Blame Contribute Delete

3.8 kB

	# GPT2-small decoding-grid comparison: step_8600 vs step_18000 vs step_18800

	Date: 2026-07-03

	## Scope

	This note compares the completed GPU decoding-grid artifacts for:

	- `step_8600`
	- `step_18000`
	- `step_18800`

	The comparison focuses on:

	- Italian behavior
	- loop / repetition resistance
	- cloze / factual prompts
	- free EN / IT prompts

	The tuning split is the main grid-selection signal. The holdout split is a confirmation check.

	## Executive reading

	- Checkpoint-level verdict: `step_18800` remains the benchmark / loss winner.
	- Preset-level verdict: `step_18000 + anti_loop` is the strongest overall decoding-grid pairing once tuning and holdout are both considered.
	- Why that split matters: the raw tuning winner is `step_18800 + anti_loop`, but the raw holdout winner is `step_18000 + anti_loop`, and the average of the two split scores favors `step_18000 + anti_loop`.

	## Score table

	\| Checkpoint \| Tuning winner \| Tuning score \| Holdout winner \| Holdout score \| Tuning+holdout avg \| Read \|
	\|---\|---\|---:\|---\|---:\|---:\|---\|
	\| `step_8600` \| `balanced` \| 2.789 \| `balanced` \| 3.188 \| 2.989 \| stable, but weakest overall \|
	\| `step_18000` \| `creative` \| 3.370 \| `anti_loop` \| 3.927 \| 3.496 \| best combined grid result \|
	\| `step_18800` \| `anti_loop` \| 3.034 \| `anti_loop` \| 3.210 \| 3.122 \| consistent, but below `step_18000` on the combined read \|

	## Category notes

	### Italian behavior

	- `step_8600` is the weakest on Italian factual prompts, but it still occasionally lands the right answer under `balanced`.
	- `step_18000` improves English factual completion materially and keeps Italian free-form output coherent, but it still misses the Italian cloze answer in the best zero-loop setting.
	- `step_18800` is the most visibly polished in style, but it regresses on the Italian cloze prompt: it keeps drifting into the wrong factual chain instead of producing `Roma`.

	### Loop / repetition

	- `greedy` is unusable on all three checkpoints.
	- `anti_loop` is the only preset that is consistently safe on repetition across the sweep.
	- `step_18000 + anti_loop` is the best compromise: no loops on either split and the strongest combined score.
	- `step_8600` is the most repetition-sensitive checkpoint overall.

	### Cloze / factual prompts

	- On the English cloze prompt, `step_18000` is the strongest of the three checkpoints.
	- On the Italian cloze prompt, none of the checkpoints is fully reliable, but `step_18800` is the most obviously wrong because it settles into the wrong entity chain rather than a near-miss.
	- The factual prompts are the clearest place where benchmark loss and decode quality diverge: `step_18800` still wins on loss, but it does not produce the best factual decoding behavior.

	### Free EN / IT prompts

	- All three checkpoints mostly preserve language identity on the free-form prompts.
	- The main distinction is robustness:
	- `step_8600` is coherent but bland and more repetition-prone.
	- `step_18000 + anti_loop` is the cleanest balanced setting for free EN / IT generation.
	- `step_18800` reads smoother, but its factual slips keep it from being the best overall operational choice.

	## Tuning / holdout mismatch

	- `step_8600` is internally stable: tuning and holdout both prefer `balanced`.
	- `step_18000` is the clearest mismatch: tuning prefers `creative`, holdout prefers `anti_loop`.
	- `step_18800` is internally consistent on the best preset (`anti_loop`), but the checkpoint still underperforms `step_18000 + anti_loop` on the combined read.

	## Bottom line

	- Checkpoint winner: `step_18800`
	- Best decoding-grid pairing: `step_18000 + anti_loop`
	- Main tradeoff: `step_18800` is the best loss checkpoint, but `step_18000 + anti_loop` is the safer decode choice for Italian + free-form usage in this sweep.