Instructions to use nazdef/1gpu-llm-medium-en-it-base-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nazdef/1gpu-llm-medium-en-it-base-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nazdef/1gpu-llm-medium-en-it-base-v2")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nazdef/1gpu-llm-medium-en-it-base-v2")
model = AutoModelForCausalLM.from_pretrained("nazdef/1gpu-llm-medium-en-it-base-v2")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nazdef/1gpu-llm-medium-en-it-base-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nazdef/1gpu-llm-medium-en-it-base-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nazdef/1gpu-llm-medium-en-it-base-v2

SGLang

How to use nazdef/1gpu-llm-medium-en-it-base-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nazdef/1gpu-llm-medium-en-it-base-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nazdef/1gpu-llm-medium-en-it-base-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use nazdef/1gpu-llm-medium-en-it-base-v2 with Docker Model Runner:
```
docker model run hf.co/nazdef/1gpu-llm-medium-en-it-base-v2
```

1gpu-llm-medium-v2

Official medium bilingual base model of the 1gpu-llm family, trained from scratch and continued on a single NVIDIA GeForce RTX 4060 Ti 16GB.

This release is a base language model, not an instruction-tuned chat model.

Release summary

Released checkpoint: step_34200
Parent/reference checkpoint: step_34000 from the main CPT run
Continuation: optimizer-preserving decay-only continuation, global steps 34000 → 35700; this release is the early scalar winner at 34200
Architecture: GPT-2-style decoder-only Transformer with pre-layernorm blocks
Repo-native configuration: architecture: gpt2, block_type: gpt2_prelayernorm, norm_order: preln
Parameters: 337,639,424 (~337.6M)
Context window and training sequence length: 2500 tokens
Languages: English and Italian

Why step_34200

The checkpoint was selected by a controlled 1000-token decoding comparison of step_34000, step_34200, and step_34800, using the same tokenizer, seed, device, precision, prompts, and generation budget across four presets.

Evaluation winner:

checkpoint: step_34200
preset: creative
tuning score: 3.1683
holdout score: 3.8405
EOS termination: 100% on tuning and holdout
truncation at 1000 tokens: 0%
strict loop rate: 0%
mean length: 410.6 tuning / 494.8 holdout tokens
distinct-2: 0.9531 tuning / 0.9536 holdout
language switches: 0%

The model remains imperfect: long completions can show factual or semantic drift. The conservative alternative is the parent step_34000 with the anti_loop preset, which is shorter and more controlled.

Official default generation settings

All official 1gpu-llm releases use the following short-response defaults in their published generation configuration and UI:

preset: balanced
max_new_tokens: 64
temperature: 0.3
top_k: 50
top_p: 0.95
repetition_penalty: 1.1
no_repeat_ngram_size: 0
early termination: model EOS only

These are the operational publication defaults. The 1000-token settings above belong to the checkpoint-selection evaluation and are intentionally separate.

Recommended generation

The following example uses the official balanced publication defaults:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo_id = "nazdef/1gpu-llm-medium-v2"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

prompt = "La capitale d'Italia è"
prompt_ids = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
bos = torch.tensor([[tokenizer.bos_token_id]], dtype=prompt_ids["input_ids"].dtype)
input_ids = torch.cat([bos, prompt_ids["input_ids"]], dim=1)
attention_mask = torch.ones_like(input_ids)

with torch.no_grad():
    outputs = model.generate(
        input_ids=input_ids,
        attention_mask=attention_mask,
        do_sample=True,
        max_new_tokens=64,
        temperature=0.3,
        top_k=50,
        top_p=0.95,
        repetition_penalty=1.1,
        no_repeat_ngram_size=0,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id,
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

For a more conservative response style, use anti_loop:

temperature=0.3
top_k=50
top_p=0.9
repetition_penalty=1.15
no_repeat_ngram_size=4

Training provenance

The model was trained on a balanced English/Italian web + wiki corpus derived from:

English FineWeb-HQ (epfml/FineWeb-HQ)
Italian FineWeb2-HQ (epfml/FineWeb2-HQ)
English and Italian Wiki40B (google/wiki40b)

The local packed dataset used a 2500-token sequence length and a 50/50 EN/IT source-balanced mix. The continuation loaded the model and AdamW optimizer state from the step_34000 checkpoint, deliberately ignored the saved scheduler state, and created a local wsd-decay-only scheduler with no warmup or rewarm.

Included provenance files:

training_config.yaml
decoding_grid_config.yaml
decoding_grid_report.md
export_manifest.json
step_34200.safetensors.json

License

The model card uses CC BY-SA 4.0 as the release license. Training data comes from mixed upstream sources with their own terms, including FineWeb/FineWeb2 and Wiki40B. Downstream users are responsible for checking the applicable upstream dataset terms, attribution requirements, share-alike obligations, and any restrictions connected to the data or generated outputs in their intended use case.

Limitations

This is a pretrained base model; it is not instruction-following aligned.
It may hallucinate facts and drift semantically during long generations.
English/Italian language consistency is strong in the selected decoding holdout, but it is not a guarantee for arbitrary prompts.
The public checkpoint is an experimental single-GPU family release, not a claim of state-of-the-art performance.

Downloads last month: -

Safetensors

Model size

0.3B params

Tensor type

F32

Datasets used to train nazdef/1gpu-llm-medium-en-it-base-v2

Collection including nazdef/1gpu-llm-medium-en-it-base-v2

1GPU LLM

Collection

Language models trained from scratch on a single consumer GPU. • 4 items • Updated about 8 hours ago