Instructions to use nazdef/1gpu-llm-medium-en-it-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nazdef/1gpu-llm-medium-en-it-base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nazdef/1gpu-llm-medium-en-it-base")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nazdef/1gpu-llm-medium-en-it-base")
model = AutoModelForCausalLM.from_pretrained("nazdef/1gpu-llm-medium-en-it-base", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nazdef/1gpu-llm-medium-en-it-base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nazdef/1gpu-llm-medium-en-it-base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nazdef/1gpu-llm-medium-en-it-base

SGLang

How to use nazdef/1gpu-llm-medium-en-it-base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nazdef/1gpu-llm-medium-en-it-base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nazdef/1gpu-llm-medium-en-it-base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use nazdef/1gpu-llm-medium-en-it-base with Docker Model Runner:
```
docker model run hf.co/nazdef/1gpu-llm-medium-en-it-base
```

nazdef commited on Jul 1

Commit

2a0960a

verified ·

1 Parent(s): 4b972d0

card: prefer balanced default for medium base release

Browse files

Files changed (3) hide show

README.md +31 -11
generation_config.json +2 -2
recommended_decoding_params.json +11 -9

README.md CHANGED Viewed

@@ -25,8 +25,6 @@ tags:
 - decay-only
 ---
-![Ih4-DhpQJv3BJTj_RG5tR (1) (1)](https://cdn-uploads.huggingface.co/production/uploads/61cf263eb7eeed127532a758/rYYwbJLnEkb8CRD_m6sWu.png)
 # 1gpu-llm Medium EN/IT Base
 This repository is the current **ready-to-use base release** for the `1gpu-llm`
@@ -184,16 +182,31 @@ Short honest read:
 The repo-native decoding sweep was run on this exact checkpoint.
-Winner:
 - tuning winner: `creative`
 - holdout winner: `creative`
-Recommended generation params:
 - `do_sample = true`
-- `temperature = 1.0`
-- `top_k = 100`
 - `top_p = 0.95`
 - `repetition_penalty = 1.1`
 - `no_repeat_ngram_size = 0`
@@ -201,14 +214,21 @@ Recommended generation params:
 Holdout metrics for the recommended preset:
-- `score = 2.6369`
 - `completion_rate = 1.0`
-- `distinct_2 = 0.9969`
-- `language_consistency_mean = 0.6250`
 - `loop_rate = 0.0`
 - `repeated_4gram_rate = 0.0`
-- `language_switch_rate_mean = 0.0`
-- `length_closeness = 0.9437`
 Both `generation_config.json` and `recommended_decoding_params.json` are
 included in the repo.

 - decay-only
 ---
 # 1gpu-llm Medium EN/IT Base
 This repository is the current **ready-to-use base release** for the `1gpu-llm`
 The repo-native decoding sweep was run on this exact checkpoint.
+Raw sweep result:
 - tuning winner: `creative`
 - holdout winner: `creative`
+Public default:
+- keep `balanced` as the recommended preset for the published family-base card
+- rationale:
+  - Naz explicitly prefers `balanced` as the default unless `creative` wins
+    clearly enough to justify the more aggressive preset
+  - on this checkpoint, `creative` does win the holdout score, but not by a
+    margin large enough to force a louder default for the public base release
+  - practical delta:
+    - `creative` holdout score = `2.6369`
+    - `balanced` holdout score = `2.4656`
+    - delta = `+0.1713`
+  - so the repo keeps the stronger exploratory preset documented, but ships the
+    calmer preset as the default recommendation
+Recommended generation params (`balanced`):
 - `do_sample = true`
+- `temperature = 0.8`
+- `top_k = 50`
 - `top_p = 0.95`
 - `repetition_penalty = 1.1`
 - `no_repeat_ngram_size = 0`
 Holdout metrics for the recommended preset:
+- `score = 2.4656`
 - `completion_rate = 1.0`
+- `distinct_2 = 0.9878`
+- `language_consistency_mean = 0.6667`
 - `loop_rate = 0.0`
 - `repeated_4gram_rate = 0.0`
+- `language_switch_rate_mean = 0.2500`
+- `length_closeness = 0.9355`
+If you want the higher-scoring exploratory preset from the sweep instead:
+- `creative`
+  - `temperature = 1.0`
+  - `top_k = 100`
+  - holdout score = `2.6369`
 Both `generation_config.json` and `recommended_decoding_params.json` are
 included in the repo.

generation_config.json CHANGED Viewed

@@ -1,7 +1,7 @@
 {
   "do_sample": true,
-  "temperature": 1.0,
-  "top_k": 100,
   "top_p": 0.95,
   "repetition_penalty": 1.1,
   "max_new_tokens": 64,

 {
   "do_sample": true,
+  "temperature": 0.8,
+  "top_k": 50,
   "top_p": 0.95,
   "repetition_penalty": 1.1,
   "max_new_tokens": 64,

recommended_decoding_params.json CHANGED Viewed

@@ -1,18 +1,20 @@
 {
-  "preset_name": "creative",
-  "selection_basis": "repo-native decoding search holdout winner for the official medium base release",
   "do_sample": true,
-  "temperature": 1.0,
-  "top_k": 100,
   "top_p": 0.95,
   "repetition_penalty": 1.1,
   "no_repeat_ngram_size": 0,
   "max_new_tokens": 64,
-  "holdout_score": 2.636900140224359,
-  "distinct_2": 0.9969230769230769,
   "loop_rate": 0.0,
   "repeated_4gram_rate": 0.0,
-  "language_consistency_mean": 0.625,
-  "language_switch_rate_mean": 0.0,
-  "length_closeness": 0.9436848958333334
 }

 {
+  "preset_name": "balanced",
+  "selection_basis": "public family-base default chosen conservatively; creative won the decoding-search holdout, but not by a large enough margin to override the balanced-default policy",
   "do_sample": true,
+  "temperature": 0.8,
+  "top_k": 50,
   "top_p": 0.95,
   "repetition_penalty": 1.1,
   "no_repeat_ngram_size": 0,
   "max_new_tokens": 64,
+  "holdout_score": 2.4656392911585368,
+  "distinct_2": 0.9878048780487805,
   "loop_rate": 0.0,
   "repeated_4gram_rate": 0.0,
+  "language_consistency_mean": 0.6666666666666666,
+  "language_switch_rate_mean": 0.25,
+  "length_closeness": 0.935546875,
+  "creative_holdout_score": 2.636900140224359,
+  "creative_minus_balanced_score_delta": 0.17126084906582216
 }