Instructions to use Nanbeige/Nanbeige4.2-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanbeige/Nanbeige4.2-3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanbeige/Nanbeige4.2-3B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Nanbeige/Nanbeige4.2-3B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Nanbeige/Nanbeige4.2-3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanbeige/Nanbeige4.2-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanbeige/Nanbeige4.2-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nanbeige/Nanbeige4.2-3B

SGLang

How to use Nanbeige/Nanbeige4.2-3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanbeige/Nanbeige4.2-3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanbeige/Nanbeige4.2-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanbeige/Nanbeige4.2-3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanbeige/Nanbeige4.2-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Nanbeige/Nanbeige4.2-3B with Docker Model Runner:
```
docker model run hf.co/Nanbeige/Nanbeige4.2-3B
```

Suitability for 8 GB graphics cards

#18

by thermi6 - opened 6 days ago

Discussion

thermi6

6 days ago

Thank you so much for your excellent model! It is very useful and usable for small contexts on 8 GB graphics cards. However, as stated in a now closed discussion, the KV cache increases in size very quickly. Then it is not possible anymore to keep the data in VRAM.

Would it be possible to aim at severly limiting the size increase of the KV cache in the near future?

Right now this works on 8 GB cards: 4 bit quant + 32768 tokens context length. Any higher and it is too large for VRAM.

I think you would agree that just 32768 tokens is a bit small. What is your opinion on the matter? Do you have any pointers for improving the situation with the current model?

leran1995

Nanbeige LLM Lab org 5 days ago

Hi, thank you for the kind words and for the detailed report!
Both improving performance and reducing KV cache are on our roadmap. As mentioned in the earlier discussion, we did try sharing the KV cache across loop passes, but it noticeably hurt performance, so we kept the full cache in Nanbeige4.2. Going forward, we are exploring Linear / Sparse Attention to bring the long-context memory cost down more fundamentally. Stay tuned!

thermi6

5 days ago

This comment has been hidden (marked as Low Quality)

thermi6

5 days ago

•

edited 5 days ago

For information, this is what I get with an RTX 5060 that has 8 GB of VRAM:

first prompt is 63.07 to 53.67 t/s

Second prompt then 20.56 to 20.20 t/s

Third prompt is 16.74 to 16.00 t/s

These are the llama.cpp settings (I have a special entrypoint):

KV_TYPE_K=q4_0
KV_TYPE_V=q4_0
KV_OFFLOAD=1
MAX_CTX=65536

Entrypoint:

#!/usr/bin/env bash
set -euo pipefail

MODEL_DIR="${MODEL_DIR:-/models/target}"
HOST="${HOST:-0.0.0.0}"
PORT="${PORT:-8080}"
NUM_THREADS="${NUM_THREADS:-6}"
GPU_LAYERS="${GPU_LAYERS:-99}"
MAX_CTX="${MAX_CTX:-16384}"
TARGET_FILE="${TARGET_FILE:-nanbeige-3b-q4_k_m.gguf}"
# KV cache: use FA-supported CUDA types (q4_0/q8_0/f16). iq4_nl is NOT in
# flash-attn kernels and burns CPU. q4_0/q4_0 is FA-native and VRAM-cheap.
KV_TYPE_K="${KV_TYPE_K:-q4_0}"
KV_TYPE_V="${KV_TYPE_V:-q4_0}"
# KV placement. Nanbeige4.2-3B loops layers (num_loops=2 -> 44 KV layers),
# so full 262k KV is ~18 GiB even quantized and cannot fit 8 GB VRAM.
# KV_OFFLOAD=0 keeps weights on GPU but stores KV in system RAM, enabling
# full 262144-token context (~2.5 GiB VRAM total, ~20 tok/s decode).
# KV_OFFLOAD=1 (default) keeps KV in VRAM: fastest, ctx up to ~64k.
KV_OFFLOAD="${KV_OFFLOAD:-1}"

KV_ARGS=()
if [ "${KV_OFFLOAD}" = "0" ]; then
  KV_ARGS+=(--no-kv-offload)
fi

echo "==> Starting llama-server on ${HOST}:${PORT}"
echo "    Model: ${MODEL_DIR}/${TARGET_FILE}"
echo "    GPU layers: ${GPU_LAYERS}"
echo "    Context: ${MAX_CTX}"

exec /opt/llama/build/bin/llama-server \
  -m "${MODEL_DIR}/${TARGET_FILE}" \
  --host "${HOST}" \
  --port "${PORT}" \
  --alias "Nanbeige4.2-3B" \
  --threads "${NUM_THREADS}" \
  --gpu-layers "${GPU_LAYERS}" \
  --ctx-size "${MAX_CTX}" \
  --cache-type-k "${KV_TYPE_K}" \
  --cache-type-v "${KV_TYPE_V}" \
  --flash-attn on \
  --no-mmap \
  "${KV_ARGS[@]}" \
  --temp 0.7 \
  --min-p 0.05 \
  --parallel 1 \
  "$@"

2026-07-27 15:41:34.475 | ==> Starting llama-server on 0.0.0.0:8080
2026-07-27 15:41:34.475 |     Model: /models/target/nanbeige-3b-q4_k_m.gguf
2026-07-27 15:41:34.475 |     GPU layers: 99
2026-07-27 15:41:34.475 |     Context: 65536
2026-07-27 15:41:34.724 | 0.00.109.328 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
2026-07-27 15:41:34.946 | 0.00.330.965 W srv  llama_server: -----------------
2026-07-27 15:41:34.946 | 0.00.330.987 W srv  llama_server: CORS is set to allow all origins ('*') and no API key is set
2026-07-27 15:41:34.946 | 0.00.330.988 W srv  llama_server: this can be a security risk (cross-origin attacks)
2026-07-27 15:41:34.946 | 0.00.330.988 W srv  llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
2026-07-27 15:41:34.946 | 0.00.330.989 W srv  llama_server: -----------------
2026-07-27 15:41:34.949 | 0.00.333.799 I srv    load_model: loading model '/models/target/nanbeige-3b-q4_k_m.gguf'
2026-07-27 15:41:36.804 | 0.02.189.293 I srv    load_model: initializing, n_slots = 1, n_ctx_slot = 65536, kv_unified = 'false'
2026-07-27 15:41:36.833 | 0.02.217.434 I srv          init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
2026-07-27 15:41:36.833 | 0.02.217.516 I srv  llama_server: model loaded
2026-07-27 15:41:36.833 | 0.02.217.522 I srv  llama_server: listening on http://0.0.0.0:8080
2026-07-27 15:42:17.132 | 0.42.516.647 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
2026-07-27 15:42:17.137 | 0.42.521.572 I slot launch_slot_: id  0 | task 0 | processing task, is_child = 0
2026-07-27 15:42:20.840 | 0.46.224.559 I slot print_timing: id  0 | task 0 | prompt processing, n_tokens =   8192, progress = 0.89, t =   3.70 s / 2212.30 tokens per second
2026-07-27 15:42:22.193 | 0.47.577.897 I slot print_timing: id  0 | task 0 | prompt eval time =    4630.91 ms /  9221 tokens (    0.50 ms per token,  1991.19 tokens per second)
2026-07-27 15:42:22.193 | 0.47.577.925 I slot print_timing: id  0 | task 0 |        eval time =     425.33 ms /    10 tokens (   42.53 ms per token,    23.51 tokens per second)
2026-07-27 15:42:22.197 | 0.47.577.926 I slot print_timing: id  0 | task 0 |       total time =    5056.24 ms /  9231 tokens
2026-07-27 15:42:22.197 | 0.47.577.936 I slot print_timing: id  0 | task 0 |    graphs reused =          9
2026-07-27 15:42:22.197 | 0.47.579.880 I slot      release: id  0 | task 0 | stop processing: n_tokens = 9230, truncated = 0
2026-07-27 15:42:22.291 | 0.47.676.175 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 47668802366
2026-07-27 15:42:22.624 | 0.48.008.540 I slot launch_slot_: id  0 | task 15 | processing task, is_child = 0
2026-07-27 15:42:25.759 | 0.51.144.099 I slot print_timing: id  0 | task 15 | n_decoded =    190, tg =  63.07 t/s, tg_3s =  63.07 t/s
2026-07-27 15:42:28.759 | 0.54.144.246 I slot print_timing: id  0 | task 15 | n_decoded =    371, tg =  61.70 t/s, tg_3s =  60.33 t/s
2026-07-27 15:42:31.774 | 0.57.158.827 I slot print_timing: id  0 | task 15 | n_decoded =    550, tg =  60.92 t/s, tg_3s =  59.36 t/s
2026-07-27 15:42:34.779 | 1.00.164.159 I slot print_timing: id  0 | task 15 | n_decoded =    723, tg =  60.08 t/s, tg_3s =  57.56 t/s
2026-07-27 15:42:37.787 | 1.03.172.296 I slot print_timing: id  0 | task 15 | n_decoded =    888, tg =  59.03 t/s, tg_3s =  54.85 t/s
2026-07-27 15:42:40.798 | 1.06.182.792 I slot print_timing: id  0 | task 15 | n_decoded =   1051, tg =  58.22 t/s, tg_3s =  54.14 t/s
2026-07-27 15:42:43.804 | 1.09.188.780 I slot print_timing: id  0 | task 15 | n_decoded =   1211, tg =  57.51 t/s, tg_3s =  53.23 t/s
2026-07-27 15:42:46.823 | 1.12.207.956 I slot print_timing: id  0 | task 15 | n_decoded =   1365, tg =  56.69 t/s, tg_3s =  51.01 t/s
2026-07-27 15:42:49.835 | 1.15.219.372 I slot print_timing: id  0 | task 15 | n_decoded =   1517, tg =  56.00 t/s, tg_3s =  50.47 t/s
2026-07-27 15:42:52.849 | 1.18.234.100 I slot print_timing: id  0 | task 15 | n_decoded =   1659, tg =  55.11 t/s, tg_3s =  47.10 t/s
2026-07-27 15:42:55.862 | 1.21.247.317 I slot print_timing: id  0 | task 15 | n_decoded =   1799, tg =  54.32 t/s, tg_3s =  46.46 t/s
2026-07-27 15:42:58.871 | 1.24.256.270 I slot print_timing: id  0 | task 15 | n_decoded =   1939, tg =  53.67 t/s, tg_3s =  46.53 t/s
2026-07-27 15:43:00.533 | 1.25.918.113 I slot print_timing: id  0 | task 15 | prompt eval time =     122.91 ms /   302 tokens (    0.41 ms per token,  2457.10 tokens per second)
2026-07-27 15:43:00.533 | 1.25.918.138 I slot print_timing: id  0 | task 15 |        eval time =   37787.78 ms /  2013 tokens (   18.77 ms per token,    53.27 tokens per second)
2026-07-27 15:43:00.533 | 1.25.918.139 I slot print_timing: id  0 | task 15 |       total time =   37910.68 ms /  2315 tokens
2026-07-27 15:43:00.533 | 1.25.918.140 I slot print_timing: id  0 | task 15 |    graphs reused =       2012
2026-07-27 15:43:00.533 | 1.25.918.213 I slot      release: id  0 | task 15 | stop processing: n_tokens = 2317, truncated = 0
2026-07-27 15:43:14.384 | 1.39.769.045 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 47707141849
2026-07-27 15:43:14.610 | 1.39.994.788 I slot launch_slot_: id  0 | task 2029 | processing task, is_child = 0
2026-07-27 15:43:20.352 | 1.45.736.595 I slot print_timing: id  0 | task 2029 | n_decoded =    100, tg =  20.56 t/s, tg_3s =  20.56 t/s
2026-07-27 15:43:23.364 | 1.48.748.861 I slot print_timing: id  0 | task 2029 | n_decoded =    161, tg =  20.44 t/s, tg_3s =  20.25 t/s
2026-07-27 15:43:26.385 | 1.51.770.126 I slot print_timing: id  0 | task 2029 | n_decoded =    222, tg =  20.37 t/s, tg_3s =  20.19 t/s
2026-07-27 15:43:29.398 | 1.54.782.486 I slot print_timing: id  0 | task 2029 | n_decoded =    283, tg =  20.35 t/s, tg_3s =  20.25 t/s
2026-07-27 15:43:32.441 | 1.57.825.916 I slot print_timing: id  0 | task 2029 | n_decoded =    345, tg =  20.35 t/s, tg_3s =  20.36 t/s
2026-07-27 15:43:35.477 | 2.00.861.730 I slot print_timing: id  0 | task 2029 | n_decoded =    405, tg =  20.26 t/s, tg_3s =  19.76 t/s
2026-07-27 15:43:38.512 | 2.03.896.819 I slot print_timing: id  0 | task 2029 | n_decoded =    465, tg =  20.20 t/s, tg_3s =  19.77 t/s
2026-07-27 15:43:41.205 | 2.06.589.738 I slot print_timing: id  0 | task 2029 | prompt eval time =     878.09 ms /  1184 tokens (    0.74 ms per token,  1348.39 tokens per second)
2026-07-27 15:43:41.205 | 2.06.589.760 I slot print_timing: id  0 | task 2029 |        eval time =   25717.86 ms /   518 tokens (   49.65 ms per token,    20.14 tokens per second)
2026-07-27 15:43:41.205 | 2.06.589.761 I slot print_timing: id  0 | task 2029 |       total time =   26595.95 ms /  1702 tokens
2026-07-27 15:43:41.205 | 2.06.589.762 I slot print_timing: id  0 | task 2029 |    graphs reused =       2526
2026-07-27 15:43:41.206 | 2.06.590.824 I slot      release: id  0 | task 2029 | stop processing: n_tokens = 10922, truncated = 0
2026-07-27 15:43:47.454 | 2.12.838.697 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.756 (> 0.100 thold), f_keep = 0.950
2026-07-27 15:43:47.455 | 2.12.839.990 I slot launch_slot_: id  0 | task 2548 | processing task, is_child = 0
2026-07-27 15:43:56.120 | 2.21.504.497 I slot print_timing: id  0 | task 2548 | n_decoded =    100, tg =  16.74 t/s, tg_3s =  16.74 t/s
2026-07-27 15:43:59.158 | 2.24.543.090 I slot print_timing: id  0 | task 2548 | n_decoded =    149, tg =  16.53 t/s, tg_3s =  16.13 t/s
2026-07-27 15:44:02.170 | 2.27.554.994 I slot print_timing: id  0 | task 2548 | n_decoded =    198, tg =  16.46 t/s, tg_3s =  16.26 t/s
2026-07-27 15:44:05.212 | 2.30.596.839 I slot print_timing: id  0 | task 2548 | n_decoded =    248, tg =  16.46 t/s, tg_3s =  16.44 t/s
2026-07-27 15:44:08.241 | 2.33.626.301 I slot print_timing: id  0 | task 2548 | n_decoded =    297, tg =  16.41 t/s, tg_3s =  16.17 t/s
2026-07-27 15:44:11.285 | 2.36.670.274 I slot print_timing: id  0 | task 2548 | n_decoded =    348, tg =  16.46 t/s, tg_3s =  16.75 t/s
2026-07-27 15:44:14.310 | 2.39.695.074 I slot print_timing: id  0 | task 2548 | n_decoded =    398, tg =  16.47 t/s, tg_3s =  16.53 t/s
2026-07-27 15:44:17.325 | 2.42.709.415 I slot print_timing: id  0 | task 2548 | n_decoded =    448, tg =  16.48 t/s, tg_3s =  16.59 t/s
2026-07-27 15:44:20.370 | 2.45.754.432 I slot print_timing: id  0 | task 2548 | n_decoded =    499, tg =  16.51 t/s, tg_3s =  16.75 t/s
2026-07-27 15:44:23.402 | 2.48.786.718 I slot print_timing: id  0 | task 2548 | n_decoded =    550, tg =  16.54 t/s, tg_3s =  16.82 t/s
2026-07-27 15:44:26.436 | 2.51.820.758 I slot print_timing: id  0 | task 2548 | n_decoded =    601, tg =  16.56 t/s, tg_3s =  16.81 t/s
2026-07-27 15:44:29.457 | 2.54.841.572 I slot print_timing: id  0 | task 2548 | n_decoded =    651, tg =  16.56 t/s, tg_3s =  16.55 t/s
2026-07-27 15:44:32.464 | 2.57.848.472 I slot print_timing: id  0 | task 2548 | n_decoded =    701, tg =  16.56 t/s, tg_3s =  16.62 t/s
2026-07-27 15:44:35.484 | 3.00.868.780 I slot print_timing: id  0 | task 2548 | n_decoded =    751, tg =  16.56 t/s, tg_3s =  16.55 t/s
2026-07-27 15:44:38.517 | 3.03.901.826 I slot print_timing: id  0 | task 2548 | n_decoded =    800, tg =  16.54 t/s, tg_3s =  16.16 t/s
2026-07-27 15:44:41.537 | 3.06.922.036 I slot print_timing: id  0 | task 2548 | n_decoded =    849, tg =  16.52 t/s, tg_3s =  16.22 t/s
2026-07-27 15:44:44.595 | 3.09.979.561 I slot print_timing: id  0 | task 2548 | n_decoded =    899, tg =  16.51 t/s, tg_3s =  16.35 t/s
2026-07-27 15:44:47.622 | 3.13.007.161 I slot print_timing: id  0 | task 2548 | n_decoded =    948, tg =  16.49 t/s, tg_3s =  16.18 t/s
2026-07-27 15:44:50.622 | 3.16.007.288 I slot print_timing: id  0 | task 2548 | n_decoded =    997, tg =  16.48 t/s, tg_3s =  16.33 t/s
2026-07-27 15:44:53.625 | 3.19.009.611 I slot print_timing: id  0 | task 2548 | n_decoded =   1046, tg =  16.48 t/s, tg_3s =  16.32 t/s
2026-07-27 15:44:56.653 | 3.22.037.637 I slot print_timing: id  0 | task 2548 | n_decoded =   1095, tg =  16.46 t/s, tg_3s =  16.18 t/s
2026-07-27 15:44:59.679 | 3.25.064.027 I slot print_timing: id  0 | task 2548 | n_decoded =   1144, tg =  16.45 t/s, tg_3s =  16.19 t/s
2026-07-27 15:45:02.709 | 3.28.094.263 I slot print_timing: id  0 | task 2548 | n_decoded =   1193, tg =  16.44 t/s, tg_3s =  16.16 t/s
2026-07-27 15:45:05.765 | 3.31.149.947 I slot print_timing: id  0 | task 2548 | n_decoded =   1242, tg =  16.42 t/s, tg_3s =  16.04 t/s
2026-07-27 15:45:08.809 | 3.34.194.253 I slot print_timing: id  0 | task 2548 | n_decoded =   1290, tg =  16.40 t/s, tg_3s =  15.77 t/s
2026-07-27 15:45:11.836 | 3.37.220.667 I slot print_timing: id  0 | task 2548 | n_decoded =   1339, tg =  16.39 t/s, tg_3s =  16.19 t/s
2026-07-27 15:45:14.888 | 3.40.272.951 I slot print_timing: id  0 | task 2548 | n_decoded =   1388, tg =  16.38 t/s, tg_3s =  16.05 t/s
2026-07-27 15:45:17.910 | 3.43.294.438 I slot print_timing: id  0 | task 2548 | n_decoded =   1436, tg =  16.36 t/s, tg_3s =  15.89 t/s
2026-07-27 15:45:20.960 | 3.46.344.568 I slot print_timing: id  0 | task 2548 | n_decoded =   1484, tg =  16.34 t/s, tg_3s =  15.74 t/s
2026-07-27 15:45:23.984 | 3.49.368.602 I slot print_timing: id  0 | task 2548 | n_decoded =   1532, tg =  16.33 t/s, tg_3s =  15.87 t/s
2026-07-27 15:45:27.007 | 3.52.391.406 I slot print_timing: id  0 | task 2548 | n_decoded =   1580, tg =  16.31 t/s, tg_3s =  15.88 t/s
2026-07-27 15:45:30.055 | 3.55.439.577 I slot print_timing: id  0 | task 2548 | n_decoded =   1628, tg =  16.29 t/s, tg_3s =  15.75 t/s
2026-07-27 15:45:33.105 | 3.58.490.110 I slot print_timing: id  0 | task 2548 | n_decoded =   1675, tg =  16.27 t/s, tg_3s =  15.40 t/s
2026-07-27 15:45:36.128 | 4.01.512.536 I slot print_timing: id  0 | task 2548 | n_decoded =   1722, tg =  16.25 t/s, tg_3s =  15.55 t/s
2026-07-27 15:45:39.132 | 4.04.517.004 I slot print_timing: id  0 | task 2548 | n_decoded =   1768, tg =  16.22 t/s, tg_3s =  15.31 t/s
2026-07-27 15:45:42.134 | 4.07.518.822 I slot print_timing: id  0 | task 2548 | n_decoded =   1814, tg =  16.20 t/s, tg_3s =  15.32 t/s
2026-07-27 15:45:45.197 | 4.10.581.377 I slot print_timing: id  0 | task 2548 | n_decoded =   1861, tg =  16.17 t/s, tg_3s =  15.35 t/s
2026-07-27 15:45:48.199 | 4.13.584.098 I slot print_timing: id  0 | task 2548 | n_decoded =   1907, tg =  16.15 t/s, tg_3s =  15.32 t/s
2026-07-27 15:45:51.248 | 4.16.633.218 I slot print_timing: id  0 | task 2548 | n_decoded =   1954, tg =  16.13 t/s, tg_3s =  15.41 t/s
2026-07-27 15:45:54.279 | 4.19.663.478 I slot print_timing: id  0 | task 2548 | n_decoded =   2000, tg =  16.11 t/s, tg_3s =  15.18 t/s
2026-07-27 15:45:57.328 | 4.22.713.223 I slot print_timing: id  0 | task 2548 | n_decoded =   2047, tg =  16.09 t/s, tg_3s =  15.41 t/s
2026-07-27 15:46:00.341 | 4.25.725.908 I slot print_timing: id  0 | task 2548 | n_decoded =   2094, tg =  16.08 t/s, tg_3s =  15.60 t/s
2026-07-27 15:46:03.376 | 4.28.761.002 I slot print_timing: id  0 | task 2548 | n_decoded =   2141, tg =  16.07 t/s, tg_3s =  15.48 t/s
2026-07-27 15:46:06.390 | 4.31.774.631 I slot print_timing: id  0 | task 2548 | n_decoded =   2187, tg =  16.05 t/s, tg_3s =  15.26 t/s
2026-07-27 15:46:09.397 | 4.34.781.966 I slot print_timing: id  0 | task 2548 | n_decoded =   2233, tg =  16.04 t/s, tg_3s =  15.30 t/s
2026-07-27 15:46:12.403 | 4.37.787.752 I slot print_timing: id  0 | task 2548 | n_decoded =   2279, tg =  16.02 t/s, tg_3s =  15.30 t/s
2026-07-27 15:46:15.447 | 4.40.831.666 I slot print_timing: id  0 | task 2548 | n_decoded =   2325, tg =  16.00 t/s, tg_3s =  15.11 t/s

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment