Instructions to use tharaka-axonect/nemotron-testing with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tharaka-axonect/nemotron-testing with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tharaka-axonect/nemotron-testing", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tharaka-axonect/nemotron-testing", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("tharaka-axonect/nemotron-testing", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use tharaka-axonect/nemotron-testing with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tharaka-axonect/nemotron-testing with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tharaka-axonect/nemotron-testing"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tharaka-axonect/nemotron-testing",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tharaka-axonect/nemotron-testing

SGLang

How to use tharaka-axonect/nemotron-testing with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tharaka-axonect/nemotron-testing" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tharaka-axonect/nemotron-testing",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tharaka-axonect/nemotron-testing" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tharaka-axonect/nemotron-testing",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tharaka-axonect/nemotron-testing with Docker Model Runner:
```
docker model run hf.co/tharaka-axonect/nemotron-testing
```

tharaka-axonect commited on 13 days ago

Commit

d47859e

verified ·

1 Parent(s): 1d17552

Upload folder using huggingface_hub

Browse files

Files changed (18) hide show

.gitattributes +0 -2
models/model-00001-of-00013.safetensors +3 -0
models/model-00002-of-00013.safetensors +3 -0
models/model-00003-of-00013.safetensors +3 -0
models/model-00004-of-00013.safetensors +3 -0
models/model-00005-of-00013.safetensors +3 -0
models/model-00006-of-00013.safetensors +3 -0
models/model-00007-of-00013.safetensors +3 -0
models/model-00008-of-00013.safetensors +3 -0
models/model-00009-of-00013.safetensors +3 -0
models/model-00010-of-00013.safetensors +3 -0
models/model-00011-of-00013.safetensors +3 -0
models/model-00012-of-00013.safetensors +3 -0
models/model-00013-of-00013.safetensors +3 -0
models/tokenizer.json +3 -0
training_data/test.jsonl +3 -0
training_data/train.jsonl +3 -0
training_data/validation.jsonl +3 -0

.gitattributes CHANGED Viewed

@@ -38,5 +38,3 @@ models/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 training_data/train.jsonl filter=lfs diff=lfs merge=lfs -text
 training_data/validation.jsonl filter=lfs diff=lfs merge=lfs -text
 training_data/test.jsonl filter=lfs diff=lfs merge=lfs -text
-.env/lib/python3.13/site-packages/pip/_vendor/rich/__pycache__/console.cpython-313.pyc filter=lfs diff=lfs merge=lfs -text
-.env/lib/python3.13/site-packages/yaml/_yaml.cpython-313-x86_64-linux-gnu.so filter=lfs diff=lfs merge=lfs -text

 training_data/train.jsonl filter=lfs diff=lfs merge=lfs -text
 training_data/validation.jsonl filter=lfs diff=lfs merge=lfs -text
 training_data/test.jsonl filter=lfs diff=lfs merge=lfs -text

models/model-00001-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d17e471fda392732090bd0330b06a51465023983ec17b7460b88ab50cf00915f
+size 4991204552

models/model-00002-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:94b2fc0297347fb20381424f0038e941bb606358b02d78c27ecbbc260ae306b7
+size 4992601016

models/model-00003-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6ef5bdacf48aedaff4d0ed05c00bcf367180408a7e7c7d15001af912af127ce9
+size 4992601360

models/model-00004-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d7d62252cd974b2295a77abe4f8ea90b8459feda646fcac3b6429d53d9ee94a3
+size 4995692800

models/model-00005-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:09021c81aed0cd75881a6b38662d9f2fcd43e15bdd3e856aca44f5dc0139cefc
+size 4980545528

models/model-00006-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c1a8efb5dfe878ddb2a2f3244df4bb2798e8d8e91f4568dfc5a5506624855a4a
+size 4999409808

models/model-00007-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d01f71e7ca548477e40e4c791c57ce84d0c61ec4a720347c7a865644386cb1ba
+size 4992601488

models/model-00008-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:35d2dea9bf03b767aadc05189dcbfc50226131fb577534e273da4ff92b6d1a43
+size 4992601520

models/model-00009-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3ff1e87f41641445950958a89c148c8eb0f7aba3f976ca2e397a5aa6c76a69ad
+size 4995692800

models/model-00010-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:454d39c8593b155a34bad393e8a06c2c8f376c4ed264f22938f9237d94af1a6b
+size 4992601520

models/model-00011-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e9c1d00c97f8b98f0dfd944628dc4a72fdebeffb1a56b8f1b084213ff3a439fd
+size 4995692800

models/model-00012-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b5b240a924aeac719e6601ae901947f6e86daf0bccfe03297d055aff1b9039ae
+size 4995692816

models/model-00013-of-00013.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a8a9f55b66a86adeb0555c386a7adfd50f3678965263b18eb623ab81fa02a038
+size 3239751000

models/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:623c34567aebb18582765289fbe23d901c62704d6518d71866e0e58db892b5b7
+size 17077484

training_data/test.jsonl ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b98a255536fe6ad0d164854fe2bcc272d07950eb6478a476b1800a83f800fe30
+size 21237514

training_data/train.jsonl ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bb3376297314a02a46a92df61609455047bbd23e503c05034881f7845f22eba7
+size 5084230127

training_data/validation.jsonl ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cb6f7f0b19e53d3921a23fb85bcf75320c277be6a3dbc8b5108a30d142e069ba
+size 11035096