Instructions to use AscendKernelGen/KernelGen-LM-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AscendKernelGen/KernelGen-LM-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AscendKernelGen/KernelGen-LM-4B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AscendKernelGen/KernelGen-LM-4B")
model = AutoModelForCausalLM.from_pretrained("AscendKernelGen/KernelGen-LM-4B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use AscendKernelGen/KernelGen-LM-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AscendKernelGen/KernelGen-LM-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AscendKernelGen/KernelGen-LM-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AscendKernelGen/KernelGen-LM-4B

SGLang

How to use AscendKernelGen/KernelGen-LM-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AscendKernelGen/KernelGen-LM-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AscendKernelGen/KernelGen-LM-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AscendKernelGen/KernelGen-LM-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AscendKernelGen/KernelGen-LM-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use AscendKernelGen/KernelGen-LM-4B with Docker Model Runner:
```
docker model run hf.co/AscendKernelGen/KernelGen-LM-4B
```

KernelGen-LM-4B

Commit History

Update README.md

5596483
verified

AscendKernelGen commited on 11 days ago

Upload 2 files

54affd5
verified

AscendKernelGen commited on 11 days ago

Update README.md

8b92771
verified

AscendKernelGen commited on 12 days ago

Update README.md

48a3071
verified

AscendKernelGen commited on 12 days ago

Update README.md

72eb425
verified

AscendKernelGen commited on 15 days ago

Update README.md

ce78088
verified

AscendKernelGen commited on 16 days ago

Add metadata and improve model card (#1)

8fd37c8

AscendKernelGen

nielsr HF Staff commited on 19 days ago

Update README.md

cf040eb
verified

AscendKernelGen commited on Mar 13

Update README.md

8cc2c58
verified

AscendKernelGen commited on Mar 13

Update README.md

df3120f
verified

AscendKernelGen commited on Mar 13

Create README.md

b410043
verified

AscendKernelGen commited on Jan 26

Upload folder using huggingface_hub

98026a0
verified

AscendKernelGen commited on Jan 24

initial commit

db508d7
verified

AscendKernelGen commited on Jan 23

Commit History

Update README.md 5596483 verified

Upload 2 files 54affd5 verified

Update README.md 8b92771 verified

Update README.md 48a3071 verified

Update README.md 72eb425 verified

Update README.md ce78088 verified

Add metadata and improve model card (#1) 8fd37c8

Update README.md cf040eb verified

Update README.md 8cc2c58 verified

Update README.md df3120f verified

Create README.md b410043 verified

Upload folder using huggingface_hub 98026a0 verified

initial commit db508d7 verified

Update README.md

5596483
verified

Upload 2 files

54affd5
verified

Update README.md

8b92771
verified

Update README.md

48a3071
verified

Update README.md

72eb425
verified

Update README.md

ce78088
verified

Add metadata and improve model card (#1)

8fd37c8

Update README.md

cf040eb
verified

Update README.md

8cc2c58
verified

Update README.md

df3120f
verified

Create README.md

b410043
verified

Upload folder using huggingface_hub

98026a0
verified

initial commit

db508d7
verified