Instructions to use Lite-Coder/LiteCoder-Terminal-30b-a3b-sft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Lite-Coder/LiteCoder-Terminal-30b-a3b-sft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Lite-Coder/LiteCoder-Terminal-30b-a3b-sft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Lite-Coder/LiteCoder-Terminal-30b-a3b-sft")
model = AutoModelForCausalLM.from_pretrained("Lite-Coder/LiteCoder-Terminal-30b-a3b-sft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Lite-Coder/LiteCoder-Terminal-30b-a3b-sft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Lite-Coder/LiteCoder-Terminal-30b-a3b-sft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Lite-Coder/LiteCoder-Terminal-30b-a3b-sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Lite-Coder/LiteCoder-Terminal-30b-a3b-sft

SGLang

How to use Lite-Coder/LiteCoder-Terminal-30b-a3b-sft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Lite-Coder/LiteCoder-Terminal-30b-a3b-sft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Lite-Coder/LiteCoder-Terminal-30b-a3b-sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Lite-Coder/LiteCoder-Terminal-30b-a3b-sft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Lite-Coder/LiteCoder-Terminal-30b-a3b-sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Lite-Coder/LiteCoder-Terminal-30b-a3b-sft with Docker Model Runner:
```
docker model run hf.co/Lite-Coder/LiteCoder-Terminal-30b-a3b-sft
```

LiteCoder-Terminal-30b-a3b-sft

Commit History

Update README.md

8d326f6
verified

Lite-Coder commited on 6 days ago

Update README.md

65e8941
verified

Lite-Coder commited on 6 days ago

Update README.md

a04f9e9
verified

Lite-Coder commited on 6 days ago

Update README.md

9aa54ae
verified

Lite-Coder commited on 6 days ago

Add library name, pipeline tag and links to paper/code (#1)

ef4daed

Lite-Coder

nielsr HF Staff commited on 6 days ago

Update README.md

11142cb
verified

Lite-Coder commited on 9 days ago

Update README.md

1b225c5
verified

Lite-Coder commited on Apr 20

Upload folder using huggingface_hub

7c1ef9e
verified

Lite-Coder commited on Apr 13

Upload folder using huggingface_hub

030e687
verified

Lite-Coder commited on Apr 12

initial commit

b82954b
verified

Lite-Coder commited on Apr 12

Commit History

Update README.md 8d326f6 verified

Update README.md 65e8941 verified

Update README.md a04f9e9 verified

Update README.md 9aa54ae verified

Add library name, pipeline tag and links to paper/code (#1) ef4daed

Update README.md 11142cb verified

Update README.md 1b225c5 verified

Upload folder using huggingface_hub 7c1ef9e verified

Upload folder using huggingface_hub 030e687 verified

initial commit b82954b verified

Update README.md

8d326f6
verified

Update README.md

65e8941
verified

Update README.md

a04f9e9
verified

Update README.md

9aa54ae
verified

Add library name, pipeline tag and links to paper/code (#1)

ef4daed

Update README.md

11142cb
verified

Update README.md

1b225c5
verified

Upload folder using huggingface_hub

7c1ef9e
verified

Upload folder using huggingface_hub

030e687
verified

initial commit

b82954b
verified