Instructions to use zerofata/L3.3-GeneticLemonade-Final-v2-70B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zerofata/L3.3-GeneticLemonade-Final-v2-70B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zerofata/L3.3-GeneticLemonade-Final-v2-70B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zerofata/L3.3-GeneticLemonade-Final-v2-70B")
model = AutoModelForCausalLM.from_pretrained("zerofata/L3.3-GeneticLemonade-Final-v2-70B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zerofata/L3.3-GeneticLemonade-Final-v2-70B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zerofata/L3.3-GeneticLemonade-Final-v2-70B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zerofata/L3.3-GeneticLemonade-Final-v2-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zerofata/L3.3-GeneticLemonade-Final-v2-70B

SGLang

How to use zerofata/L3.3-GeneticLemonade-Final-v2-70B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zerofata/L3.3-GeneticLemonade-Final-v2-70B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zerofata/L3.3-GeneticLemonade-Final-v2-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zerofata/L3.3-GeneticLemonade-Final-v2-70B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zerofata/L3.3-GeneticLemonade-Final-v2-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use zerofata/L3.3-GeneticLemonade-Final-v2-70B with Docker Model Runner:
```
docker model run hf.co/zerofata/L3.3-GeneticLemonade-Final-v2-70B
```

Template wrong?

by Hrre54353543 - opened Jun 6, 2025

Discussion

Hrre54353543

Jun 6, 2025

You suggest "Llama-3-Instruct-Names but you will need to uncheck "System same as user"." but if I use that template it just produces 1 sentence long stuff and not even talking had to switch back to the Llama-3.3-T4 for it to actually make several sentences and talk.

zerofata

Owner Jun 6, 2025

Don't know what the Llama-3.3-T4 template is (it isn't a default ST template), but any Llama 3 template will work.

Currently working on making this model a bit more steerable and coherent, but that probably won't be the cause of one liner responses. Check your system prompt, samplers and a few different characters / scenarios to try narrow down the issue.

Hrre54353543

Jun 6, 2025

I used the same settings as you suggested, although left system blank because there isn't any llama-3 instruct names template for system and it just didn't want to create long response nor talk. the Llama-3.3-t4 template is this one https://huggingface.co/sleepdeprived3/Llama-3.3-T4/tree/main

I'll try a few more times

zerofata

Owner Jun 6, 2025

Yep, just had a look at the one you linked and that'll work fine.

Llama-3-Instruct-Names template specifically you need to go in and uncheck the "System same as user" setting so that it enables the system role. Then it can be used as normal, I don't know why ST defaults to having it off but it is what it is. Without a system prompt / role the model will definitely underperform, as all the training I've done had system prompts in mind.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment