Instructions to use ViorikaAI-org/MicroLlama-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ViorikaAI-org/MicroLlama-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ViorikaAI-org/MicroLlama-v2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ViorikaAI-org/MicroLlama-v2")
model = AutoModelForCausalLM.from_pretrained("ViorikaAI-org/MicroLlama-v2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ViorikaAI-org/MicroLlama-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ViorikaAI-org/MicroLlama-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ViorikaAI-org/MicroLlama-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ViorikaAI-org/MicroLlama-v2

SGLang

How to use ViorikaAI-org/MicroLlama-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ViorikaAI-org/MicroLlama-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ViorikaAI-org/MicroLlama-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ViorikaAI-org/MicroLlama-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ViorikaAI-org/MicroLlama-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ViorikaAI-org/MicroLlama-v2 with Docker Model Runner:
```
docker model run hf.co/ViorikaAI-org/MicroLlama-v2
```

MicroLlama-v2 / README.md

Dmitriy-Zemskov

Update README.md

90f9b73 verified about 5 hours ago

preview code

Raw

History Blame Contribute Delete

1.75 kB

metadata

language:
  - en
tags:
  - llama
  - tiny-lm
library_name: transformers
model_creator: ViorikaAI
pipeline_tag: text-generation
license: mit

MicroLlama-v2 (45M Parameters)

The second generation of the MicroLlama series. This is an ultra-compact, 45-million parameter language model trained from scratch and fine-tuned using Supervised Fine-Tuning (SFT) on the clean Alpaca dataset.

Model Features

Size: ~45M parameters — runs lightning-fast even on a potato.
Vocabulary: Custom LLaMA-based tokenizer.
Grammar Over Mind: Thanks to SFT, the model has pristine English grammar and strict formatting, though its world-knowledge is hilariously limited by its scale.
Lore: Has a strange obsession with cats, parks, and existential AI humor.

Example Dialogue

User: Tell me a joke.
Assistant: One example of an AI system could be that use AI to learn more about the potential consequences or disadvantages of humanity and what is expected? It can also cause data in both languages.

User: Write a sentence about a dog.
Assistant: A cake has an easy way to stay in the top of the mountain, but there are plenty of food that can have many benefits and resources available on people's favorite?

How To Use

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("ViorikaAI-org/MicroLlama-v2")
model = AutoModelForCausalLM.from_pretrained("ViorikaAI-org/MicroLlama-v2")

prompt = "user: How are you?\nassistant: <s> "
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0]))