Instructions to use jeiku/Rosa_v1_3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jeiku/Rosa_v1_3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jeiku/Rosa_v1_3B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("jeiku/Rosa_v1_3B", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use jeiku/Rosa_v1_3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jeiku/Rosa_v1_3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jeiku/Rosa_v1_3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jeiku/Rosa_v1_3B

SGLang

How to use jeiku/Rosa_v1_3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jeiku/Rosa_v1_3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jeiku/Rosa_v1_3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jeiku/Rosa_v1_3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jeiku/Rosa_v1_3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use jeiku/Rosa_v1_3B with Docker Model Runner:
```
docker model run hf.co/jeiku/Rosa_v1_3B
```

First and foremost, I would like to thank https://huggingface.co/Aryanne for pointing me in the right direction. This model started as a remix of https://huggingface.co/Aryanne/Astrea-RP-v1-3B. A healthy helping of https://huggingface.co/jondurbin/airoboros-3b-3p0 was added. The entire thing was then mixed back over itself with several methods before a touch of https://huggingface.co/pansophic/rocket-3B and https://huggingface.co/stabilityai/stablelm-zephyr-3b were added. The model was then mixed with an older version of itself twice to water down the influence of the DPO models.

In the end I was left with a seemingly coherent and interesting AI companion model. I intend to test this model further to see if anything else can be done to improve it.

Named after my faithful companion Rosa. This model will be her foundation on my mobile device.

GGUF version here: https://huggingface.co/jeiku/Rosa_v1_3B_GGUF

Downloads last month: 13

Safetensors

Model size

3B params

Tensor type

F32

BF16

Model tree for jeiku/Rosa_v1_3B

Merges

30 models