Instructions to use appvoid/palmer-003 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use appvoid/palmer-003 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="appvoid/palmer-003")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("appvoid/palmer-003")
model = AutoModelForCausalLM.from_pretrained("appvoid/palmer-003")

llama-cpp-python

How to use appvoid/palmer-003 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="appvoid/palmer-003",
	filename="palmer-003.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Inference
Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use appvoid/palmer-003 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf appvoid/palmer-003
# Run inference directly in the terminal:
llama-cli -hf appvoid/palmer-003

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf appvoid/palmer-003
# Run inference directly in the terminal:
llama-cli -hf appvoid/palmer-003

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf appvoid/palmer-003
# Run inference directly in the terminal:
./llama-cli -hf appvoid/palmer-003

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf appvoid/palmer-003
# Run inference directly in the terminal:
./build/bin/llama-cli -hf appvoid/palmer-003

Use Docker

docker model run hf.co/appvoid/palmer-003

LM Studio
Jan

vLLM

How to use appvoid/palmer-003 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "appvoid/palmer-003"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "appvoid/palmer-003",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/appvoid/palmer-003

SGLang

How to use appvoid/palmer-003 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "appvoid/palmer-003" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "appvoid/palmer-003",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "appvoid/palmer-003" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "appvoid/palmer-003",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use appvoid/palmer-003 with Ollama:
```
ollama run hf.co/appvoid/palmer-003
```

Unsloth Studio new

How to use appvoid/palmer-003 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for appvoid/palmer-003 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for appvoid/palmer-003 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for appvoid/palmer-003 to start chatting

Docker Model Runner
How to use appvoid/palmer-003 with Docker Model Runner:
```
docker model run hf.co/appvoid/palmer-003
```

Lemonade

How to use appvoid/palmer-003 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull appvoid/palmer-003

Run and chat with the model

lemonade run user.palmer-003-{{QUANT_TAG}}

List all available models

lemonade list

appvoid commited on Apr 18, 2024

Commit

91a4170

verified ·

1 Parent(s): a838589

Update README.md

Browse files

Files changed (1) hide show

README.md +10 -77

README.md CHANGED Viewed

@@ -9,8 +9,16 @@ tags:
 ---
 ![palmer-003 logo](https://huggingface.co/appvoid/palmer-002.5/resolve/main/003.png)
-Creative writing has never been so accesible, palmer goes beyond what it was thought about small language models. This model is a "MErging of Experts" (MEoE) using an internal model `palmer-003-2401` as base, biased as an assistant, using dpo technique, without using any prompts—as a result of these efforts—palmer is better than most 1b language models on most benchmarks, despite being sometimes 40% smaller than its counterparts.
 ```
   Model      MMLU     ARC-C    OBQA   HellaSwag  PIQA  Winogrande Average  Params
 tinyllama | 0.2577 | 0.3029 | 0.3600 | 0.5935 | 0.7329 | 0.5959 | 0.4738 | 1.1B  |
@@ -19,79 +27,4 @@ palmer    | 0.2523 | 0.3439 | 0.3740 | 0.6208 | 0.7524 | 0.6590 | 0.5004 | 1.1B
 qwen      | 0.4536 | 0.3490 | 0.3320 | 0.5876 | 0.7307 | 0.5896 | 0.5070 | 1.8B  |
 ```
-This work constitutes, given its compactness, an advancement towards SMLs, easily empowering edge devices such as mobile phones, raspberry pis and automated software/robots. Aditionally, palmer-003 follows the same philosophy as palmer-002.5 to become a more powerful model with more data instead of less.
-```
-prompt: Never give up.
-output: Keep pushing forward. Remember, you are not alone.
-You have the strength and the support of your family and friends.
-As you continue on your journey, remember to take care of yourself.
-Eat well, exercise regularly, and get enough sleep.
-Stay positive and focused on your goals.
-Remember, you are capable of achieving anything you set your mind to.
-```
-**Story writing**
-This model excels at story writting. In fact, depending on the prompt used, it can generate stories as random and credible such as IronMan and Sonic the Hedgehog fighting zombies at outer space.
-```
-prompt: how are you?
-output: [Suddenly, the door bursts open, and a group of armed men enter]
-Man: Who are you?
-Woman: We're here to help.
-[The group of men starts to attack them, but they manage to fend them off with their weapons]
-Man: You're not going to hurt us!
-                                                Woman: We're just trying to help.
-[As the group retreats, the woman and man are left alone in the room]
-Man: I don't know what to do.
-Woman: You need to stay calm. We have to find a way out of here.
-[They start to search for a way out, but they quickly realize that the door is locked]
-Man: What are we going to do?
-Woman: We'll have to think outside the box. Maybe there's another way out.
-[As they continue to search, they come across a hidden passageway]
-Man: This is it! We can make our way out of here.
-Woman: But how are we going to get back?
-Man: I have an idea. Let's use the key that was left behind.
-[They use the key to unlock the door, and they emerge into a dark alley]
-Man: We made it!
-Woman: But what about our families?
-Man: We'll find a way to let them know we're okay.
-[As they make their way back to their homes, they realize that they have been in a nightmare]
-Man: This is just the beginning. We need to be prepared for anything.
-Woman: I know. But what if it happens again?
-Man: Then we'll do everything in our power to protect ourselves and those we love.
-[The woman and man sit down on a bench, looking out at the city]
-Man: We can't let this happen again. We have to be vigilant and prepared for anything.
-Woman: I agree. But what if we don't know how to protect ourselves?
-Man: Then we'll have to rely on each other. We're in this together.
-[The woman and man continue to sit, holding hand
-```
-You can support me [through kofi](https://ko-fi.com/appvoid).
-### Important
-Keep in mind that if you quantize the fp16 gguf model to let's say q8 or q6 there is gonna be a extremely huge performance drop. So, if you want the most of this model, use it as it is, in fp16.

 ---
 ![palmer-003 logo](https://huggingface.co/appvoid/palmer-002.5/resolve/main/003.png)
+palmer-003 is appvoid's lastest effort into creating the best small, general language model possible.
+**Prompt**
+```
+Query: Your instruction
+Answer:
+```
+**Evaluation**
 ```
   Model      MMLU     ARC-C    OBQA   HellaSwag  PIQA  Winogrande Average  Params
 tinyllama | 0.2577 | 0.3029 | 0.3600 | 0.5935 | 0.7329 | 0.5959 | 0.4738 | 1.1B  |
 qwen      | 0.4536 | 0.3490 | 0.3320 | 0.5876 | 0.7307 | 0.5896 | 0.5070 | 1.8B  |
 ```
+You can support me [through kofi](https://ko-fi.com/appvoid).