Instructions to use athirdpath/Iambe-20b-DARE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use athirdpath/Iambe-20b-DARE with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="athirdpath/Iambe-20b-DARE")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("athirdpath/Iambe-20b-DARE")
model = AutoModelForCausalLM.from_pretrained("athirdpath/Iambe-20b-DARE")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use athirdpath/Iambe-20b-DARE with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "athirdpath/Iambe-20b-DARE"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "athirdpath/Iambe-20b-DARE",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/athirdpath/Iambe-20b-DARE

SGLang

How to use athirdpath/Iambe-20b-DARE with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "athirdpath/Iambe-20b-DARE" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "athirdpath/Iambe-20b-DARE",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "athirdpath/Iambe-20b-DARE" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "athirdpath/Iambe-20b-DARE",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use athirdpath/Iambe-20b-DARE with Docker Model Runner:
```
docker model run hf.co/athirdpath/Iambe-20b-DARE
```

Model comparison

by anmol989 - opened Nov 29, 2023

Discussion

anmol989

Nov 29, 2023

•

edited Nov 30, 2023

I wish there was a benchmark to test a model story writing capabilities automatically like there is a ayumi benchmark for roleplay

Self testing the model is always biased. I am using hestia model previously to generate my stories. At first i was happy since the model writing capabilities are really good

But i am not satisfied with the reasoning capability of hestia. I hope this model performs better!

Hestia vs Iambe-20b-DARE

anmol989

Nov 30, 2023

•

edited Nov 30, 2023

@athirdpath

The new model Iambe is not good for story writing. It follows instructions but generate rubbish and stupid scenarios. I don't know anything about model merging process but something went wrong and the noromaid unique writing style is not present in this model. The old model hestia is more creative and have a unique story writing style!

athirdpath

Owner Nov 30, 2023

I think the trimming away that this new DARE process introduces removed a lot of Noromaid's little human details. You helped me to pinpoint the shift, thank you, I appreciate it.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment