Instructions to use LoneStriker/Solstice-11B-v1-AWQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LoneStriker/Solstice-11B-v1-AWQ with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LoneStriker/Solstice-11B-v1-AWQ")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LoneStriker/Solstice-11B-v1-AWQ")
model = AutoModelForCausalLM.from_pretrained("LoneStriker/Solstice-11B-v1-AWQ", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LoneStriker/Solstice-11B-v1-AWQ with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LoneStriker/Solstice-11B-v1-AWQ"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LoneStriker/Solstice-11B-v1-AWQ",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/LoneStriker/Solstice-11B-v1-AWQ

SGLang

How to use LoneStriker/Solstice-11B-v1-AWQ with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LoneStriker/Solstice-11B-v1-AWQ" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LoneStriker/Solstice-11B-v1-AWQ",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LoneStriker/Solstice-11B-v1-AWQ" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LoneStriker/Solstice-11B-v1-AWQ",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use LoneStriker/Solstice-11B-v1-AWQ with Docker Model Runner:
```
docker model run hf.co/LoneStriker/Solstice-11B-v1-AWQ
```

Configuration Parsing Warning:In config.json: "quantization_config.modules_to_not_convert" must be an array

===== Solstice-11B-v1 =====

A model trained with the sole goal of NSFW.

That is it.

Results are to be as expected.

Finetuned off several instruct datasets that are NSFW. Example subset Below, other instruct datasets modified are private for now. Outputs were manually verified by me and two good friends.

Lewd-Assistant-v1 ---> Used a combination of Claude 2.0, GPT-4-Turbo and WinterGoddess-1.4x to reformat instead of a small 10B model [Fimbulvetr-v1] like the example dataset shown, which resulted in less errors and better answers. Private for now.

Ruled Out names:
Solarslut
Solascivious
Sultry
Sundress
Scorch

Prompt Format: Alpaca

There are several issues with the model, but this is an experimental one so :shrug:
----> May speak as {{user}} sometimes. I know what causes it. I kinda like it though that way.
----> May ramble on or give small outputs. Sampler settings.
----> May be a little inconsistent at times. Yeah its inevitable due to the nature of the data.
----> Steers towards NSFW --> As Expected.

GGUF: https://huggingface.co/Sao10K/Solstice-11B-v1-GGUF

Downloads last month: 5

Safetensors

Model size

11B params

Tensor type

I32

F16

LoneStriker
/

Solstice-11B-v1-AWQ

Dataset used to train LoneStriker/Solstice-11B-v1-AWQ