Instructions to use breadlicker45/discordLLama-460m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use breadlicker45/discordLLama-460m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="breadlicker45/discordLLama-460m")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("breadlicker45/discordLLama-460m")
model = AutoModelForCausalLM.from_pretrained("breadlicker45/discordLLama-460m")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use breadlicker45/discordLLama-460m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "breadlicker45/discordLLama-460m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "breadlicker45/discordLLama-460m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/breadlicker45/discordLLama-460m

SGLang

How to use breadlicker45/discordLLama-460m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "breadlicker45/discordLLama-460m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "breadlicker45/discordLLama-460m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "breadlicker45/discordLLama-460m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "breadlicker45/discordLLama-460m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use breadlicker45/discordLLama-460m with Docker Model Runner:
```
docker model run hf.co/breadlicker45/discordLLama-460m
```

discordLLama-460m

Commit History

Create README.md

461880d

breadlicker45 commited on Jun 22, 2023

Rename tokenizer_config (1).json to tokenizer_config.json

f72f032

breadlicker45 commited on Jun 22, 2023

Upload tokenizer_config (1).json

458a2f6

breadlicker45 commited on Jun 22, 2023

Rename tokenizer (6).json to tokenizer.json

83d029f

breadlicker45 commited on Jun 22, 2023

Upload tokenizer (6).json

284eecf

breadlicker45 commited on Jun 22, 2023

Rename pytorch_model (7).bin to pytorch_model.bin

c356227

breadlicker45 commited on Jun 22, 2023

Upload 2 files

62b2e09

breadlicker45 commited on Jun 22, 2023

initial commit

def4ff2

bread null commited on Jun 22, 2023

Commit History

Create README.md 461880d

Rename tokenizer_config (1).json to tokenizer_config.json f72f032

Upload tokenizer_config (1).json 458a2f6

Rename tokenizer (6).json to tokenizer.json 83d029f

Upload tokenizer (6).json 284eecf

Rename pytorch_model (7).bin to pytorch_model.bin c356227

Upload 2 files 62b2e09

initial commit def4ff2

Create README.md

461880d

Rename tokenizer_config (1).json to tokenizer_config.json

f72f032

Upload tokenizer_config (1).json

458a2f6

Rename tokenizer (6).json to tokenizer.json

83d029f

Upload tokenizer (6).json

284eecf

Rename pytorch_model (7).bin to pytorch_model.bin

c356227

Upload 2 files

62b2e09

initial commit

def4ff2