Instructions to use Geralt-Targaryen/FantasyGPT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Geralt-Targaryen/FantasyGPT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Geralt-Targaryen/FantasyGPT")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Geralt-Targaryen/FantasyGPT")
model = AutoModelForCausalLM.from_pretrained("Geralt-Targaryen/FantasyGPT", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Geralt-Targaryen/FantasyGPT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Geralt-Targaryen/FantasyGPT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Geralt-Targaryen/FantasyGPT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Geralt-Targaryen/FantasyGPT

SGLang

How to use Geralt-Targaryen/FantasyGPT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Geralt-Targaryen/FantasyGPT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Geralt-Targaryen/FantasyGPT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Geralt-Targaryen/FantasyGPT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Geralt-Targaryen/FantasyGPT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Geralt-Targaryen/FantasyGPT with Docker Model Runner:
```
docker model run hf.co/Geralt-Targaryen/FantasyGPT
```

YAML Metadata Error:"widget[0]" must be of type object

YAML Metadata Error:"widget[1]" must be of type object

YAML Metadata Error:"widget[2]" must be of type object

YAML Metadata Error:"widget[3]" must be of type object

YAML Metadata Error:"widget[4]" must be of type object

YAML Metadata Error:"widget[5]" must be of type object

YAML Metadata Error:"widget[6]" must be of type object

GPT-2 (including tokenizer) trained from scratch on some of my favorite books (about 31M words in total).

It's only trained on an RTX 3090 for three hours, so don't take it seriously, just have fun!

peak lr: 4e-4
global batch size: 32
weight decay: 0.01
training steps: 25k
warmup steps: 1k
lr decay: cosine

Example usage:

from transformers import AutoTokenizer, GPT2LMHeadModel

tokenizer = AutoTokenizer.from_pretrained('Geralt-Targaryen/FantasyGPT')
model = GPT2LMHeadModel.from_pretrained('Geralt-Targaryen/FantasyGPT')

input_text = ["Daenerys kissed Gandalf, as the witcher hacked off Lord Voldemort's head with a brutal swing of Longclaw."]
input_tokenized = tokenizer(input_text, return_tensors='pt')

output = model.generate(inputs=input_tokenized.input_ids, max_new_tokens=256, do_sample=True, top_p=0.9, temperature=1)

print(tokenizer.decode(output[0]))

Sample output:

Daenerys kissed Gandalf, as the witcher hacked off Lord Voldemort's head with a brutal swing of Longclaw. “Do you know what a warrior like that will do?”

“I am a knight of seven,” Geralt said. “And how is this knight?”

“Prince, it is known,” replied the witcher, “I am a knight of Solamnia, not a Knight. A knight of Solamnia in the name of Reorx, with elven armies at the head of his knights and knights. You do not even remember my name, elf. It was a good call.”

“That name,” said the witcher, “what does it mean, elf?”

“Some story,” said the old knight. “A good story from the Cataclysm.”

The witcher snorted and looked at the witcher. “That is how you feel, elf. You don't understand why the knights won't be executed for them.”

“That's because the knight is a king,” Chireadan said finally, “the king's elder son. Because he is, the knights would also be honorable with the title of knighthood. But a warrior would be better suited to have the honor to win.”

“But…”

“It's true,” said the old knight, “that, isn't entirely what you want. The lance of your sword is one of the best

Downloads last month: 13