Instructions to use evilfreelancer/ruGPT3XL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use evilfreelancer/ruGPT3XL with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="evilfreelancer/ruGPT3XL", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("evilfreelancer/ruGPT3XL", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use evilfreelancer/ruGPT3XL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "evilfreelancer/ruGPT3XL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "evilfreelancer/ruGPT3XL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/evilfreelancer/ruGPT3XL

SGLang

How to use evilfreelancer/ruGPT3XL with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "evilfreelancer/ruGPT3XL" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "evilfreelancer/ruGPT3XL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "evilfreelancer/ruGPT3XL" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "evilfreelancer/ruGPT3XL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use evilfreelancer/ruGPT3XL with Docker Model Runner:
```
docker model run hf.co/evilfreelancer/ruGPT3XL
```

ruGPT3XL

Commit History

Update README.md

d35fdf9
verified

evilfreelancer commited on Mar 30

SDPA added

a25e87a

Pavel Rykov commited on Mar 30

Fix of sparse attention for training

9d4a84b

Pavel Rykov commited on Mar 30

Fix

fd0515a

Pavel Rykov commited on Mar 29

Sparse attention fixed

a942c12

Pavel Rykov commited on Mar 29

Update README.md

bfac750
verified

evilfreelancer commited on Mar 28

Update README.md

f6a8301
verified

evilfreelancer commited on Mar 28

Typo fix

511ef2e

Pavel Rykov commited on Mar 27

Merge branch 'main' of hf.co:evilfreelancer/ruGPT3XL

d01e7f2

Pavel Rykov commited on Mar 27

Readme updated

35af8a3

Pavel Rykov commited on Mar 27

Update README.md

00c8d90
verified

evilfreelancer commited on Mar 25

Update README.md

96707b5
verified

evilfreelancer commited on Mar 25

Upload generate.py

6efb4d2
verified

evilfreelancer commited on Mar 25

Update README.md

77b12f5
verified

evilfreelancer commited on Mar 25

Update README.md

5bfbc1c
verified

evilfreelancer commited on Mar 25

Update README.md

27ea67e
verified

evilfreelancer commited on Mar 25

Update README.md

eeae566
verified

evilfreelancer commited on Mar 25

Upload folder using huggingface_hub

a3802f5
verified

evilfreelancer commited on Mar 25

initial commit

7e83c14
verified

evilfreelancer commited on Mar 25

Commit History

Update README.md d35fdf9 verified

SDPA added a25e87a

Fix of sparse attention for training 9d4a84b

Fix fd0515a

Sparse attention fixed a942c12

Update README.md bfac750 verified

Update README.md f6a8301 verified

Typo fix 511ef2e

Merge branch 'main' of hf.co:evilfreelancer/ruGPT3XL d01e7f2

Readme updated 35af8a3

Update README.md 00c8d90 verified

Update README.md 96707b5 verified

Upload generate.py 6efb4d2 verified

Update README.md 77b12f5 verified

Update README.md 5bfbc1c verified

Update README.md 27ea67e verified

Update README.md eeae566 verified

Upload folder using huggingface_hub a3802f5 verified

initial commit 7e83c14 verified

Update README.md

d35fdf9
verified

SDPA added

a25e87a

Fix of sparse attention for training

9d4a84b

Fix

fd0515a

Sparse attention fixed

a942c12

Update README.md

bfac750
verified

Update README.md

f6a8301
verified

Typo fix

511ef2e

Merge branch 'main' of hf.co:evilfreelancer/ruGPT3XL

d01e7f2

Readme updated

35af8a3

Update README.md

00c8d90
verified

Update README.md

96707b5
verified

Upload generate.py

6efb4d2
verified

Update README.md

77b12f5
verified

Update README.md

5bfbc1c
verified

Update README.md

27ea67e
verified

Update README.md

eeae566
verified

Upload folder using huggingface_hub

a3802f5
verified

initial commit

7e83c14
verified