Instructions to use 3ndetz/fred-t5-xl-remastered with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 3ndetz/fred-t5-xl-remastered with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="3ndetz/fred-t5-xl-remastered")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("3ndetz/fred-t5-xl-remastered")
model = AutoModelForSeq2SeqLM.from_pretrained("3ndetz/fred-t5-xl-remastered")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use 3ndetz/fred-t5-xl-remastered with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "3ndetz/fred-t5-xl-remastered"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "3ndetz/fred-t5-xl-remastered",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/3ndetz/fred-t5-xl-remastered

SGLang

How to use 3ndetz/fred-t5-xl-remastered with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "3ndetz/fred-t5-xl-remastered" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "3ndetz/fred-t5-xl-remastered",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "3ndetz/fred-t5-xl-remastered" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "3ndetz/fred-t5-xl-remastered",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 3ndetz/fred-t5-xl-remastered with Docker Model Runner:
```
docker model run hf.co/3ndetz/fred-t5-xl-remastered
```

LegendaryFred Modernization

The FredT5 XL. The legendary. The BEST nick analyzer ever!

TODO add cool picture of the FRED

End of 2025 year: still NO models (even 100B+) can beat FRED-T5-1.7b (2023 SOTA for Russian language) in sense of trash nicknames analyzing...

I added FRED-compatible jinja2 prompt template and fixed special tokens like <extra_id_0> (added to tokenizer) to remove it from outputs for better expirience.

So, now I just want this model to revive. This endless creativity of FRED will be reminded forever!

Lower you can see example outputs, installing and loading example.

Of course, I AM NOT creator of the model! All credits to the respectful authors!

The best model inference way for now - using model via something like text-generation-webui.

Example outputs

Хабрастатьи тут и немного тут.

Фред - невероятно креативная модель. Многие ещё его хвалят за маск-токены и возможность продолжать текст с середины. Но у него есть смертельная проблема для старых LLM (ну и многих новых тоже касается) - он нереально любит повторяться, а если repetition penalty перебрать, то будет нести совсем чушь...

TODO добавить жирных примеров прям сюда

Getting started

You can run the model via webui. Run their server, open Download model section and enter "3ndetz/fred-t5-xl-remastered". Runnable in all envs supporting transformers safetensors.

TODO add transformers usage example with using prompt chat template with nick analyzing

Origin

Base model
Finetune of Base model I used for this convertation

Developing

Converted with PAIN using ctranslate2

Downloads last month: 2

Safetensors

Model size

2B params

Tensor type

F32

F16

Model tree for 3ndetz/fred-t5-xl-remastered

Base model

SiberiaSoft/SiberianFredT5-instructor

Finetuned

(2)

this model