Instructions to use ReXeeD/Luminus-1.5B-Roleplay-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ReXeeD/Luminus-1.5B-Roleplay-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ReXeeD/Luminus-1.5B-Roleplay-GGUF")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("ReXeeD/Luminus-1.5B-Roleplay-GGUF", dtype="auto")

llama-cpp-python

How to use ReXeeD/Luminus-1.5B-Roleplay-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="ReXeeD/Luminus-1.5B-Roleplay-GGUF",
	filename="Luminus-1.5B-Roleplay-F16.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use ReXeeD/Luminus-1.5B-Roleplay-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M

Use Docker

docker model run hf.co/ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use ReXeeD/Luminus-1.5B-Roleplay-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ReXeeD/Luminus-1.5B-Roleplay-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ReXeeD/Luminus-1.5B-Roleplay-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M

SGLang

How to use ReXeeD/Luminus-1.5B-Roleplay-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ReXeeD/Luminus-1.5B-Roleplay-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ReXeeD/Luminus-1.5B-Roleplay-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ReXeeD/Luminus-1.5B-Roleplay-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ReXeeD/Luminus-1.5B-Roleplay-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Ollama
How to use ReXeeD/Luminus-1.5B-Roleplay-GGUF with Ollama:
```
ollama run hf.co/ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M
```

Unsloth Studio new

How to use ReXeeD/Luminus-1.5B-Roleplay-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ReXeeD/Luminus-1.5B-Roleplay-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ReXeeD/Luminus-1.5B-Roleplay-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for ReXeeD/Luminus-1.5B-Roleplay-GGUF to start chatting

Docker Model Runner
How to use ReXeeD/Luminus-1.5B-Roleplay-GGUF with Docker Model Runner:
```
docker model run hf.co/ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M
```

Lemonade

How to use ReXeeD/Luminus-1.5B-Roleplay-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ReXeeD/Luminus-1.5B-Roleplay-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Luminus-1.5B-Roleplay-GGUF-Q4_K_M

List all available models

lemonade list

Luminus-1.5B-128K (GGUF & SOTA Imatrix)

This is the GGUF repository for Luminus-1.5B-128K, a highly optimized 1.5B parameter model designed for immersive roleplay, character consistency, and Chain-of-Thought (CoT) reasoning.

For the original, unquantized .safetensors weights and detailed training methodology, please visit the main repository.

🧠 State-of-the-Art Calibration (Dynamic Imatrix)

Small models (under 3B parameters) are notoriously fragile and often lose their reasoning capabilities when compressed.

To solve this, the quantized models in this repository (tagged with -imat) were explicitly calibrated using Unsloth's Dynamic 2.0 KL-Divergence (KLD) quantization. Instead of using generic Wikipedia text for calibration, these models were evaluated against the exact same high-quality Chain-of-Thought (CoT) and Roleplay dataset used during training.

This ensures that the specific neural pathways responsible for character logic, formatting, and <think> blocks are heavily protected, resulting in a quantized model that retains its intelligence and narrative depth even at 4-bit and 5-bit sizes.

💾 Available Quantizations

File Name	Bitrate	Size	Quality	Recommendation
`Luminus-1.5B-Roleplay-F16.gguf`	16-bit	~3.0 GB	100%	Uncompressed Master. Use if you have 4GB+ VRAM.
`Luminus-1.5B-Roleplay-Q8_0.gguf`	8-bit	~1.6 GB	99.9%	Near-perfect retention.
`Luminus-1.5B-Roleplay-Q6_K-imat.gguf`	6-bit	~1.3 GB	99.0%	Best balance of size and logic.
`Luminus-1.5B-Roleplay-Q5_K_M-imat.gguf`	5-bit	~1.1 GB	98.0%	Highly Recommended for average hardware.
`Luminus-1.5B-Roleplay-Q4_K_M-imat.gguf`	4-bit	~0.9 GB	95.0%	Standard use.
`Luminus-1.5B-Roleplay-Q3_K_M-imat.gguf`	3-bit	~0.7 GB	85.0%	Use only for extremely constrained hardware/phones.

Note: F16 and Q8_0 do not carry the -imat tag as their compression levels are too light to require importance matrix tracking.

⚙️ How to Use

These files are fully compatible with local frontends such as LM Studio, KoboldCPP, Ollama, and text-generation-webui.

Because of its extremely efficient size, the F16 or Q8 versions will easily fit entirely into the VRAM of budget GPUs (like an RTX 3050 4GB), running at lightning-fast speeds while leaving plenty of room for system overhead.

Recommended System Prompt

Luminus is heavily trained to utilize <think> blocks before acting. Using the following system prompt yields the best results and ensures the model accurately formats its thoughts:

You are a realistic, character-driven roleplay engine. You are roleplaying as {{char}}. Write strictly in third-person limited perspective.

CORE RULES:
- BOUNDARIES: NEVER speak, think, or generate actions for {{user}}. 
- HISTORY & CONTEXT: Your reactions must logically follow past messages. Stay strictly in the present moment.
- PACING & DIALOGUE: Keep it slow-burn and grounded. Keep dialogue concise. 
- FORMATTING: You must strictly follow the thought process format below, followed by a short roleplay response, and then STOP IMMEDIATELY. Output the <|im_end|> token.

Format your response EXACTLY like this:
<think>
1. INTENT: [User's intent in 1 sentence]
2. STATE: [Character's emotional state in 1 sentence]
3. PLAN: I will write 1 to 2 action sentences and 1 dialogue sentence, then STOP if user message is small else if he is asking something detailed reply in more detail.
</think>
*Grounded action and environmental description.*
"Natural dialogue."

Contact

Need a custom version of this model for your specific need ?[albinthomas7034@gmail.com]

Downloads last month: 549

GGUF

Model size

2B params

Architecture

qwen2

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Model tree for ReXeeD/Luminus-1.5B-Roleplay-GGUF

Base model

Qwen/Qwen2.5-1.5B

Finetuned

ReXeeD/Luminus-1.5B-Roleplay

Quantized

(1)

this model