Instructions to use LucidityAI/Astral-4B-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LucidityAI/Astral-4B-Preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LucidityAI/Astral-4B-Preview")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LucidityAI/Astral-4B-Preview")
model = AutoModelForCausalLM.from_pretrained("LucidityAI/Astral-4B-Preview")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use LucidityAI/Astral-4B-Preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LucidityAI/Astral-4B-Preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LucidityAI/Astral-4B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LucidityAI/Astral-4B-Preview

SGLang

How to use LucidityAI/Astral-4B-Preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LucidityAI/Astral-4B-Preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LucidityAI/Astral-4B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LucidityAI/Astral-4B-Preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LucidityAI/Astral-4B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LucidityAI/Astral-4B-Preview with Docker Model Runner:
```
docker model run hf.co/LucidityAI/Astral-4B-Preview
```

sd-inf commited on Oct 4, 2025

Commit

b29408e

verified ·

1 Parent(s): addc713

Delete license

Browse files

Files changed (1) hide show

license +0 -22

license DELETED Viewed

@@ -1,22 +0,0 @@
-Copyright (c) 2025 Lucidity AI
-**License Grant:** Permission is hereby granted, free of charge, to any person obtaining a copy of this model (the "Model") and associated documentation files (collectively, the "Materials"), derived from a model (licensed under Apache 2.0), to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Materials, subject to the following conditions:
-1. **Usage Restrictions:**
-   The Materials shall not be used, directly or indirectly, for:
-   * Automated or systematic replacement or displacement of human employees. ( Example: Firing a employee with the intent to utilize a Artificial intelligence tool in their place )
-   * Activities violating human rights, dignity, privacy, or civil liberties.
-   * Promoting discrimination based on race, ethnicity, gender, sexual orientation, religion, disability, or socioeconomic status.
-   * Generating or disseminating misleading information or content intended to deceive, defraud, or manipulate public perception, democracy, or safety.
-2. **Attribution and Apache 2.0 Compliance:**
-   All redistributed or derivative versions of the Materials must include:
-   * A clear notice of attribution to the original Qwen model, specifying it is licensed under the Apache 2.0 license.
-   * The full text or a link to the Apache 2.0 license for those parts originating from the Qwen model.
-3. **Warranty Disclaimer:**
-   THE MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY ARISING FROM OR IN CONNECTION WITH THE USE OR OTHER DEALINGS IN THE MATERIALS.
-By utilizing or redistributing the Materials, you acknowledge your understanding and acceptance of these conditions.