Instructions to use alphaaico/AAI-1.5B-Thought-16-Bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use alphaaico/AAI-1.5B-Thought-16-Bit with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="alphaaico/AAI-1.5B-Thought-16-Bit")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("alphaaico/AAI-1.5B-Thought-16-Bit")
model = AutoModelForCausalLM.from_pretrained("alphaaico/AAI-1.5B-Thought-16-Bit")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use alphaaico/AAI-1.5B-Thought-16-Bit with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "alphaaico/AAI-1.5B-Thought-16-Bit"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "alphaaico/AAI-1.5B-Thought-16-Bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/alphaaico/AAI-1.5B-Thought-16-Bit

SGLang

How to use alphaaico/AAI-1.5B-Thought-16-Bit with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "alphaaico/AAI-1.5B-Thought-16-Bit" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "alphaaico/AAI-1.5B-Thought-16-Bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "alphaaico/AAI-1.5B-Thought-16-Bit" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "alphaaico/AAI-1.5B-Thought-16-Bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Unsloth Studio

How to use alphaaico/AAI-1.5B-Thought-16-Bit with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for alphaaico/AAI-1.5B-Thought-16-Bit to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for alphaaico/AAI-1.5B-Thought-16-Bit to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for alphaaico/AAI-1.5B-Thought-16-Bit to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="alphaaico/AAI-1.5B-Thought-16-Bit",
    max_seq_length=2048,
)

Docker Model Runner
How to use alphaaico/AAI-1.5B-Thought-16-Bit with Docker Model Runner:
```
docker model run hf.co/alphaaico/AAI-1.5B-Thought-16-Bit
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Uploaded model - AlphaAI-1.5B-Thought

Developed by: alphaaico
License: apache-2.0
Finetuned from model : Qwen2.5-1.5B

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Overview

AlphaAI-1.5B-Thought is a fine-tuned version of Qwen2.5-1.5B, optimized for chain-of-thought (CoT) reasoning and structured problem-solving. This model has been trained on a custom CoT dataset, enhancing its ability to perform step-by-step logical reasoning, multi-step inference, and contextual understanding across various domains.

Designed for local AI deployments, it supports efficient inference on personal hardware while maintaining high reasoning capabilities. The training process was accelerated using Unsloth and Hugging Face's TRL library, allowing for 2x faster fine-tuning.

Model Details

Model: Qwen2.5-1.5B
Fine-tuned By: Alpha AI
Training Framework: Unsloth + Hugging Face TRL
License: Apache-2.0
Format: GGUF (Optimized for local use)

Quantization Levels Available:

q4_k_m
q5_k_m
q8_0
16-bit (This)

Others https://huggingface.co/alphaaico/AAI-1.5B-Thought

Use Cases

Complex Reasoning & Problem Solving – Ideal for tasks requiring logical deductions, multi-step inference, and structured decision-making.
Conversational AI with Deep Thought – Enhances chatbots, virtual assistants, and customer support agents with structured responses.
Mathematical & Scientific Analysis – Useful for AI-assisted research, theorem verification, and structured problem decomposition.
Code and Workflow Generation – Helps in AI-driven programming assistance and process automation.

Model Performance

Enhanced Chain-of-Thought Reasoning – Generates step-by-step logical deductions.
Efficient Local Inference – Optimized for deployment on consumer GPUs and edge devices.
Balanced Creativity & Precision – Ensures structured yet flexible responses for diverse reasoning tasks.

Limitations & Biases

As with any AI model, AlphaAI-1.5B-Thought may reflect biases present in its training data. Users should validate responses for critical applications and fine-tune further for domain-specific tasks.

Acknowledgments

Special thanks to:

Unsloth for the optimized training pipeline.
Hugging Face TRL for providing robust tools for fine-tuning large models efficiently.

Downloads last month: 5

Safetensors

Model size

2B params

Tensor type

F16

Model tree for alphaaico/AAI-1.5B-Thought-16-Bit

Quantizations

1 model