---
pretty_name: VibeThinker-1.5B GGUF
author: siruncahined
license: apache-2.0
tags:
- qwen2
- qwen
- gguf
- quantized
- VibeThinker
- Math
- 1.5B
- text-generation
- q4_k_m
- q5_k_m
- q6_k
- q8_0
widget:
- text: Tell me about the VibeThinker-1.5B model.
---
# VibeThinker-1.5B GGUF

This is a quantized version of the **[WeiboAI/VibeThinker-1.5B](https://huggingface.co/WeiboAI/VibeThinker-1.5B)** model, converted to GGUF format for use with `llama.cpp` and compatible tools.

## Quantization Details

The model was quantized using `llama.cpp` to the following formats:

- **FP16**: The original model was converted to FP16 GGUF as an intermediate step.
- **Q4_K_M**: Quantized using the Q4_K_M method.
- **Q5_K_M**: Quantized using the Q5_K_M method.
- **Q6_K**: Quantized using the Q6_K method.
- **Q8_0**: Quantized using the Q8_0 method.

### Original Model Card Summary:

*   **Model ID**: `WeiboAI/VibeThinker-1.5B`
*   **Original Repository**: [https://huggingface.co/WeiboAI/VibeThinker-1.5B](https://huggingface.co/WeiboAI/VibeThinker-1.5B)

## How to Use

You can use these GGUF files with `llama.cpp` or other tools that support the GGUF format. Download the desired quantization level and use it with your `llama.cpp` compatible inference engine.

Example usage with `llama.cpp` (replace `[quant_level]` with your desired quantization, e.g., `q4_k_m`):

```bash
./main -m VibeThinker-1.5B-[quant_level].gguf -p "Hello, what is your name?" -n 128
```

## Files Provided

*   `VibeThinker-1.5B-f16.gguf` (FP16)
*   `VibeThinker-1.5B-q4_k_m.gguf` (Q4_K_M)
*   `VibeThinker-1.5B-q5_k_m.gguf` (Q5_K_M)
*   `VibeThinker-1.5B-q6_k.gguf` (Q6_K)
*   `VibeThinker-1.5B-q8_0.gguf` (Q8_0)