---
license: mit
language:
- ru
---

# PozdGPT

Peak of humanity technologies. New breath in neuroslop world.

![PozdGPT](./pozdgpt.jpg)

## Usage

Via llama.cpp + GGUF
```python
# !pip install llama-cpp-python
from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="sodeeplearning/pozdgpt",
	filename="PozdGPT-Q4_K_M.gguf", # Or Q6_K, Q8_0, f16
)
```

## AWQ + vLLM
To launch 4bit AWQ version you need to download this 
[folder](https://huggingface.co/sodeeplearning/pozdgpt/tree/main/PozdGPT-awq-4bit)
and launch your vLLM server:

```bash
# !pip install vllm

vllm serve ./PozdGPT-awq-4bit \
  --served-model-name pozdgpt \
  --quantization compressed-tensors \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.88 \
  --max-num-seqs 6 \
  --kv-cache-dtype fp8 \
  --enable-prefix-caching \
  --api-key key \
  --port 8148
```

## Test via telegram bot

You can test this bot in official [telegram bot](https://t.me/pozdgpt_bot)

# Contacts

- [Github](https://github.com/sodeeplearning/PozdGPT)
- [Team Telegram](https://t.me/Notfag)
- [Telegram bot](https://t.me/pozdgpt_bot)
- Email: vitaliy.petreev@gmail.com