README.md

---
license: apache-2.0
datasets:
- Knixee/atlas-ru-dataset-v2
language:
- ru
base_model:
- t-tech/T-lite-it-2.1
pipeline_tag: text-generation
tags:
- atlas
- nms
- no-mans-sky
---

# Atlas

Local Atlas from the game No Man's Sky.

<img src="./atlas.jpg" alt="Atlas" width="600">

# Serving GGUF & AWQ Models with vLLM

Examples for running GGUF (via llama.cpp) and AWQ (via vLLM).

## GGUF Version (via llama.cpp)

```python
# pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
repo_id="Knixee/Atlas-nms-v2",
filename="Atlas-2.0-Q4_K_M.gguf", # Or Q4_K_S, Q8_0, BF16 
)

output = llm("User: Hello!\nAssistant:", max_tokens=512)
print(output["choices"][0]["text"])
```

## AWQ Version (via vLLM)
To launch 4bit AWQ version you need to download this 
[folder](https://huggingface.co/Knixee/Atlas-nms-v2/tree/main/Atlas-2.0-AWQ-4bit)

```bash
# pip install vllm

vllm serve Knixee/Atlas-nms-v2 \
  --subfolder Atlas-2.0-AWQ-4bit \
  --served-model-name Atlas-nms-v2 \
  --quantization awq \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.88 \
  --max-num-seqs 6 \
  --port 8148
```

API Request Example:

```bash
curl http://localhost:8148/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "Atlas-nms-v2", "messages": [{"role": "user", "content": "Hello!"}]}'
```

# Contacts

- [Github](https://github.com/Knixee)
- [Telegram](https://t.me/Norm_imya)
- Email: kantserovegor@gmail.com