Instructions to use maletsidibeOS/Anos_AI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use maletsidibeOS/Anos_AI with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="maletsidibeOS/Anos_AI")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("maletsidibeOS/Anos_AI", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use maletsidibeOS/Anos_AI with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "maletsidibeOS/Anos_AI"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maletsidibeOS/Anos_AI",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/maletsidibeOS/Anos_AI

SGLang

How to use maletsidibeOS/Anos_AI with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "maletsidibeOS/Anos_AI" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maletsidibeOS/Anos_AI",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "maletsidibeOS/Anos_AI" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maletsidibeOS/Anos_AI",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use maletsidibeOS/Anos_AI with Docker Model Runner:
```
docker model run hf.co/maletsidibeOS/Anos_AI
```

Anos_AI / inference /README.md

maletsidibeOS

GeeeekExplorer

Duplicate from deepseek-ai/DeepSeek-V4.1-Flash

ebdc3f0 21 days ago

preview code

Raw History Blame Contribute Delete

2.03 kB

# Minimal inference

A readable reference implementation rather than a production serving engine. The
model code covers the vision encoder and aligner, sliding-window plus compressed
sparse attention with its two-level indexer, engram n-gram lookups, MoE,
Hyper-Connections, and the DSpark forward path. Generation itself is plain
autoregressive sampling.

## Install

```bash
python -m pip install -r requirements.txt
```

## Convert Hugging Face weights

The runtime uses one converted checkpoint file per tensor-parallel rank. From
this directory:

```bash
export HF_CKPT_PATH=/path/to/DeepSeek-V4.1-Flash-HF
export SAVE_PATH=/path/to/DeepSeek-V4.1-Flash-TP8
export MP=8
python convert.py \
  --hf-ckpt-path "${HF_CKPT_PATH}" \
  --save-path "${SAVE_PATH}" \
  --model-parallel "${MP}" \
  --expert-dtype fp4 \
  --tokenizer-path "${HF_CKPT_PATH}"
```

Expert counts are inferred from the weight names, so they do not need to be
passed. `--tokenizer-path` points at whichever directory holds `tokenizer.json`
and `tokenizer_config.json`; they are copied into the converted checkpoint.

## Run the equivalent TXT and JSON examples

```bash
export CKPT_PATH=/path/to/DeepSeek-V4.1-Flash-TP8
export MP=8
INPUT_FILE=examples/example.txt ./run.sh
INPUT_FILE=examples/example_harmony.json ./run.sh
```

The two files express the same interleaved two-image prompt, so they produce
identical encoded prompts and input token IDs.

For interactive chat:

```bash
torchrun --nproc-per-node "${MP}" generate.py \
  --ckpt-path "${CKPT_PATH}" \
  --config config.json \
  --interactive \
  --temperature 0.6
```

For multi-node execution, pass the usual `torchrun --nnodes`, `--node-rank`,
`--master-addr`, and `--master-port` arguments before `generate.py`.

## Self-test

`model.py` builds a small model from the `ModelArgs` defaults and runs a prefill
plus 22 decode steps, exercising the real dense-fp8 / MoE-fp4 kernels. Weights
are uninitialized, so it checks shapes and kernel plumbing, not numerics:

```bash
python model.py
```

	# Minimal inference

	A readable reference implementation rather than a production serving engine. The
	model code covers the vision encoder and aligner, sliding-window plus compressed
	sparse attention with its two-level indexer, engram n-gram lookups, MoE,
	Hyper-Connections, and the DSpark forward path. Generation itself is plain
	autoregressive sampling.

	## Install

	```bash
	python -m pip install -r requirements.txt
	```

	## Convert Hugging Face weights

	The runtime uses one converted checkpoint file per tensor-parallel rank. From
	this directory:

	```bash
	export HF_CKPT_PATH=/path/to/DeepSeek-V4.1-Flash-HF
	export SAVE_PATH=/path/to/DeepSeek-V4.1-Flash-TP8
	export MP=8

	python convert.py \
	--hf-ckpt-path "${HF_CKPT_PATH}" \
	--save-path "${SAVE_PATH}" \
	--model-parallel "${MP}" \
	--expert-dtype fp4 \
	--tokenizer-path "${HF_CKPT_PATH}"
	```

	Expert counts are inferred from the weight names, so they do not need to be
	passed. `--tokenizer-path` points at whichever directory holds `tokenizer.json`
	and `tokenizer_config.json`; they are copied into the converted checkpoint.

	## Run the equivalent TXT and JSON examples

	```bash
	export CKPT_PATH=/path/to/DeepSeek-V4.1-Flash-TP8
	export MP=8

	INPUT_FILE=examples/example.txt ./run.sh
	INPUT_FILE=examples/example_harmony.json ./run.sh
	```

	The two files express the same interleaved two-image prompt, so they produce
	identical encoded prompts and input token IDs.

	For interactive chat:

	```bash
	torchrun --nproc-per-node "${MP}" generate.py \
	--ckpt-path "${CKPT_PATH}" \
	--config config.json \
	--interactive \
	--temperature 0.6
	```

	For multi-node execution, pass the usual `torchrun --nnodes`, `--node-rank`,
	`--master-addr`, and `--master-port` arguments before `generate.py`.

	## Self-test

	`model.py` builds a small model from the `ModelArgs` defaults and runs a prefill
	plus 22 decode steps, exercising the real dense-fp8 / MoE-fp4 kernels. Weights
	are uninitialized, so it checks shapes and kernel plumbing, not numerics:

	```bash
	python model.py
	```