Upload vllm_borealis/README.md with huggingface_hub

vllm_borealis/README.md

@@ -0,0 +1,101 @@
+# vLLM Plugin for Borealis
+
+vLLM plugin to enable inference with Borealis Audio-Language Model.
+
+## Installation
+
+```bash
+pip install -e .
+```
+
+## Usage
+
+After installation, the Borealis model will be automatically registered with vLLM.
+
+```python
+import numpy as np
+from vllm import LLM, SamplingParams
+
+# Load model
+llm = LLM(
+    model="Vikhrmodels/Borealis-5b-it",
+    trust_remote_code=True,
+    dtype="bfloat16",
+    limit_mm_per_prompt={"audio": 1},
+)
+
+# Load audio (16kHz expected)
+import librosa
+audio, sr = librosa.load("audio.wav", sr=16000)
+
+# Create prompt with audio placeholder
+prompt = "<|AUDIO|>Transcribe this audio."
+
+# Inference
+sampling_params = SamplingParams(temperature=0.7, max_tokens=512)
+outputs = llm.generate(
+    {
+        "prompt": prompt,
+        "multi_modal_data": {"audio": audio},
+    },
+    sampling_params=sampling_params,
+)
+
+print(outputs[0].outputs[0].text)
+```
+
+### With Chat Template
+
+```python
+from vllm import LLM, SamplingParams
+import librosa
+
+llm = LLM(
+    model="Vikhrmodels/Borealis-5b-it",
+    trust_remote_code=True,
+    dtype="bfloat16",
+    limit_mm_per_prompt={"audio": 1},
+)
+
+audio, sr = librosa.load("audio.wav", sr=16000)
+
+messages = [
+    {"role": "system", "content": "You are a helpful voice assistant."},
+    {"role": "user", "content": "<|AUDIO|>What is being said in this audio?"},
+]
+
+# Apply chat template
+prompt = llm.get_tokenizer().apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+
+sampling_params = SamplingParams(temperature=0.7, max_tokens=512)
+outputs = llm.generate(
+    {
+        "prompt": prompt,
+        "multi_modal_data": {"audio": audio},
+    },
+    sampling_params=sampling_params,
+)
+
+print(outputs[0].outputs[0].text)
+```
+
+## Architecture
+
+Borealis combines:
+- **Whisper Large V3** encoder for audio processing (1280-dim, 1500 frames)
+- **Qwen3-4B** LLM for text generation (2560-dim hidden size)
+- **Audio Adapter** that downsamples by 4x and projects to LLM space (375 tokens per 30s audio)
+
+## Model
+
+- HuggingFace: [Vikhrmodels/Borealis-5b-it](https://huggingface.co/Vikhrmodels/Borealis-5b-it)
+
+## Requirements
+
+- vLLM >= 0.12.0
+- transformers
+- torch