Update README.md

README.md

@@ -12,9 +12,60 @@ base_model:
 
 > Notes: Keep `lm_head` in high precision; calibrate on long, domain-relevant sequences.
 
-Check the original model card for information about this model.
+📘 About This Model
+
+This is a quantized NVFP4 (W4A4) version of Step-Audio-R1, an open-weights Qwen2-Audio–based multimodal model for audio understanding and reasoning.
+The original BF16 model requires ~67 GB VRAM.
+
+Step-Audio-R1 combines:
+
+A high-capacity audio encoder
+
+A projection layer that maps audio features into the transformer
+
+A Qwen2 language backbone for reasoning and text generation
+
+The model is designed for:
+
+Speech transcription and interpretation
+
+Emotional / tonal analysis
+
+Speaker characteristics
+
+Music and sound-scene understanding
+
+High-quality step-by-step reasoning about audio inputs
+
+It does not generate audio; it produces text based on audio input.
+
+📦 What This Quantized Version Enables
+
+This NVFP4 quantized version reduces memory requirements significantly:
+
+Size: ~22 GB (down from ~67 GB)
+
+Runs comfortably on a single RTX 5090
+
+Fully compatible with vLLM (including streaming text output)
+
+Preserves most reasoning performance from the BF16 release
+
+Because of this, anyone with a high-end consumer GPU can experiment with advanced audio reasoning locally.
+
+🖥 Supported Audio Behavior
+
+The model supports:
+
+✔ Streaming text output through vLLM
+✔ Reading uploaded audio files (WAV/MP3/etc) via ffmpeg
+✘ It does not synthesize audio
+✘ It does not require pre-burned waveforms — any user-provided audio file works
+
+Check the original model card for information about this model for more info.
 
 # Running the model with VLLM in Docker
+It requires a specific vllm container released by the model authors.
 ```sh
 docker run --rm -ti --gpus all \
     -v $(pwd)/Step-Audio-R1:/Step-Audio-R1 \
@@ -30,6 +81,39 @@ docker run --rm -ti --gpus all \
     --interleave-mm-strings \
     --trust-remote-code
 ```
+
+This example script should allow an audio wave full to be streamed to the model and get a response based on the prompt.
+```py
+import requests
+import base64
+
+with open("audio.wav", "rb") as f:
+    audio_b64 = base64.b64encode(f.read()).decode()
+
+payload = {
+    "model": "Step-Audio-R1",
+    "stream": True,
+    "messages": [
+        {
+            "role": "user",
+            "content": [
+                {"type": "input_audio", "audio_data": audio_b64, "mime_type": "audio/wav"},
+                {"type": "text", "text": "Transcribe this and describe the speaker."}
+            ]
+        }
+    ]
+}
+
+with requests.post(
+    "http://localhost:9999/v1/chat/completions",
+    json=payload,
+    stream=True,
+) as r:
+    for line in r.iter_lines():
+        if line:
+            print(line.decode())
+```
+
 This was tested on an RTX Pro 6000 Blackwell cloud instance.
 
 If there are other models you're interested in seeing quantized to NVFP4 for use on the DGX Spark, or other modern Blackwell (or newer) cards let me know. I'm trying to make more NVFP4 models available to allow more people to try them out.