a-m-team
/

AM-Thinking-v1

@@ -1,153 +1,142 @@
 ---
-license: apache-2.0
-pipeline_tag: text-generation
 library_name: transformers
 ---
-# AM‑Thinking‑v1: Advancing the Frontier of Reasoning at 32B Scale
-* 2025-05-10 · a-m‑team
 <p align="center">
-🤗 <a href="https://huggingface.co/a-m-team">Hugging Face</a>&nbsp&nbsp | &nbsp&nbsp 📑 <a href="https://arxiv.org/abs/2505.08311"> Paper</a>  &nbsp&nbsp | &nbsp&nbsp 📑 <a href="https://a-m-team.github.io/am-thinking-v1/">Blog</a> &nbsp&nbsp
 </p>
-## 🚀 Introduction
-We release **AM-Thinking‑v1**, a 32B dense language model focused on enhancing reasoning capabilities.
-Built on Qwen 2.5‑32B‑Base, AM-Thinking‑v1 shows strong performance on reasoning benchmarks, comparable to much larger MoE models like **DeepSeek‑R1**, **Qwen3‑235B‑A22B**, **Seed1.5-Thinking**, and larger dense model like **Nemotron-Ultra-253B-v1**.
-<div style="text-align: center;">
-    <img src="assets/benchmark.png" alt="benchmark" style="width: 90%;">
-</div>
-## 🧩 Why Another 32B Reasoning Model Matters?
-Large Mixture‑of‑Experts (MoE) models such as **DeepSeek‑R1** or **Qwen3‑235B‑A22B** dominate leaderboards—but they also demand clusters of high‑end GPUs. Many teams just need *the best dense model that fits on a single card*.
-**AM‑Thinking‑v1** fills that gap **while remaining fully based on open-source components**:
-* **Outperforms DeepSeek‑R1** on AIME’24/’25 & LiveCodeBench and **approaches Qwen3‑235B‑A22B** despite being 1/7‑th the parameter count.
-* **Built on the publicly available Qwen 2.5‑32B‑Base**, as well as the RL training queries.
-* Shows that with a **well‑designed post‑training pipeline** ( SFT + dual‑stage RL ) you can squeeze flagship‑level reasoning out of a 32 B dense model.
-* **Deploys on one A100‑80 GB** with deterministic latency—no MoE routing overhead.
-<div style="text-align: center;">
-  <img src="assets/param-aime2024.jpeg" alt="AIME 2024" style="width: 90%; margin-bottom: 20px;">
-  <img src="assets/param-lcb.jpeg" alt="LiveCodeBench" style="width: 90%;">
-  <div style="margin-top: 10px;">
-    <em>AM-Thinking-v1 achieves strong reasoning performance with significantly fewer parameters.</em>
-  </div>
-</div>
-## 🛠️ Use Cases
-### 1) Code Generation
-<pre style="font-family: 'Times New Roman', serif; font-size: 12px; border: 1px solid black; padding: 10px; font-style: italic;">
-PROMPT :
-write a python script for a bouncing red ball within a triangle, make sure to handle collision detection properly. make the triangle slowly rotate. implement it in python. make sure ball stays within the triangle
-</pre>
-<div style="text-align: center;">
-    <img src="assets/ball.gif" alt="Bouncing Red Ball" width="50%">
-</div>
-### 2) Logic
-<div style="text-align: center;">
-    <img src="assets/diamond.png" alt="diamond" width="90%">
-</div>
-### 3) Writing
-<div style="text-align: center;">
-    <img src="assets/writing.png" alt="sushi" width="90%">
-</div>
-## ⚡ Quick start
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-model_name = "a-m-team/AM-Thinking-v1"
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-model = AutoModelForCausalLM.from_pretrained(
-    model_name,
-    torch_dtype="auto",
-    device_map="auto"
-)
-prompt = "How can I find inner peace?"
-messages = [
-    {"role": "user", "content": prompt}
-]
-text = tokenizer.apply_chat_template(
-    messages,
-    tokenize=False,
-    add_generation_prompt=True
-)
-model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
-generated_ids = model.generate(
-    **model_inputs,
-    max_new_tokens=49152
-)
-output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
-response = tokenizer.decode(output_ids, skip_special_tokens=True)
-think_content = response.split("<think>")[1].split("</think>")[0]
-answer_content = response.split("<answer>")[1].split("</answer>")[0]
-print (f"user prompt: {prompt}")
-print (f"model thinking: {think_content}")
-print (f"model answer: {answer_content}")
 ```
-> Note: We have included the system prompt in the tokenizer configuration, as it was used during both the SFT and RL stages. To ensure consistent output quality, we recommend including the same system prompt during actual usage; otherwise, the model's responses may be significantly affected.
-### Quantized versions for compact devices
-A series of quantized versions for [AM-Thinking-v1](https://huggingface.co/a-m-team/AM-Thinking-v1-gguf) model.
-For use with [llama.cpp](https://github.com/ggml-org/llama.cpp) and [Ollama](https://github.com/ollama/ollama)
-is available at [AM-Thinking-v1-gguf](https://huggingface.co/a-m-team/AM-Thinking-v1-gguf).
-## 🔧 Post-training pipeline
-To achieve its strong reasoning ability, AM‑Thinking‑v1 goes through a carefully designed post-training pipeline.
-Below we describe the key stages involved in turning a base model into a high-performing reasoner:
-**Step 1 – Cold‑start SFT.**
-We begin with the open-sourced **Qwen 2.5‑32B‑Base** and run a broad supervised fine‑tune on a blended training dataset of math, code and open‑domain chat. This endows the model with a "think‑then‑answer" behavioural pattern and equips it with an initial capacity for reasoning.
-**Step 2 – Pass‑rate‑aware data curation.**
-Before any RL, the SFT model is evaluated on every math‑ and code‑oriented training query. For each item we log a pass rate; only those with **0 < pass‑rate < 1** are kept. In effect we discard problems the model already masters and those it utterly fails, concentrating learning on genuinely informative cases.
-**Step 3 – Reinforcement learning .**
-We adopt a two‑stage GRPO scheme: Stage 1 trains only on math and code queries. Once it converges, stage 2 starts by removing every query the model answered 100% correctly in Stage 1 and adjusting key hyper‑parameters such as maximum generation length and learning rate.
-## ⚠️ Limitations
-While AM‑Thinking‑v1 excels at pure language reasoning and open‑domain chat, it has not yet been trained for structured function‑calling or tool‑use workflows, which restricts its usefulness in agent‑style applications that must act on external systems.
-Improving the model's ability to follow complex instructions is also an important direction for our future work.
-In addition, our safety alignment is still at an early stage, so more rigorous red‑teaming are required to reduce potential harms.
-## 📚 Citation
-The a-m-team is an internal team at Beike (Ke.com), dedicated to exploring AGI technology.
-If you find our work helpful, feel free to give us a cite.
 ```
-@misc{ji2025amthinkingv1advancingfrontierreasoning,
-      title={AM-Thinking-v1: Advancing the Frontier of Reasoning at 32B Scale},
-      author={Yunjie Ji and Xiaoyu Tian and Sitong Zhao and Haotian Wang and Shuaiting Chen and Yiping Peng and Han Zhao and Xiangang Li},
-      year={2025},
-      eprint={2505.08311},
-      archivePrefix={arXiv},
-      primaryClass={cs.CL},
-      url={https://arxiv.org/abs/2505.08311},
 }
 ```

 ---
+datasets:
+- maitrix-org/Voila-Benchmark
+- maitrix-org/Voila-million-voice
+language:
+- en
+- zh
+- fr
+- de
+- ja
+- ko
 library_name: transformers
+license: mit
+pipeline_tag: audio-text-to-text
 ---
 <p align="center">
+    <img src="https://voila.maitrix.org/static/images/logo.png" width="400"/><br/>
+    <b>Voila: <span style="color:#ca00f9">Voi</span>ce-<span style="color:#ca00f9">La</span>nguage Foundation Models</b><br/><br/>
+    💜 <a href="https://voila.maitrix.org"><b>Project Page</b></a> &nbsp&nbsp ｜ &nbsp&nbsp 🖥️ <a href="https://github.com/maitrix-org/Voila">GitHub</a> &nbsp&nbsp  | &nbsp&nbsp🤗 <a href="https://huggingface.co/collections/maitrix-org/voila-67e0d96962c19f221fc73fa5">Hugging Face</a>&nbsp&nbsp | &nbsp&nbsp 📑 <a href="http://arxiv.org/abs/2505.02707">Paper</a> &nbsp&nbsp | &nbsp&nbsp 🌐 <a href="https://huggingface.co/spaces/maitrix-org/Voila-demo">Online Demo</a> &nbsp&nbsp| &nbsp&nbsp 🏠<a href="https://maitrix.org">Maitrix.org</a>
 </p>
+Voila is a new family of large voice-language foundation models aiming to lift human-AI interaction experiences to the next level. Breaking away from the constraints of traditional voice AI systems—high latency, loss of vocal nuances, and mechanical responses—Voila employs an innovative end-to-end model design and a novel hierarchical Transformer architecture. This approach enables real-time, autonomous, and rich voice interactions, with latency as low as 195 ms, surpassing average human response times. Combining advanced voice and language modeling, Voila offers customizable, persona-driven engagements and excels in a range of audio tasks from ASR and TTS to speech translation across six languages. With the online [web demo](https://huggingface.co/spaces/maitrix-org/Voila-demo), Voila invites you to explore a transformative, natural dialogue experience between human and AI.
+# ✨ Highlights
+- ⭐ High-fidelity, low-latency, real-time streaming audio processing
+- ⭐ Effective integration of voice and language modeling capabilities
+- ⭐ Millions of pre-built and custom voices, fast voice switching during conversation
+- ⭐ Unified model for various audio tasks
+# 🎥 Video Demo
+[![Voila Demo](https://img.youtube.com/vi/J27M9-g5KL0/0.jpg)](https://www.youtube.com/watch?v=J27M9-g5KL0)
+# 🔥 Latest News!!
+* April 28, 2025: 👋 We've released the inference code and model weights of Voila.
+# ⚙️ Foundation Models
+| Model | Description | Download Link |
+|--------|-----------|-----------------|
+|Voila-base|Voila base model|https://huggingface.co/maitrix-org/Voila-base|
+|Voila-Chat|End-to-end audio chat model|https://huggingface.co/maitrix-org/Voila-chat|
+|Voila-Autonomous (preview)|Full-duplex audio chat model|https://huggingface.co/maitrix-org/Voila-autonomous-preview|
+|Voila-Audio-alpha|Empowering LLM with raw audio input|https://huggingface.co/maitrix-org/Voila-audio-alpha|
+|Voila-Tokenizer|Audio tokenizer|https://huggingface.co/maitrix-org/Voila-Tokenizer|
+## Usage
+### CLI demo
+```shell
+for model_name in "maitrix-org/Voila-audio-alpha" "maitrix-org/Voila-base" "maitrix-org/Voila-chat"; do
+    # Text chat
+    python infer.py \
+        --model-name ${model_name} \
+	    --instruction "" \
+	    --input-text "Hello" \
+	    --task-type chat_tito
+    # Voice chat
+    python infer.py \
+        --model-name ${model_name} \
+	    --instruction "" \
+	    --input-audio "examples/test1.mp3" \
+	    --task-type chat_aiao
+done
+# Autonomous mode
+python infer.py \
+    --model-name "maitrix-org/Voila-autonomous-preview" \
+	--instruction "" \
+	--input-audio "examples/test_autonomous1.mp3" \
+	--task-type chat_aiao_auto
 ```
+### Gradio demo
+```shell
+python gradio_demo.py
+```
+For more information, please refer to the [code repository](https://github.com/maitrix-org/Voila).
+# 📁 Datasets
+We publish the following two datasets: Voila Benchmark and Voila Voice Library. Voila-Benchmark is a novel speech evaluation benchmark, while Voila Voice Library provides millions of pre-built and customizable voices.
+| Dataset | Description | Download Link |
+|--------|-----------|-----------------|
+|Voila Benchmark| Evaluation of Voila Benchmark | https://huggingface.co/datasets/maitrix-org/Voila-Benchmark |
+|Voila Voice Library| Millons of pre-build voices | https://huggingface.co/datasets/maitrix-org/Voila-million-voice
+# 📊 Benchmark
+## 1. Voila Benchmark
+We introduce a novel speech evaluation benchmark called the VoilaBenchmark. The Voila Benchmark is constructed by sampling from five widely used language model evaluation datasets: MMLU, MATH, OpenAI HumanEval, NQ-Open, and GSM8k. We compare our results with SpeechGPT and Moshi.
+| Model | Voila Benchmark |
+|-------|----------------|
+|SpeechGPT| 13.29|
+|Moshi | 11.45 |
+|**Voila** | **30.56** |
+_(higher is better)_
+For detailed scores of Voila Benchmark on each specific domain, please refer to our paper (Section 5.1 "Evaluation of Voila Benchmark").
+## 2. Evaluation of ASR
+As Voila supports multiple tasks, including Automatic Speech Recognition (ASR), Text-to-Speech(TTS), and spoken question answering, we also evaluate the performance of ASR and TTS.
+For ASR, we assess performance on the LibriSpeech test-clean dataset, using Word Error Rate (WER) as our metric. Voila attains a word error rate (WER) of 4.8%, outperforming the 5.7% reported by Moshi. In scenarios where both models utilize LibriSpeech training data, Voila achieves an impressive WER of 2.7%.
+| Model | LibriSpeech test-clean (WER) |
+|-------|-----------------------|
+|Whisper large v2|2.7|
+|Whisper large v3|2.2|
+|FastConformer|3.6|
+|VoxtLM |2.7|
+|Moshi |5.7|
+|**Voila (w/o LibriSpeech train split)** |**4.8**|
+|**Voila (with LibriSpeech train split)**|**2.7**|
+_(lower is better)_
+## 3. Evaluation of TTS
+For TTS, we follow the evaluation metrics proposed in Vall-E, which involves transcribing the generated audio using HuBERT-Large.
+Voila once again leads with a WER of 3.2% (and 2.8% when using LibriSpeech training data).
+| Model | LibriSpeech test-clean (WER) |
+|-------|-----------------------|
+|YourTTS |7.7|
+|Vall-E|5.9|
+|Moshi|4.7|
+|**Voila (w/o LibriSpeech train split)** |**3.2**|
+|**Voila (with LibriSpeech train split)** |**2.8**|
+_(lower is better)_
+# 📝 Citation
+If you find our work helpful, please cite us.
 ```
+@article{voila2025,
+  author    = {Yemin Shi, Yu Shu, Siwei Dong, Guangyi Liu, Jaward Sesay, Jingwen Li, Zhiting Hu},
+  title     = {Voila: Voice-Language Foundation Models for Real-Time Autonomous Interaction and Voice Roleplay},
+  eprint={2505.02707},
+  archivePrefix={arXiv},
+  primaryClass={cs.CL},
+  year      = {2025}
 }
 ```