File size: 2,884 Bytes
f2ad81a 7970a1a f2ad81a cc00fe2 f2ad81a 80c9ef5 2904bfe 3dcc37f f2ad81a 2904bfe f2ad81a 1d19a7d f2ad81a f95675a f2ad81a 2904bfe f2ad81a f362eb8 f2ad81a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | ---
language:
- en
pipeline_tag: audio-text-to-text
tags:
- audio
- speech
- voice-assistant
- voicebench
- gemma
---
LFG-3 is an audio-language model that fuses the best-in-class conversational intelligence of Gemma 4 31B with a Parakeet audio encoder through a trained projection layer. Speech goes in, text comes out.
The model is the third iteration in a personal learning journey to answer the question: "Can I stand on the shoulders of giants and use limited compute resources to build a standout model that can understand what and how you say things, not just speech to text?"
The answer is Yes. LFG-3 was trained entirely on a single H100 from my house on nights and weekends. It ranks #1 on Voice Bench at the time of this release.
LFG-3 serves as an open exploration of just how adaptable the Gemma architecture can be for independent developers building multimodal applications.
## Voice Bench Results
| Subset |Metric | Score |
| --- | --- | ---: |
| AlpacaEval | (1-5, GPT) | 4.73 |
| CommonEval | (1-5, GPT) | 4.40 |
| WildVoice | (1-5, GPT) | 4.45 |
| SD-QA | (% GPT majority) | 78.12 |
| MMSU | (% accuracy) | 85.52 |
| OpenBookQA | (% accuracy) | 94.73 |
| BBH | (% accuracy) | 92.20 |
| IFEval |(% strict-loose avg) | 88.54 |
|AdvBench | (% refusal rate) | 98.27 |
| Overall | | 89.88 |
## Usage
```python
import soundfile as sf
from transformers import AutoModelForMultimodalLM, AutoProcessor
MODEL = "glenn2/LFG-3"
processor = AutoProcessor.from_pretrained(MODEL, trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained(
MODEL, trust_remote_code=True, dtype="bfloat16", device_map="cuda"
)
audio, sr = sf.read("question.wav") # 16 kHz mono
messages = [
{"role": "system", "content": [{"type": "text", "text": "You are a helpful voice assistant. The user is speaking to you, and your reply will be read aloud."}]},
{"role": "user", "content": [{"type": "audio", "audio": audio}]}, # Audio should be 16 kHz mono.
]
# Process input
inputs = processor.apply_chat_template(
messages,
tokenize=True,
return_dict=True,
return_tensors="pt",
add_generation_prompt=True,
enable_thinking=True
).to(model.device)
input_len = inputs["input_ids"].shape[-1]
# Generate output
outputs = model.generate(**inputs, max_new_tokens=4096)
response = processor.decode(outputs[0][input_len:], skip_special_tokens=False)
# Parse output
print(processor.parse_response(response)["content"])
```
## Intended use
- Designed for **English** spoken questions/instructions → text answers.
- Inherits knowledge from the Gemma 4 31B IT model.
|