---
license: apache-2.0
language:
- en
base_model:
- google/gemma-4-26B-A4B-it
tags:
- Audio-Text-to-Text
- multimodal
---
![Gemini_Generated_Image_4lbcxb4lbcxb4lbc](https://cdn-uploads.huggingface.co/production/uploads/61b37e66986f43ddf4956d21/5saouXyVqseQS1XpI-HY7.png)

## Model Details

* **Model Name:** LFG-1 (Listening Fusion Gemma)
* **Model Type:** Multimodal Conversational Audio-Language Model
* **Background:** LFG-1 was built as a personal learning project, trained entirely on Apple Silicon at my house on nights and weekends. It isn't an official model.
* **Architecture:** LFG-1 bridges the **Gemma 4 E2B audio encoder** with the **Gemma 4 26B-A4B text model**. The connection is made via a custom-trained projection layer, allowing the language model to natively ingest and understand raw acoustic features without relying on an intermediate text transcript.
* **Framework:** MLX (`mlx-vlm`)
* **License:** Apache 2.0

## Intended Use

* **Primary Use Case:** Real-time conversational audio applications running locally on Mac hardware.
* **Capabilities:** Native speech-to-response generation, streaming text output, and multimodal support (simultaneous audio and image input). Because the model processes raw acoustic data, it captures conversational pacing, pauses, and tone that traditional Speech-to-Text (STT) pipelines strip away.
* **Untouched Text Reasoning:** LFG-1 keeps Gemma 4’s text backbone frozen during audio-projection training, so its core language and reasoning capabilities are expected to remain essentially unchanged while gaining a trained audio input pathway.

## Language Support

* **Current Status:** The audio projection layer is currently trained exclusively on **English**. 
* **Future Roadmap:** This is an active learning project (hence the "1" in LFG-1). Future iterations will focus on expanding the audio training data. Requests, suggestions, and dataset recommendations for additional languages are highly welcome in the repository Issues!

## Hardware Requirements

Due to the size of the combined weights (~48 GB), LFG-1 requires substantial unified memory.
* **Minimum Requirement:** Apple Silicon with at least **64 GB of Unified Memory**.
* **Storage:** ~50 GB of disk space for local weights.

## Install

Install `mlx-vlm`, then clone and install the LFG wrapper code:

```bash
pip install mlx-vlm
git clone https://github.com/codemadeio/LFG-1.git
cd LFG-1
pip install -e 
```

## Usage

LFG-1 is designed as a drop-in model type for `mlx-vlm`. Once the `lfg` module is registered, it can be loaded like any other Hugging Face model.

```python

from lfg import register
register()

from mlx_vlm import load, generate

model, processor = load("glenn2/LFG-1")

response = generate(
    model, processor,
    prompt="<|audio|>",
    audio=["recording.wav"],
    max_tokens=512,
)
print(response.text)

```

## Example
```python
git clone https://github.com/codemadeio/LFG-1.git
cd LFG-1/examples
pip install flask
brew install ffmpeg
python app.py
# open http://127.0.0.1:5001
```