| --- |
| license: mit |
| datasets: |
| - inesc-id/FalAR |
| - inesc-id/camoes_asr |
| language: |
| - pt |
| base_model: |
| - amalia-llm/AMALIA-9B-1225-SFT |
| - facebook/seamless-m4t-v2-large |
| pipeline_tag: automatic-speech-recognition |
| --- |
| # Model card |
|
|
| AMALIA-FALA-ABERTO is a Portuguese ASR checkpoint based on the [SLAM-ASR architecture](https://arxiv.org/abs/2402.08846). It connects a pre-trained speech encoder to a large language model through a learned projection module, allowing the LLM to generate text transcriptions from speech representations. |
|
|
| ## Architecture |
|
|
| The model follows the SLAM-ASR design: |
|
|
| ```text |
| audio → speech encoder → projector → LLM → transcription |
| ``` |
|
|
| The speech encoder extracts acoustic representations from the input audio. These representations are mapped by a trainable projector into the embedding space expected by the language model. The LLM then autoregressively generates the transcription. |
|
|
| ## Components |
|
|
| * **Base architecture:** SLAM-ASR |
| * **Speech encoder:** `Seamless`, also available on Hugging Face |
| * **Language model:** `AMALIA-9B-1225-SFT` |
| * **Task:** Automatic Speech Recognition |
| * **Language:** Portuguese, with focus on European Portuguese |
|
|
| ## Intended Use |
|
|
| This model is intended for research and experimentation on European Portuguese ASR, especially in settings where speech representations are connected to an LLM instead of using a standalone encoder-decoder ASR model. |
|
|
| Example use cases include: |
|
|
| * transcribing Portuguese speech; |
| * evaluating LLM-augmented ASR systems; |
| * comparing SLAM-style ASR with conventional ASR models; |
| * research on European Portuguese speech processing. |
|
|
| ## AMALIA-FALA Usage |
|
|
| AMALIA-FALA models are split into two parts: |
|
|
| 1. **Model repository**: contains the model weights and example assets. |
| 2. **Loader repository**: contains the lightweight Python code used to load and run the model. |
|
|
| The model weights are **not cloned manually** by the user. They are downloaded automatically from Hugging Face the first time the model is loaded. |
|
|
| --- |
|
|
| ### 1. Install the loader package |
|
|
| Install the lightweight AMALIA-FALA loader directly from Hugging Face: |
|
|
| ```bash |
| pip install git+https://huggingface.co/amalia-llm/amalia-fala-loader |
| ``` |
|
|
| --- |
|
|
| ### 2. Run inference on your own audio file |
|
|
| ```python |
| from amalia_fala import pipeline |
| |
| asr = pipeline( |
| "amalia-llm/AMALIA-FALA-ABERTO", |
| device="cuda:0", |
| ) |
| |
| result = asr("/path/to/audio.wav") |
| print(result["text"]) |
| ``` |
|
|
| The checkpoint is downloaded automatically from: |
|
|
| ```text |
| amalia-llm/AMALIA-FALA-ABERTO |
| ``` |
|
|
| and cached locally by Hugging Face. Future runs reuse the cached checkpoint. |
|
|
| --- |
|
|
| ### 3. Load the model directly |
|
|
| If you need access to the underlying model and tokenizer: |
|
|
| ```python |
| from amalia_fala import load_model |
| |
| model, tokenizer, model_config, dataset_config = load_model( |
| "amalia-llm/AMALIA-FALA-ABERTO", |
| device="cuda:0", |
| ) |
| ``` |
|
|
| --- |
|
|
| ### Notes |
|
|
| - The loader package contains only the Python loading code. |
| - The model repositories contain the actual checkpoints. |
| - The checkpoint is downloaded automatically on first use. |
| - The Hugging Face cache is reused after the first download. |
| - A GPU is recommended for practical inference speed. |
|
|
| ## Training Data |
|
|
| The model was developed in the context of European Portuguese ASR experiments using [FalAR](https://arxiv.org/abs/2605.27062) and [CAMÕES](https://arxiv.org/abs/2508.19721) ASR data. |
| FalAR contains more than 5000 hours of parliamentary speech from the *Assambleia da República*. |
| CAMÕES consists of a curated collection of up to 14 sub-corpora containing different domains and speech styles, totaling approximately 425 hours of speech. |
| The encoder `AMALIA-speech-encoder` was trained separately and reused as the speech encoder in this SLAM-ASR setup. |
|
|
| ## Limitations |
|
|
| This checkpoint is intended as a research artifact. Performance may vary depending on audio quality, speaker domain, recording conditions, and transcription style. The model may be less reliable on noisy audio, long-form speech, code-switching, or domains that differ from the training data. |
|
|
| ## Citation / Acknowledgements |
|
|
| This model builds on the SLAM-ASR idea of connecting a speech encoder to a large language model for automatic speech recognition. |
