Updated readme

README.md

@@ -1,24 +1,116 @@
+---
+library_name: pytorch
+tags:
+- chimera-ml
+- oragen
+- pytorch
+- audio
+- image
+- multimodal
+- age-estimation
+- gender-recognition
+- wav2vec2
+- vit
+datasets:
+- AGENDER
+- CommonVoice
+- TIMIT
+- LAGENDA
+- IMDB-clean
+- AFEW
+- VoxCeleb2
+- BRAVE-MASKS
+base_model:
+- facebook/wav2vec2-large-robust
+- nateraw/vit-age-classifier
+---
+
 # ORAGEN Models
 
-This directory contains exported ORAGEN-based model weights for [`chimera-ml`](https://github.com/markitantov/chimera_ml).
+This repository contains exported ORAGEN-based model weights for [`chimera-ml`](https://github.com/markitantov/chimera_ml/).
 
-ORAGEN is an audio-visual modeling approach designed for age estimation and gender recognition from human speech and face data. These models are intended for research and inference pipelines built on top of `chimera-ml`.
+These checkpoints are used for age estimation and gender recognition from speech, face images, and combined audio-visual inputs. In the `chimera-ml` ORAGEN pipeline, the multimodal model operates on intermediate audio and visual features extracted from the unimodal branches.
 
 ## Files
 
-- `audio_model.pt` — audio-only ORAGEN model for prediction from speech.
-- `image_model.pt` — image-only ORAGEN model for prediction from face images.
-- `multimodal_model.pt` — audio-visual ORAGEN model that combines both modalities.
+- `audio_model.pt` — audio-only checkpoint used for speech-based age estimation and gender recognition.
+- `image_model.pt` — image-only checkpoint used for face-based feature extraction and prediction in the ORAGEN pipeline.
+- `multimodal_model.pt` — audio-visual checkpoint that combines audio and image features for multimodal prediction.
 
 ## What They Predict
 
-These models are used to predict:
-- age
-- gender [female, male]
+These models predict:
 
-In the broader ORAGEN setup, multimodal models may also be used for protective mask-related prediction, depending on the training configuration.
+- age (0-100)
+- gender (`female`, `male`)
 
+The ORAGEN codebase also contains support for mask-related prediction in some model variants, but the exported multimodal configuration used here has `include_mask: false`.
 
----
-license: apache-2.0
----
+## Training Setup
+
+According to the training configs in `examples/oragen/configs`:
+
+- Audio training uses `facebook/wav2vec2-large-robust` as the backbone.
+- The multimodal setup uses `agender_multimodal_model_v3`.
+- The visual branch is used as an image feature extractor in the fusion pipeline and is referenced together with `nateraw/vit-age-classifier`-based ORAGEN visual weights.
+- Training and inference use `16 kHz` audio and `4s` windows with `2s` shift.
+
+Datasets referenced by the configs:
+
+- Audio: `AGENDER`, `CommonVoice`, `TIMIT`
+- Image: `LAGENDA`, `IMDB-Clean`, `AFEW`
+- Multimodal: `VoxCeleb2`, `BRAVE-MASKS`
+
+## Per-Corpus Results
+
+The training logs do not report raw accuracy directly. For gender prediction, the reported classification metrics are `gen_precision`, `gen_uar`, and `gen_macro_f1`. For age prediction, the reported regression metrics are `age_mae` and `age_pcc`.
+
+## Results from the original paper
+
+### Audio Model
+
+| Corpus | Age MAE | Age PCC | Gender UAR, % | Gender Macro F1, % |
+|--------|---------|---------|------------|-----------------|
+| AGENDER | 10.60 | 0.83 | 87.17 | 86.25 |
+| CommonVoice | 10.47 | 0.81 | 92.59 | 92.64 |
+| TIMIT | 6.90 | 0.91 | 98.60 | 98.58 |
+| VoxCeleb2 | 9.91 | 0.60 | 90.00 | 88.71 |
+| BRAVE-MASKS (test) | 11.89 | 0.64 | 86.22 | 85.18 |
+
+### Image Model
+
+| Corpus | Age MAE | Age PCC | Gender UAR, % | Gender Macro F1, % |
+|--------|---------|---------|------------|-----------------|
+| LAGENDA | 5.18 | 0.95 | 92.89 | 92.90 |
+| AFEW | 5.62 | 0.82 | 95.16 | 94.98 |
+| IMDB-Clean (test) | 5.47 | 0.84 | 98.37 | 98.26 |
+| VoxCeleb2 | 5.97 | 0.64 | 98.37 | 98.16 |
+| BRAVE-MASKS (test) | 8.71 | 0.74 | 94.44 | 94.43 |
+
+### Multimodal Model (intermediate fusion)
+
+| Corpus | Age MAE | Age PCC | Gender UAR, % | Gender Macro F1, % |
+|--------|---------|---------|------------|-----------------|
+| VoxCeleb2 | 5.68 | 0.66 | 99.11 | 99.02 |
+| BRAVE-MASKS (test) | 8.73 | 0.74 | 94.95 | 94.89 |
+
+
+## 6) Related publications
+
+Markitantov M., Ryumina E., Karpov A. Audio-visual occlusion-robust gender recognition and age estimation approach based on multi-task cross-modal attention. // Expert Systems with Applications. 2026. vol. 296. ID 127473. https://doi.org/10.1016/j.eswa.2025.127473
+
+BibTeX:
+
+```bibtex
+@article{markitantov2026oragen,
+  author = {Markitantov, Maxim and Ryumina, Elena and Karpov, Alexey},
+  title = {Audio-visual occlusion-robust gender recognition and age estimation approach based on multi-task cross-modal attention},
+  journal = {Expert Systems with Applications},
+  volume = {296},
+  pages = {127473},
+  year = {2026},
+  month = jan,
+  doi = {10.1016/j.eswa.2025.127473},
+  url = {https://doi.org/10.1016/j.eswa.2025.127473}
+}
+```