README.md

metadata

license: apache-2.0
language:
  - en
  - hi
  - ta
  - te
  - ml
  - bn
  - mr
  - kn
base_model:
  - Aratako/MioCodec-25Hz-24kHz
pipeline_tag: audio-to-audio

Overview

• What: Lightweight toolkit to run MioCodec decoding in two modes: streaming (real‑time token→audio from an LLM websocket) and offline (JSONL of embeddings → WAV).
• Why: You can avoid pushing the entire MioCodec source to your HF model repo — push only checkpoints and a few small helper files instead.

Requirements

• See MioCodec-stream/requirements.txt for the base list. Minimal packages:
  • torch, numpy
  • git+https://github.com/Aratako/MioCodec.git (or miocodec package if available)
  • websockets (streaming)
  • soundfile (offline WAV write) or sounddevice (optional playback)

Repository contents (what to push to HF)

• weights/ — your lora_windowed checkpoint files (use Git LFS).
• streaming_istft_head.py — streaming-capable ISTFT head (small helper).
• windowed_streamer.py — windowed streamer implementation.
• mio_codec_wrapper.py — wrapper script for streaming and offline decode. See mio_codec_wrapper.py.
• requirements.txt — environment dependencies. See MioCodec-stream/requirements.txt.

Install (inference host)

1. Create a fresh venv and activate it:

python3 -m venv ~/mio_env
source ~/mio_env/bin/activate

2. Install dependencies:

pip install -r MioCodec-stream/requirements.txt

If you prefer to install MioCodec from the GitHub repo directly:

pip install git+https://github.com/Aratako/MioCodec.git

Usage — Streaming (LLM websocket → audio)

• Start/point your LLM (vLLM or other) to stream embeddings to a websocket endpoint (JSON messages). Expected message format (per message):

{"embedding": [0.12, 0.34, ...]}

Send a final message { "finished": true } when the utterance completes.

Run the wrapper in streaming mode (plays audio locally if --play):

python mydrive2/Indic-Mio/MioCodec/mio_codec_wrapper.py --mode stream \
  --uri ws://localhost:8000/stream --device cpu --play

Notes:

• The wrapper prefers local helper files (windowed_streamer.py, streaming_istft_head.py) when present; otherwise it will attempt to use the installed miocodec package and swap in the streaming head if available.
• Adjust --chunk-size, --left, --right options to tune streaming latency/quality.

Usage — Offline (JSONL tokens → WAV)

• Prepare a JSONL file where each line is a JSON object with an embedding array, e.g.:

{"embedding": [ ... ]}
{"embedding": [ ... ]}

Run offline decode:

python MioCodec-stream/mio_codec_wrapper.py --mode offline --input tokens.jsonl --out out.wav --device cpu

The wrapper will prefer the windowed_streamer workflow (exact frame accounting). If the streamer is not available it falls back to a best-effort one-shot decode (less exact).

Pushing weights to Hugging Face

• Only push the small helper files and the weights/ folder to your HF repo. Use Git LFS for large checkpoint files:

git lfs install
git lfs track "weights/*"
git add .gitattributes weights/ streaming_istft_head.py windowed_streamer.py mio_codec_wrapper.py requirements.txt
git commit -m "Add weights + streaming helpers"
git push

When calling MioCodecModel.from_pretrained() in the wrapper, you can pass the HF repo id (for example PRIYANSHUDHAKED/Mio-Codec-Stream) so the model code downloads the weights from the hub automatically.

Troubleshooting

• Import errors (e.g. missing BloomPreTrainedModel) usually come from incompatible transformers versions — use the version recommended by the model package or install inside a fresh venv.
• If audio shows seams, increase --right lookahead or ensure windowed_streamer.py is present so exact frame trimming is used.
• If playback is silent but files exist, inspect out.wav in a player; the wrapper may write raw float32 if soundfile is unavailable.

Next steps / options

• I can add a tiny README section that shows how to configure a vLLM or other LLM server to emit the expected websocket JSON format. Want that added?