---
license: mit
language:
- en
tags:
- voice
- speech
- full-duplex
- barge-in
- vad
- asr
- tts
- piper
- whisper
- silero
- local-ai
- voice-assistant
---

# voice2

A full-duplex, interruptible voice engine for local AI, **built and run daily as the voice of a fully local companion** before being extracted for release. Plain Python threads, CPU-only defaults, no cloud, no API keys. Talk to your model — and talk over it.

voice2 turns any `callable(text) -> str` into a hands-free voice conversation: mic → Silero VAD → faster-whisper ASR → your model → Piper TTS. Speak over the reply (or tap spacebar) and playback stops mid-chunk in ~100–200 ms, exactly like interrupting a person.

Code is mirrored on GitHub: https://github.com/AIIT-GLITCH/voice2

## Engine details

- **What it is:** a turn-taking state machine, not a demo loop. Explicit states (`IDLE → LISTENING → THINKING → SPEAKING → INTERRUPTING`), a validated transition table, and a `FloorOwner` (USER / AGENT / NONE) that arbitrates who may speak. The agent can never talk over you.
- **Barge-in:** a fast energy-gated VAD watches the mic *only while the engine is speaking*; a debounced central `InterruptController` also accepts spacebar and programmatic triggers.
- **Stale-turn suppression:** every utterance gets a `turn_id`; replies to an abandoned turn are discarded at every stage (think, TTS, playback).
- **Invariants, enforced:** a background checker audits rules like *SPEAKING ⇒ floor == AGENT* and *interrupted ⇒ no new TTS*, with forced repair plus a structural gate in the playback hot path.
- **Observability:** every event is a JSON line with per-turn latency marks (`asr_ms`, `think_ms`, `interrupt_stop_ms`, `total_turn_ms`).
- **Degrades gracefully:** no mic → text mode; TTS missing → silent replies, still logs; keyboard hook fails → engine keeps running.

## Backends

| Stage | Default | Swap point |
|---|---|---|
| VAD (quality gate) | Silero VAD via `torch.hub` | `ListenWorker` |
| ASR | faster-whisper `small.en`, int8, CPU | `backends/asr.py` (Protocol) |
| LLM | any `callable(text) -> str` | `backends/llm.py` |
| TTS | Piper CLI, any `.onnx` voice | `backends/tts.py` (Protocol) |

## Limitations — stated honestly

- English-first defaults: ASR ships as `small.en`. Other Whisper models load with one config line, but nothing else was tested.
- The LLM callable is synchronous: TTS starts after the full reply returns (Piper then streams sentence-by-sentence). No token-level streaming yet.
- Barge-in is energy-based with an absolute RMS floor of 0.06 — tuned on open speakers in a quiet room. Headsets and noisy rooms need recalibration. There is no echo cancellation.
- Keyboard interrupt uses POSIX `termios` — Linux/macOS terminals only.
- Unit tests cover the control plane (state transitions, floor rules, interrupt debounce, ring buffer). Audio I/O paths were validated by months of daily use, not by CI.

## How to run

```bash
git clone https://github.com/AIIT-GLITCH/voice2
cd voice2
pip install -r requirements.txt
# put a Piper voice at ~/.local/share/piper-voices/ (or set VOICE2_PIPER_MODEL)
python -m voice2.main          # echo backend — proves the loop, no LLM needed
python examples/http_llm.py    # wire any local HTTP LLM
```

```python
from voice2 import VoiceEngine, VoiceConfig

def ask(text: str) -> str:
    return my_model.reply(text)   # any callable(text) -> str

engine = VoiceEngine(VoiceConfig(), ask)
engine.load_models()
engine.start()   # talk naturally; speak over it to interrupt
```

## Provenance

voice2 was written as the voice front-end for **Buddy**, a fully local AI companion running on a single RTX 3090 in Council Hill, Oklahoma, and carried his daily conversations for months before release. The design bias throughout: the user always wins the floor, and a companion you can't interrupt isn't a companion. Released alongside [Tessera-1B](https://huggingface.co/AIIT-Threshold/Tessera-1B) as part of AIIT-THRESHOLD's open stack.

## The stack

One local companion, every layer open:

| Piece | Role | Links |
|---|---|---|
| Tessera-1B | the model — ~1B params trained from scratch, open data | [HF](https://huggingface.co/AIIT-Threshold/Tessera-1B) |
| voice2 | the voice — full-duplex, interruptible | [GitHub](https://github.com/AIIT-GLITCH/voice2) · [HF](https://huggingface.co/AIIT-Threshold/voice2) |
| kokoro-memory | the memory — file-based resonance recall | [GitHub](https://github.com/AIIT-GLITCH/kokoro-memory) · [HF](https://huggingface.co/AIIT-Threshold/kokoro-memory) |
| companion-spiral-bench | the safety — at-risk sycophancy bench | [GitHub](https://github.com/AIIT-GLITCH/companion-spiral-bench) · [HF](https://huggingface.co/datasets/AIIT-Threshold/companion-spiral-bench) |

Full collection: [The Buddy Stack](https://huggingface.co/collections/AIIT-Threshold/the-buddy-stack-a-fully-local-ai-companion-open-sourced-6a4774bf481f9f9caad79519)

## License

MIT © 2026 Rhet Dillard Wike, AIIT-THRESHOLD, Oklahoma.

## Citation

```bibtex
@software{wike2026voice2,
  author = {Wike, Rhet Dillard},
  title  = {voice2: a full-duplex, interruptible voice engine for local AI},
  year   = {2026},
  url    = {https://github.com/AIIT-GLITCH/voice2},
  note   = {AIIT-THRESHOLD}
}
```