---
license: apache-2.0
library_name: pytorch
tags:
  - audio-to-video
  - talking-head
  - digital-human
  - omnirt
  - opentalking
---

# QuickTalk

This repository contains the QuickTalk model files used by the OmniRT `quicktalk` audio-to-video runtime.

## Files

```text
quicktalk.pth   # QuickTalk PyTorch checkpoint
repair.npy      # QuickTalk repair parameters used by the runtime post-processing stage
```

This repository intentionally does **not** include third-party dependency weights such as HuBERT or InsightFace `buffalo_l`. Download those dependencies from their original sources according to their own licenses and place them next to these files when running OmniRT.

Expected runtime layout:

```text
$OMNIRT_MODEL_ROOT/quicktalk/
  quicktalk.pth
  repair.npy
  chinese-hubert-large/
    config.json
    preprocessor_config.json
    pytorch_model.bin
  auxiliary/models/buffalo_l/
    <InsightFace model files>
```

## Repair Parameters

`repair.npy` is a required QuickTalk runtime parameter file. It is not a standalone neural network checkpoint. The runtime loads it after the main QuickTalk model to apply fixed repair parameters during post-processing, helping map the model output back to the template face more consistently.

Keep `repair.npy` in the same QuickTalk model directory as `quicktalk.pth`.

## Usage With OmniRT

Set the model root and start the QuickTalk runtime through OmniRT:

```bash
export OMNIRT_MODEL_ROOT=/path/to/models
export OMNIRT_QUICKTALK_MODEL_ROOT="$OMNIRT_MODEL_ROOT/quicktalk"
export OMNIRT_QUICKTALK_RUNTIME=1
export OMNIRT_QUICKTALK_DEVICE=cuda:0
export OMNIRT_QUICKTALK_HUBERT_DEVICE=cuda:1

omnirt-serve --host 0.0.0.0 --port 9000
```

OpenTalking can then connect through the unified OmniRT endpoint:

```bash
export OMNIRT_ENDPOINT=http://127.0.0.1:9000
```

The QuickTalk WebSocket route is:

```text
/v1/audio2video/quicktalk
```

## Related Projects

- OpenTalking: <https://github.com/datascale-ai/opentalking>
- OmniRT: <https://github.com/datascale-ai/omnirt>

## Security Notes

`quicktalk.pth` is a PyTorch checkpoint. PyTorch checkpoint files are pickle-based, and unpickling arbitrary `.pth` files can execute code. For that reason, Hugging Face security scanners may flag this file as suspicious or unsafe, for example via Picklescan or generic Python obfuscation signatures.

This warning is expected for many pickle-based PyTorch checkpoints and does not by itself prove that the file is malicious. It does mean you should treat the file as executable input:

- only load it in a trusted environment;
- only use it with the intended OmniRT QuickTalk loader;
- do not load it with generic `torch.load` in an untrusted or multi-tenant environment;
- prefer isolated runtime users, containers, or virtual machines for production deployments.

The uploaded files in this repository are limited to the QuickTalk checkpoint and repair parameters:

```text
quicktalk.pth
repair.npy
```

No runtime caches, template caches, HuBERT weights, or InsightFace weights are included.