Hugging Face's logo

datascale-ai
/
quicktalk

PyTorch
audio-to-video
talking-head
digital-human
omnirt
opentalking
Model card Files
xet
 Community
quicktalk / README.md
ZyaireYao's picture
ZyaireYao
Add related project links
4111522 verified
preview code
|
raw
history blame contribute delete
3.05 kB
metadata 
license: apache-2.0
library_name: pytorch
tags:
  - audio-to-video
  - talking-head
  - digital-human
  - omnirt
  - opentalking

QuickTalk

This repository contains the QuickTalk model files used by the OmniRT quicktalk audio-to-video runtime.

Files 

quicktalk.pth   # QuickTalk PyTorch checkpoint
repair.npy      # QuickTalk repair parameters used by the runtime post-processing stage

This repository intentionally does not include third-party dependency weights such as HuBERT or InsightFace buffalo_l. Download those dependencies from their original sources according to their own licenses and place them next to these files when running OmniRT.

Expected runtime layout:

$OMNIRT_MODEL_ROOT/quicktalk/
  quicktalk.pth
  repair.npy
  chinese-hubert-large/
    config.json
    preprocessor_config.json
    pytorch_model.bin
  auxiliary/models/buffalo_l/
    <InsightFace model files>

Repair Parameters

repair.npy is a required QuickTalk runtime parameter file. It is not a standalone neural network checkpoint. The runtime loads it after the main QuickTalk model to apply fixed repair parameters during post-processing, helping map the model output back to the template face more consistently.

Keep repair.npy in the same QuickTalk model directory as quicktalk.pth.

Usage With OmniRT

Set the model root and start the QuickTalk runtime through OmniRT:

export OMNIRT_MODEL_ROOT=/path/to/models
export OMNIRT_QUICKTALK_MODEL_ROOT="$OMNIRT_MODEL_ROOT/quicktalk"
export OMNIRT_QUICKTALK_RUNTIME=1
export OMNIRT_QUICKTALK_DEVICE=cuda:0
export OMNIRT_QUICKTALK_HUBERT_DEVICE=cuda:1

omnirt-serve --host 0.0.0.0 --port 9000

OpenTalking can then connect through the unified OmniRT endpoint:

export OMNIRT_ENDPOINT=http://127.0.0.1:9000

The QuickTalk WebSocket route is:

/v1/audio2video/quicktalk

Related Projects

Security Notes

quicktalk.pth is a PyTorch checkpoint. PyTorch checkpoint files are pickle-based, and unpickling arbitrary .pth files can execute code. For that reason, Hugging Face security scanners may flag this file as suspicious or unsafe, for example via Picklescan or generic Python obfuscation signatures.

This warning is expected for many pickle-based PyTorch checkpoints and does not by itself prove that the file is malicious. It does mean you should treat the file as executable input:

  • only load it in a trusted environment;
  • only use it with the intended OmniRT QuickTalk loader;
  • do not load it with generic torch.load in an untrusted or multi-tenant environment;
  • prefer isolated runtime users, containers, or virtual machines for production deployments.

The uploaded files in this repository are limited to the QuickTalk checkpoint and repair parameters:

quicktalk.pth
repair.npy

No runtime caches, template caches, HuBERT weights, or InsightFace weights are included.