Image-to-Video
ZyaireYao commited on
Commit
31c6a45
·
verified ·
1 Parent(s): 76b1259

Update QuickTalk model card

Browse files
Files changed (1) hide show
  1. README.md +80 -0
README.md CHANGED
@@ -1,3 +1,83 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: pytorch
4
+ tags:
5
+ - audio-to-video
6
+ - talking-head
7
+ - digital-human
8
+ - omnirt
9
+ - opentalking
10
  ---
11
+
12
+ # QuickTalk
13
+
14
+ This repository contains the QuickTalk model files used by the OmniRT `quicktalk` audio-to-video runtime.
15
+
16
+ ## Files
17
+
18
+ ```text
19
+ quicktalk.pth # QuickTalk PyTorch checkpoint
20
+ repair.npy # QuickTalk repair parameters
21
+ ```
22
+
23
+ This repository intentionally does **not** include third-party dependency weights such as HuBERT or InsightFace `buffalo_l`. Download those dependencies from their original sources according to their own licenses and place them next to these files when running OmniRT.
24
+
25
+ Expected runtime layout:
26
+
27
+ ```text
28
+ $OMNIRT_MODEL_ROOT/quicktalk/
29
+ quicktalk.pth
30
+ repair.npy
31
+ chinese-hubert-large/
32
+ config.json
33
+ preprocessor_config.json
34
+ pytorch_model.bin
35
+ auxiliary/models/buffalo_l/
36
+ *.onnx
37
+ ```
38
+
39
+ ## Usage With OmniRT
40
+
41
+ Set the model root and start the QuickTalk runtime through OmniRT:
42
+
43
+ ```bash
44
+ export OMNIRT_MODEL_ROOT=/path/to/models
45
+ export OMNIRT_QUICKTALK_MODEL_ROOT="$OMNIRT_MODEL_ROOT/quicktalk"
46
+ export OMNIRT_QUICKTALK_RUNTIME=1
47
+ export OMNIRT_QUICKTALK_DEVICE=cuda:0
48
+ export OMNIRT_QUICKTALK_HUBERT_DEVICE=cuda:1
49
+
50
+ omnirt-serve --host 0.0.0.0 --port 9000
51
+ ```
52
+
53
+ OpenTalking can then connect through the unified OmniRT endpoint:
54
+
55
+ ```bash
56
+ export OMNIRT_ENDPOINT=http://127.0.0.1:9000
57
+ ```
58
+
59
+ The QuickTalk WebSocket route is:
60
+
61
+ ```text
62
+ /v1/audio2video/quicktalk
63
+ ```
64
+
65
+ ## Security Notes
66
+
67
+ `quicktalk.pth` is a PyTorch checkpoint. PyTorch checkpoint files are pickle-based, and unpickling arbitrary `.pth` files can execute code. For that reason, Hugging Face security scanners may flag this file as suspicious or unsafe, for example via Picklescan or generic Python obfuscation signatures.
68
+
69
+ This warning is expected for many pickle-based PyTorch checkpoints and does not by itself prove that the file is malicious. It does mean you should treat the file as executable input:
70
+
71
+ - only load it in a trusted environment;
72
+ - only use it with the intended OmniRT QuickTalk loader;
73
+ - do not load it with generic `torch.load` in an untrusted or multi-tenant environment;
74
+ - prefer isolated runtime users, containers, or virtual machines for production deployments.
75
+
76
+ The uploaded files in this repository are limited to the QuickTalk checkpoint and repair parameters:
77
+
78
+ ```text
79
+ quicktalk.pth
80
+ repair.npy
81
+ ```
82
+
83
+ No runtime caches, template caches, HuBERT weights, or InsightFace weights are included.