J.B-Lin commited on
Commit
6230c98
Β·
1 Parent(s): e770d07

fix: remove --voxcpm2-base-lm/acoustic args (not valid for llama-server CLI)

Browse files

llama-server does not have voxcpm2 CLI args. TTS model paths are passed
via omni_init JSON body (model_dir/tts_bin_dir).

Files changed (2) hide show
  1. modal_deploy/deploy_omni.py +0 -4
  2. tmp_omni_output.txt +25 -0
modal_deploy/deploy_omni.py CHANGED
@@ -171,8 +171,6 @@ def serve():
171
  llama_server_bin,
172
  "-m", paths["main"],
173
  "--mmproj", paths["vision"],
174
- "--voxcpm2-base-lm", paths["tts_base_lm"],
175
- "--voxcpm2-acoustic", paths["tts_acoustic"],
176
  "--host", "127.0.0.1",
177
  "--port", str(LLAMA_SERVER_PORT),
178
  "-ngl", "99",
@@ -894,8 +892,6 @@ def test_omni():
894
  ls_bin,
895
  "-m", paths["main"],
896
  "--mmproj", paths["vision"],
897
- "--voxcpm2-base-lm", paths["tts_base_lm"],
898
- "--voxcpm2-acoustic", paths["tts_acoustic"],
899
  "--host", "127.0.0.1",
900
  "--port", str(LLAMA_SERVER_PORT),
901
  "-ngl", "99",
 
171
  llama_server_bin,
172
  "-m", paths["main"],
173
  "--mmproj", paths["vision"],
 
 
174
  "--host", "127.0.0.1",
175
  "--port", str(LLAMA_SERVER_PORT),
176
  "-ngl", "99",
 
892
  ls_bin,
893
  "-m", paths["main"],
894
  "--mmproj", paths["vision"],
 
 
895
  "--host", "127.0.0.1",
896
  "--port", str(LLAMA_SERVER_PORT),
897
  "-ngl", "99",
tmp_omni_output.txt ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ο»Ώβœ“ Initialized. View run at
2
+ https://modal.com/apps/andrew-jiabin/main/ap-Oe0oWTtAGqGqJ6ZS8Pesj5
3
+ βœ“ Created objects.
4
+ β”œβ”€β”€ πŸ”¨ Created mount PythonPackage:modal_deploy
5
+ β”œβ”€β”€ πŸ”¨ Created mount
6
+ β”‚ C:\Users\Andre\codes\LJB\hackthon\for_qclaw\PregoPal\modal_deploy\llamacpp_
7
+ β”‚ omni
8
+ β”œβ”€β”€ πŸ”¨ Created web function serve =>
9
+ β”‚ https://andrew-jiabin--prego-pal-minicpm-omni-serve-dev.modal.run
10
+ β”œβ”€β”€ πŸ”¨ Created function diagnose_volume.
11
+ β”œβ”€β”€ πŸ”¨ Created function test_inference.
12
+ └── πŸ”¨ Created function test_omni.
13
+ [PregoPal] ========== TEST OMNI (llama-server built-in endpoints) ==========
14
+ [PregoPal] main: /models/MiniCPM-o-4_5-gguf/MiniCPM-o-4_5-Q4_K_M.gguf (exists=True)
15
+ [PregoPal] vision: /models/MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf (exists=True)
16
+ [PregoPal] audio: /models/MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf (exists=True)
17
+ [PregoPal] tts_base_lm: /models/MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (exists=True)
18
+ [PregoPal] tts_acoustic: /models/MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf (exists=True)
19
+ [PregoPal] token2wav_dir: /models/MiniCPM-o-4_5-gguf/token2wav-gguf (exists=True)
20
+ Starting llama-server with omni support...
21
+ [FAIL] llama-server not ready after 90s
22
+ [llama-server stderr] (1 lines):
23
+ error: invalid argument: --voxcpm2-base-lm
24
+ βœ“ App completed. View run at
25
+ https://modal.com/apps/andrew-jiabin/main/ap-Oe0oWTtAGqGqJ6ZS8Pesj5