ArGrigorov commited on
Commit
bced3e1
Β·
verified Β·
1 Parent(s): eba01d2

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +45 -0
  2. src/llama-cpp-src.tar.gz +3 -0
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # llama.cpp (fork) β€” Gemma 4 + audio (mtmd)
2
+
3
+ Source snapshot of the private llama.cpp fork used by AGIWS inference workers.
4
+
5
+ ## What is inside
6
+
7
+ - `src/models/gemma4.cpp`, `gemma4-assistant.cpp` β€” text models `gemma4`, `gemma4-assistant`.
8
+ - `tools/mtmd/clip.cpp` β€” graphs `clip_graph_gemma4a` / `clip_graph_gemma4ua` (audio encoders).
9
+ - `tools/mtmd/mtmd-audio.cpp` β€” preprocessors `gemma4a` / `gemma4ua`, mel filters, streaming.
10
+ - `tools/mtmd/mtmd.h` β€” `mtmd_support_audio`, `mtmd_bitmap_init_from_audio`.
11
+ - `tools/server` β€” OpenAI-compatible server accepting `input_audio` content parts.
12
+ - `conversion/gemma.py` β€” `Gemma4Model` GGUF converter (`Gemma4ForConditionalGeneration`).
13
+
14
+ ## Files
15
+
16
+ - `src/llama-cpp-src.tar.gz` β€” full source tree (no `.git`, no build artifacts).
17
+ - `lib/*.so` β€” prebuilt CUDA libraries and `llama-server` for T4 (sm_75), uploaded by the build job.
18
+ - `lib/BUILD_INFO.txt` β€” compiler, CUDA version, git-less snapshot hash, sha256 of every artifact.
19
+
20
+ ## Build (Colab T4)
21
+
22
+ ```bash
23
+ tar -xzf src/llama-cpp-src.tar.gz -C /content/llama-cpp-fork
24
+ cmake -S /content/llama-cpp-fork -B /content/llama-cpp-fork/build \
25
+ -DCMAKE_BUILD_TYPE=Release \
26
+ -DGGML_CUDA=ON \
27
+ -DGGML_NATIVE=OFF \
28
+ -DCMAKE_CUDA_ARCHITECTURES=75 \
29
+ -DBUILD_SHARED_LIBS=ON \
30
+ -DLLAMA_CURL=OFF \
31
+ -DLLAMA_BUILD_TESTS=OFF \
32
+ -DLLAMA_BUILD_EXAMPLES=OFF
33
+ cmake --build /content/llama-cpp-fork/build -j"$(nproc)" --target llama-server
34
+ ```
35
+
36
+ Artifacts needed at runtime: `llama-server` plus `libllama.so*`, `libmtmd.so*`, `libggml*.so*`
37
+ (`libggml-base`, `libggml-cpu`, `libggml-cuda`, `libggml`).
38
+
39
+ Runtime flags for Gemma audio:
40
+
41
+ ```bash
42
+ LD_LIBRARY_PATH=/content/llama/lib \
43
+ /content/llama/lib/llama-server -m .../gemma-4-12b-it-Q4_K_M.gguf \
44
+ --mmproj .../mmproj-F16.gguf -ngl 99 -c 4096 --host 127.0.0.1 --port 8091
45
+ ```
src/llama-cpp-src.tar.gz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:654fd863f3de8fcf3bb12d47a065a0bda02e48e752010f194c0f5ff783a796b4
3
+ size 15055639