Nova-Zero commited on
Commit
6a42fce
·
verified ·
1 Parent(s): 7bcbd9a

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ llama_cpp_python-0.3.36-py3-none-linux_x86_64.whl filter=lfs diff=lfs merge=lfs -text
37
+ llama_cpp_python-0.3.36+cu130.gemma2-py3-none-linux_x86_64.whl filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - llama-cpp-python
5
+ - llama.cpp
6
+ - cuda
7
+ - gguf
8
+ - embeddings
9
+ ---
10
+
11
+ # llama-cpp-python 0.3.36 (CUDA 13) prebuilt wheel
12
+
13
+ Prebuilt `llama-cpp-python` with CUDA, so you can install in seconds instead of compiling for 10-20 minutes. Made to run `unsloth/embeddinggemma-2-GGUF` (architecture `gemma-embedding2`), which the PyPI build cannot load.
14
+
15
+ ## Install
16
+
17
+ !pip install https://huggingface.co/Nova-Zero/llama-cpp-python-cuda-wheels/resolve/main/llama_cpp_python-0.3.36%2Bcu130.gemma2-py3-none-linux_x86_64.whl
18
+
19
+
20
+ Note: the `+` in the file name is written as `%2B` in the URL. `llama_cpp.__version__` prints `0.3.36`; `pip show llama-cpp-python` shows `0.3.36+cu130.gemma2`.
21
+
22
+ ## Works on
23
+ - Linux x86_64
24
+ - Python 3.13 (tested; the wheel is tagged `py3`, other 3.x versions are untested)
25
+ - CUDA 13 runtime (built with CUDA 13.0)
26
+ - GPUs with compute capability 7.5 (Tesla T4, as in Google Colab)
27
+
28
+ ## Does not work on
29
+ - Windows, macOS, ARM
30
+ - Other GPU generations (built only with `-DCMAKE_CUDA_ARCHITECTURES=75`)
31
+
32
+ ## What is inside
33
+ - llama-cpp-python 0.3.36
34
+ - llama.cpp commit `fc9ce6b9d52a8504edcb262abc92737c2289f96c` (2026-10-08)
35
+ - Built with `CMAKE_ARGS="-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=75"`
36
+ - Two small fixes to the Python bindings, needed because that llama.cpp commit changed its C structs:
37
+ 1. Added the missing `moe_cache_size` field to `llama_context_params` (without it, `embed()` crashes with a NULL pointer error).
38
+ 2. `embed()` now uses `llama_model_n_embd_out` for the vector size (otherwise embeddinggemma-2 returns 512 numbers instead of 768).
39
+
40
+ ## Tested
41
+ `embeddinggemma-2-BF16.gguf` with `embedding=True, n_gpu_layers=-1` on a Colab T4: 768-number embeddings, all layers offloaded to the GPU.
42
+
43
+ ## Credits
44
+ Built from [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) and [llama.cpp](https://github.com/ggml-org/llama.cpp), both MIT licensed.
llama_cpp_python-0.3.36+cu130.gemma2-py3-none-linux_x86_64.whl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:162d81a2a9458dc3fdd6fbcb0040485f1743336c19720e0f87112e60d1e6105e
3
+ size 271311101