Spaces:
Configuration error
Configuration error
Update README.md
Browse files
README.md
CHANGED
|
@@ -9,31 +9,17 @@
|
|
| 9 |
[docs-shield]: https://img.shields.io/badge/DeepWiki-009485?style=for-the-badge&logo=readthedocs&logoColor=white
|
| 10 |
[docs-url]: https://deepwiki.com/cactus-compute/cactus
|
| 11 |
|
| 12 |
-
|
| 13 |
|
| 14 |
-
## 
|
| 28 |
-
|
| 29 |
-
- APIs are increasingly becoming expensive, especially at scale
|
| 30 |
-
- Private and local, data do not leave the device whatsoever
|
| 31 |
-
- Low-latency anf fault-tolerant, no need for users to have internet connections
|
| 32 |
-
- Small models excell at most tasks, big APIs are often only better at enterprise tasks like coding
|
| 33 |
-
- Freedom to use any GGUF model, unlike Apple Foundations and Google AI Core
|
| 34 |
-
- React-Native and Flutter APIs, no need for separate Swift and Android setups
|
| 35 |
-
- iOS xcframework and JNILibs ifworking in native setup
|
| 36 |
-
- Neat and tiny C++ build for custom hardware
|
| 37 |
|
| 38 |
## 
|
| 39 |
|
|
@@ -226,40 +212,6 @@ Cactus backend is written in C/C++ and can run directly on any ARM/X86/Raspberry
|
|
| 226 |
```
|
| 227 |
To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/cactus). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
|
| 228 |
|
| 229 |
-
|
| 230 |
-
## 
|
| 231 |
-
|
| 232 |
-
First, clone the repo with `git clone https://github.com/cactus-compute/cactus.git`, cd into it and make all scripts executable with `chmod +x scripts/*.sh`
|
| 233 |
-
|
| 234 |
-
1. **Flutter**
|
| 235 |
-
- Build the Android JNILibs with `scripts/build-flutter-android.sh`.
|
| 236 |
-
- Build the Flutter Plugin with `scripts/build-flutter-android.sh`.
|
| 237 |
-
- Navigate to the example app with `cd flutter/example`.
|
| 238 |
-
- Open your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
|
| 239 |
-
- Always start app with this combo `flutter clean && flutter pub get && flutter run`.
|
| 240 |
-
- Play with the app, and make changes either to the example app or plugin as desired.
|
| 241 |
-
|
| 242 |
-
2. **React Native**
|
| 243 |
-
- Build the Android JNILibs with `scripts/build-react-android.sh`.
|
| 244 |
-
- Build the Flutter Plugin with `scripts/build-react-android.sh`.
|
| 245 |
-
- Navigate to the example app with `cd react/example`.
|
| 246 |
-
- Setup your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
|
| 247 |
-
- Always start app with this combo `yarn && yarn ios` or `yarn && yarn android`.
|
| 248 |
-
- Play with the app, and make changes either to the example app or package as desired.
|
| 249 |
-
- For now, if changes are made in the package, you would manually copy the files/folders into the `examples/react/node_modules/cactus-react-native`.
|
| 250 |
-
|
| 251 |
-
2. **C/C++**
|
| 252 |
-
- Navigate to the example app with `cd cactus/example`.
|
| 253 |
-
- There are multiple main files `main_vlm, main_llm, main_embed, main_tts`.
|
| 254 |
-
- Build both the libraries and executable using `build.sh`.
|
| 255 |
-
- Run with one of the executables `./cactus_vlm`, `./cactus_llm`, `./cactus_embed`, `./cactus_tts`.
|
| 256 |
-
- Try different models and make changes as desired.
|
| 257 |
-
|
| 258 |
-
4. **Contributing**
|
| 259 |
-
- To contribute a bug fix, create a branch after making your changes with `git checkout -b <branch-name>` and submit a PR.
|
| 260 |
-
- To contribute a feature, please raise as issue first so it can be discussed, to avoid intersecting with someone else.
|
| 261 |
-
- [Join our discord](https://discord.gg/SdZjmfWQ)
|
| 262 |
-
|
| 263 |
## 
|
| 264 |
|
| 265 |
| Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
|
|
@@ -287,12 +239,6 @@ First, clone the repo with `git clone https://github.com/cactus-compute/cactus.g
|
|
| 287 |
| Samsung Galaxy Z Fold 4 | 22 | 7 |
|
| 288 |
| Xiaomi Poco F6 5G | 22 | 6 |
|
| 289 |
|
| 290 |
-
|
| 291 |
-
|
| 292 |
-
We created a demo chat app we use for benchmarking:
|
| 293 |
-
|
| 294 |
-
[](https://apps.apple.com/gb/app/cactus-chat/id6744444212)
|
| 295 |
-
[](https://play.google.com/store/apps/details?id=com.rshemetsubuser.myapp&pcampaignid=web_share)
|
| 296 |
|
| 297 |
-
|
| 298 |
-
We provide a colleaction of recommended models on our [HuggingFace Page](https://huggingface.co/Cactus-Compute?sort_models=alphabetical#models)
|
|
|
|
| 9 |
[docs-shield]: https://img.shields.io/badge/DeepWiki-009485?style=for-the-badge&logo=readthedocs&logoColor=white
|
| 10 |
[docs-url]: https://deepwiki.com/cactus-compute/cactus
|
| 11 |
|
| 12 |
+
A cross-platform framework for deploying LLMs, VLMs, Embedding Models, TTS models and more locally on smartphones.
|
| 13 |
|
| 14 |
+
## 
|
| 15 |
|
| 16 |
+
- Available in Flutter and React-Native for cross-platform developers.
|
| 17 |
+
- Supports any GGUF model you can find on Huggingface; Qwen, Gemma, Llama, DeepSeek etc.
|
| 18 |
+
- Accommodates from FP32 to as low as 2-bit quantized models, for efficiency and less device strain.
|
| 19 |
+
- MCP tool-calls to make AI performant and helpful (set reminder, gallery search, reply messages) etc.
|
| 20 |
+
- iOS xcframework and JNILibs for native setups
|
| 21 |
+
- Neat and tiny C++ build for custom hardware
|
| 22 |
- Chat templates with Jinja2 support
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## 
|
| 25 |
|
|
|
|
| 212 |
```
|
| 213 |
To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/cactus). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
|
| 214 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 215 |
## 
|
| 216 |
|
| 217 |
| Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
|
|
|
|
| 239 |
| Samsung Galaxy Z Fold 4 | 22 | 7 |
|
| 240 |
| Xiaomi Poco F6 5G | 22 | 6 |
|
| 241 |
|
| 242 |
+
We are completely open-source and would appreciate feedback!
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 243 |
|
| 244 |
+
Repo: https://github.com/cactus-compute/cactus
|
|
|