hmunachii commited on
Commit
f1bce23
·
verified ·
1 Parent(s): 64dd92f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -64
README.md CHANGED
@@ -9,31 +9,17 @@
9
  [docs-shield]: https://img.shields.io/badge/DeepWiki-009485?style=for-the-badge&logo=readthedocs&logoColor=white
10
  [docs-url]: https://deepwiki.com/cactus-compute/cactus
11
 
12
- Cactus is a lightweight, high-performance framework for running AI models on mobile devices, with simple and consistent APIs across C/C++, Dart/Flutter and Ts/React-Native. Cactus currently leverages GGML backends to support any GGUF model already compatible with Llama.cpp.
13
 
14
- ## ![Features](https://img.shields.io/badge/Features-grey.svg?style=for-the-badge)
15
 
16
- - Text completion and chat completion
17
- - Vision Language Models
18
- - Streaming token generation
19
- - Embedding generation
20
- - Text-to-speech model support (early stages)
21
- - JSON mode with schema validation
22
  - Chat templates with Jinja2 support
23
- - Low memory footprint
24
- - Battery-efficient inference
25
- - Background processing
26
-
27
- ## ![Why Cactus?](https://img.shields.io/badge/WHy_Cactus-grey.svg?style=for-the-badge)
28
-
29
- - APIs are increasingly becoming expensive, especially at scale
30
- - Private and local, data do not leave the device whatsoever
31
- - Low-latency anf fault-tolerant, no need for users to have internet connections
32
- - Small models excell at most tasks, big APIs are often only better at enterprise tasks like coding
33
- - Freedom to use any GGUF model, unlike Apple Foundations and Google AI Core
34
- - React-Native and Flutter APIs, no need for separate Swift and Android setups
35
- - iOS xcframework and JNILibs ifworking in native setup
36
- - Neat and tiny C++ build for custom hardware
37
 
38
  ## ![Flutter](https://img.shields.io/badge/Flutter-grey.svg?style=for-the-badge&logo=Flutter&logoColor=white)
39
 
@@ -226,40 +212,6 @@ Cactus backend is written in C/C++ and can run directly on any ARM/X86/Raspberry
226
  ```
227
  To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/cactus). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
228
 
229
-
230
- ## ![Using this Repo & Example Apps](https://img.shields.io/badge/Using_Repo_And_Examples-grey.svg?style=for-the-badge)
231
-
232
- First, clone the repo with `git clone https://github.com/cactus-compute/cactus.git`, cd into it and make all scripts executable with `chmod +x scripts/*.sh`
233
-
234
- 1. **Flutter**
235
- - Build the Android JNILibs with `scripts/build-flutter-android.sh`.
236
- - Build the Flutter Plugin with `scripts/build-flutter-android.sh`.
237
- - Navigate to the example app with `cd flutter/example`.
238
- - Open your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
239
- - Always start app with this combo `flutter clean && flutter pub get && flutter run`.
240
- - Play with the app, and make changes either to the example app or plugin as desired.
241
-
242
- 2. **React Native**
243
- - Build the Android JNILibs with `scripts/build-react-android.sh`.
244
- - Build the Flutter Plugin with `scripts/build-react-android.sh`.
245
- - Navigate to the example app with `cd react/example`.
246
- - Setup your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
247
- - Always start app with this combo `yarn && yarn ios` or `yarn && yarn android`.
248
- - Play with the app, and make changes either to the example app or package as desired.
249
- - For now, if changes are made in the package, you would manually copy the files/folders into the `examples/react/node_modules/cactus-react-native`.
250
-
251
- 2. **C/C++**
252
- - Navigate to the example app with `cd cactus/example`.
253
- - There are multiple main files `main_vlm, main_llm, main_embed, main_tts`.
254
- - Build both the libraries and executable using `build.sh`.
255
- - Run with one of the executables `./cactus_vlm`, `./cactus_llm`, `./cactus_embed`, `./cactus_tts`.
256
- - Try different models and make changes as desired.
257
-
258
- 4. **Contributing**
259
- - To contribute a bug fix, create a branch after making your changes with `git checkout -b <branch-name>` and submit a PR.
260
- - To contribute a feature, please raise as issue first so it can be discussed, to avoid intersecting with someone else.
261
- - [Join our discord](https://discord.gg/SdZjmfWQ)
262
-
263
  ## ![Performance](https://img.shields.io/badge/Performance-grey.svg?style=for-the-badge)
264
 
265
  | Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
@@ -287,12 +239,6 @@ First, clone the repo with `git clone https://github.com/cactus-compute/cactus.g
287
  | Samsung Galaxy Z Fold 4 | 22 | 7 |
288
  | Xiaomi Poco F6 5G | 22 | 6 |
289
 
290
- ## ![Demo](https://img.shields.io/badge/Demo-grey.svg?style=for-the-badge)
291
-
292
- We created a demo chat app we use for benchmarking:
293
-
294
- [![Download App](https://img.shields.io/badge/Download_iOS_App-grey?style=for-the-badge&logo=apple&logoColor=white)](https://apps.apple.com/gb/app/cactus-chat/id6744444212)
295
- [![Download App](https://img.shields.io/badge/Download_Android_App-grey?style=for-the-badge&logo=android&logoColor=white)](https://play.google.com/store/apps/details?id=com.rshemetsubuser.myapp&pcampaignid=web_share)
296
 
297
- ## ![Recommendations](https://img.shields.io/badge/Our_Recommendations-grey.svg?style=for-the-badge)
298
- We provide a colleaction of recommended models on our [HuggingFace Page](https://huggingface.co/Cactus-Compute?sort_models=alphabetical#models)
 
9
  [docs-shield]: https://img.shields.io/badge/DeepWiki-009485?style=for-the-badge&logo=readthedocs&logoColor=white
10
  [docs-url]: https://deepwiki.com/cactus-compute/cactus
11
 
12
+ A cross-platform framework for deploying LLMs, VLMs, Embedding Models, TTS models and more locally on smartphones.
13
 
14
+ ## ![Features](https://img.shields.io/badge/WHy_Cactus-grey.svg?style=for-the-badge)
15
 
16
+ - Available in Flutter and React-Native for cross-platform developers.
17
+ - Supports any GGUF model you can find on Huggingface; Qwen, Gemma, Llama, DeepSeek etc.
18
+ - Accommodates from FP32 to as low as 2-bit quantized models, for efficiency and less device strain.
19
+ - MCP tool-calls to make AI performant and helpful (set reminder, gallery search, reply messages) etc.
20
+ - iOS xcframework and JNILibs for native setups
21
+ - Neat and tiny C++ build for custom hardware
22
  - Chat templates with Jinja2 support
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
  ## ![Flutter](https://img.shields.io/badge/Flutter-grey.svg?style=for-the-badge&logo=Flutter&logoColor=white)
25
 
 
212
  ```
213
  To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/cactus). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
214
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
215
  ## ![Performance](https://img.shields.io/badge/Performance-grey.svg?style=for-the-badge)
216
 
217
  | Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
 
239
  | Samsung Galaxy Z Fold 4 | 22 | 7 |
240
  | Xiaomi Poco F6 5G | 22 | 6 |
241
 
242
+ We are completely open-source and would appreciate feedback!
 
 
 
 
 
243
 
244
+ Repo: https://github.com/cactus-compute/cactus