hmunachii commited on
Commit
64dd92f
·
verified ·
1 Parent(s): 32b7920

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +82 -42
README.md CHANGED
@@ -50,32 +50,48 @@ Cactus is a lightweight, high-performance framework for running AI models on mob
50
  ```bash
51
  flutter pub get
52
  ```
53
- 3. **Basic Flutter Text Completion**
54
  ```dart
55
  import 'package:cactus/cactus.dart';
56
 
57
- Future<String> basicCompletion() async {
58
- // Initialize context
59
- final context = await CactusContext.init(CactusInitParams(
60
- modelPath: '/path/to/model.gguf',
61
- contextSize: 2048,
62
- threads: 4,
63
- ));
64
-
65
- // Generate response
66
- final result = await context.completion(CactusCompletionParams(
67
- messages: [
68
- ChatMessage(role: 'user', content: 'Hello, how are you?')
69
- ],
70
- maxPredictedTokens: 100,
71
- temperature: 0.7,
72
- ));
73
-
74
- context.free();
75
- return result.text;
76
- }
77
  ```
78
- To learn more, see the [Flutter Docs](https://github.com/cactus-compute/cactus/blob/main/docs/flutter.md). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
 
80
  ## ![React Native](https://img.shields.io/badge/React%20Native-grey.svg?style=for-the-badge&logo=react&logoColor=%2361DAFB)
81
 
@@ -94,29 +110,45 @@ Cactus is a lightweight, high-performance framework for running AI models on mob
94
  npx pod-install
95
  ```
96
 
97
- 3. **Basic React-Native Text Completion**
98
  ```typescript
99
- import { initLlama } from 'cactus-react-native';
100
-
101
- // Initialize a context
102
- const context = await initLlama({
103
- model: '/path/to/your/model.gguf',
104
  n_ctx: 2048,
105
- n_threads: 4,
106
  });
107
 
108
- // Generate text
109
- const result = await context.completion({
110
- messages: [
111
- { role: 'user', content: 'Hello, how are you?' }
112
- ],
113
- n_predict: 100,
114
- temperature: 0.7,
 
 
 
 
 
 
 
 
 
 
115
  });
116
 
117
- console.log(result.text);
 
 
 
 
 
 
 
 
 
118
  ```
119
- To learn more, see the [React Docs](https://github.com/cactus-compute/cactus/blob/main/docs/react.md). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
120
 
121
  ## ![C++](https://img.shields.io/badge/C%2B%2B-grey.svg?style=for-the-badge&logo=c%2B%2B&logoColor=white)
122
 
@@ -192,7 +224,7 @@ Cactus backend is written in C/C++ and can run directly on any ARM/X86/Raspberry
192
  return 0;
193
  }
194
  ```
195
- To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/docs/cpp.md). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
196
 
197
 
198
  ## ![Using this Repo & Example Apps](https://img.shields.io/badge/Using_Repo_And_Examples-grey.svg?style=for-the-badge)
@@ -202,7 +234,7 @@ First, clone the repo with `git clone https://github.com/cactus-compute/cactus.g
202
  1. **Flutter**
203
  - Build the Android JNILibs with `scripts/build-flutter-android.sh`.
204
  - Build the Flutter Plugin with `scripts/build-flutter-android.sh`.
205
- - Navigate to the example app with `cd examples/flutter`.
206
  - Open your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
207
  - Always start app with this combo `flutter clean && flutter pub get && flutter run`.
208
  - Play with the app, and make changes either to the example app or plugin as desired.
@@ -210,19 +242,24 @@ First, clone the repo with `git clone https://github.com/cactus-compute/cactus.g
210
  2. **React Native**
211
  - Build the Android JNILibs with `scripts/build-react-android.sh`.
212
  - Build the Flutter Plugin with `scripts/build-react-android.sh`.
213
- - Navigate to the example app with `cd examples/react`.
214
  - Setup your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
215
  - Always start app with this combo `yarn && yarn ios` or `yarn && yarn android`.
216
  - Play with the app, and make changes either to the example app or package as desired.
217
  - For now, if changes are made in the package, you would manually copy the files/folders into the `examples/react/node_modules/cactus-react-native`.
218
 
219
  2. **C/C++**
220
- - Navigate to the example app with `cd examples/cpp`.
221
  - There are multiple main files `main_vlm, main_llm, main_embed, main_tts`.
222
  - Build both the libraries and executable using `build.sh`.
223
  - Run with one of the executables `./cactus_vlm`, `./cactus_llm`, `./cactus_embed`, `./cactus_tts`.
224
  - Try different models and make changes as desired.
225
 
 
 
 
 
 
226
  ## ![Performance](https://img.shields.io/badge/Performance-grey.svg?style=for-the-badge)
227
 
228
  | Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
@@ -256,3 +293,6 @@ We created a demo chat app we use for benchmarking:
256
 
257
  [![Download App](https://img.shields.io/badge/Download_iOS_App-grey?style=for-the-badge&logo=apple&logoColor=white)](https://apps.apple.com/gb/app/cactus-chat/id6744444212)
258
  [![Download App](https://img.shields.io/badge/Download_Android_App-grey?style=for-the-badge&logo=android&logoColor=white)](https://play.google.com/store/apps/details?id=com.rshemetsubuser.myapp&pcampaignid=web_share)
 
 
 
 
50
  ```bash
51
  flutter pub get
52
  ```
53
+ 3. **Flutter Text Completion**
54
  ```dart
55
  import 'package:cactus/cactus.dart';
56
 
57
+ // Initialize
58
+ final lm = await CactusLM.init(
59
+ modelUrl: 'huggingface/gguf/link',
60
+ nCtx: 2048,
61
+ );
62
+
63
+ // Completion
64
+ final messages = [CactusMessage(role: CactusMessageRole.user, content: 'Hello!')];
65
+ final params = CactusCompletionParams(nPredict: 100, temperature: 0.7);
66
+ final response = await lm.completion(messages, params);
67
+
68
+ // Embedding
69
+ final text = 'Your text to embed';
70
+ final params = CactusEmbeddingParams(normalize: true);
71
+ final result = await lm.embedding(text, params);
 
 
 
 
 
72
  ```
73
+ 4. **Flutter VLM Completion**
74
+ ```dart
75
+ import 'package:cactus/cactus.dart';
76
+
77
+ // Initialize (Flutter handles downloads automatically)
78
+ final vlm = await CactusVLM.init(
79
+ modelUrl: 'huggingface/gguf/link',
80
+ mmprojUrl: 'huggingface/gguf/mmproj/link',
81
+ );
82
+
83
+ // Multimodal Completion (can add multiple images)
84
+ final messages = [CactusMessage(role: CactusMessageRole.user, content: 'Describe this image')];
85
+
86
+ final params = CactusVLMParams(
87
+ images: ['/absolute/path/to/image.jpg'],
88
+ nPredict: 200,
89
+ temperature: 0.3,
90
+ );
91
+
92
+ final response = await vlm.completion(messages, params);
93
+ ```
94
+ N/B: See the [Flutter Docs](https://github.com/cactus-compute/cactus/blob/main/flutter). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
95
 
96
  ## ![React Native](https://img.shields.io/badge/React%20Native-grey.svg?style=for-the-badge&logo=react&logoColor=%2361DAFB)
97
 
 
110
  npx pod-install
111
  ```
112
 
113
+ 3. **React-Native Text Completion**
114
  ```typescript
115
+ // Initialize
116
+ const lm = await CactusLM.init({
117
+ model: '/path/to/model.gguf',
 
 
118
  n_ctx: 2048,
 
119
  });
120
 
121
+ // Completion
122
+ const messages = [{ role: 'user', content: 'Hello!' }];
123
+ const params = { n_predict: 100, temperature: 0.7 };
124
+ const response = await lm.completion(messages, params);
125
+
126
+ // Embedding
127
+ const text = 'Your text to embed';
128
+ const params = { normalize: true };
129
+ const result = await lm.embedding(text, params);
130
+ ```
131
+
132
+ 4. **React-Native VLM**
133
+ ```typescript
134
+ // Initialize
135
+ const vlm = await CactusVLM.init({
136
+ model: '/path/to/vision-model.gguf',
137
+ mmproj: '/path/to/mmproj.gguf',
138
  });
139
 
140
+ // Multimodal Completion (can add multiple images)
141
+ const messages = [{ role: 'user', content: 'Describe this image' }];
142
+
143
+ const params = {
144
+ images: ['/absolute/path/to/image.jpg'],
145
+ n_predict: 200,
146
+ temperature: 0.3,
147
+ };
148
+
149
+ const response = await vlm.completion(messages, params);
150
  ```
151
+ N/B: See the [React Docs](https://github.com/cactus-compute/cactus/blob/main/react). It covers chat design, embeddings, multimodal models, text-to-speech, and various options.
152
 
153
  ## ![C++](https://img.shields.io/badge/C%2B%2B-grey.svg?style=for-the-badge&logo=c%2B%2B&logoColor=white)
154
 
 
224
  return 0;
225
  }
226
  ```
227
+ To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/cactus). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
228
 
229
 
230
  ## ![Using this Repo & Example Apps](https://img.shields.io/badge/Using_Repo_And_Examples-grey.svg?style=for-the-badge)
 
234
  1. **Flutter**
235
  - Build the Android JNILibs with `scripts/build-flutter-android.sh`.
236
  - Build the Flutter Plugin with `scripts/build-flutter-android.sh`.
237
+ - Navigate to the example app with `cd flutter/example`.
238
  - Open your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
239
  - Always start app with this combo `flutter clean && flutter pub get && flutter run`.
240
  - Play with the app, and make changes either to the example app or plugin as desired.
 
242
  2. **React Native**
243
  - Build the Android JNILibs with `scripts/build-react-android.sh`.
244
  - Build the Flutter Plugin with `scripts/build-react-android.sh`.
245
+ - Navigate to the example app with `cd react/example`.
246
  - Setup your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
247
  - Always start app with this combo `yarn && yarn ios` or `yarn && yarn android`.
248
  - Play with the app, and make changes either to the example app or package as desired.
249
  - For now, if changes are made in the package, you would manually copy the files/folders into the `examples/react/node_modules/cactus-react-native`.
250
 
251
  2. **C/C++**
252
+ - Navigate to the example app with `cd cactus/example`.
253
  - There are multiple main files `main_vlm, main_llm, main_embed, main_tts`.
254
  - Build both the libraries and executable using `build.sh`.
255
  - Run with one of the executables `./cactus_vlm`, `./cactus_llm`, `./cactus_embed`, `./cactus_tts`.
256
  - Try different models and make changes as desired.
257
 
258
+ 4. **Contributing**
259
+ - To contribute a bug fix, create a branch after making your changes with `git checkout -b <branch-name>` and submit a PR.
260
+ - To contribute a feature, please raise as issue first so it can be discussed, to avoid intersecting with someone else.
261
+ - [Join our discord](https://discord.gg/SdZjmfWQ)
262
+
263
  ## ![Performance](https://img.shields.io/badge/Performance-grey.svg?style=for-the-badge)
264
 
265
  | Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
 
293
 
294
  [![Download App](https://img.shields.io/badge/Download_iOS_App-grey?style=for-the-badge&logo=apple&logoColor=white)](https://apps.apple.com/gb/app/cactus-chat/id6744444212)
295
  [![Download App](https://img.shields.io/badge/Download_Android_App-grey?style=for-the-badge&logo=android&logoColor=white)](https://play.google.com/store/apps/details?id=com.rshemetsubuser.myapp&pcampaignid=web_share)
296
+
297
+ ## ![Recommendations](https://img.shields.io/badge/Our_Recommendations-grey.svg?style=for-the-badge)
298
+ We provide a colleaction of recommended models on our [HuggingFace Page](https://huggingface.co/Cactus-Compute?sort_models=alphabetical#models)