Spaces:
Configuration error
Configuration error
Update README.md
Browse files
README.md
CHANGED
|
@@ -50,32 +50,48 @@ Cactus is a lightweight, high-performance framework for running AI models on mob
|
|
| 50 |
```bash
|
| 51 |
flutter pub get
|
| 52 |
```
|
| 53 |
-
3. **
|
| 54 |
```dart
|
| 55 |
import 'package:cactus/cactus.dart';
|
| 56 |
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
final
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
));
|
| 73 |
-
|
| 74 |
-
context.free();
|
| 75 |
-
return result.text;
|
| 76 |
-
}
|
| 77 |
```
|
| 78 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
|
| 80 |
## 
|
| 81 |
|
|
@@ -94,29 +110,45 @@ Cactus is a lightweight, high-performance framework for running AI models on mob
|
|
| 94 |
npx pod-install
|
| 95 |
```
|
| 96 |
|
| 97 |
-
3. **
|
| 98 |
```typescript
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
const context = await initLlama({
|
| 103 |
-
model: '/path/to/your/model.gguf',
|
| 104 |
n_ctx: 2048,
|
| 105 |
-
n_threads: 4,
|
| 106 |
});
|
| 107 |
|
| 108 |
-
//
|
| 109 |
-
const
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 115 |
});
|
| 116 |
|
| 117 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
```
|
| 119 |
-
|
| 120 |
|
| 121 |
## 
|
| 122 |
|
|
@@ -192,7 +224,7 @@ Cactus backend is written in C/C++ and can run directly on any ARM/X86/Raspberry
|
|
| 192 |
return 0;
|
| 193 |
}
|
| 194 |
```
|
| 195 |
-
To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/
|
| 196 |
|
| 197 |
|
| 198 |
## 
|
|
@@ -202,7 +234,7 @@ First, clone the repo with `git clone https://github.com/cactus-compute/cactus.g
|
|
| 202 |
1. **Flutter**
|
| 203 |
- Build the Android JNILibs with `scripts/build-flutter-android.sh`.
|
| 204 |
- Build the Flutter Plugin with `scripts/build-flutter-android.sh`.
|
| 205 |
-
- Navigate to the example app with `cd
|
| 206 |
- Open your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
|
| 207 |
- Always start app with this combo `flutter clean && flutter pub get && flutter run`.
|
| 208 |
- Play with the app, and make changes either to the example app or plugin as desired.
|
|
@@ -210,19 +242,24 @@ First, clone the repo with `git clone https://github.com/cactus-compute/cactus.g
|
|
| 210 |
2. **React Native**
|
| 211 |
- Build the Android JNILibs with `scripts/build-react-android.sh`.
|
| 212 |
- Build the Flutter Plugin with `scripts/build-react-android.sh`.
|
| 213 |
-
- Navigate to the example app with `cd
|
| 214 |
- Setup your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
|
| 215 |
- Always start app with this combo `yarn && yarn ios` or `yarn && yarn android`.
|
| 216 |
- Play with the app, and make changes either to the example app or package as desired.
|
| 217 |
- For now, if changes are made in the package, you would manually copy the files/folders into the `examples/react/node_modules/cactus-react-native`.
|
| 218 |
|
| 219 |
2. **C/C++**
|
| 220 |
-
- Navigate to the example app with `cd
|
| 221 |
- There are multiple main files `main_vlm, main_llm, main_embed, main_tts`.
|
| 222 |
- Build both the libraries and executable using `build.sh`.
|
| 223 |
- Run with one of the executables `./cactus_vlm`, `./cactus_llm`, `./cactus_embed`, `./cactus_tts`.
|
| 224 |
- Try different models and make changes as desired.
|
| 225 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 226 |
## 
|
| 227 |
|
| 228 |
| Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
|
|
@@ -256,3 +293,6 @@ We created a demo chat app we use for benchmarking:
|
|
| 256 |
|
| 257 |
[](https://apps.apple.com/gb/app/cactus-chat/id6744444212)
|
| 258 |
[](https://play.google.com/store/apps/details?id=com.rshemetsubuser.myapp&pcampaignid=web_share)
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
```bash
|
| 51 |
flutter pub get
|
| 52 |
```
|
| 53 |
+
3. **Flutter Text Completion**
|
| 54 |
```dart
|
| 55 |
import 'package:cactus/cactus.dart';
|
| 56 |
|
| 57 |
+
// Initialize
|
| 58 |
+
final lm = await CactusLM.init(
|
| 59 |
+
modelUrl: 'huggingface/gguf/link',
|
| 60 |
+
nCtx: 2048,
|
| 61 |
+
);
|
| 62 |
+
|
| 63 |
+
// Completion
|
| 64 |
+
final messages = [CactusMessage(role: CactusMessageRole.user, content: 'Hello!')];
|
| 65 |
+
final params = CactusCompletionParams(nPredict: 100, temperature: 0.7);
|
| 66 |
+
final response = await lm.completion(messages, params);
|
| 67 |
+
|
| 68 |
+
// Embedding
|
| 69 |
+
final text = 'Your text to embed';
|
| 70 |
+
final params = CactusEmbeddingParams(normalize: true);
|
| 71 |
+
final result = await lm.embedding(text, params);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
```
|
| 73 |
+
4. **Flutter VLM Completion**
|
| 74 |
+
```dart
|
| 75 |
+
import 'package:cactus/cactus.dart';
|
| 76 |
+
|
| 77 |
+
// Initialize (Flutter handles downloads automatically)
|
| 78 |
+
final vlm = await CactusVLM.init(
|
| 79 |
+
modelUrl: 'huggingface/gguf/link',
|
| 80 |
+
mmprojUrl: 'huggingface/gguf/mmproj/link',
|
| 81 |
+
);
|
| 82 |
+
|
| 83 |
+
// Multimodal Completion (can add multiple images)
|
| 84 |
+
final messages = [CactusMessage(role: CactusMessageRole.user, content: 'Describe this image')];
|
| 85 |
+
|
| 86 |
+
final params = CactusVLMParams(
|
| 87 |
+
images: ['/absolute/path/to/image.jpg'],
|
| 88 |
+
nPredict: 200,
|
| 89 |
+
temperature: 0.3,
|
| 90 |
+
);
|
| 91 |
+
|
| 92 |
+
final response = await vlm.completion(messages, params);
|
| 93 |
+
```
|
| 94 |
+
N/B: See the [Flutter Docs](https://github.com/cactus-compute/cactus/blob/main/flutter). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
|
| 95 |
|
| 96 |
## 
|
| 97 |
|
|
|
|
| 110 |
npx pod-install
|
| 111 |
```
|
| 112 |
|
| 113 |
+
3. **React-Native Text Completion**
|
| 114 |
```typescript
|
| 115 |
+
// Initialize
|
| 116 |
+
const lm = await CactusLM.init({
|
| 117 |
+
model: '/path/to/model.gguf',
|
|
|
|
|
|
|
| 118 |
n_ctx: 2048,
|
|
|
|
| 119 |
});
|
| 120 |
|
| 121 |
+
// Completion
|
| 122 |
+
const messages = [{ role: 'user', content: 'Hello!' }];
|
| 123 |
+
const params = { n_predict: 100, temperature: 0.7 };
|
| 124 |
+
const response = await lm.completion(messages, params);
|
| 125 |
+
|
| 126 |
+
// Embedding
|
| 127 |
+
const text = 'Your text to embed';
|
| 128 |
+
const params = { normalize: true };
|
| 129 |
+
const result = await lm.embedding(text, params);
|
| 130 |
+
```
|
| 131 |
+
|
| 132 |
+
4. **React-Native VLM**
|
| 133 |
+
```typescript
|
| 134 |
+
// Initialize
|
| 135 |
+
const vlm = await CactusVLM.init({
|
| 136 |
+
model: '/path/to/vision-model.gguf',
|
| 137 |
+
mmproj: '/path/to/mmproj.gguf',
|
| 138 |
});
|
| 139 |
|
| 140 |
+
// Multimodal Completion (can add multiple images)
|
| 141 |
+
const messages = [{ role: 'user', content: 'Describe this image' }];
|
| 142 |
+
|
| 143 |
+
const params = {
|
| 144 |
+
images: ['/absolute/path/to/image.jpg'],
|
| 145 |
+
n_predict: 200,
|
| 146 |
+
temperature: 0.3,
|
| 147 |
+
};
|
| 148 |
+
|
| 149 |
+
const response = await vlm.completion(messages, params);
|
| 150 |
```
|
| 151 |
+
N/B: See the [React Docs](https://github.com/cactus-compute/cactus/blob/main/react). It covers chat design, embeddings, multimodal models, text-to-speech, and various options.
|
| 152 |
|
| 153 |
## 
|
| 154 |
|
|
|
|
| 224 |
return 0;
|
| 225 |
}
|
| 226 |
```
|
| 227 |
+
To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/cactus). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
|
| 228 |
|
| 229 |
|
| 230 |
## 
|
|
|
|
| 234 |
1. **Flutter**
|
| 235 |
- Build the Android JNILibs with `scripts/build-flutter-android.sh`.
|
| 236 |
- Build the Flutter Plugin with `scripts/build-flutter-android.sh`.
|
| 237 |
+
- Navigate to the example app with `cd flutter/example`.
|
| 238 |
- Open your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
|
| 239 |
- Always start app with this combo `flutter clean && flutter pub get && flutter run`.
|
| 240 |
- Play with the app, and make changes either to the example app or plugin as desired.
|
|
|
|
| 242 |
2. **React Native**
|
| 243 |
- Build the Android JNILibs with `scripts/build-react-android.sh`.
|
| 244 |
- Build the Flutter Plugin with `scripts/build-react-android.sh`.
|
| 245 |
+
- Navigate to the example app with `cd react/example`.
|
| 246 |
- Setup your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
|
| 247 |
- Always start app with this combo `yarn && yarn ios` or `yarn && yarn android`.
|
| 248 |
- Play with the app, and make changes either to the example app or package as desired.
|
| 249 |
- For now, if changes are made in the package, you would manually copy the files/folders into the `examples/react/node_modules/cactus-react-native`.
|
| 250 |
|
| 251 |
2. **C/C++**
|
| 252 |
+
- Navigate to the example app with `cd cactus/example`.
|
| 253 |
- There are multiple main files `main_vlm, main_llm, main_embed, main_tts`.
|
| 254 |
- Build both the libraries and executable using `build.sh`.
|
| 255 |
- Run with one of the executables `./cactus_vlm`, `./cactus_llm`, `./cactus_embed`, `./cactus_tts`.
|
| 256 |
- Try different models and make changes as desired.
|
| 257 |
|
| 258 |
+
4. **Contributing**
|
| 259 |
+
- To contribute a bug fix, create a branch after making your changes with `git checkout -b <branch-name>` and submit a PR.
|
| 260 |
+
- To contribute a feature, please raise as issue first so it can be discussed, to avoid intersecting with someone else.
|
| 261 |
+
- [Join our discord](https://discord.gg/SdZjmfWQ)
|
| 262 |
+
|
| 263 |
## 
|
| 264 |
|
| 265 |
| Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
|
|
|
|
| 293 |
|
| 294 |
[](https://apps.apple.com/gb/app/cactus-chat/id6744444212)
|
| 295 |
[](https://play.google.com/store/apps/details?id=com.rshemetsubuser.myapp&pcampaignid=web_share)
|
| 296 |
+
|
| 297 |
+
## 
|
| 298 |
+
We provide a colleaction of recommended models on our [HuggingFace Page](https://huggingface.co/Cactus-Compute?sort_models=alphabetical#models)
|