Spaces:
Configuration error
Configuration error
Update README.md
Browse files
README.md
CHANGED
|
@@ -43,55 +43,39 @@ Cactus is a lightweight, high-performance framework for running AI models on mob
|
|
| 43 |
dependencies:
|
| 44 |
flutter:
|
| 45 |
sdk: flutter
|
| 46 |
-
cactus: ^0.1.
|
| 47 |
```
|
| 48 |
2. **Install dependencies:**
|
| 49 |
Execute the following command in your project terminal:
|
| 50 |
```bash
|
| 51 |
flutter pub get
|
| 52 |
```
|
| 53 |
-
3. **Flutter Text Completion**
|
| 54 |
```dart
|
| 55 |
import 'package:cactus/cactus.dart';
|
| 56 |
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
final
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
// Initialize (Flutter handles downloads automatically)
|
| 78 |
-
final vlm = await CactusVLM.init(
|
| 79 |
-
modelUrl: 'huggingface/gguf/link',
|
| 80 |
-
mmprojUrl: 'huggingface/gguf/mmproj/link',
|
| 81 |
-
);
|
| 82 |
-
|
| 83 |
-
// Multimodal Completion (can add multiple images)
|
| 84 |
-
final messages = [CactusMessage(role: CactusMessageRole.user, content: 'Describe this image')];
|
| 85 |
-
|
| 86 |
-
final params = CactusVLMParams(
|
| 87 |
-
images: ['/absolute/path/to/image.jpg'],
|
| 88 |
-
nPredict: 200,
|
| 89 |
-
temperature: 0.3,
|
| 90 |
-
);
|
| 91 |
-
|
| 92 |
-
final response = await vlm.completion(messages, params);
|
| 93 |
```
|
| 94 |
-
|
| 95 |
|
| 96 |
## 
|
| 97 |
|
|
@@ -110,45 +94,29 @@ Cactus is a lightweight, high-performance framework for running AI models on mob
|
|
| 110 |
npx pod-install
|
| 111 |
```
|
| 112 |
|
| 113 |
-
3. **React-Native Text Completion**
|
| 114 |
```typescript
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
|
|
|
|
|
|
| 118 |
n_ctx: 2048,
|
|
|
|
| 119 |
});
|
| 120 |
|
| 121 |
-
//
|
| 122 |
-
const
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
const params = { normalize: true };
|
| 129 |
-
const result = await lm.embedding(text, params);
|
| 130 |
-
```
|
| 131 |
-
|
| 132 |
-
4. **React-Native VLM**
|
| 133 |
-
```typescript
|
| 134 |
-
// Initialize
|
| 135 |
-
const vlm = await CactusVLM.init({
|
| 136 |
-
model: '/path/to/vision-model.gguf',
|
| 137 |
-
mmproj: '/path/to/mmproj.gguf',
|
| 138 |
});
|
| 139 |
|
| 140 |
-
|
| 141 |
-
const messages = [{ role: 'user', content: 'Describe this image' }];
|
| 142 |
-
|
| 143 |
-
const params = {
|
| 144 |
-
images: ['/absolute/path/to/image.jpg'],
|
| 145 |
-
n_predict: 200,
|
| 146 |
-
temperature: 0.3,
|
| 147 |
-
};
|
| 148 |
-
|
| 149 |
-
const response = await vlm.completion(messages, params);
|
| 150 |
```
|
| 151 |
-
|
| 152 |
|
| 153 |
## 
|
| 154 |
|
|
@@ -224,17 +192,17 @@ Cactus backend is written in C/C++ and can run directly on any ARM/X86/Raspberry
|
|
| 224 |
return 0;
|
| 225 |
}
|
| 226 |
```
|
| 227 |
-
To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/
|
| 228 |
|
| 229 |
|
| 230 |
-
##  if you have not done this before.
|
| 239 |
- Always start app with this combo `flutter clean && flutter pub get && flutter run`.
|
| 240 |
- Play with the app, and make changes either to the example app or plugin as desired.
|
|
@@ -242,24 +210,19 @@ First, clone the repo with `git clone https://github.com/cactus-compute/cactus.g
|
|
| 242 |
2. **React Native**
|
| 243 |
- Build the Android JNILibs with `scripts/build-react-android.sh`.
|
| 244 |
- Build the Flutter Plugin with `scripts/build-react-android.sh`.
|
| 245 |
-
- Navigate to the example app with `cd react
|
| 246 |
- Setup your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
|
| 247 |
- Always start app with this combo `yarn && yarn ios` or `yarn && yarn android`.
|
| 248 |
- Play with the app, and make changes either to the example app or package as desired.
|
| 249 |
- For now, if changes are made in the package, you would manually copy the files/folders into the `examples/react/node_modules/cactus-react-native`.
|
| 250 |
|
| 251 |
2. **C/C++**
|
| 252 |
-
- Navigate to the example app with `cd
|
| 253 |
- There are multiple main files `main_vlm, main_llm, main_embed, main_tts`.
|
| 254 |
- Build both the libraries and executable using `build.sh`.
|
| 255 |
- Run with one of the executables `./cactus_vlm`, `./cactus_llm`, `./cactus_embed`, `./cactus_tts`.
|
| 256 |
- Try different models and make changes as desired.
|
| 257 |
|
| 258 |
-
4. **Contributing**
|
| 259 |
-
- To contribute a bug fix, create a branch after making your changes with `git checkout -b <branch-name>` and submit a PR.
|
| 260 |
-
- To contribute a feature, please raise as issue first so it can be discussed, to avoid intersecting with someone else.
|
| 261 |
-
- [Join our discord](https://discord.gg/SdZjmfWQ)
|
| 262 |
-
|
| 263 |
## 
|
| 264 |
|
| 265 |
| Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
|
|
|
|
| 43 |
dependencies:
|
| 44 |
flutter:
|
| 45 |
sdk: flutter
|
| 46 |
+
cactus: ^0.1.3
|
| 47 |
```
|
| 48 |
2. **Install dependencies:**
|
| 49 |
Execute the following command in your project terminal:
|
| 50 |
```bash
|
| 51 |
flutter pub get
|
| 52 |
```
|
| 53 |
+
3. **Basic Flutter Text Completion**
|
| 54 |
```dart
|
| 55 |
import 'package:cactus/cactus.dart';
|
| 56 |
|
| 57 |
+
Future<String> basicCompletion() async {
|
| 58 |
+
// Initialize context
|
| 59 |
+
final context = await CactusContext.init(CactusInitParams(
|
| 60 |
+
modelPath: '/path/to/model.gguf',
|
| 61 |
+
contextSize: 2048,
|
| 62 |
+
threads: 4,
|
| 63 |
+
));
|
| 64 |
+
|
| 65 |
+
// Generate response
|
| 66 |
+
final result = await context.completion(CactusCompletionParams(
|
| 67 |
+
messages: [
|
| 68 |
+
ChatMessage(role: 'user', content: 'Hello, how are you?')
|
| 69 |
+
],
|
| 70 |
+
maxPredictedTokens: 100,
|
| 71 |
+
temperature: 0.7,
|
| 72 |
+
));
|
| 73 |
+
|
| 74 |
+
context.free();
|
| 75 |
+
return result.text;
|
| 76 |
+
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
```
|
| 78 |
+
To learn more, see the [Flutter Docs](https://github.com/cactus-compute/cactus/blob/main/docs/flutter.md). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
|
| 79 |
|
| 80 |
## 
|
| 81 |
|
|
|
|
| 94 |
npx pod-install
|
| 95 |
```
|
| 96 |
|
| 97 |
+
3. **Basic React-Native Text Completion**
|
| 98 |
```typescript
|
| 99 |
+
import { initLlama } from 'cactus-react-native';
|
| 100 |
+
|
| 101 |
+
// Initialize a context
|
| 102 |
+
const context = await initLlama({
|
| 103 |
+
model: '/path/to/your/model.gguf',
|
| 104 |
n_ctx: 2048,
|
| 105 |
+
n_threads: 4,
|
| 106 |
});
|
| 107 |
|
| 108 |
+
// Generate text
|
| 109 |
+
const result = await context.completion({
|
| 110 |
+
messages: [
|
| 111 |
+
{ role: 'user', content: 'Hello, how are you?' }
|
| 112 |
+
],
|
| 113 |
+
n_predict: 100,
|
| 114 |
+
temperature: 0.7,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 115 |
});
|
| 116 |
|
| 117 |
+
console.log(result.text);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
```
|
| 119 |
+
To learn more, see the [React Docs](https://github.com/cactus-compute/cactus/blob/main/docs/react.md). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
|
| 120 |
|
| 121 |
## 
|
| 122 |
|
|
|
|
| 192 |
return 0;
|
| 193 |
}
|
| 194 |
```
|
| 195 |
+
To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/docs/cpp.md). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
|
| 196 |
|
| 197 |
|
| 198 |
+
## 
|
| 199 |
|
| 200 |
First, clone the repo with `git clone https://github.com/cactus-compute/cactus.git`, cd into it and make all scripts executable with `chmod +x scripts/*.sh`
|
| 201 |
|
| 202 |
1. **Flutter**
|
| 203 |
- Build the Android JNILibs with `scripts/build-flutter-android.sh`.
|
| 204 |
- Build the Flutter Plugin with `scripts/build-flutter-android.sh`.
|
| 205 |
+
- Navigate to the example app with `cd examples/flutter`.
|
| 206 |
- Open your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
|
| 207 |
- Always start app with this combo `flutter clean && flutter pub get && flutter run`.
|
| 208 |
- Play with the app, and make changes either to the example app or plugin as desired.
|
|
|
|
| 210 |
2. **React Native**
|
| 211 |
- Build the Android JNILibs with `scripts/build-react-android.sh`.
|
| 212 |
- Build the Flutter Plugin with `scripts/build-react-android.sh`.
|
| 213 |
+
- Navigate to the example app with `cd examples/react`.
|
| 214 |
- Setup your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
|
| 215 |
- Always start app with this combo `yarn && yarn ios` or `yarn && yarn android`.
|
| 216 |
- Play with the app, and make changes either to the example app or package as desired.
|
| 217 |
- For now, if changes are made in the package, you would manually copy the files/folders into the `examples/react/node_modules/cactus-react-native`.
|
| 218 |
|
| 219 |
2. **C/C++**
|
| 220 |
+
- Navigate to the example app with `cd examples/cpp`.
|
| 221 |
- There are multiple main files `main_vlm, main_llm, main_embed, main_tts`.
|
| 222 |
- Build both the libraries and executable using `build.sh`.
|
| 223 |
- Run with one of the executables `./cactus_vlm`, `./cactus_llm`, `./cactus_embed`, `./cactus_tts`.
|
| 224 |
- Try different models and make changes as desired.
|
| 225 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 226 |
## 
|
| 227 |
|
| 228 |
| Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
|