hmunachii commited on
Commit
32b7920
·
verified ·
1 Parent(s): be7e782

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +44 -81
README.md CHANGED
@@ -43,55 +43,39 @@ Cactus is a lightweight, high-performance framework for running AI models on mob
43
  dependencies:
44
  flutter:
45
  sdk: flutter
46
- cactus: ^0.1.2
47
  ```
48
  2. **Install dependencies:**
49
  Execute the following command in your project terminal:
50
  ```bash
51
  flutter pub get
52
  ```
53
- 3. **Flutter Text Completion**
54
  ```dart
55
  import 'package:cactus/cactus.dart';
56
 
57
- // Initialize
58
- final lm = await CactusLM.init(
59
- modelUrl: 'huggingface/gguf/link',
60
- nCtx: 2048,
61
- );
62
-
63
- // Completion
64
- final messages = [CactusMessage(role: CactusMessageRole.user, content: 'Hello!')];
65
- final params = CactusCompletionParams(nPredict: 100, temperature: 0.7);
66
- final response = await lm.completion(messages, params);
67
-
68
- // Embedding
69
- final text = 'Your text to embed';
70
- final params = CactusEmbeddingParams(normalize: true);
71
- final result = await lm.embedding(text, params);
72
- ```
73
- 4. **Flutter VLM Completion**
74
- ```dart
75
- import 'package:cactus/cactus.dart';
76
-
77
- // Initialize (Flutter handles downloads automatically)
78
- final vlm = await CactusVLM.init(
79
- modelUrl: 'huggingface/gguf/link',
80
- mmprojUrl: 'huggingface/gguf/mmproj/link',
81
- );
82
-
83
- // Multimodal Completion (can add multiple images)
84
- final messages = [CactusMessage(role: CactusMessageRole.user, content: 'Describe this image')];
85
-
86
- final params = CactusVLMParams(
87
- images: ['/absolute/path/to/image.jpg'],
88
- nPredict: 200,
89
- temperature: 0.3,
90
- );
91
-
92
- final response = await vlm.completion(messages, params);
93
  ```
94
- N/B: See the [Flutter Docs](https://github.com/cactus-compute/cactus/blob/main/flutter). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
95
 
96
  ## ![React Native](https://img.shields.io/badge/React%20Native-grey.svg?style=for-the-badge&logo=react&logoColor=%2361DAFB)
97
 
@@ -110,45 +94,29 @@ Cactus is a lightweight, high-performance framework for running AI models on mob
110
  npx pod-install
111
  ```
112
 
113
- 3. **React-Native Text Completion**
114
  ```typescript
115
- // Initialize
116
- const lm = await CactusLM.init({
117
- model: '/path/to/model.gguf',
 
 
118
  n_ctx: 2048,
 
119
  });
120
 
121
- // Completion
122
- const messages = [{ role: 'user', content: 'Hello!' }];
123
- const params = { n_predict: 100, temperature: 0.7 };
124
- const response = await lm.completion(messages, params);
125
-
126
- // Embedding
127
- const text = 'Your text to embed';
128
- const params = { normalize: true };
129
- const result = await lm.embedding(text, params);
130
- ```
131
-
132
- 4. **React-Native VLM**
133
- ```typescript
134
- // Initialize
135
- const vlm = await CactusVLM.init({
136
- model: '/path/to/vision-model.gguf',
137
- mmproj: '/path/to/mmproj.gguf',
138
  });
139
 
140
- // Multimodal Completion (can add multiple images)
141
- const messages = [{ role: 'user', content: 'Describe this image' }];
142
-
143
- const params = {
144
- images: ['/absolute/path/to/image.jpg'],
145
- n_predict: 200,
146
- temperature: 0.3,
147
- };
148
-
149
- const response = await vlm.completion(messages, params);
150
  ```
151
- N/B: See the [React Docs](https://github.com/cactus-compute/cactus/blob/main/react). It covers chat design, embeddings, multimodal models, text-to-speech, and various options.
152
 
153
  ## ![C++](https://img.shields.io/badge/C%2B%2B-grey.svg?style=for-the-badge&logo=c%2B%2B&logoColor=white)
154
 
@@ -224,17 +192,17 @@ Cactus backend is written in C/C++ and can run directly on any ARM/X86/Raspberry
224
  return 0;
225
  }
226
  ```
227
- To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/cactus). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
228
 
229
 
230
- ## ![Using Cactus Repo & Example Apps](https://img.shields.io/badge/Using_Repo_And_Examples-grey.svg?style=for-the-badge)
231
 
232
  First, clone the repo with `git clone https://github.com/cactus-compute/cactus.git`, cd into it and make all scripts executable with `chmod +x scripts/*.sh`
233
 
234
  1. **Flutter**
235
  - Build the Android JNILibs with `scripts/build-flutter-android.sh`.
236
  - Build the Flutter Plugin with `scripts/build-flutter-android.sh`.
237
- - Navigate to the example app with `cd flutter/example`.
238
  - Open your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
239
  - Always start app with this combo `flutter clean && flutter pub get && flutter run`.
240
  - Play with the app, and make changes either to the example app or plugin as desired.
@@ -242,24 +210,19 @@ First, clone the repo with `git clone https://github.com/cactus-compute/cactus.g
242
  2. **React Native**
243
  - Build the Android JNILibs with `scripts/build-react-android.sh`.
244
  - Build the Flutter Plugin with `scripts/build-react-android.sh`.
245
- - Navigate to the example app with `cd react/example`.
246
  - Setup your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
247
  - Always start app with this combo `yarn && yarn ios` or `yarn && yarn android`.
248
  - Play with the app, and make changes either to the example app or package as desired.
249
  - For now, if changes are made in the package, you would manually copy the files/folders into the `examples/react/node_modules/cactus-react-native`.
250
 
251
  2. **C/C++**
252
- - Navigate to the example app with `cd cactus/example`.
253
  - There are multiple main files `main_vlm, main_llm, main_embed, main_tts`.
254
  - Build both the libraries and executable using `build.sh`.
255
  - Run with one of the executables `./cactus_vlm`, `./cactus_llm`, `./cactus_embed`, `./cactus_tts`.
256
  - Try different models and make changes as desired.
257
 
258
- 4. **Contributing**
259
- - To contribute a bug fix, create a branch after making your changes with `git checkout -b <branch-name>` and submit a PR.
260
- - To contribute a feature, please raise as issue first so it can be discussed, to avoid intersecting with someone else.
261
- - [Join our discord](https://discord.gg/SdZjmfWQ)
262
-
263
  ## ![Performance](https://img.shields.io/badge/Performance-grey.svg?style=for-the-badge)
264
 
265
  | Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |
 
43
  dependencies:
44
  flutter:
45
  sdk: flutter
46
+ cactus: ^0.1.3
47
  ```
48
  2. **Install dependencies:**
49
  Execute the following command in your project terminal:
50
  ```bash
51
  flutter pub get
52
  ```
53
+ 3. **Basic Flutter Text Completion**
54
  ```dart
55
  import 'package:cactus/cactus.dart';
56
 
57
+ Future<String> basicCompletion() async {
58
+ // Initialize context
59
+ final context = await CactusContext.init(CactusInitParams(
60
+ modelPath: '/path/to/model.gguf',
61
+ contextSize: 2048,
62
+ threads: 4,
63
+ ));
64
+
65
+ // Generate response
66
+ final result = await context.completion(CactusCompletionParams(
67
+ messages: [
68
+ ChatMessage(role: 'user', content: 'Hello, how are you?')
69
+ ],
70
+ maxPredictedTokens: 100,
71
+ temperature: 0.7,
72
+ ));
73
+
74
+ context.free();
75
+ return result.text;
76
+ }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
  ```
78
+ To learn more, see the [Flutter Docs](https://github.com/cactus-compute/cactus/blob/main/docs/flutter.md). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
79
 
80
  ## ![React Native](https://img.shields.io/badge/React%20Native-grey.svg?style=for-the-badge&logo=react&logoColor=%2361DAFB)
81
 
 
94
  npx pod-install
95
  ```
96
 
97
+ 3. **Basic React-Native Text Completion**
98
  ```typescript
99
+ import { initLlama } from 'cactus-react-native';
100
+
101
+ // Initialize a context
102
+ const context = await initLlama({
103
+ model: '/path/to/your/model.gguf',
104
  n_ctx: 2048,
105
+ n_threads: 4,
106
  });
107
 
108
+ // Generate text
109
+ const result = await context.completion({
110
+ messages: [
111
+ { role: 'user', content: 'Hello, how are you?' }
112
+ ],
113
+ n_predict: 100,
114
+ temperature: 0.7,
 
 
 
 
 
 
 
 
 
 
115
  });
116
 
117
+ console.log(result.text);
 
 
 
 
 
 
 
 
 
118
  ```
119
+ To learn more, see the [React Docs](https://github.com/cactus-compute/cactus/blob/main/docs/react.md). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
120
 
121
  ## ![C++](https://img.shields.io/badge/C%2B%2B-grey.svg?style=for-the-badge&logo=c%2B%2B&logoColor=white)
122
 
 
192
  return 0;
193
  }
194
  ```
195
+ To learn more, see the [C++ Docs](https://github.com/cactus-compute/cactus/blob/main/docs/cpp.md). It covers chat design, embeddings, multimodal models, text-to-speech, and more.
196
 
197
 
198
+ ## ![Using this Repo & Example Apps](https://img.shields.io/badge/Using_Repo_And_Examples-grey.svg?style=for-the-badge)
199
 
200
  First, clone the repo with `git clone https://github.com/cactus-compute/cactus.git`, cd into it and make all scripts executable with `chmod +x scripts/*.sh`
201
 
202
  1. **Flutter**
203
  - Build the Android JNILibs with `scripts/build-flutter-android.sh`.
204
  - Build the Flutter Plugin with `scripts/build-flutter-android.sh`.
205
+ - Navigate to the example app with `cd examples/flutter`.
206
  - Open your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
207
  - Always start app with this combo `flutter clean && flutter pub get && flutter run`.
208
  - Play with the app, and make changes either to the example app or plugin as desired.
 
210
  2. **React Native**
211
  - Build the Android JNILibs with `scripts/build-react-android.sh`.
212
  - Build the Flutter Plugin with `scripts/build-react-android.sh`.
213
+ - Navigate to the example app with `cd examples/react`.
214
  - Setup your simulator via Xcode or Android Studio, [walkthrough](https://medium.com/@daspinola/setting-up-android-and-ios-emulators-22d82494deda) if you have not done this before.
215
  - Always start app with this combo `yarn && yarn ios` or `yarn && yarn android`.
216
  - Play with the app, and make changes either to the example app or package as desired.
217
  - For now, if changes are made in the package, you would manually copy the files/folders into the `examples/react/node_modules/cactus-react-native`.
218
 
219
  2. **C/C++**
220
+ - Navigate to the example app with `cd examples/cpp`.
221
  - There are multiple main files `main_vlm, main_llm, main_embed, main_tts`.
222
  - Build both the libraries and executable using `build.sh`.
223
  - Run with one of the executables `./cactus_vlm`, `./cactus_llm`, `./cactus_embed`, `./cactus_tts`.
224
  - Try different models and make changes as desired.
225
 
 
 
 
 
 
226
  ## ![Performance](https://img.shields.io/badge/Performance-grey.svg?style=for-the-badge)
227
 
228
  | Device | Gemma3 1B Q4 (toks/sec) | Qwen3 4B Q4 (toks/sec) |