Update README.md
Browse files
README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
**FlagOS** is a unified heterogeneous computing software stack for large models, co-developed with leading global chip manufacturers. With core technologies such as the **FlagScale** distributed training/inference framework, **FlagGems** universal operator library, **FlagCX** communication library, and **FlagTree** unified compiler, the **FlagRelease** platform leverages the FlagOS stack to automatically produce and release various combinations of <chip + open-source model>. This enables efficient and automated model migration across diverse chips, opening a new chapter for large model deployment and application.
|
| 4 |
|
| 5 |
-
Based on this, the **RoboBrain2.0-7B-
|
| 6 |
|
| 7 |
### Integrated Deployment
|
| 8 |
|
|
@@ -43,9 +43,18 @@ FlagEval (Libra)** is a comprehensive evaluation system and open platform for la
|
|
| 43 |
|
| 44 |
## Benchmark Result
|
| 45 |
|
| 46 |
-
| Metrics
|
| 47 |
-
| ----------- | ------------------------- | --------------------------------- |
|
| 48 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
# User Guide
|
| 51 |
|
|
@@ -53,10 +62,10 @@ FlagEval (Libra)** is a comprehensive evaluation system and open platform for la
|
|
| 53 |
|
| 54 |
**Basic Information**
|
| 55 |
|
| 56 |
-
| Type | Location
|
| 57 |
-
| --------------- | ----------- |
|
| 58 |
-
| Model Weights |
|
| 59 |
-
| Container Image |
|
| 60 |
|
| 61 |
**Environment Setup**
|
| 62 |
|
|
@@ -65,7 +74,7 @@ FlagEval (Libra)** is a comprehensive evaluation system and open platform for la
|
|
| 65 |
| Accelerator Card Driver Version | Driver Version: 535.183.06 |
|
| 66 |
| Docker Version | Docker version 20.10.5, build 55c4c88 |
|
| 67 |
| Operating System | Description: Ubuntu 22.04.4 LTS |
|
| 68 |
-
| FlagScale | Version: 0.
|
| 69 |
| FlagGems | Version: 2.2 |
|
| 70 |
|
| 71 |
## Operation Steps
|
|
@@ -74,13 +83,13 @@ FlagEval (Libra)** is a comprehensive evaluation system and open platform for la
|
|
| 74 |
|
| 75 |
```python
|
| 76 |
pip install modelscope
|
| 77 |
-
modelscope download --model
|
| 78 |
```
|
| 79 |
|
| 80 |
### Download FlagOS Image
|
| 81 |
|
| 82 |
```python
|
| 83 |
-
docker pull
|
| 84 |
```
|
| 85 |
|
| 86 |
### Start the inference service
|
|
@@ -98,12 +107,25 @@ docker run --rm --init --detach \
|
|
| 98 |
-v /share:/share \
|
| 99 |
--gpus all \
|
| 100 |
--name flagos \
|
| 101 |
-
|
| 102 |
sleep infinity
|
| 103 |
|
| 104 |
docker exec -it flagos bash
|
| 105 |
```
|
| 106 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
### Serve
|
| 108 |
|
| 109 |
```python
|
|
@@ -123,7 +145,7 @@ flagscale serve robobrain2
|
|
| 123 |
import openai
|
| 124 |
openai.api_key = "EMPTY"
|
| 125 |
openai.base_url = "http://<server_ip>:9010/v1/"
|
| 126 |
-
model = "RoboBrain2
|
| 127 |
messages = [
|
| 128 |
{"role": "system", "content": "You are a helpful assistant."},
|
| 129 |
{"role": "user", "content": "What's the weather like today?"}
|
|
@@ -192,4 +214,4 @@ We warmly welcome global developers to join us:
|
|
| 192 |
|
| 193 |
This project and related model weights are licensed under the MIT License.
|
| 194 |
|
| 195 |
-
Release Date: 2025.07.
|
|
|
|
| 2 |
|
| 3 |
**FlagOS** is a unified heterogeneous computing software stack for large models, co-developed with leading global chip manufacturers. With core technologies such as the **FlagScale** distributed training/inference framework, **FlagGems** universal operator library, **FlagCX** communication library, and **FlagTree** unified compiler, the **FlagRelease** platform leverages the FlagOS stack to automatically produce and release various combinations of <chip + open-source model>. This enables efficient and automated model migration across diverse chips, opening a new chapter for large model deployment and application.
|
| 4 |
|
| 5 |
+
Based on this, the **RoboBrain2.0-7B-FP8Dynamic-FlagOS** model is adapted for the Metax chip using the FlagOS software stack, enabling:
|
| 6 |
|
| 7 |
### Integrated Deployment
|
| 8 |
|
|
|
|
| 43 |
|
| 44 |
## Benchmark Result
|
| 45 |
|
| 46 |
+
| Metrics | RoboBrain2.0-7B-H100-CUDA | RoboBrain2.0-7B-FP8Dynamic-FlagOS |
|
| 47 |
+
| --------------------- | ------------------------- | --------------------------------- |
|
| 48 |
+
| SAT | 75.330 | 72.000 |
|
| 49 |
+
| all_angles_bench | 47.700 | 46.480 |
|
| 50 |
+
| Where2Place | 63.590 | 63.060 |
|
| 51 |
+
| blink_val_ev | 56.360 | 55.200 |
|
| 52 |
+
| robo_spatial_home_all | 54.227 | 54.312 |
|
| 53 |
+
| egoplan_bench2 | 33.230 | 33.310 |
|
| 54 |
+
| erqa | 38.750 | 39.750 |
|
| 55 |
+
| cv_bench_test | 85.750 | 85.770 |
|
| 56 |
+
| embspatial_bench | 76.320 | 75.270 |
|
| 57 |
+
| vsi_bench_tiny | 36.100 | 38.700 |
|
| 58 |
|
| 59 |
# User Guide
|
| 60 |
|
|
|
|
| 62 |
|
| 63 |
**Basic Information**
|
| 64 |
|
| 65 |
+
| Type | Location |
|
| 66 |
+
| --------------- | ------------------------------------------------------------ |
|
| 67 |
+
| Model Weights | https://huggingface.co/FlagRelease/RoboBrain2.0-7B-FP8Dynamic-FlagOS/files |
|
| 68 |
+
| Container Image | flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:flagrelease_nv_robobrain2_32b |
|
| 69 |
|
| 70 |
**Environment Setup**
|
| 71 |
|
|
|
|
| 74 |
| Accelerator Card Driver Version | Driver Version: 535.183.06 |
|
| 75 |
| Docker Version | Docker version 20.10.5, build 55c4c88 |
|
| 76 |
| Operating System | Description: Ubuntu 22.04.4 LTS |
|
| 77 |
+
| FlagScale | Version: 0.8.0 |
|
| 78 |
| FlagGems | Version: 2.2 |
|
| 79 |
|
| 80 |
## Operation Steps
|
|
|
|
| 83 |
|
| 84 |
```python
|
| 85 |
pip install modelscope
|
| 86 |
+
modelscope download --model FlagRelease/RoboBrain2.0-7B-FP8Dynamic-FlagOS --local_dir /share/RoboBrain2.0-7B-FP8Dynamic
|
| 87 |
```
|
| 88 |
|
| 89 |
### Download FlagOS Image
|
| 90 |
|
| 91 |
```python
|
| 92 |
+
docker pull flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:flagrelease_nv_robobrain2_32b
|
| 93 |
```
|
| 94 |
|
| 95 |
### Start the inference service
|
|
|
|
| 107 |
-v /share:/share \
|
| 108 |
--gpus all \
|
| 109 |
--name flagos \
|
| 110 |
+
flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:flagrelease_nv_robobrain2_32b \
|
| 111 |
sleep infinity
|
| 112 |
|
| 113 |
docker exec -it flagos bash
|
| 114 |
```
|
| 115 |
|
| 116 |
+
### **Modify configuration files**
|
| 117 |
+
|
| 118 |
+
```
|
| 119 |
+
#Use 'pip show flag_scale' to find the installation path of FlagScale.
|
| 120 |
+
pip show flag_scale
|
| 121 |
+
# Modify the 7b.yaml file located at flag_scale/examples/robobrain2/conf/serve
|
| 122 |
+
set the 【model path】 to /share/RoboBrain2.0-7B-FP8Dynamic
|
| 123 |
+
set the 【tensor_parallel_size】 to 4
|
| 124 |
+
set the 【served-model-name】 to RoboBrain2-7B-nvidia-flagos-FP8Dynamic
|
| 125 |
+
# Modify the serve.yaml file located at flag_scale/examples/robobrain2/conf
|
| 126 |
+
Change all the 32b to 7b in it.
|
| 127 |
+
```
|
| 128 |
+
|
| 129 |
### Serve
|
| 130 |
|
| 131 |
```python
|
|
|
|
| 145 |
import openai
|
| 146 |
openai.api_key = "EMPTY"
|
| 147 |
openai.base_url = "http://<server_ip>:9010/v1/"
|
| 148 |
+
model = "RoboBrain2-7B-nvidia-flagos-FP8Dynamic"
|
| 149 |
messages = [
|
| 150 |
{"role": "system", "content": "You are a helpful assistant."},
|
| 151 |
{"role": "user", "content": "What's the weather like today?"}
|
|
|
|
| 214 |
|
| 215 |
This project and related model weights are licensed under the MIT License.
|
| 216 |
|
| 217 |
+
Release Date: 2025.07.15
|