YummyYum commited on
Commit
e71d30a
·
verified ·
1 Parent(s): 5e6c0a7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -14
README.md CHANGED
@@ -2,7 +2,7 @@
2
 
3
  **FlagOS** is a unified heterogeneous computing software stack for large models, co-developed with leading global chip manufacturers. With core technologies such as the **FlagScale** distributed training/inference framework, **FlagGems** universal operator library, **FlagCX** communication library, and **FlagTree** unified compiler, the **FlagRelease** platform leverages the FlagOS stack to automatically produce and release various combinations of <chip + open-source model>. This enables efficient and automated model migration across diverse chips, opening a new chapter for large model deployment and application.
4
 
5
- Based on this, the **RoboBrain2.0-7B-FlagOS-FP8Dynamic** model is adapted for the Metax chip using the FlagOS software stack, enabling:
6
 
7
  ### Integrated Deployment
8
 
@@ -43,9 +43,18 @@ FlagEval (Libra)** is a comprehensive evaluation system and open platform for la
43
 
44
  ## Benchmark Result
45
 
46
- | Metrics | RoboBrain2.0-7B-H100-CUDA | RoboBrain2.0-7B-FlagOS-FP8Dynamic |
47
- | ----------- | ------------------------- | --------------------------------- |
48
- | coming soon | coming soon | coming soon |
 
 
 
 
 
 
 
 
 
49
 
50
  # User Guide
51
 
@@ -53,10 +62,10 @@ FlagEval (Libra)** is a comprehensive evaluation system and open platform for la
53
 
54
  **Basic Information**
55
 
56
- | Type | Location |
57
- | --------------- | ----------- |
58
- | Model Weights | (https://huggingface.co/FlagRelease/RoboBrain2.0-7B-FlagOS-FP8Dynamic/tree/main) |
59
- | Container Image | coming soon |
60
 
61
  **Environment Setup**
62
 
@@ -65,7 +74,7 @@ FlagEval (Libra)** is a comprehensive evaluation system and open platform for la
65
  | Accelerator Card Driver Version | Driver Version: 535.183.06 |
66
  | Docker Version | Docker version 20.10.5, build 55c4c88 |
67
  | Operating System | Description: Ubuntu 22.04.4 LTS |
68
- | FlagScale | Version: 0.6.0 |
69
  | FlagGems | Version: 2.2 |
70
 
71
  ## Operation Steps
@@ -74,13 +83,13 @@ FlagEval (Libra)** is a comprehensive evaluation system and open platform for la
74
 
75
  ```python
76
  pip install modelscope
77
- modelscope download --model <model path> --local_dir /share/RoboBrain2.0-7B
78
  ```
79
 
80
  ### Download FlagOS Image
81
 
82
  ```python
83
- docker pull <image>
84
  ```
85
 
86
  ### Start the inference service
@@ -98,12 +107,25 @@ docker run --rm --init --detach \
98
  -v /share:/share \
99
  --gpus all \
100
  --name flagos \
101
- <image> \
102
  sleep infinity
103
 
104
  docker exec -it flagos bash
105
  ```
106
 
 
 
 
 
 
 
 
 
 
 
 
 
 
107
  ### Serve
108
 
109
  ```python
@@ -123,7 +145,7 @@ flagscale serve robobrain2
123
  import openai
124
  openai.api_key = "EMPTY"
125
  openai.base_url = "http://<server_ip>:9010/v1/"
126
- model = "RoboBrain2.0-7B-nv-flagos-FP8Dynamic"
127
  messages = [
128
  {"role": "system", "content": "You are a helpful assistant."},
129
  {"role": "user", "content": "What's the weather like today?"}
@@ -192,4 +214,4 @@ We warmly welcome global developers to join us:
192
 
193
  This project and related model weights are licensed under the MIT License.
194
 
195
- Release Date: 2025.07.12
 
2
 
3
  **FlagOS** is a unified heterogeneous computing software stack for large models, co-developed with leading global chip manufacturers. With core technologies such as the **FlagScale** distributed training/inference framework, **FlagGems** universal operator library, **FlagCX** communication library, and **FlagTree** unified compiler, the **FlagRelease** platform leverages the FlagOS stack to automatically produce and release various combinations of <chip + open-source model>. This enables efficient and automated model migration across diverse chips, opening a new chapter for large model deployment and application.
4
 
5
+ Based on this, the **RoboBrain2.0-7B-FP8Dynamic-FlagOS** model is adapted for the Metax chip using the FlagOS software stack, enabling:
6
 
7
  ### Integrated Deployment
8
 
 
43
 
44
  ## Benchmark Result
45
 
46
+ | Metrics | RoboBrain2.0-7B-H100-CUDA | RoboBrain2.0-7B-FP8Dynamic-FlagOS |
47
+ | --------------------- | ------------------------- | --------------------------------- |
48
+ | SAT | 75.330 | 72.000 |
49
+ | all_angles_bench | 47.700 | 46.480 |
50
+ | Where2Place | 63.590 | 63.060 |
51
+ | blink_val_ev | 56.360 | 55.200 |
52
+ | robo_spatial_home_all | 54.227 | 54.312 |
53
+ | egoplan_bench2 | 33.230 | 33.310 |
54
+ | erqa | 38.750 | 39.750 |
55
+ | cv_bench_test | 85.750 | 85.770 |
56
+ | embspatial_bench | 76.320 | 75.270 |
57
+ | vsi_bench_tiny | 36.100 | 38.700 |
58
 
59
  # User Guide
60
 
 
62
 
63
  **Basic Information**
64
 
65
+ | Type | Location |
66
+ | --------------- | ------------------------------------------------------------ |
67
+ | Model Weights | https://huggingface.co/FlagRelease/RoboBrain2.0-7B-FP8Dynamic-FlagOS/files |
68
+ | Container Image | flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:flagrelease_nv_robobrain2_32b |
69
 
70
  **Environment Setup**
71
 
 
74
  | Accelerator Card Driver Version | Driver Version: 535.183.06 |
75
  | Docker Version | Docker version 20.10.5, build 55c4c88 |
76
  | Operating System | Description: Ubuntu 22.04.4 LTS |
77
+ | FlagScale | Version: 0.8.0 |
78
  | FlagGems | Version: 2.2 |
79
 
80
  ## Operation Steps
 
83
 
84
  ```python
85
  pip install modelscope
86
+ modelscope download --model FlagRelease/RoboBrain2.0-7B-FP8Dynamic-FlagOS --local_dir /share/RoboBrain2.0-7B-FP8Dynamic
87
  ```
88
 
89
  ### Download FlagOS Image
90
 
91
  ```python
92
+ docker pull flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:flagrelease_nv_robobrain2_32b
93
  ```
94
 
95
  ### Start the inference service
 
107
  -v /share:/share \
108
  --gpus all \
109
  --name flagos \
110
+ flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:flagrelease_nv_robobrain2_32b \
111
  sleep infinity
112
 
113
  docker exec -it flagos bash
114
  ```
115
 
116
+ ### **Modify configuration files**
117
+
118
+ ```
119
+ #Use 'pip show flag_scale' to find the installation path of FlagScale.
120
+ pip show flag_scale
121
+ # Modify the 7b.yaml file located at flag_scale/examples/robobrain2/conf/serve
122
+ set the 【model path】 to /share/RoboBrain2.0-7B-FP8Dynamic
123
+ set the 【tensor_parallel_size】 to 4
124
+ set the 【served-model-name】 to RoboBrain2-7B-nvidia-flagos-FP8Dynamic
125
+ # Modify the serve.yaml file located at flag_scale/examples/robobrain2/conf
126
+ Change all the 32b to 7b in it.
127
+ ```
128
+
129
  ### Serve
130
 
131
  ```python
 
145
  import openai
146
  openai.api_key = "EMPTY"
147
  openai.base_url = "http://<server_ip>:9010/v1/"
148
+ model = "RoboBrain2-7B-nvidia-flagos-FP8Dynamic"
149
  messages = [
150
  {"role": "system", "content": "You are a helpful assistant."},
151
  {"role": "user", "content": "What's the weather like today?"}
 
214
 
215
  This project and related model weights are licensed under the MIT License.
216
 
217
+ Release Date: 2025.07.15