ACE-Brain commited on
Commit
95285b0
·
verified ·
1 Parent(s): 8381d83

Update model card

Browse files
Files changed (1) hide show
  1. README.md +110 -0
README.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - ACE-Brain/ACE-Brain-0.5-8B
4
+ library_name: transformers
5
+ ---
6
+
7
+ <div align="center">
8
+ <img src="https://raw.githubusercontent.com/ACE-Brain-Team/ACE-Brain-0.5/main/assets/logo.png" width=600>
9
+ </div>
10
+
11
+ <br/>
12
+
13
+ <div align="center" style="line-height: 1;">
14
+ |
15
+ <a href="https://huggingface.co/ACE-Brain/ACE-Brain-0.5-8B" target="_blank">🤗 HuggingFace</a>
16
+ &nbsp;|
17
+ <a href="https://raw.githubusercontent.com/ACE-Brain-Team/ACE-Brain-0.5/main/assets/ACE-Brain-0.5.pdf" target="_blank">📔 Technical Report</a>
18
+ &nbsp;|
19
+ <a href="https://github.com/ACE-Brain-Team/ACE-Brain-0.5" target="_blank">🤖 Github</a>
20
+ &nbsp;|
21
+ <br/>
22
+ </div>
23
+
24
+ ## Overview
25
+
26
+ **ACE-Brain-0.5** is a unified embodied foundation model for Physical Agentic AI. It extends ACE-Brain-0 from an understanding-centric spatial model into a closed-loop embodied model that can perceive the physical world, plan under goals, act through robot bodies, monitor execution progress, and improve from accumulated experience.
27
+
28
+ ACE-Brain-0.5 organizes robot intelligence into five tightly coupled cognitive functions: **Spatial Perception**, **Decision Making**, **Embodied Interaction**, **Self Monitoring**, and **Self Improvement**. A single 8B backbone instantiates the core perception-planning-action-evaluation loop, supporting object and affordance grounding, 3D and egocentric spatial reasoning, long-horizon task planning, navigation and manipulation action generation, and progress estimation for verification and recovery.
29
+
30
+ <div align="center">
31
+ <img src="https://raw.githubusercontent.com/ACE-Brain-Team/ACE-Brain-0.5/main/assets/teaser.png" width=800>
32
+ </div>
33
+
34
+ ## Key Features
35
+
36
+ - Unified embodied foundation model for Physical Agentic AI
37
+ - Closed-loop cognitive model spanning Spatial Perception, Decision Making, Embodied Interaction, Self Monitoring, and Self Improvement
38
+ - SSR+ training paradigm extending Scaffold-Specialize-Reconcile with a Reactivate stage
39
+ - Unified coverage across spatial grounding, executable planning, navigation, manipulation, and progress estimation
40
+
41
+ ## Method & Architecture
42
+
43
+ ACE-Brain-0.5 uses a shared embodied backbone to encode heterogeneous inputs and maintain a unified scene-and-task representation, while dedicated interfaces decode this shared state into spatial grounding, executable subgoal planning, navigation and manipulation actions, and progress-estimation signals. Training follows **SSR+**, which inherits the spatial scaffold from ACE-Brain-0, specializes domain capabilities, reconciles task vectors through model merging, and applies a lightweight Reactivate stage to align output conventions across grounding, navigation, manipulation, and progress estimation.
44
+
45
+ <div align="center">
46
+ <img src="https://raw.githubusercontent.com/ACE-Brain-Team/ACE-Brain-0.5/main/assets/architecture.png" width=800>
47
+ </div>
48
+
49
+ ## Capability Evaluation
50
+
51
+ ACE-Brain-0.5 is evaluated as a unified embodied foundation model rather than as a collection of task-specific specialists. The goal is to verify whether a single model can preserve broad spatial understanding while extending to planning, action generation, execution monitoring, and self-improvement.
52
+
53
+ | Capability | Summary |
54
+ |------------|---------|
55
+ | **Spatial Perception** | Preserves strong spatial reasoning while extending to embodied grounding and affordance understanding. |
56
+ | **Decision Making** | Evaluates planning and decision reasoning under embodied and driving scenarios. |
57
+ | **Embodied Interaction** | Supports executable navigation decisions and continuous manipulation control. |
58
+ | **Self Monitoring** | Estimates task progress for execution assessment and recovery. |
59
+ | **Self Improvement** | Uses rollout feedback to improve behavior beyond static imitation. |
60
+
61
+ ## Inference Example
62
+
63
+ ```python
64
+ from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
65
+
66
+ model = Qwen3VLForConditionalGeneration.from_pretrained(
67
+ "ACE-Brain/ACE-Brain-0.5-8B",
68
+ dtype="auto",
69
+ device_map="auto",
70
+ )
71
+
72
+ processor = AutoProcessor.from_pretrained("ACE-Brain/ACE-Brain-0.5-8B")
73
+
74
+ messages = [
75
+ {
76
+ "role": "user",
77
+ "content": [
78
+ {
79
+ "type": "image",
80
+ "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
81
+ },
82
+ {"type": "text", "text": "Describe this image."},
83
+ ],
84
+ }
85
+ ]
86
+
87
+ inputs = processor.apply_chat_template(
88
+ messages,
89
+ tokenize=True,
90
+ add_generation_prompt=True,
91
+ return_dict=True,
92
+ return_tensors="pt",
93
+ )
94
+ inputs = inputs.to(model.device)
95
+
96
+ generated_ids = model.generate(**inputs, max_new_tokens=128)
97
+ generated_ids_trimmed = [
98
+ out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
99
+ ]
100
+ output_text = processor.batch_decode(
101
+ generated_ids_trimmed,
102
+ skip_special_tokens=True,
103
+ clean_up_tokenization_spaces=False,
104
+ )
105
+ print(output_text)
106
+ ```
107
+
108
+ ## Citation
109
+
110
+ If you find ACE-Brain-0.5 useful for your research and applications, please consider citing the technical report. BibTeX will be added once the public citation metadata is finalized.