chingshuai commited on
Commit
b3c5ec9
·
1 Parent(s): 82276ad
Files changed (3) hide show
  1. README.md +44 -5
  2. app.py +300 -4
  3. requirements.txt +6 -0
README.md CHANGED
@@ -1,13 +1,52 @@
1
  ---
2
- title: Test Text2MotionPrompter
3
- emoji: 🏆
4
  colorFrom: purple
5
  colorTo: blue
6
  sdk: gradio
7
- sdk_version: 6.2.0
8
  app_file: app.py
9
  pinned: false
10
- short_description: Test Text2MotionPrompter
 
 
 
 
 
 
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Text2Motion Prompter
3
+ emoji: 🎬
4
  colorFrom: purple
5
  colorTo: blue
6
  sdk: gradio
7
+ sdk_version: 5.9.1
8
  app_file: app.py
9
  pinned: false
10
+ short_description: Transform action descriptions into structured motion prompts
11
+ license: mit
12
+ tags:
13
+ - motion-generation
14
+ - text-to-motion
15
+ - animation
16
+ - 3d-motion
17
  ---
18
 
19
+ # 🎬 Text2Motion Prompter
20
+
21
+ Transform action descriptions into structured motion prompts for 3D animation systems.
22
+
23
+ ## Features
24
+
25
+ - **Duration Estimation**: Analyzes action complexity to estimate execution time in frames (30fps)
26
+ - **Caption Refinement**: Generates refined, grammatically correct English descriptions
27
+ - **Multi-language Support**: Accepts inputs in multiple languages (English, Chinese, etc.)
28
+ - **ZeroGPU Powered**: Utilizes Hugging Face's ZeroGPU for efficient inference
29
+
30
+ ## Usage
31
+
32
+ 1. Enter an action description in the input field (e.g., "walking forward", "跳跃")
33
+ 2. Adjust the max tokens if needed
34
+ 3. Click "Generate Motion Prompt"
35
+ 4. Get a structured JSON response with duration and refined caption
36
+
37
+ ## Output Format
38
+
39
+ ```json
40
+ {
41
+ "duration": 90,
42
+ "short_caption": "A person walks forward at a steady pace."
43
+ }
44
+ ```
45
+
46
+ ## Model
47
+
48
+ This Space uses the [Text2MotionPrompter](https://huggingface.co/Text2MotionPrompter/Text2MotionPrompter) model.
49
+
50
+ ## License
51
+
52
+ MIT License
app.py CHANGED
@@ -1,7 +1,303 @@
1
  import gradio as gr
 
 
 
2
 
3
- def greet(name):
4
- return "Hello " + name + "!!"
5
 
6
- demo = gr.Interface(fn=greet, inputs="text", outputs="text")
7
- demo.launch()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  import gradio as gr
2
+ import spaces
3
+ import torch
4
+ from transformers import AutoModelForCausalLM, AutoTokenizer
5
 
6
+ # Model configuration
7
+ MODEL_NAME = "Text2MotionPrompter/Text2MotionPrompter"
8
 
9
+ # Prompt template
10
+ TEMPLATE = """
11
+ # Role
12
+ You are an expert in 3D motion analysis, animation timing, and choreography. Your task is to analyze textual action descriptions to estimate execution time and standardize the language for motion generation systems.
13
+
14
+ # Task
15
+ Analyze the user-provided [Input Action] and generate a structured JSON response containing a duration estimate and a refined caption.
16
+
17
+ # Instructions
18
+
19
+ ### 1. Duration Estimation (frame_count)
20
+ - Analyze the complexity, speed, and physical constraints of the described action.
21
+ - Estimate the time required to perform the action in a **smooth, natural, and realistic manner**.
22
+ - Calculate the total duration in frames based on a **30 fps** (frames per second) standard.
23
+ - Output strictly as an Integer.
24
+
25
+ ### 2. Caption Refinement (short_caption)
26
+ - Generate a refined, grammatically correct version of the input description in **English**.
27
+ - **Strict Constraints**:
28
+ - You must **PRESERVE** the original sequence of events (chronological order).
29
+ - You must **RETAIN** all original spatial modifiers (e.g., "left," "upward," "quickly").
30
+ - **DO NOT** add new sub-actions or hallucinate details not present in the input.
31
+ - **DO NOT** delete any specific movements.
32
+ - The goal is to improve clarity and flow while maintaining 100% semantic fidelity to the original request.
33
+
34
+ ### 3. Output Format
35
+ - Return **ONLY** a raw JSON object.
36
+ - Do not use Markdown formatting (i.e., do not use ```json ... ```).
37
+ - Ensure the JSON is valid and parsable.
38
+
39
+ # JSON Structure
40
+ {{
41
+ "duration": <Integer, frames at 30fps>,
42
+ "short_caption": "<String, the refined English description>"
43
+ }}
44
+
45
+ # Input
46
+ {}
47
+ """
48
+
49
+ # Global variables for lazy loading
50
+ tokenizer = None
51
+ model = None
52
+
53
+
54
+ def load_model():
55
+ """Load the model and tokenizer (lazy loading)"""
56
+ global tokenizer, model
57
+ if tokenizer is None:
58
+ tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
59
+ if model is None:
60
+ model = AutoModelForCausalLM.from_pretrained(
61
+ MODEL_NAME,
62
+ torch_dtype=torch.float16,
63
+ device_map="auto"
64
+ )
65
+ return tokenizer, model
66
+
67
+
68
+ @spaces.GPU(duration=120)
69
+ def generate_motion_prompt(action_input: str, max_new_tokens: int = 512) -> str:
70
+ """
71
+ Generate motion prompt from action description.
72
+
73
+ Args:
74
+ action_input: The action description to analyze
75
+ max_new_tokens: Maximum number of tokens to generate
76
+
77
+ Returns:
78
+ Generated JSON response with duration and refined caption
79
+ """
80
+ if not action_input.strip():
81
+ return "Please enter an action description."
82
+
83
+ # Load model (will use cached version if already loaded)
84
+ tokenizer, model = load_model()
85
+
86
+ # Prepare the prompt
87
+ messages = [
88
+ {"role": "user", "content": TEMPLATE.format(action_input)}
89
+ ]
90
+
91
+ text = tokenizer.apply_chat_template(
92
+ messages,
93
+ tokenize=False,
94
+ add_generation_prompt=True,
95
+ )
96
+
97
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
98
+
99
+ # Generate response
100
+ with torch.no_grad():
101
+ generated_ids = model.generate(
102
+ **model_inputs,
103
+ max_new_tokens=max_new_tokens,
104
+ do_sample=True,
105
+ temperature=0.7,
106
+ top_p=0.9,
107
+ )
108
+
109
+ # Decode only the new tokens
110
+ output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
111
+ content = tokenizer.decode(output_ids, skip_special_tokens=True)
112
+
113
+ return content
114
+
115
+
116
+ # Custom CSS for a distinctive look
117
+ custom_css = """
118
+ @import url('https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;600&family=Space+Grotesk:wght@400;500;600&display=swap');
119
+
120
+ .gradio-container {
121
+ font-family: 'Space Grotesk', sans-serif !important;
122
+ background: linear-gradient(135deg, #0f0f1a 0%, #1a1a2e 50%, #16213e 100%) !important;
123
+ min-height: 100vh;
124
+ }
125
+
126
+ .main-title {
127
+ background: linear-gradient(90deg, #00d4ff, #7c3aed, #f472b6);
128
+ -webkit-background-clip: text;
129
+ -webkit-text-fill-color: transparent;
130
+ background-clip: text;
131
+ font-size: 2.5rem !important;
132
+ font-weight: 600 !important;
133
+ text-align: center;
134
+ margin-bottom: 0.5rem !important;
135
+ font-family: 'Space Grotesk', sans-serif !important;
136
+ }
137
+
138
+ .subtitle {
139
+ color: #94a3b8 !important;
140
+ text-align: center;
141
+ font-size: 1.1rem !important;
142
+ margin-bottom: 2rem !important;
143
+ }
144
+
145
+ .input-container, .output-container {
146
+ background: rgba(30, 41, 59, 0.7) !important;
147
+ border: 1px solid rgba(100, 116, 139, 0.3) !important;
148
+ border-radius: 16px !important;
149
+ backdrop-filter: blur(10px);
150
+ }
151
+
152
+ textarea, input[type="text"] {
153
+ background: rgba(15, 23, 42, 0.8) !important;
154
+ border: 1px solid rgba(100, 116, 139, 0.4) !important;
155
+ color: #e2e8f0 !important;
156
+ font-family: 'JetBrains Mono', monospace !important;
157
+ border-radius: 12px !important;
158
+ }
159
+
160
+ textarea:focus, input[type="text"]:focus {
161
+ border-color: #7c3aed !important;
162
+ box-shadow: 0 0 0 3px rgba(124, 58, 237, 0.2) !important;
163
+ }
164
+
165
+ .primary-btn {
166
+ background: linear-gradient(135deg, #7c3aed 0%, #a855f7 100%) !important;
167
+ border: none !important;
168
+ color: white !important;
169
+ font-weight: 600 !important;
170
+ padding: 12px 32px !important;
171
+ border-radius: 12px !important;
172
+ font-size: 1rem !important;
173
+ transition: all 0.3s ease !important;
174
+ box-shadow: 0 4px 15px rgba(124, 58, 237, 0.4) !important;
175
+ }
176
+
177
+ .primary-btn:hover {
178
+ transform: translateY(-2px) !important;
179
+ box-shadow: 0 6px 20px rgba(124, 58, 237, 0.6) !important;
180
+ }
181
+
182
+ .example-btn {
183
+ background: rgba(51, 65, 85, 0.6) !important;
184
+ border: 1px solid rgba(100, 116, 139, 0.4) !important;
185
+ color: #cbd5e1 !important;
186
+ border-radius: 8px !important;
187
+ transition: all 0.2s ease !important;
188
+ }
189
+
190
+ .example-btn:hover {
191
+ background: rgba(71, 85, 105, 0.8) !important;
192
+ border-color: #7c3aed !important;
193
+ }
194
+
195
+ label {
196
+ color: #94a3b8 !important;
197
+ font-weight: 500 !important;
198
+ }
199
+
200
+ .output-json {
201
+ font-family: 'JetBrains Mono', monospace !important;
202
+ background: rgba(15, 23, 42, 0.9) !important;
203
+ color: #22d3ee !important;
204
+ padding: 1.5rem !important;
205
+ border-radius: 12px !important;
206
+ border: 1px solid rgba(34, 211, 238, 0.2) !important;
207
+ }
208
+
209
+ .footer {
210
+ text-align: center;
211
+ color: #64748b;
212
+ margin-top: 2rem;
213
+ padding: 1rem;
214
+ font-size: 0.9rem;
215
+ }
216
+
217
+ .slider-container input[type="range"] {
218
+ accent-color: #7c3aed !important;
219
+ }
220
+ """
221
+
222
+ # Build the Gradio interface
223
+ with gr.Blocks(css=custom_css, theme=gr.themes.Base()) as demo:
224
+ gr.HTML("""
225
+ <h1 class="main-title">🎬 Text2Motion Prompter</h1>
226
+ <p class="subtitle">Transform action descriptions into structured motion prompts for 3D animation</p>
227
+ """)
228
+
229
+ with gr.Row():
230
+ with gr.Column(scale=1):
231
+ action_input = gr.Textbox(
232
+ label="🎯 Action Description",
233
+ placeholder="Enter an action description (e.g., 'walking forward', '向前走', 'jumping up and down')",
234
+ lines=4,
235
+ elem_classes=["input-container"]
236
+ )
237
+
238
+ max_tokens = gr.Slider(
239
+ label="🔧 Max Tokens",
240
+ minimum=128,
241
+ maximum=2048,
242
+ value=512,
243
+ step=64,
244
+ elem_classes=["slider-container"]
245
+ )
246
+
247
+ submit_btn = gr.Button(
248
+ "✨ Generate Motion Prompt",
249
+ variant="primary",
250
+ elem_classes=["primary-btn"]
251
+ )
252
+
253
+ gr.Markdown("### 📝 Examples")
254
+ examples = gr.Examples(
255
+ examples=[
256
+ ["走路"],
257
+ ["A person walks forward slowly, then turns left"],
258
+ ["跳跃并挥手"],
259
+ ["Running quickly and then stopping suddenly"],
260
+ ["坐下来休息一会儿,然后站起来"],
261
+ ["Dancing with arms raised above the head"],
262
+ ],
263
+ inputs=action_input,
264
+ elem_id="examples"
265
+ )
266
+
267
+ with gr.Column(scale=1):
268
+ output = gr.Textbox(
269
+ label="📤 Generated Motion Prompt (JSON)",
270
+ lines=12,
271
+ elem_classes=["output-container", "output-json"]
272
+ )
273
+
274
+ gr.Markdown("""
275
+ ### 📖 Output Format
276
+ The model generates a JSON response containing:
277
+ - **duration**: Estimated frames at 30fps
278
+ - **short_caption**: Refined English description
279
+ """)
280
+
281
+ gr.HTML("""
282
+ <div class="footer">
283
+ <p>Powered by <strong>Text2MotionPrompter</strong> | Using Hugging Face ZeroGPU</p>
284
+ </div>
285
+ """)
286
+
287
+ # Set up the action
288
+ submit_btn.click(
289
+ fn=generate_motion_prompt,
290
+ inputs=[action_input, max_tokens],
291
+ outputs=output
292
+ )
293
+
294
+ # Allow Enter key to submit
295
+ action_input.submit(
296
+ fn=generate_motion_prompt,
297
+ inputs=[action_input, max_tokens],
298
+ outputs=output
299
+ )
300
+
301
+ # Launch the app
302
+ if __name__ == "__main__":
303
+ demo.launch()
requirements.txt ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ gradio>=4.0.0
2
+ torch>=2.0.0
3
+ transformers>=4.40.0
4
+ accelerate>=0.27.0
5
+ spaces
6
+