chenemii commited on
Commit
7680d90
·
1 Parent(s): d86b170

try to fix permission denied issue in community cloud

Browse files
README.md CHANGED
@@ -9,7 +9,7 @@ A tool for analyzing golf swings using computer vision and AI.
9
  - Pose estimation and tracking
10
  - Swing phase segmentation
11
  - Club and ball trajectory analysis
12
- - LLM-powered swing analysis and coaching tips
13
  - Annotated video generation
14
  - Side-by-side comparison with professional golfer
15
  - Improvement recommendations from AI analysis
@@ -29,10 +29,32 @@ A tool for analyzing golf swings using computer vision and AI.
29
  - Save a video of a professional golfer's swing as `pro_golfer.mp4` in the `downloads` directory
30
  - This will be used for the side-by-side comparison feature
31
 
32
- 5. Set your OpenAI API key as an environment variable:
33
- ```
34
- export OPENAI_API_KEY="your-api-key"
35
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
  ## Running the Application
38
 
@@ -61,7 +83,7 @@ The application uses:
61
  - YOLOv8 for object detection
62
  - MediaPipe for pose estimation
63
  - OpenCV for video processing
64
- - OpenAI GPT-4 for swing analysis
65
  - Streamlit for the web interface
66
 
67
  ## Directory Structure
 
9
  - Pose estimation and tracking
10
  - Swing phase segmentation
11
  - Club and ball trajectory analysis
12
+ - LLM-powered swing analysis and coaching tips (OpenAI GPT-4/3.5 or local Ollama models)
13
  - Annotated video generation
14
  - Side-by-side comparison with professional golfer
15
  - Improvement recommendations from AI analysis
 
29
  - Save a video of a professional golfer's swing as `pro_golfer.mp4` in the `downloads` directory
30
  - This will be used for the side-by-side comparison feature
31
 
32
+ 5. Set up LLM services for analysis (optional):
33
+
34
+ **Option 1: OpenAI**
35
+ - Set your OpenAI API key in `.streamlit/secrets.toml`:
36
+ ```toml
37
+ [openai]
38
+ api_key = "your-openai-api-key-here"
39
+ ```
40
+
41
+ **Option 2: Ollama (Local LLM)**
42
+ - Install and run Ollama locally: https://ollama.ai/
43
+ - Configure in `.streamlit/secrets.toml`:
44
+ ```toml
45
+ [ollama]
46
+ base_url = "http://localhost:11434/v1"
47
+ model = "llama2" # or your preferred model
48
+ ```
49
+
50
+ **Option 3: Both Services**
51
+ - Configure both in `.streamlit/secrets.toml` for automatic fallback
52
+ - The app will try Ollama first, then OpenAI if Ollama fails
53
+
54
+ **No Configuration**
55
+ - The app works without any LLM configuration using sample analysis mode
56
+
57
+ See `.streamlit/secrets.toml.example` for a complete configuration template.
58
 
59
  ## Running the Application
60
 
 
83
  - YOLOv8 for object detection
84
  - MediaPipe for pose estimation
85
  - OpenCV for video processing
86
+ - OpenAI GPT-4/3.5 or Ollama for swing analysis
87
  - Streamlit for the web interface
88
 
89
  ## Directory Structure
app/models/llm_analyzer.py CHANGED
@@ -8,6 +8,49 @@ from openai import OpenAI
8
  import streamlit as st
9
 
10
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  def generate_swing_analysis(pose_data, swing_phases, trajectory_data):
12
  """
13
  Generate swing analysis and coaching tips using LLM
@@ -20,111 +63,162 @@ def generate_swing_analysis(pose_data, swing_phases, trajectory_data):
20
  Returns:
21
  str: Detailed swing analysis and coaching tips
22
  """
23
- # Check if OpenAI API key is available from secrets
24
- try:
25
- api_key = st.secrets["openai"]["api_key"]
26
- except (KeyError, FileNotFoundError):
27
- # Return a sample analysis instead of an error message
28
- return """
29
- ## Swing Analysis Summary
30
-
31
- Based on the video analysis, here are some observations about your swing:
32
-
33
- ### Setup Phase
34
- - Your stance appears slightly wider than shoulder-width, which can provide good stability
35
- - Your posture shows a good spine angle, though you could bend slightly more from the hips
36
- - The ball position looks appropriate for the club you're using
37
-
38
- ### Backswing
39
- - Your takeaway is smooth with good tempo
40
- - Your wrist hinge develops appropriately in the backswing
41
- - Your right elbow could be kept a bit closer to your body for better consistency
42
-
43
- ### Downswing
44
- - Good weight transfer from back foot to front foot during the transition
45
- - Your hips are rotating well through impact
46
- - The swing plane looks consistent throughout the downswing
47
 
48
- ### Impact
49
- - Club face alignment at impact appears slightly open
50
- - Your head position is stable through impact
51
- - The club path is on a good line toward the target
52
 
53
- ### Follow Through
54
- - Good balance maintained through the finish
55
- - Full extension of arms after impact
56
- - Complete rotation of the body toward the target
57
 
58
- ## Areas for Improvement
 
 
 
 
 
 
 
 
59
 
60
- 1. **Club Face Control**: The slightly open club face at impact suggests you may be prone to slicing the ball. Focus on maintaining a square club face through impact.
 
 
 
 
 
 
 
 
 
61
 
62
- 2. **Right Elbow Position**: Keeping your right elbow closer to your body during the backswing will help create a more consistent swing plane.
 
63
 
64
- 3. **Hip Rotation**: While your hip rotation is good, increasing the speed of rotation could generate more power in your swing.
65
 
66
- 4. **Wrist Release**: Your wrist release could be more active through impact to generate additional club head speed.
 
 
 
 
 
 
 
 
 
 
 
 
 
67
 
68
- These adjustments should help improve both consistency and distance in your swing.
69
- """
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
70
 
71
- # Prepare data for LLM
72
- analysis_data = prepare_data_for_llm(pose_data, swing_phases,
73
- trajectory_data)
74
 
75
- # Generate prompt for LLM
76
- prompt = create_llm_prompt(analysis_data)
77
 
 
 
 
 
 
 
 
 
 
 
 
78
  try:
79
  # Create a custom httpx client without proxies
80
  http_client = httpx.Client()
81
-
82
- # Initialize the OpenAI client with the custom http client
83
- # This avoids any proxy settings that might be causing issues
84
- client = OpenAI(
85
- api_key=api_key,
86
- http_client=http_client
87
- )
88
-
89
  try:
90
  # Try with GPT-4 first
91
  response = client.chat.completions.create(
92
  model="gpt-4-turbo",
93
- messages=[
94
- {"role": "system", "content": "You are a professional golf coach with expertise in analyzing golf swings. Provide detailed, actionable feedback based on the swing data provided."},
95
- {"role": "user", "content": prompt}
96
- ],
 
 
 
 
 
97
  temperature=0.7,
98
- max_tokens=1000
99
- )
100
-
101
- # Extract content from the response
102
- analysis = response.choices[0].message.content
103
- return analysis
104
-
105
  except Exception as gpt4_error:
106
  # If there's an error with GPT-4 (like quota exceeded), try GPT-3.5
107
- print(f"Error with GPT-4: {str(gpt4_error)}. Falling back to GPT-3.5-turbo...")
108
-
 
 
109
  try:
110
  response = client.chat.completions.create(
111
  model="gpt-3.5-turbo",
112
- messages=[
113
- {"role": "system", "content": "You are a professional golf coach with expertise in analyzing golf swings. Provide detailed, actionable feedback based on the swing data provided."},
114
- {"role": "user", "content": prompt}
115
- ],
 
 
 
 
 
116
  temperature=0.7,
117
- max_tokens=1000
118
- )
119
-
120
- # Extract content from the response
121
- analysis = response.choices[0].message.content
122
- return analysis
123
-
124
  except Exception as gpt35_error:
125
- # Both models failed, return the sample analysis
126
- print(f"Error with GPT-3.5: {str(gpt35_error)}. Using sample analysis instead.")
127
- return """
 
 
 
 
 
 
 
 
 
 
 
 
 
128
  ## Swing Analysis Summary
129
 
130
  Based on the video analysis, here are some observations about your swing:
@@ -167,9 +261,6 @@ Based on the video analysis, here are some observations about your swing:
167
  These adjustments should help improve both consistency and distance in your swing.
168
  """
169
 
170
- except Exception as e:
171
- return f"Error generating swing analysis: {str(e)}"
172
-
173
 
174
  def prepare_data_for_llm(pose_data, swing_phases, trajectory_data):
175
  """
@@ -214,18 +305,18 @@ def prepare_data_for_llm(pose_data, swing_phases, trajectory_data):
214
  # Calculate backswing and downswing durations if available
215
  backswing_frames = swing_phases.get("backswing", [])
216
  downswing_frames = swing_phases.get("downswing", [])
217
-
218
  backswing_duration = None
219
  downswing_duration = None
220
-
221
  if backswing_frames:
222
  # Assuming 30 fps video
223
  backswing_duration = len(backswing_frames) / 30.0
224
-
225
  if downswing_frames:
226
  # Assuming 30 fps video
227
  downswing_duration = len(downswing_frames) / 30.0
228
-
229
  # Calculate tempo ratio if both durations are available
230
  tempo_ratio = None
231
  if backswing_duration and downswing_duration and downswing_duration > 0:
@@ -241,32 +332,34 @@ def prepare_data_for_llm(pose_data, swing_phases, trajectory_data):
241
  "hip_rotation": 45, # degrees
242
  "shoulder_rotation": 90, # degrees
243
  "posture_score": 0.8, # 0-1 scale
244
-
245
  # Upper body mechanics
246
  "arm_extension": 0.8, # 0-1 scale
247
  "wrist_hinge": 80, # degrees
248
  "chest_rotation_efficiency": 0.75, # 0-1 scale
249
  "head_movement_lateral": 2.5, # inches
250
  "head_movement_vertical": 1.8, # inches
251
-
252
  # Lower body mechanics
253
  "knee_flexion_address": 25, # degrees
254
  "knee_flexion_impact": 30, # degrees
255
  "hip_thrust": 0.6, # 0-1 scale
256
  "ground_force_efficiency": 0.7, # 0-1 scale
257
-
258
  # Club path and face metrics
259
- "swing_path": 2.5, # degrees (positive = out-to-in, negative = in-to-out)
 
260
  "clubface_angle": 2.1, # degrees (positive = open, negative = closed)
261
- "attack_angle": -4.2, # degrees (negative = descending, positive = ascending)
 
262
  "club_path_consistency": 0.78, # 0-1 scale
263
-
264
  # Tempo and timing metrics
265
  "transition_smoothness": 0.75, # 0-1 scale
266
  "backswing_duration": backswing_duration or 0.9, # seconds
267
  "downswing_duration": downswing_duration or 0.3, # seconds
268
  "kinematic_sequence": 0.82, # 0-1 scale
269
-
270
  # Efficiency and power metrics
271
  "energy_transfer": 0.78, # 0-1 scale
272
  "potential_distance": 240, # yards
@@ -299,62 +392,89 @@ I've analyzed a golf swing and extracted the following data:
299
 
300
  # Add trajectory information
301
  prompt += "\n## Trajectory Data\n"
302
- if "trajectory" in analysis_data and "club_speed_mph" in analysis_data["trajectory"]:
 
303
  prompt += f"- Club Speed: {analysis_data['trajectory']['club_speed_mph']:.1f} mph\n"
304
-
305
  # Add detailed biomechanical metrics
306
  prompt += "\n## Swing Mechanics\n"
307
-
308
  # Core body mechanics
309
  prompt += "\n### Body Mechanics\n"
310
- prompt += "- Tempo Ratio (Backswing:Downswing): {:.1f}\n".format(analysis_data["metrics"].get("tempo_ratio", 0))
311
- prompt += "- Hip Rotation (degrees): {}\n".format(analysis_data["metrics"].get("hip_rotation", 0))
312
- prompt += "- Shoulder Rotation (degrees): {}\n".format(analysis_data["metrics"].get("shoulder_rotation", 0))
313
- prompt += "- Posture Score: {}%\n".format(int(analysis_data["metrics"].get("posture_score", 0) * 100))
314
-
 
 
 
 
315
  # Upper body mechanics
316
  prompt += "\n### Upper Body Mechanics\n"
317
- prompt += "- Arm Extension (impact): {}%\n".format(int(analysis_data["metrics"].get("arm_extension", 0.8) * 100))
318
- prompt += "- Wrist Hinge (degrees): {}\n".format(analysis_data["metrics"].get("wrist_hinge", 0))
319
- prompt += "- Shoulder Plane Consistency: {}%\n".format(int(analysis_data["metrics"].get("swing_plane_consistency", 0) * 100))
320
- prompt += "- Chest Rotation Efficiency: {}%\n".format(int(analysis_data["metrics"].get("chest_rotation_efficiency", 0.75) * 100))
321
- prompt += "- Head Movement (lateral): {}in\n".format(analysis_data["metrics"].get("head_movement_lateral", 2.5))
322
- prompt += "- Head Movement (vertical): {}in\n".format(analysis_data["metrics"].get("head_movement_vertical", 1.8))
323
-
 
 
 
 
 
 
 
324
  # Lower body mechanics
325
  prompt += "\n### Lower Body Mechanics\n"
326
- prompt += "- Weight Shift (lead foot at impact): {}%\n".format(int(analysis_data["metrics"].get("weight_shift", 0) * 100))
327
- prompt += "- Knee Flexion (address): {}°\n".format(analysis_data["metrics"].get("knee_flexion_address", 25))
328
- prompt += "- Knee Flexion (impact): {}°\n".format(analysis_data["metrics"].get("knee_flexion_impact", 30))
329
- prompt += "- Hip Thrust (impact): {}%\n".format(int(analysis_data["metrics"].get("hip_thrust", 0.6) * 100))
330
- prompt += "- Ground Force Efficiency: {}%\n".format(int(analysis_data["metrics"].get("ground_force_efficiency", 0.7) * 100))
331
-
 
 
 
 
 
 
332
  # Swing path and clubface metrics
333
  prompt += "\n### Club Path & Face Metrics\n"
334
  prompt += "- Swing Path (degrees): {} ({})\n".format(
335
- analysis_data["metrics"].get("swing_path", 2.5),
336
- "Out-to-In" if analysis_data["metrics"].get("swing_path", 0) > 0 else "In-to-Out")
337
  prompt += "- Clubface Angle (degrees): {} ({})\n".format(
338
- analysis_data["metrics"].get("clubface_angle", 2.1),
339
- "Open" if analysis_data["metrics"].get("clubface_angle", 0) > 0 else "Closed")
340
  prompt += "- Attack Angle (degrees): {} ({})\n".format(
341
- analysis_data["metrics"].get("attack_angle", -4.2),
342
- "Descending" if analysis_data["metrics"].get("attack_angle", 0) < 0 else "Ascending")
343
- prompt += "- Club Path Consistency: {}%\n".format(int(analysis_data["metrics"].get("club_path_consistency", 0.78) * 100))
344
-
 
345
  # Tempo and timing metrics
346
  prompt += "\n### Tempo & Timing\n"
347
- prompt += "- Transition Smoothness: {}%\n".format(int(analysis_data["metrics"].get("transition_smoothness", 0.75) * 100))
348
- prompt += "- Backswing Duration: {} seconds\n".format(analysis_data["metrics"].get("backswing_duration", 0.9))
349
- prompt += "- Downswing Duration: {} seconds\n".format(analysis_data["metrics"].get("downswing_duration", 0.3))
350
- prompt += "- Sequential Kinematic Sequence: {}%\n".format(int(analysis_data["metrics"].get("kinematic_sequence", 0.82) * 100))
351
-
 
 
 
 
352
  # Efficiency and power metrics
353
  prompt += "\n### Efficiency & Power Metrics\n"
354
- prompt += "- Energy Transfer Efficiency: {}%\n".format(int(analysis_data["metrics"].get("energy_transfer", 0.78) * 100))
355
- prompt += "- Potential Distance: {} yards\n".format(analysis_data["metrics"].get("potential_distance", 240))
356
- prompt += "- Power Accumulation: {}%\n".format(int(analysis_data["metrics"].get("power_accumulation", 0.75) * 100))
357
- prompt += "- Speed Generation Method: {}\n".format(analysis_data["metrics"].get("speed_generation", "Arms-dominant"))
 
 
 
 
358
 
359
  prompt += """
360
 
 
8
  import streamlit as st
9
 
10
 
11
+ def check_llm_services():
12
+ """
13
+ Check which LLM services are configured
14
+
15
+ Returns:
16
+ dict: Dictionary with service availability and configuration
17
+ """
18
+ services = {
19
+ 'ollama': {
20
+ 'available': False,
21
+ 'config': {}
22
+ },
23
+ 'openai': {
24
+ 'available': False,
25
+ 'config': {}
26
+ }
27
+ }
28
+
29
+ # Check Ollama configuration
30
+ try:
31
+ ollama_url = st.secrets.get("ollama", {}).get("base_url", "")
32
+ ollama_model = st.secrets.get("ollama", {}).get("model", "")
33
+ if ollama_url and ollama_model:
34
+ services['ollama']['available'] = True
35
+ services['ollama']['config'] = {
36
+ 'base_url': ollama_url,
37
+ 'model': ollama_model
38
+ }
39
+ except (KeyError, FileNotFoundError, AttributeError):
40
+ pass
41
+
42
+ # Check OpenAI configuration
43
+ try:
44
+ openai_key = st.secrets.get("openai", {}).get("api_key", "")
45
+ if openai_key:
46
+ services['openai']['available'] = True
47
+ services['openai']['config'] = {'api_key': openai_key}
48
+ except (KeyError, FileNotFoundError, AttributeError):
49
+ pass
50
+
51
+ return services
52
+
53
+
54
  def generate_swing_analysis(pose_data, swing_phases, trajectory_data):
55
  """
56
  Generate swing analysis and coaching tips using LLM
 
63
  Returns:
64
  str: Detailed swing analysis and coaching tips
65
  """
66
+ # Check available services
67
+ services = check_llm_services()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
 
69
+ # If no services are available, return sample analysis
70
+ if not services['ollama']['available'] and not services['openai'][
71
+ 'available']:
72
+ return get_sample_analysis()
73
 
74
+ # Prepare data for LLM
75
+ analysis_data = prepare_data_for_llm(pose_data, swing_phases,
76
+ trajectory_data)
77
+ prompt = create_llm_prompt(analysis_data)
78
 
79
+ # Try Ollama first if available
80
+ if services['ollama']['available']:
81
+ try:
82
+ analysis = call_ollama_service(prompt,
83
+ services['ollama']['config'])
84
+ if analysis:
85
+ return analysis
86
+ except Exception as e:
87
+ print(f"Error with Ollama: {str(e)}. Falling back to OpenAI...")
88
 
89
+ # Try OpenAI if available
90
+ if services['openai']['available']:
91
+ try:
92
+ analysis = call_openai_service(prompt,
93
+ services['openai']['config'])
94
+ if analysis:
95
+ return analysis
96
+ except Exception as e:
97
+ print(
98
+ f"Error with OpenAI: {str(e)}. Using sample analysis instead.")
99
 
100
+ # If both services failed, return sample analysis
101
+ return get_sample_analysis()
102
 
 
103
 
104
+ def call_ollama_service(prompt, config):
105
+ """
106
+ Call Ollama service for analysis
107
+
108
+ Args:
109
+ prompt (str): The analysis prompt
110
+ config (dict): Ollama configuration
111
+
112
+ Returns:
113
+ str: Analysis result or None if failed
114
+ """
115
+ try:
116
+ # Create a custom httpx client
117
+ http_client = httpx.Client()
118
 
119
+ # Initialize OpenAI client with Ollama endpoint
120
+ client = OpenAI(
121
+ base_url=config['base_url'],
122
+ api_key="ollama", # Ollama doesn't need a real API key
123
+ http_client=http_client)
124
+
125
+ response = client.chat.completions.create(
126
+ model=config['model'],
127
+ messages=[{
128
+ "role":
129
+ "system",
130
+ "content":
131
+ "You are a professional golf coach with expertise in analyzing golf swings. Provide detailed, actionable feedback based on the swing data provided."
132
+ }, {
133
+ "role": "user",
134
+ "content": prompt
135
+ }],
136
+ temperature=0.7,
137
+ max_tokens=1000)
138
+
139
+ return response.choices[0].message.content
140
 
141
+ except Exception as e:
142
+ print(f"Ollama service error: {str(e)}")
143
+ return None
144
 
 
 
145
 
146
+ def call_openai_service(prompt, config):
147
+ """
148
+ Call OpenAI service for analysis
149
+
150
+ Args:
151
+ prompt (str): The analysis prompt
152
+ config (dict): OpenAI configuration
153
+
154
+ Returns:
155
+ str: Analysis result or None if failed
156
+ """
157
  try:
158
  # Create a custom httpx client without proxies
159
  http_client = httpx.Client()
160
+
161
+ # Initialize the OpenAI client
162
+ client = OpenAI(api_key=config['api_key'], http_client=http_client)
163
+
 
 
 
 
164
  try:
165
  # Try with GPT-4 first
166
  response = client.chat.completions.create(
167
  model="gpt-4-turbo",
168
+ messages=[{
169
+ "role":
170
+ "system",
171
+ "content":
172
+ "You are a professional golf coach with expertise in analyzing golf swings. Provide detailed, actionable feedback based on the swing data provided."
173
+ }, {
174
+ "role": "user",
175
+ "content": prompt
176
+ }],
177
  temperature=0.7,
178
+ max_tokens=1000)
179
+
180
+ return response.choices[0].message.content
181
+
 
 
 
182
  except Exception as gpt4_error:
183
  # If there's an error with GPT-4 (like quota exceeded), try GPT-3.5
184
+ print(
185
+ f"Error with GPT-4: {str(gpt4_error)}. Falling back to GPT-3.5-turbo..."
186
+ )
187
+
188
  try:
189
  response = client.chat.completions.create(
190
  model="gpt-3.5-turbo",
191
+ messages=[{
192
+ "role":
193
+ "system",
194
+ "content":
195
+ "You are a professional golf coach with expertise in analyzing golf swings. Provide detailed, actionable feedback based on the swing data provided."
196
+ }, {
197
+ "role": "user",
198
+ "content": prompt
199
+ }],
200
  temperature=0.7,
201
+ max_tokens=1000)
202
+
203
+ return response.choices[0].message.content
204
+
 
 
 
205
  except Exception as gpt35_error:
206
+ print(f"Error with GPT-3.5: {str(gpt35_error)}")
207
+ return None
208
+
209
+ except Exception as e:
210
+ print(f"OpenAI service error: {str(e)}")
211
+ return None
212
+
213
+
214
+ def get_sample_analysis():
215
+ """
216
+ Return sample analysis when no LLM services are available
217
+
218
+ Returns:
219
+ str: Sample swing analysis
220
+ """
221
+ return """
222
  ## Swing Analysis Summary
223
 
224
  Based on the video analysis, here are some observations about your swing:
 
261
  These adjustments should help improve both consistency and distance in your swing.
262
  """
263
 
 
 
 
264
 
265
  def prepare_data_for_llm(pose_data, swing_phases, trajectory_data):
266
  """
 
305
  # Calculate backswing and downswing durations if available
306
  backswing_frames = swing_phases.get("backswing", [])
307
  downswing_frames = swing_phases.get("downswing", [])
308
+
309
  backswing_duration = None
310
  downswing_duration = None
311
+
312
  if backswing_frames:
313
  # Assuming 30 fps video
314
  backswing_duration = len(backswing_frames) / 30.0
315
+
316
  if downswing_frames:
317
  # Assuming 30 fps video
318
  downswing_duration = len(downswing_frames) / 30.0
319
+
320
  # Calculate tempo ratio if both durations are available
321
  tempo_ratio = None
322
  if backswing_duration and downswing_duration and downswing_duration > 0:
 
332
  "hip_rotation": 45, # degrees
333
  "shoulder_rotation": 90, # degrees
334
  "posture_score": 0.8, # 0-1 scale
335
+
336
  # Upper body mechanics
337
  "arm_extension": 0.8, # 0-1 scale
338
  "wrist_hinge": 80, # degrees
339
  "chest_rotation_efficiency": 0.75, # 0-1 scale
340
  "head_movement_lateral": 2.5, # inches
341
  "head_movement_vertical": 1.8, # inches
342
+
343
  # Lower body mechanics
344
  "knee_flexion_address": 25, # degrees
345
  "knee_flexion_impact": 30, # degrees
346
  "hip_thrust": 0.6, # 0-1 scale
347
  "ground_force_efficiency": 0.7, # 0-1 scale
348
+
349
  # Club path and face metrics
350
+ "swing_path":
351
+ 2.5, # degrees (positive = out-to-in, negative = in-to-out)
352
  "clubface_angle": 2.1, # degrees (positive = open, negative = closed)
353
+ "attack_angle":
354
+ -4.2, # degrees (negative = descending, positive = ascending)
355
  "club_path_consistency": 0.78, # 0-1 scale
356
+
357
  # Tempo and timing metrics
358
  "transition_smoothness": 0.75, # 0-1 scale
359
  "backswing_duration": backswing_duration or 0.9, # seconds
360
  "downswing_duration": downswing_duration or 0.3, # seconds
361
  "kinematic_sequence": 0.82, # 0-1 scale
362
+
363
  # Efficiency and power metrics
364
  "energy_transfer": 0.78, # 0-1 scale
365
  "potential_distance": 240, # yards
 
392
 
393
  # Add trajectory information
394
  prompt += "\n## Trajectory Data\n"
395
+ if "trajectory" in analysis_data and "club_speed_mph" in analysis_data[
396
+ "trajectory"]:
397
  prompt += f"- Club Speed: {analysis_data['trajectory']['club_speed_mph']:.1f} mph\n"
398
+
399
  # Add detailed biomechanical metrics
400
  prompt += "\n## Swing Mechanics\n"
401
+
402
  # Core body mechanics
403
  prompt += "\n### Body Mechanics\n"
404
+ prompt += "- Tempo Ratio (Backswing:Downswing): {:.1f}\n".format(
405
+ analysis_data["metrics"].get("tempo_ratio", 0))
406
+ prompt += "- Hip Rotation (degrees): {}\n".format(
407
+ analysis_data["metrics"].get("hip_rotation", 0))
408
+ prompt += "- Shoulder Rotation (degrees): {}\n".format(
409
+ analysis_data["metrics"].get("shoulder_rotation", 0))
410
+ prompt += "- Posture Score: {}%\n".format(
411
+ int(analysis_data["metrics"].get("posture_score", 0) * 100))
412
+
413
  # Upper body mechanics
414
  prompt += "\n### Upper Body Mechanics\n"
415
+ prompt += "- Arm Extension (impact): {}%\n".format(
416
+ int(analysis_data["metrics"].get("arm_extension", 0.8) * 100))
417
+ prompt += "- Wrist Hinge (degrees): {}\n".format(
418
+ analysis_data["metrics"].get("wrist_hinge", 0))
419
+ prompt += "- Shoulder Plane Consistency: {}%\n".format(
420
+ int(analysis_data["metrics"].get("swing_plane_consistency", 0) * 100))
421
+ prompt += "- Chest Rotation Efficiency: {}%\n".format(
422
+ int(analysis_data["metrics"].get("chest_rotation_efficiency", 0.75) *
423
+ 100))
424
+ prompt += "- Head Movement (lateral): {}in\n".format(
425
+ analysis_data["metrics"].get("head_movement_lateral", 2.5))
426
+ prompt += "- Head Movement (vertical): {}in\n".format(
427
+ analysis_data["metrics"].get("head_movement_vertical", 1.8))
428
+
429
  # Lower body mechanics
430
  prompt += "\n### Lower Body Mechanics\n"
431
+ prompt += "- Weight Shift (lead foot at impact): {}%\n".format(
432
+ int(analysis_data["metrics"].get("weight_shift", 0) * 100))
433
+ prompt += "- Knee Flexion (address): {}°\n".format(
434
+ analysis_data["metrics"].get("knee_flexion_address", 25))
435
+ prompt += "- Knee Flexion (impact): {}°\n".format(
436
+ analysis_data["metrics"].get("knee_flexion_impact", 30))
437
+ prompt += "- Hip Thrust (impact): {}%\n".format(
438
+ int(analysis_data["metrics"].get("hip_thrust", 0.6) * 100))
439
+ prompt += "- Ground Force Efficiency: {}%\n".format(
440
+ int(analysis_data["metrics"].get("ground_force_efficiency", 0.7) *
441
+ 100))
442
+
443
  # Swing path and clubface metrics
444
  prompt += "\n### Club Path & Face Metrics\n"
445
  prompt += "- Swing Path (degrees): {} ({})\n".format(
446
+ analysis_data["metrics"].get("swing_path", 2.5), "Out-to-In"
447
+ if analysis_data["metrics"].get("swing_path", 0) > 0 else "In-to-Out")
448
  prompt += "- Clubface Angle (degrees): {} ({})\n".format(
449
+ analysis_data["metrics"].get("clubface_angle", 2.1), "Open"
450
+ if analysis_data["metrics"].get("clubface_angle", 0) > 0 else "Closed")
451
  prompt += "- Attack Angle (degrees): {} ({})\n".format(
452
+ analysis_data["metrics"].get("attack_angle", -4.2), "Descending" if
453
+ analysis_data["metrics"].get("attack_angle", 0) < 0 else "Ascending")
454
+ prompt += "- Club Path Consistency: {}%\n".format(
455
+ int(analysis_data["metrics"].get("club_path_consistency", 0.78) * 100))
456
+
457
  # Tempo and timing metrics
458
  prompt += "\n### Tempo & Timing\n"
459
+ prompt += "- Transition Smoothness: {}%\n".format(
460
+ int(analysis_data["metrics"].get("transition_smoothness", 0.75) * 100))
461
+ prompt += "- Backswing Duration: {} seconds\n".format(
462
+ analysis_data["metrics"].get("backswing_duration", 0.9))
463
+ prompt += "- Downswing Duration: {} seconds\n".format(
464
+ analysis_data["metrics"].get("downswing_duration", 0.3))
465
+ prompt += "- Sequential Kinematic Sequence: {}%\n".format(
466
+ int(analysis_data["metrics"].get("kinematic_sequence", 0.82) * 100))
467
+
468
  # Efficiency and power metrics
469
  prompt += "\n### Efficiency & Power Metrics\n"
470
+ prompt += "- Energy Transfer Efficiency: {}%\n".format(
471
+ int(analysis_data["metrics"].get("energy_transfer", 0.78) * 100))
472
+ prompt += "- Potential Distance: {} yards\n".format(
473
+ analysis_data["metrics"].get("potential_distance", 240))
474
+ prompt += "- Power Accumulation: {}%\n".format(
475
+ int(analysis_data["metrics"].get("power_accumulation", 0.75) * 100))
476
+ prompt += "- Speed Generation Method: {}\n".format(
477
+ analysis_data["metrics"].get("speed_generation", "Arms-dominant"))
478
 
479
  prompt += """
480
 
app/models/pose_estimator.py CHANGED
@@ -15,7 +15,7 @@ class PoseEstimator:
15
  """Initialize the pose estimator"""
16
  self.mp_pose = mp.solutions.pose
17
  self.pose = self.mp_pose.Pose(static_image_mode=False,
18
- model_complexity=2,
19
  enable_segmentation=False,
20
  min_detection_confidence=0.5,
21
  min_tracking_confidence=0.5)
 
15
  """Initialize the pose estimator"""
16
  self.mp_pose = mp.solutions.pose
17
  self.pose = self.mp_pose.Pose(static_image_mode=False,
18
+ model_complexity=1,
19
  enable_segmentation=False,
20
  min_detection_confidence=0.5,
21
  min_tracking_confidence=0.5)
app/streamlit_app.py CHANGED
@@ -21,7 +21,7 @@ from app.utils.video_downloader import download_youtube_video
21
  from app.utils.video_processor import process_video
22
  from app.models.pose_estimator import analyze_pose
23
  from app.models.swing_analyzer import segment_swing, analyze_trajectory
24
- from app.models.llm_analyzer import generate_swing_analysis, create_llm_prompt, prepare_data_for_llm
25
  from app.utils.visualizer import create_annotated_video
26
 
27
  # Set page config
@@ -59,8 +59,7 @@ def display_video(video_path, width=300):
59
  # Create a container with custom width
60
  video_container = st.container()
61
  # Apply CSS to control the width and ensure it's centered
62
- video_container.markdown(
63
- f"""
64
  <style>
65
  .element-container:has(video) {{
66
  max-width: {width}px;
@@ -71,10 +70,9 @@ def display_video(video_path, width=300):
71
  height: auto !important;
72
  }}
73
  </style>
74
- """,
75
- unsafe_allow_html=True
76
- )
77
-
78
  # Display video using st.video with bytes
79
  with video_container:
80
  st.video(video_bytes)
@@ -109,37 +107,56 @@ def main():
109
  # Sidebar for configuration
110
  st.sidebar.title("Configuration")
111
 
 
 
 
 
 
112
  # Option to enable/disable GPT analysis with better explanation
113
- st.sidebar.markdown("### GPT Analysis Settings")
114
- enable_gpt = st.sidebar.checkbox(
115
- "Enable GPT Analysis",
116
- value=False, # Disabled by default
117
- help="When enabled, uses OpenAI's API for personalized analysis. Requires API key."
118
- )
119
-
120
- if enable_gpt:
121
- # Check for OpenAI API key in Streamlit secrets
122
- api_key_available = False
123
- try:
124
- if st.secrets["openai"]["api_key"]:
125
- api_key_available = True
126
- st.sidebar.success("✅ OpenAI API key configured in Streamlit secrets")
127
- except (KeyError, FileNotFoundError):
128
- # Fallback to environment variable
129
- api_key = os.getenv("OPENAI_API_KEY")
130
- if api_key:
131
- api_key_available = True
132
- st.sidebar.success("✅ OpenAI API key configured in environment variables")
133
-
134
- if not api_key_available:
135
- st.sidebar.warning(
136
- "⚠️ OpenAI API key not found. Add it to your .streamlit/secrets.toml file."
137
- )
138
  else:
139
- st.sidebar.info(
140
- "Using sample analysis mode (no API key required)"
 
 
141
  )
142
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
143
  # Frame skip rate for YOLO
144
  sample_rate = st.sidebar.slider(
145
  "Frame Skip Rate (YOLO)",
@@ -238,16 +255,20 @@ def main():
238
  'analysis_data': analysis_data,
239
  'prompt': prompt
240
  }
241
-
242
  # Present the two options after analysis
243
  st.subheader("What would you like to do next?")
244
  options_col1, options_col2 = st.columns(2)
245
-
246
  with options_col1:
247
- st.info("**Option 1: Generate Annotated Video**\n\nCreate a video with visual feedback showing your swing phases, body positioning, and key metrics.")
248
-
 
 
249
  with options_col2:
250
- st.info("**Option 2: Generate Improvement Recommendations**\n\nGet AI-powered analysis of your swing with specific tips for improvement.")
 
 
251
 
252
  except Exception as e:
253
  st.error(f"Error during analysis: {str(e)}")
@@ -264,7 +285,7 @@ def main():
264
  with phase_cols[i]:
265
  st.metric(label=phase.capitalize(),
266
  value=f"{len(frames_in_phase)} frames")
267
-
268
  # Display club speed if available
269
  if 'trajectory_data' in st.session_state.analysis_data and 'swing_phases' in st.session_state.analysis_data:
270
  trajectory_data = st.session_state.analysis_data['trajectory_data']
@@ -272,27 +293,34 @@ def main():
272
  impact_frames = swing_phases.get("impact", [])
273
  if impact_frames:
274
  impact_frame = impact_frames[len(impact_frames) // 2]
275
- if impact_frame in trajectory_data and trajectory_data[impact_frame].get("club_speed"):
 
276
  st.subheader("Club Speed")
277
  st.metric(
278
  label="Estimated Club Speed",
279
- value=f"{trajectory_data[impact_frame]['club_speed']:.1f} mph"
 
280
  )
281
-
282
  # Display the GPT prompt in an expander
283
  if 'prompt' in st.session_state.analysis_data:
284
- with st.expander("View GPT Prompt", expanded=False):
285
- st.code(st.session_state.analysis_data['prompt'], language="text")
286
-
 
287
  # Create columns for the two action buttons
288
  button_col1, button_col2 = st.columns(2)
289
-
290
  with button_col1:
291
- annotated_video_clicked = st.button("Generate Annotated Video", key="create_annotated", use_container_width=True)
292
-
 
 
293
  with button_col2:
294
- improvements_clicked = st.button("Generate Improvements", key="gpt_recommendations", use_container_width=True)
295
-
 
 
296
  # Handle annotated video creation
297
  if annotated_video_clicked:
298
  try:
@@ -320,68 +348,79 @@ def main():
320
 
321
  # Store the annotated video path in session state
322
  st.session_state.annotated_video_path = output_path
323
-
324
  # Display success message and video after spinner completes
325
  st.success("Annotated video created successfully!")
326
  display_video(output_path, width=400)
327
-
328
  # Show download button
329
  with open(output_path, "rb") as file:
330
  video_bytes = file.read()
331
- st.download_button(
332
- label="Download Annotated Video",
333
- data=video_bytes,
334
- file_name=os.path.basename(output_path),
335
- mime="video/mp4"
336
- )
337
 
338
  except Exception as e:
339
  st.error(f"Error creating annotated video: {str(e)}")
340
  st.error(
341
  "Please check if the downloads directory exists and is writable"
342
  )
343
-
344
  # Handle improvement recommendations generation
345
  if improvements_clicked:
346
- with st.spinner("Analyzing your swing and generating recommendations..."):
 
347
  # Get data from session state
348
  data = st.session_state.analysis_data
349
  pose_data = data['pose_data']
350
  swing_phases = data['swing_phases']
351
  trajectory_data = data['trajectory_data']
352
-
353
  # Generate detailed analysis with recommendations
354
- analysis = generate_swing_analysis(pose_data, swing_phases, trajectory_data)
355
-
 
356
  # Display the analysis
357
  st.subheader("Swing Analysis and Recommendations")
358
-
359
- # Check if we're using the sample analysis (no API key)
360
- api_key_available = False
361
- try:
362
- if st.secrets["openai"]["api_key"]:
363
- api_key_available = True
364
- except (KeyError, FileNotFoundError):
365
- api_key = os.getenv("OPENAI_API_KEY")
366
- if api_key:
367
- api_key_available = True
368
-
369
- if not api_key_available and not enable_gpt:
370
- st.info("ℹ️ **Using sample analysis mode**. The recommendations below are general examples and not personalized to your specific swing.")
371
-
 
 
 
 
 
 
 
372
  st.markdown(analysis)
373
-
374
  # Add some example drills based on the analysis
375
  if "Error:" not in analysis: # Only show drills if analysis was successful
376
  st.subheader("Recommended Drills")
377
  drill1, drill2 = st.columns(2)
378
-
379
  with drill1:
380
  st.markdown("**Posture Drill**")
381
  st.markdown("- Stand with your back against a wall")
382
- st.markdown("- Take your golf stance while maintaining contact")
383
- st.markdown("- Practice maintaining this posture during your swing")
384
-
 
 
 
 
385
  with drill2:
386
  st.markdown("**Tempo Drill**")
387
  st.markdown("- Count '1-2-3' for your backswing")
 
21
  from app.utils.video_processor import process_video
22
  from app.models.pose_estimator import analyze_pose
23
  from app.models.swing_analyzer import segment_swing, analyze_trajectory
24
+ from app.models.llm_analyzer import generate_swing_analysis, create_llm_prompt, prepare_data_for_llm, check_llm_services
25
  from app.utils.visualizer import create_annotated_video
26
 
27
  # Set page config
 
59
  # Create a container with custom width
60
  video_container = st.container()
61
  # Apply CSS to control the width and ensure it's centered
62
+ video_container.markdown(f"""
 
63
  <style>
64
  .element-container:has(video) {{
65
  max-width: {width}px;
 
70
  height: auto !important;
71
  }}
72
  </style>
73
+ """,
74
+ unsafe_allow_html=True)
75
+
 
76
  # Display video using st.video with bytes
77
  with video_container:
78
  st.video(video_bytes)
 
107
  # Sidebar for configuration
108
  st.sidebar.title("Configuration")
109
 
110
+ # Check available LLM services
111
+ llm_services = check_llm_services()
112
+ any_service_available = llm_services['ollama'][
113
+ 'available'] or llm_services['openai']['available']
114
+
115
  # Option to enable/disable GPT analysis with better explanation
116
+ st.sidebar.markdown("### LLM Analysis Settings")
117
+
118
+ # Show service status
119
+ if llm_services['ollama']['available']:
120
+ st.sidebar.success(
121
+ f"✅ Ollama configured: {llm_services['ollama']['config']['model']}"
122
+ )
123
+
124
+ if llm_services['openai']['available']:
125
+ st.sidebar.success("✅ OpenAI configured")
126
+
127
+ if not any_service_available:
128
+ st.sidebar.info("ℹ️ No LLM services configured")
129
+
130
+ # Automatically enable if services are available, otherwise allow manual control
131
+ if any_service_available:
132
+ enable_gpt = st.sidebar.checkbox(
133
+ "Enable LLM Analysis",
134
+ value=True, # Automatically enabled when services are available
135
+ help=
136
+ "Uses configured LLM services (Ollama/OpenAI) for personalized analysis."
137
+ )
 
 
 
138
  else:
139
+ enable_gpt = st.sidebar.checkbox(
140
+ "Enable LLM Analysis",
141
+ value=False, # Disabled by default when no services
142
+ help="Configure Ollama or OpenAI in secrets to enable LLM analysis."
143
  )
144
 
145
+ if enable_gpt and not any_service_available:
146
+ st.sidebar.warning(
147
+ "⚠️ No LLM services configured. Configure Ollama or OpenAI in your .streamlit/secrets.toml file."
148
+ )
149
+ elif enable_gpt:
150
+ if llm_services['ollama']['available'] and llm_services['openai'][
151
+ 'available']:
152
+ st.sidebar.info("🔄 Will try Ollama first, then OpenAI as fallback")
153
+ elif llm_services['ollama']['available']:
154
+ st.sidebar.info("🦙 Using Ollama for analysis")
155
+ elif llm_services['openai']['available']:
156
+ st.sidebar.info("🤖 Using OpenAI for analysis")
157
+ else:
158
+ st.sidebar.info("Using sample analysis mode (no LLM required)")
159
+
160
  # Frame skip rate for YOLO
161
  sample_rate = st.sidebar.slider(
162
  "Frame Skip Rate (YOLO)",
 
255
  'analysis_data': analysis_data,
256
  'prompt': prompt
257
  }
258
+
259
  # Present the two options after analysis
260
  st.subheader("What would you like to do next?")
261
  options_col1, options_col2 = st.columns(2)
262
+
263
  with options_col1:
264
+ st.info(
265
+ "**Option 1: Generate Annotated Video**\n\nCreate a video with visual feedback showing your swing phases, body positioning, and key metrics."
266
+ )
267
+
268
  with options_col2:
269
+ st.info(
270
+ "**Option 2: Generate Improvement Recommendations**\n\nGet AI-powered analysis of your swing with specific tips for improvement."
271
+ )
272
 
273
  except Exception as e:
274
  st.error(f"Error during analysis: {str(e)}")
 
285
  with phase_cols[i]:
286
  st.metric(label=phase.capitalize(),
287
  value=f"{len(frames_in_phase)} frames")
288
+
289
  # Display club speed if available
290
  if 'trajectory_data' in st.session_state.analysis_data and 'swing_phases' in st.session_state.analysis_data:
291
  trajectory_data = st.session_state.analysis_data['trajectory_data']
 
293
  impact_frames = swing_phases.get("impact", [])
294
  if impact_frames:
295
  impact_frame = impact_frames[len(impact_frames) // 2]
296
+ if impact_frame in trajectory_data and trajectory_data[
297
+ impact_frame].get("club_speed"):
298
  st.subheader("Club Speed")
299
  st.metric(
300
  label="Estimated Club Speed",
301
+ value=
302
+ f"{trajectory_data[impact_frame]['club_speed']:.1f} mph"
303
  )
304
+
305
  # Display the GPT prompt in an expander
306
  if 'prompt' in st.session_state.analysis_data:
307
+ with st.expander("View LLM Prompt", expanded=False):
308
+ st.code(st.session_state.analysis_data['prompt'],
309
+ language="text")
310
+
311
  # Create columns for the two action buttons
312
  button_col1, button_col2 = st.columns(2)
313
+
314
  with button_col1:
315
+ annotated_video_clicked = st.button("Generate Annotated Video",
316
+ key="create_annotated",
317
+ use_container_width=True)
318
+
319
  with button_col2:
320
+ improvements_clicked = st.button("Generate Improvements",
321
+ key="gpt_recommendations",
322
+ use_container_width=True)
323
+
324
  # Handle annotated video creation
325
  if annotated_video_clicked:
326
  try:
 
348
 
349
  # Store the annotated video path in session state
350
  st.session_state.annotated_video_path = output_path
351
+
352
  # Display success message and video after spinner completes
353
  st.success("Annotated video created successfully!")
354
  display_video(output_path, width=400)
355
+
356
  # Show download button
357
  with open(output_path, "rb") as file:
358
  video_bytes = file.read()
359
+ st.download_button(label="Download Annotated Video",
360
+ data=video_bytes,
361
+ file_name=os.path.basename(output_path),
362
+ mime="video/mp4")
 
 
363
 
364
  except Exception as e:
365
  st.error(f"Error creating annotated video: {str(e)}")
366
  st.error(
367
  "Please check if the downloads directory exists and is writable"
368
  )
369
+
370
  # Handle improvement recommendations generation
371
  if improvements_clicked:
372
+ with st.spinner(
373
+ "Analyzing your swing and generating recommendations..."):
374
  # Get data from session state
375
  data = st.session_state.analysis_data
376
  pose_data = data['pose_data']
377
  swing_phases = data['swing_phases']
378
  trajectory_data = data['trajectory_data']
379
+
380
  # Generate detailed analysis with recommendations
381
+ analysis = generate_swing_analysis(pose_data, swing_phases,
382
+ trajectory_data)
383
+
384
  # Display the analysis
385
  st.subheader("Swing Analysis and Recommendations")
386
+
387
+ # Check available services to show appropriate message
388
+ llm_services = check_llm_services()
389
+ any_service_available = llm_services['ollama'][
390
+ 'available'] or llm_services['openai']['available']
391
+
392
+ if not any_service_available or not enable_gpt:
393
+ st.info(
394
+ "ℹ️ **Using sample analysis mode**. The recommendations below are general examples and not personalized to your specific swing."
395
+ )
396
+ else:
397
+ if llm_services['ollama']['available'] and llm_services[
398
+ 'openai']['available']:
399
+ st.info(
400
+ "🔄 **Analysis generated using available LLM services** (tried Ollama first, OpenAI as fallback)"
401
+ )
402
+ elif llm_services['ollama']['available']:
403
+ st.info("🦙 **Analysis generated using Ollama**")
404
+ elif llm_services['openai']['available']:
405
+ st.info("🤖 **Analysis generated using OpenAI**")
406
+
407
  st.markdown(analysis)
408
+
409
  # Add some example drills based on the analysis
410
  if "Error:" not in analysis: # Only show drills if analysis was successful
411
  st.subheader("Recommended Drills")
412
  drill1, drill2 = st.columns(2)
413
+
414
  with drill1:
415
  st.markdown("**Posture Drill**")
416
  st.markdown("- Stand with your back against a wall")
417
+ st.markdown(
418
+ "- Take your golf stance while maintaining contact"
419
+ )
420
+ st.markdown(
421
+ "- Practice maintaining this posture during your swing"
422
+ )
423
+
424
  with drill2:
425
  st.markdown("**Tempo Drill**")
426
  st.markdown("- Count '1-2-3' for your backswing")