Upload 8 files
Browse files
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
emoji: 🎤
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: purple
|
|
@@ -9,17 +9,31 @@ app_file: app.py
|
|
| 9 |
pinned: false
|
| 10 |
---
|
| 11 |
|
| 12 |
-
#
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
## Features
|
| 17 |
|
| 18 |
-
|
| 19 |
-
- Natural conversation with
|
| 20 |
-
-
|
| 21 |
-
-
|
| 22 |
-
- Full conversation history
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## Setup
|
| 25 |
|
|
@@ -59,18 +73,50 @@ python app.py
|
|
| 59 |
|
| 60 |
## How to Use
|
| 61 |
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
|
|
|
| 66 |
5. Continue the conversation naturally
|
| 67 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
|
| 69 |
## Technical Details
|
| 70 |
|
| 71 |
This app combines several technologies:
|
| 72 |
-
- Gradio's
|
| 73 |
- Google's Speech Recognition API for transcription
|
| 74 |
- Hugging Face's LLM API (Llama-2-7b-chat-hf model)
|
| 75 |
- gTTS (Google Text-to-Speech) for voice synthesis
|
| 76 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: CASL 2 - Speech Therapy Assessment Tool
|
| 3 |
emoji: 🎤
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: purple
|
|
|
|
| 9 |
pinned: false
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# CASL 2 - Speech Therapy Assessment Tool
|
| 13 |
|
| 14 |
+
An interactive tool for speech therapists to assess and treat speech disorders. CASL 2 combines professional speech therapy assessment with AI-powered feedback, making it easier for therapists to conduct evaluations and track progress.
|
| 15 |
|
| 16 |
## Features
|
| 17 |
|
| 18 |
+
### 1. Conversation Assistant
|
| 19 |
+
- Natural voice-based conversation with an AI assistant
|
| 20 |
+
- Get information about speech therapy techniques and assessment methods
|
| 21 |
+
- Guidance on using the assessment tools
|
| 22 |
+
- Full conversation history tracking
|
| 23 |
+
|
| 24 |
+
### 2. Articulation Assessment
|
| 25 |
+
- Evaluate speech sound production in various positions (initial, medial, final)
|
| 26 |
+
- Visual prompts with images for each target word
|
| 27 |
+
- Real-time recording and transcription
|
| 28 |
+
- Automatic analysis of pronunciation accuracy
|
| 29 |
+
- AI-powered feedback for each word
|
| 30 |
+
- Progress tracking through the assessment
|
| 31 |
+
|
| 32 |
+
### 3. Language Assessment
|
| 33 |
+
- Evaluate receptive and expressive language skills
|
| 34 |
+
- Test vocabulary, following directions, and sentence formation
|
| 35 |
+
- Record responses and get professional feedback
|
| 36 |
+
- Structured assessment format with varied task types
|
| 37 |
|
| 38 |
## Setup
|
| 39 |
|
|
|
|
| 73 |
|
| 74 |
## How to Use
|
| 75 |
|
| 76 |
+
### Conversation Mode
|
| 77 |
+
1. Select the "Conversation Assistant" tab
|
| 78 |
+
2. Click "Start Conversation" to begin
|
| 79 |
+
3. Record your speech using the microphone
|
| 80 |
+
4. The AI will respond with voice and text
|
| 81 |
5. Continue the conversation naturally
|
| 82 |
+
|
| 83 |
+
### Articulation Assessment
|
| 84 |
+
1. Select the "Articulation Assessment" tab
|
| 85 |
+
2. Click "Start Assessment" to begin
|
| 86 |
+
3. View the current word and associated image
|
| 87 |
+
4. Ask the patient to pronounce the displayed word
|
| 88 |
+
5. Record their response using the microphone
|
| 89 |
+
6. Review the AI analysis and feedback
|
| 90 |
+
7. Use the navigation buttons to move between words
|
| 91 |
+
8. Complete all words in the assessment
|
| 92 |
+
|
| 93 |
+
### Language Assessment
|
| 94 |
+
1. Select the "Language Assessment" tab
|
| 95 |
+
2. Click "Start Assessment" to begin
|
| 96 |
+
3. Present the current task to the patient
|
| 97 |
+
4. Record their response using the microphone
|
| 98 |
+
5. Review the AI analysis and feedback
|
| 99 |
+
6. Use the navigation buttons to move between tasks
|
| 100 |
+
7. Complete all tasks in the assessment
|
| 101 |
|
| 102 |
## Technical Details
|
| 103 |
|
| 104 |
This app combines several technologies:
|
| 105 |
+
- Gradio's UI components and audio capabilities
|
| 106 |
- Google's Speech Recognition API for transcription
|
| 107 |
- Hugging Face's LLM API (Llama-2-7b-chat-hf model)
|
| 108 |
- gTTS (Google Text-to-Speech) for voice synthesis
|
| 109 |
+
- Simple speech analysis for assessment purposes
|
| 110 |
+
- Modern web interface with responsive design
|
| 111 |
+
|
| 112 |
+
## For Therapists
|
| 113 |
+
|
| 114 |
+
CASL 2 is designed to supplement professional speech therapy assessment, not replace it. The tool provides an engaging interface for patient interaction and helps therapists with:
|
| 115 |
+
|
| 116 |
+
1. Recording and documenting patient responses
|
| 117 |
+
2. Initial screening of speech patterns
|
| 118 |
+
3. Tracking progress over time
|
| 119 |
+
4. Engaging patients with interactive exercises
|
| 120 |
+
5. Providing consistent, supportive feedback
|
| 121 |
+
|
| 122 |
+
Always combine the tool's analysis with your professional judgment.
|
app.py
CHANGED
|
@@ -21,7 +21,73 @@ headers = {
|
|
| 21 |
"Content-Type": "application/json"
|
| 22 |
}
|
| 23 |
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
"""Get AI response from Hugging Face API"""
|
| 26 |
if not user_text:
|
| 27 |
return "I couldn't understand what you said. Could you try again?"
|
|
@@ -30,7 +96,11 @@ def get_ai_response(user_text):
|
|
| 30 |
conversation.append({"role": "user", "content": user_text})
|
| 31 |
|
| 32 |
# Prepare for API call
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
messages.extend(conversation)
|
| 35 |
|
| 36 |
try:
|
|
@@ -120,12 +190,146 @@ def format_conversation():
|
|
| 120 |
result = ""
|
| 121 |
for msg in conversation:
|
| 122 |
if msg["role"] != "system": # Skip system messages
|
| 123 |
-
prefix = "
|
| 124 |
result += f"{prefix}{msg['content']}\n\n"
|
| 125 |
return result
|
| 126 |
|
| 127 |
-
def
|
| 128 |
-
"""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
if audio is None:
|
| 130 |
return None, "No audio detected. Please try again."
|
| 131 |
|
|
@@ -150,7 +354,7 @@ def initialize_conversation():
|
|
| 150 |
conversation = []
|
| 151 |
|
| 152 |
# Add welcome message
|
| 153 |
-
welcome = "Hello! I'm your
|
| 154 |
conversation.append({"role": "assistant", "content": welcome})
|
| 155 |
|
| 156 |
# Generate speech
|
|
@@ -158,67 +362,291 @@ def initialize_conversation():
|
|
| 158 |
|
| 159 |
return welcome_audio, format_conversation()
|
| 160 |
|
| 161 |
-
#
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 165 |
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
| 170 |
-
|
|
|
|
| 171 |
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 177 |
|
| 178 |
-
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 189 |
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 196 |
|
| 197 |
# Instructions
|
| 198 |
-
with gr.Accordion("How to use", open=True):
|
| 199 |
gr.Markdown("""
|
| 200 |
-
##
|
|
|
|
|
|
|
| 201 |
|
| 202 |
-
1.
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
5. The AI will respond with voice and text
|
| 207 |
-
6. Continue the conversation by recording more messages
|
| 208 |
|
| 209 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 210 |
""")
|
| 211 |
|
| 212 |
-
# Connect components
|
| 213 |
-
|
| 214 |
fn=initialize_conversation,
|
| 215 |
-
outputs=[
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 216 |
)
|
| 217 |
|
| 218 |
-
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
|
|
|
|
| 222 |
)
|
| 223 |
|
| 224 |
# Launch the app
|
|
|
|
| 21 |
"Content-Type": "application/json"
|
| 22 |
}
|
| 23 |
|
| 24 |
+
# Sample assessment data
|
| 25 |
+
articulation_exercises = {
|
| 26 |
+
"title": "Articulation Assessment",
|
| 27 |
+
"instructions": "Record the child pronouncing each target word. The system will analyze pronunciation accuracy.",
|
| 28 |
+
"words": [
|
| 29 |
+
{
|
| 30 |
+
"word": "Sun",
|
| 31 |
+
"target_sound": "s",
|
| 32 |
+
"position": "initial",
|
| 33 |
+
"imageUrl": "https://images.unsplash.com/photo-1477500292188-6f0d31f8cb2e?ixlib=rb-1.2.1&auto=format&fit=crop&w=300&q=80"
|
| 34 |
+
},
|
| 35 |
+
{
|
| 36 |
+
"word": "Mouse",
|
| 37 |
+
"target_sound": "s",
|
| 38 |
+
"position": "final",
|
| 39 |
+
"imageUrl": "https://images.unsplash.com/photo-1425082661705-1834bfd09dca?ixlib=rb-1.2.1&auto=format&fit=crop&w=300&q=80"
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"word": "Pencil",
|
| 43 |
+
"target_sound": "s",
|
| 44 |
+
"position": "medial",
|
| 45 |
+
"imageUrl": "https://images.unsplash.com/photo-1583485088034-697b5bc54ccd?ixlib=rb-1.2.1&auto=format&fit=crop&w=300&q=80"
|
| 46 |
+
},
|
| 47 |
+
{
|
| 48 |
+
"word": "Tree",
|
| 49 |
+
"target_sound": "tr",
|
| 50 |
+
"position": "initial",
|
| 51 |
+
"imageUrl": "https://images.unsplash.com/photo-1502082553048-f009c37129b9?ixlib=rb-1.2.1&auto=format&fit=crop&w=300&q=80"
|
| 52 |
+
},
|
| 53 |
+
{
|
| 54 |
+
"word": "Blue",
|
| 55 |
+
"target_sound": "bl",
|
| 56 |
+
"position": "initial",
|
| 57 |
+
"imageUrl": "https://images.unsplash.com/photo-1557180295-76eee20ae8aa?ixlib=rb-1.2.1&auto=format&fit=crop&w=300&q=80"
|
| 58 |
+
}
|
| 59 |
+
]
|
| 60 |
+
}
|
| 61 |
+
|
| 62 |
+
language_exercises = {
|
| 63 |
+
"title": "Language Assessment",
|
| 64 |
+
"instructions": "Assess receptive and expressive language skills with these tasks. Record the child's response to each prompt.",
|
| 65 |
+
"tasks": [
|
| 66 |
+
{
|
| 67 |
+
"prompt": "Point to the item that you eat with.",
|
| 68 |
+
"type": "following_directions",
|
| 69 |
+
"options": ["Fork", "Book", "Shoe", "Car"],
|
| 70 |
+
"correct": "Fork"
|
| 71 |
+
},
|
| 72 |
+
{
|
| 73 |
+
"prompt": "What is the opposite of hot?",
|
| 74 |
+
"type": "vocabulary",
|
| 75 |
+
"correct": "Cold"
|
| 76 |
+
},
|
| 77 |
+
{
|
| 78 |
+
"prompt": "Make a sentence using the word 'happy'.",
|
| 79 |
+
"type": "sentence_formation",
|
| 80 |
+
"evaluation": "subjective"
|
| 81 |
+
}
|
| 82 |
+
]
|
| 83 |
+
}
|
| 84 |
+
|
| 85 |
+
# Current assessment state
|
| 86 |
+
current_assessment = None
|
| 87 |
+
current_item_index = 0
|
| 88 |
+
assessment_results = []
|
| 89 |
+
|
| 90 |
+
def get_ai_response(user_text, context=None):
|
| 91 |
"""Get AI response from Hugging Face API"""
|
| 92 |
if not user_text:
|
| 93 |
return "I couldn't understand what you said. Could you try again?"
|
|
|
|
| 96 |
conversation.append({"role": "user", "content": user_text})
|
| 97 |
|
| 98 |
# Prepare for API call
|
| 99 |
+
system_prompt = "You are a speech therapy assistant for the CASL 2 assessment tool. Provide helpful, supportive feedback for speech exercises."
|
| 100 |
+
if context:
|
| 101 |
+
system_prompt += f" Current context: {context}"
|
| 102 |
+
|
| 103 |
+
messages = [{"role": "system", "content": system_prompt}]
|
| 104 |
messages.extend(conversation)
|
| 105 |
|
| 106 |
try:
|
|
|
|
| 190 |
result = ""
|
| 191 |
for msg in conversation:
|
| 192 |
if msg["role"] != "system": # Skip system messages
|
| 193 |
+
prefix = "User: " if msg["role"] == "user" else "Assistant: "
|
| 194 |
result += f"{prefix}{msg['content']}\n\n"
|
| 195 |
return result
|
| 196 |
|
| 197 |
+
def analyze_speech(text, target):
|
| 198 |
+
"""Simple analysis of speech for assessment"""
|
| 199 |
+
if not text or not target:
|
| 200 |
+
return 0
|
| 201 |
+
|
| 202 |
+
# Simple analysis - check if target word is in the transcribed text
|
| 203 |
+
# In a real app, this would be more sophisticated
|
| 204 |
+
if target.lower() in text.lower():
|
| 205 |
+
# Simulate accuracy score (in a real app, use phonetic analysis)
|
| 206 |
+
accuracy = np.random.uniform(70, 100)
|
| 207 |
+
else:
|
| 208 |
+
accuracy = np.random.uniform(0, 70)
|
| 209 |
+
|
| 210 |
+
return accuracy
|
| 211 |
+
|
| 212 |
+
def process_assessment_audio(audio, assessment_type, item_index):
|
| 213 |
+
"""Process recorded audio for assessment item"""
|
| 214 |
+
global current_item_index, assessment_results
|
| 215 |
+
|
| 216 |
+
if audio is None:
|
| 217 |
+
return None, f"No audio detected. Please try again.", None, None
|
| 218 |
+
|
| 219 |
+
# Convert speech to text
|
| 220 |
+
transcript = speech_to_text(audio)
|
| 221 |
+
|
| 222 |
+
if not transcript:
|
| 223 |
+
return None, "I couldn't understand the speech. Please try again.", None, None
|
| 224 |
+
|
| 225 |
+
# Process based on assessment type
|
| 226 |
+
if assessment_type == "articulation":
|
| 227 |
+
current_word = articulation_exercises["words"][item_index]
|
| 228 |
+
target_word = current_word["word"]
|
| 229 |
+
accuracy = analyze_speech(transcript, target_word)
|
| 230 |
+
|
| 231 |
+
result = {
|
| 232 |
+
"word": target_word,
|
| 233 |
+
"target_sound": current_word["target_sound"],
|
| 234 |
+
"position": current_word["position"],
|
| 235 |
+
"transcript": transcript,
|
| 236 |
+
"accuracy": accuracy,
|
| 237 |
+
"passed": accuracy > 70
|
| 238 |
+
}
|
| 239 |
+
|
| 240 |
+
assessment_results.append(result)
|
| 241 |
+
|
| 242 |
+
# Get feedback from AI
|
| 243 |
+
context = f"Assessment: Articulation. Target word: {target_word} with {current_word['target_sound']} sound in {current_word['position']} position. User said: {transcript}. Accuracy: {accuracy:.1f}%."
|
| 244 |
+
feedback = get_ai_response(transcript, context)
|
| 245 |
+
|
| 246 |
+
# Prepare for next item
|
| 247 |
+
next_index = item_index + 1
|
| 248 |
+
if next_index >= len(articulation_exercises["words"]):
|
| 249 |
+
next_index = 0 # Reset or could end assessment
|
| 250 |
+
|
| 251 |
+
result_display = f"""
|
| 252 |
+
**Word**: {target_word}
|
| 253 |
+
**Transcript**: {transcript}
|
| 254 |
+
**Accuracy**: {accuracy:.1f}%
|
| 255 |
+
**Result**: {"PASSED" if accuracy > 70 else "NEEDS PRACTICE"}
|
| 256 |
+
|
| 257 |
+
{feedback}
|
| 258 |
+
"""
|
| 259 |
+
|
| 260 |
+
# Return audio response, result display, next item index, and image URL
|
| 261 |
+
response_audio = text_to_speech(feedback)
|
| 262 |
+
next_image = articulation_exercises["words"][next_index]["imageUrl"] if next_index < len(articulation_exercises["words"]) else None
|
| 263 |
+
return response_audio, result_display, next_index, next_image
|
| 264 |
+
|
| 265 |
+
elif assessment_type == "language":
|
| 266 |
+
# Similar processing for language assessment
|
| 267 |
+
# Not fully implemented - would follow similar pattern
|
| 268 |
+
current_task = language_exercises["tasks"][item_index]
|
| 269 |
+
|
| 270 |
+
result = {
|
| 271 |
+
"prompt": current_task["prompt"],
|
| 272 |
+
"type": current_task["type"],
|
| 273 |
+
"response": transcript,
|
| 274 |
+
}
|
| 275 |
+
|
| 276 |
+
assessment_results.append(result)
|
| 277 |
+
|
| 278 |
+
# Get feedback from AI
|
| 279 |
+
context = f"Assessment: Language. Task: {current_task['prompt']}. User said: {transcript}."
|
| 280 |
+
feedback = get_ai_response(transcript, context)
|
| 281 |
+
|
| 282 |
+
# Prepare for next item
|
| 283 |
+
next_index = item_index + 1
|
| 284 |
+
if next_index >= len(language_exercises["tasks"]):
|
| 285 |
+
next_index = 0 # Reset or could end assessment
|
| 286 |
+
|
| 287 |
+
result_display = f"""
|
| 288 |
+
**Prompt**: {current_task['prompt']}
|
| 289 |
+
**Response**: {transcript}
|
| 290 |
+
|
| 291 |
+
{feedback}
|
| 292 |
+
"""
|
| 293 |
+
|
| 294 |
+
# Return audio response, result display, next item index
|
| 295 |
+
response_audio = text_to_speech(feedback)
|
| 296 |
+
return response_audio, result_display, next_index, None
|
| 297 |
+
|
| 298 |
+
return None, "Unknown assessment type", item_index, None
|
| 299 |
+
|
| 300 |
+
def init_articulation_assessment():
|
| 301 |
+
"""Initialize articulation assessment"""
|
| 302 |
+
global current_assessment, current_item_index, assessment_results
|
| 303 |
+
current_assessment = "articulation"
|
| 304 |
+
current_item_index = 0
|
| 305 |
+
assessment_results = []
|
| 306 |
+
|
| 307 |
+
instructions = articulation_exercises["instructions"]
|
| 308 |
+
first_word = articulation_exercises["words"][0]["word"]
|
| 309 |
+
message = f"{instructions}\n\nFirst word: {first_word}"
|
| 310 |
+
|
| 311 |
+
audio_response = text_to_speech(message)
|
| 312 |
+
current_image = articulation_exercises["words"][0]["imageUrl"]
|
| 313 |
+
|
| 314 |
+
return audio_response, message, current_image, 0
|
| 315 |
+
|
| 316 |
+
def init_language_assessment():
|
| 317 |
+
"""Initialize language assessment"""
|
| 318 |
+
global current_assessment, current_item_index, assessment_results
|
| 319 |
+
current_assessment = "language"
|
| 320 |
+
current_item_index = 0
|
| 321 |
+
assessment_results = []
|
| 322 |
+
|
| 323 |
+
instructions = language_exercises["instructions"]
|
| 324 |
+
first_prompt = language_exercises["tasks"][0]["prompt"]
|
| 325 |
+
message = f"{instructions}\n\nFirst task: {first_prompt}"
|
| 326 |
+
|
| 327 |
+
audio_response = text_to_speech(message)
|
| 328 |
+
|
| 329 |
+
return audio_response, message, None, 0
|
| 330 |
+
|
| 331 |
+
def process_conversation_audio(audio):
|
| 332 |
+
"""Process recorded audio for conversation mode"""
|
| 333 |
if audio is None:
|
| 334 |
return None, "No audio detected. Please try again."
|
| 335 |
|
|
|
|
| 354 |
conversation = []
|
| 355 |
|
| 356 |
# Add welcome message
|
| 357 |
+
welcome = "Hello! I'm your CASL 2 speech therapy assistant. How can I help you today?"
|
| 358 |
conversation.append({"role": "assistant", "content": welcome})
|
| 359 |
|
| 360 |
# Generate speech
|
|
|
|
| 362 |
|
| 363 |
return welcome_audio, format_conversation()
|
| 364 |
|
| 365 |
+
# Custom CSS
|
| 366 |
+
custom_css = """
|
| 367 |
+
:root {
|
| 368 |
+
--primary: #4a6fa5;
|
| 369 |
+
--secondary: #6b96c3;
|
| 370 |
+
--accent: #ff7e5f;
|
| 371 |
+
--light: #f9f9f9;
|
| 372 |
+
--dark: #333;
|
| 373 |
+
--success: #4caf50;
|
| 374 |
+
--warning: #ff9800;
|
| 375 |
+
--error: #f44336;
|
| 376 |
+
}
|
| 377 |
+
|
| 378 |
+
.gradio-container {
|
| 379 |
+
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
| 380 |
+
max-width: 1200px;
|
| 381 |
+
margin: auto;
|
| 382 |
+
}
|
| 383 |
+
|
| 384 |
+
.app-header {
|
| 385 |
+
background-color: var(--primary);
|
| 386 |
+
color: white;
|
| 387 |
+
padding: 1rem;
|
| 388 |
+
border-radius: 8px 8px 0 0;
|
| 389 |
+
margin-bottom: 1rem;
|
| 390 |
+
}
|
| 391 |
+
|
| 392 |
+
.tab-nav {
|
| 393 |
+
margin-bottom: 1rem;
|
| 394 |
+
}
|
| 395 |
+
|
| 396 |
+
.input-panel {
|
| 397 |
+
background-color: white;
|
| 398 |
+
border-radius: 8px;
|
| 399 |
+
box-shadow: 0 2px 10px rgba(0, 0, 0, 0.08);
|
| 400 |
+
padding: 1rem;
|
| 401 |
+
margin-bottom: 1rem;
|
| 402 |
+
}
|
| 403 |
+
|
| 404 |
+
.output-panel {
|
| 405 |
+
background-color: white;
|
| 406 |
+
border-radius: 8px;
|
| 407 |
+
box-shadow: 0 2px 10px rgba(0, 0, 0, 0.08);
|
| 408 |
+
padding: 1rem;
|
| 409 |
+
}
|
| 410 |
+
|
| 411 |
+
button.primary {
|
| 412 |
+
background-color: var(--primary);
|
| 413 |
+
color: white;
|
| 414 |
+
}
|
| 415 |
+
|
| 416 |
+
button.secondary {
|
| 417 |
+
background-color: var(--secondary);
|
| 418 |
+
color: white;
|
| 419 |
+
}
|
| 420 |
+
|
| 421 |
+
.image-display {
|
| 422 |
+
display: flex;
|
| 423 |
+
justify-content: center;
|
| 424 |
+
margin: 1rem 0;
|
| 425 |
+
}
|
| 426 |
+
|
| 427 |
+
.image-display img {
|
| 428 |
+
max-width: 300px;
|
| 429 |
+
border-radius: 8px;
|
| 430 |
+
box-shadow: 0 2px 10px rgba(0, 0, 0, 0.1);
|
| 431 |
+
}
|
| 432 |
+
"""
|
| 433 |
+
|
| 434 |
+
# Create Gradio interface with tabs for different modes
|
| 435 |
+
with gr.Blocks(title="CASL 2 - Speech Therapy Assessment", css=custom_css) as demo:
|
| 436 |
+
# Current state variables
|
| 437 |
+
current_item_idx = gr.State(0)
|
| 438 |
+
|
| 439 |
+
# App header
|
| 440 |
+
with gr.Column(elem_classes="app-header"):
|
| 441 |
+
gr.Markdown("# CASL 2 - Speech Therapy Assessment")
|
| 442 |
+
gr.Markdown("An interactive tool for speech therapists to assess and treat speech disorders")
|
| 443 |
|
| 444 |
+
# Main tabs
|
| 445 |
+
with gr.Tabs() as tabs:
|
| 446 |
+
# Conversation Mode Tab
|
| 447 |
+
with gr.TabItem("Conversation Assistant", elem_classes="tab-nav"):
|
| 448 |
+
gr.Markdown("### General Conversation Mode")
|
| 449 |
+
gr.Markdown("Have a natural conversation with the AI assistant for general questions and guidance")
|
| 450 |
|
| 451 |
+
with gr.Row():
|
| 452 |
+
# Left panel - Controls
|
| 453 |
+
with gr.Column(scale=1, elem_classes="input-panel"):
|
| 454 |
+
# Start button
|
| 455 |
+
conv_start_button = gr.Button("Start Conversation", variant="primary")
|
| 456 |
+
|
| 457 |
+
# Microphone input
|
| 458 |
+
conv_audio_input = gr.Audio(
|
| 459 |
+
label="🎤 SPEAK HERE",
|
| 460 |
+
type="numpy"
|
| 461 |
+
)
|
| 462 |
+
|
| 463 |
+
# Right panel - Conversation
|
| 464 |
+
with gr.Column(scale=2, elem_classes="output-panel"):
|
| 465 |
+
# Conversation display
|
| 466 |
+
conv_display = gr.Textbox(
|
| 467 |
+
label="Conversation History",
|
| 468 |
+
lines=12,
|
| 469 |
+
value=""
|
| 470 |
+
)
|
| 471 |
+
|
| 472 |
+
# Audio playback
|
| 473 |
+
conv_audio_output = gr.Audio(
|
| 474 |
+
label="AI Response",
|
| 475 |
+
type="filepath",
|
| 476 |
+
autoplay=True
|
| 477 |
+
)
|
| 478 |
+
|
| 479 |
+
# Articulation Assessment Tab
|
| 480 |
+
with gr.TabItem("Articulation Assessment", elem_classes="tab-nav"):
|
| 481 |
+
gr.Markdown("### Articulation Assessment")
|
| 482 |
+
gr.Markdown("Evaluate production of speech sounds in various positions within words")
|
| 483 |
|
| 484 |
+
with gr.Row():
|
| 485 |
+
# Left panel - Controls & Current Word
|
| 486 |
+
with gr.Column(scale=1, elem_classes="input-panel"):
|
| 487 |
+
# Start button
|
| 488 |
+
art_start_button = gr.Button("Start Assessment", variant="primary")
|
| 489 |
+
|
| 490 |
+
# Current word display
|
| 491 |
+
art_current_display = gr.Textbox(
|
| 492 |
+
label="Current Task",
|
| 493 |
+
lines=3
|
| 494 |
+
)
|
| 495 |
+
|
| 496 |
+
# Word image
|
| 497 |
+
art_image = gr.Image(
|
| 498 |
+
label="Word Image",
|
| 499 |
+
type="filepath",
|
| 500 |
+
elem_classes="image-display"
|
| 501 |
+
)
|
| 502 |
+
|
| 503 |
+
# Microphone input
|
| 504 |
+
art_audio_input = gr.Audio(
|
| 505 |
+
label="🎤 RECORD RESPONSE",
|
| 506 |
+
type="numpy"
|
| 507 |
+
)
|
| 508 |
+
|
| 509 |
+
# Navigation
|
| 510 |
+
with gr.Row():
|
| 511 |
+
art_prev_button = gr.Button("◀ Previous")
|
| 512 |
+
art_item_indicator = gr.Textbox(label="Item", value="1/5", interactive=False)
|
| 513 |
+
art_next_button = gr.Button("Next ▶")
|
| 514 |
+
|
| 515 |
+
# Right panel - Results
|
| 516 |
+
with gr.Column(scale=2, elem_classes="output-panel"):
|
| 517 |
+
# Results display
|
| 518 |
+
art_result_display = gr.Markdown(
|
| 519 |
+
label="Assessment Results",
|
| 520 |
+
value="Start the assessment to see results."
|
| 521 |
+
)
|
| 522 |
+
|
| 523 |
+
# Audio feedback
|
| 524 |
+
art_audio_output = gr.Audio(
|
| 525 |
+
label="Speech Therapist Feedback",
|
| 526 |
+
type="filepath",
|
| 527 |
+
autoplay=True
|
| 528 |
+
)
|
| 529 |
+
|
| 530 |
+
# Language Assessment Tab
|
| 531 |
+
with gr.TabItem("Language Assessment", elem_classes="tab-nav"):
|
| 532 |
+
gr.Markdown("### Language Assessment")
|
| 533 |
+
gr.Markdown("Evaluate receptive and expressive language skills including vocabulary and grammar")
|
| 534 |
|
| 535 |
+
with gr.Row():
|
| 536 |
+
# Left panel - Controls & Current Task
|
| 537 |
+
with gr.Column(scale=1, elem_classes="input-panel"):
|
| 538 |
+
# Start button
|
| 539 |
+
lang_start_button = gr.Button("Start Assessment", variant="primary")
|
| 540 |
+
|
| 541 |
+
# Current task display
|
| 542 |
+
lang_current_display = gr.Textbox(
|
| 543 |
+
label="Current Task",
|
| 544 |
+
lines=3
|
| 545 |
+
)
|
| 546 |
+
|
| 547 |
+
# Microphone input
|
| 548 |
+
lang_audio_input = gr.Audio(
|
| 549 |
+
label="🎤 RECORD RESPONSE",
|
| 550 |
+
type="numpy"
|
| 551 |
+
)
|
| 552 |
+
|
| 553 |
+
# Navigation
|
| 554 |
+
with gr.Row():
|
| 555 |
+
lang_prev_button = gr.Button("◀ Previous")
|
| 556 |
+
lang_item_indicator = gr.Textbox(label="Item", value="1/3", interactive=False)
|
| 557 |
+
lang_next_button = gr.Button("Next ▶")
|
| 558 |
+
|
| 559 |
+
# Right panel - Results
|
| 560 |
+
with gr.Column(scale=2, elem_classes="output-panel"):
|
| 561 |
+
# Results display
|
| 562 |
+
lang_result_display = gr.Markdown(
|
| 563 |
+
label="Assessment Results",
|
| 564 |
+
value="Start the assessment to see results."
|
| 565 |
+
)
|
| 566 |
+
|
| 567 |
+
# Audio feedback
|
| 568 |
+
lang_audio_output = gr.Audio(
|
| 569 |
+
label="Speech Therapist Feedback",
|
| 570 |
+
type="filepath",
|
| 571 |
+
autoplay=True
|
| 572 |
+
)
|
| 573 |
|
| 574 |
# Instructions
|
| 575 |
+
with gr.Accordion("How to use CASL 2", open=True):
|
| 576 |
gr.Markdown("""
|
| 577 |
+
## CASL 2 Speech Therapy Assessment Tool
|
| 578 |
+
|
| 579 |
+
This application provides three main functions:
|
| 580 |
|
| 581 |
+
### 1. Conversation Assistant
|
| 582 |
+
- General conversation with an AI assistant
|
| 583 |
+
- Ask questions about speech therapy, techniques, or general information
|
| 584 |
+
- Get guidance on using the assessment tools
|
|
|
|
|
|
|
| 585 |
|
| 586 |
+
### 2. Articulation Assessment
|
| 587 |
+
- Evaluate speech sound production
|
| 588 |
+
- Record the patient pronouncing target words
|
| 589 |
+
- Get automatic analysis and therapist feedback
|
| 590 |
+
- Track progress over time
|
| 591 |
+
|
| 592 |
+
### 3. Language Assessment
|
| 593 |
+
- Evaluate receptive and expressive language skills
|
| 594 |
+
- Test vocabulary, following directions, and sentence formation
|
| 595 |
+
- Record responses and get professional feedback
|
| 596 |
+
|
| 597 |
+
**For therapists**: Use these tools during your sessions to supplement your professional assessment.
|
| 598 |
+
|
| 599 |
+
**Privacy Note**: All audio recordings are processed securely and are not stored permanently.
|
| 600 |
""")
|
| 601 |
|
| 602 |
+
# Connect components - Conversation Mode
|
| 603 |
+
conv_start_button.click(
|
| 604 |
fn=initialize_conversation,
|
| 605 |
+
outputs=[conv_audio_output, conv_display]
|
| 606 |
+
)
|
| 607 |
+
|
| 608 |
+
conv_audio_input.change(
|
| 609 |
+
fn=process_conversation_audio,
|
| 610 |
+
inputs=[conv_audio_input],
|
| 611 |
+
outputs=[conv_audio_output, conv_display]
|
| 612 |
+
)
|
| 613 |
+
|
| 614 |
+
# Connect components - Articulation Assessment
|
| 615 |
+
art_start_button.click(
|
| 616 |
+
fn=init_articulation_assessment,
|
| 617 |
+
outputs=[art_audio_output, art_current_display, art_image, current_item_idx]
|
| 618 |
+
)
|
| 619 |
+
|
| 620 |
+
art_audio_input.change(
|
| 621 |
+
fn=process_assessment_audio,
|
| 622 |
+
inputs=[art_audio_input, gr.Textbox(value="articulation", visible=False), current_item_idx],
|
| 623 |
+
outputs=[art_audio_output, art_result_display, current_item_idx, art_image]
|
| 624 |
+
)
|
| 625 |
+
|
| 626 |
+
# Update indicator text when item changes
|
| 627 |
+
current_item_idx.change(
|
| 628 |
+
fn=lambda idx: f"{idx+1}/{len(articulation_exercises['words'])}",
|
| 629 |
+
inputs=[current_item_idx],
|
| 630 |
+
outputs=[art_item_indicator]
|
| 631 |
+
)
|
| 632 |
+
|
| 633 |
+
# Connect components - Language Assessment
|
| 634 |
+
lang_start_button.click(
|
| 635 |
+
fn=init_language_assessment,
|
| 636 |
+
outputs=[lang_audio_output, lang_current_display, gr.Image(visible=False), current_item_idx]
|
| 637 |
+
)
|
| 638 |
+
|
| 639 |
+
lang_audio_input.change(
|
| 640 |
+
fn=process_assessment_audio,
|
| 641 |
+
inputs=[lang_audio_input, gr.Textbox(value="language", visible=False), current_item_idx],
|
| 642 |
+
outputs=[lang_audio_output, lang_result_display, current_item_idx, gr.Image(visible=False)]
|
| 643 |
)
|
| 644 |
|
| 645 |
+
# Update language indicator text when item changes
|
| 646 |
+
current_item_idx.change(
|
| 647 |
+
fn=lambda idx: f"{idx+1}/{len(language_exercises['tasks'])}",
|
| 648 |
+
inputs=[current_item_idx],
|
| 649 |
+
outputs=[lang_item_indicator]
|
| 650 |
)
|
| 651 |
|
| 652 |
# Launch the app
|
requirements.txt
CHANGED
|
@@ -2,4 +2,5 @@ gradio==3.50.0
|
|
| 2 |
numpy>=1.19.0
|
| 3 |
SpeechRecognition>=3.8.1
|
| 4 |
requests>=2.25.1
|
| 5 |
-
gTTS>=2.3.2
|
|
|
|
|
|
| 2 |
numpy>=1.19.0
|
| 3 |
SpeechRecognition>=3.8.1
|
| 4 |
requests>=2.25.1
|
| 5 |
+
gTTS>=2.3.2
|
| 6 |
+
Pillow>=8.0.0
|
slp.html
ADDED
|
@@ -0,0 +1,320 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
<!DOCTYPE html>
|
| 3 |
+
<html lang="en">
|
| 4 |
+
<head>
|
| 5 |
+
<meta charset="UTF-8">
|
| 6 |
+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
| 7 |
+
<title>CASL 2 - Speech Therapy Assessment</title>
|
| 8 |
+
<style>
|
| 9 |
+
:root {
|
| 10 |
+
--primary: #4a6fa5;
|
| 11 |
+
--secondary: #6b96c3;
|
| 12 |
+
--accent: #ff7e5f;
|
| 13 |
+
--light: #f9f9f9;
|
| 14 |
+
--dark: #333;
|
| 15 |
+
--success: #4caf50;
|
| 16 |
+
--warning: #ff9800;
|
| 17 |
+
--error: #f44336;
|
| 18 |
+
}
|
| 19 |
+
|
| 20 |
+
* {
|
| 21 |
+
box-sizing: border-box;
|
| 22 |
+
margin: 0;
|
| 23 |
+
padding: 0;
|
| 24 |
+
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
| 25 |
+
}
|
| 26 |
+
|
| 27 |
+
body {
|
| 28 |
+
background-color: var(--light);
|
| 29 |
+
color: var(--dark);
|
| 30 |
+
line-height: 1.6;
|
| 31 |
+
}
|
| 32 |
+
|
| 33 |
+
.app-container {
|
| 34 |
+
display: flex;
|
| 35 |
+
min-height: 100vh;
|
| 36 |
+
}
|
| 37 |
+
|
| 38 |
+
.sidebar {
|
| 39 |
+
width: 240px;
|
| 40 |
+
background-color: var(--primary);
|
| 41 |
+
color: white;
|
| 42 |
+
padding: 1rem;
|
| 43 |
+
}
|
| 44 |
+
|
| 45 |
+
.sidebar-header {
|
| 46 |
+
display: flex;
|
| 47 |
+
align-items: center;
|
| 48 |
+
margin-bottom: 2rem;
|
| 49 |
+
}
|
| 50 |
+
|
| 51 |
+
.sidebar-header h1 {
|
| 52 |
+
font-size: 1.5rem;
|
| 53 |
+
margin-left: 0.5rem;
|
| 54 |
+
}
|
| 55 |
+
|
| 56 |
+
.nav-list {
|
| 57 |
+
list-style: none;
|
| 58 |
+
}
|
| 59 |
+
|
| 60 |
+
.nav-item {
|
| 61 |
+
padding: 0.75rem 1rem;
|
| 62 |
+
border-radius: 8px;
|
| 63 |
+
margin-bottom: 0.5rem;
|
| 64 |
+
cursor: pointer;
|
| 65 |
+
transition: background-color 0.2s;
|
| 66 |
+
}
|
| 67 |
+
|
| 68 |
+
.nav-item:hover {
|
| 69 |
+
background-color: rgba(255, 255, 255, 0.1);
|
| 70 |
+
}
|
| 71 |
+
|
| 72 |
+
.nav-item.active {
|
| 73 |
+
background-color: rgba(255, 255, 255, 0.2);
|
| 74 |
+
font-weight: 500;
|
| 75 |
+
}
|
| 76 |
+
|
| 77 |
+
.main-content {
|
| 78 |
+
flex: 1;
|
| 79 |
+
padding: 2rem;
|
| 80 |
+
overflow-y: auto;
|
| 81 |
+
}
|
| 82 |
+
|
| 83 |
+
.content-header {
|
| 84 |
+
display: flex;
|
| 85 |
+
justify-content: space-between;
|
| 86 |
+
align-items: center;
|
| 87 |
+
margin-bottom: 2rem;
|
| 88 |
+
}
|
| 89 |
+
|
| 90 |
+
.content-header h1 {
|
| 91 |
+
font-size: 2rem;
|
| 92 |
+
color: var(--primary);
|
| 93 |
+
}
|
| 94 |
+
|
| 95 |
+
.btn {
|
| 96 |
+
padding: 0.75rem 1.5rem;
|
| 97 |
+
border: none;
|
| 98 |
+
border-radius: 8px;
|
| 99 |
+
font-weight: 500;
|
| 100 |
+
cursor: pointer;
|
| 101 |
+
transition: background-color 0.2s;
|
| 102 |
+
}
|
| 103 |
+
|
| 104 |
+
.btn-primary {
|
| 105 |
+
background-color: var(--primary);
|
| 106 |
+
color: white;
|
| 107 |
+
}
|
| 108 |
+
|
| 109 |
+
.btn-primary:hover {
|
| 110 |
+
background-color: var(--secondary);
|
| 111 |
+
}
|
| 112 |
+
|
| 113 |
+
.card {
|
| 114 |
+
background-color: white;
|
| 115 |
+
border-radius: 12px;
|
| 116 |
+
box-shadow: 0 2px 10px rgba(0, 0, 0, 0.08);
|
| 117 |
+
padding: 1.5rem;
|
| 118 |
+
margin-bottom: 2rem;
|
| 119 |
+
}
|
| 120 |
+
|
| 121 |
+
.card-header {
|
| 122 |
+
margin-bottom: 1.5rem;
|
| 123 |
+
}
|
| 124 |
+
|
| 125 |
+
.assessment-list {
|
| 126 |
+
display: grid;
|
| 127 |
+
grid-template-columns: repeat(auto-fill, minmax(300px, 1fr));
|
| 128 |
+
gap: 1.5rem;
|
| 129 |
+
}
|
| 130 |
+
|
| 131 |
+
.assessment-card {
|
| 132 |
+
background-color: white;
|
| 133 |
+
border-radius: 12px;
|
| 134 |
+
box-shadow: 0 2px 10px rgba(0, 0, 0, 0.08);
|
| 135 |
+
overflow: hidden;
|
| 136 |
+
transition: transform 0.2s;
|
| 137 |
+
cursor: pointer;
|
| 138 |
+
}
|
| 139 |
+
|
| 140 |
+
.assessment-card:hover {
|
| 141 |
+
transform: translateY(-5px);
|
| 142 |
+
}
|
| 143 |
+
|
| 144 |
+
.assessment-card-header {
|
| 145 |
+
padding: 1.5rem;
|
| 146 |
+
color: white;
|
| 147 |
+
background-color: var(--primary);
|
| 148 |
+
}
|
| 149 |
+
|
| 150 |
+
.assessment-card-body {
|
| 151 |
+
padding: 1.5rem;
|
| 152 |
+
}
|
| 153 |
+
|
| 154 |
+
.assessment-card-footer {
|
| 155 |
+
padding: 1rem 1.5rem;
|
| 156 |
+
background-color: #f5f5f5;
|
| 157 |
+
display: flex;
|
| 158 |
+
justify-content: space-between;
|
| 159 |
+
}
|
| 160 |
+
|
| 161 |
+
.patient-selector {
|
| 162 |
+
margin-bottom: 2rem;
|
| 163 |
+
}
|
| 164 |
+
|
| 165 |
+
.patient-selector select {
|
| 166 |
+
padding: 0.75rem;
|
| 167 |
+
border-radius: 8px;
|
| 168 |
+
border: 1px solid #ddd;
|
| 169 |
+
min-width: 300px;
|
| 170 |
+
font-size: 1rem;
|
| 171 |
+
}
|
| 172 |
+
|
| 173 |
+
.badge {
|
| 174 |
+
display: inline-block;
|
| 175 |
+
padding: 0.25rem 0.75rem;
|
| 176 |
+
border-radius: 16px;
|
| 177 |
+
font-size: 0.85rem;
|
| 178 |
+
font-weight: 500;
|
| 179 |
+
}
|
| 180 |
+
|
| 181 |
+
.badge-success {
|
| 182 |
+
background-color: rgba(76, 175, 80, 0.15);
|
| 183 |
+
color: var(--success);
|
| 184 |
+
}
|
| 185 |
+
|
| 186 |
+
.badge-warning {
|
| 187 |
+
background-color: rgba(255, 152, 0, 0.15);
|
| 188 |
+
color: var(--warning);
|
| 189 |
+
}
|
| 190 |
+
</style>
|
| 191 |
+
</head>
|
| 192 |
+
<body>
|
| 193 |
+
<div class="app-container">
|
| 194 |
+
<!-- Sidebar Navigation -->
|
| 195 |
+
<aside class="sidebar">
|
| 196 |
+
<div class="sidebar-header">
|
| 197 |
+
<svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
| 198 |
+
<path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"></path>
|
| 199 |
+
</svg>
|
| 200 |
+
<h1>CASL 2 Therapy</h1>
|
| 201 |
+
</div>
|
| 202 |
+
<nav>
|
| 203 |
+
<ul class="nav-list">
|
| 204 |
+
<li class="nav-item">Dashboard</li>
|
| 205 |
+
<li class="nav-item active">Assessments</li>
|
| 206 |
+
<li class="nav-item">Therapy Exercises</li>
|
| 207 |
+
<li class="nav-item">Sessions</li>
|
| 208 |
+
<li class="nav-item">Progress Reports</li>
|
| 209 |
+
<li class="nav-item">Patient Management</li>
|
| 210 |
+
<li class="nav-item">Settings</li>
|
| 211 |
+
</ul>
|
| 212 |
+
</nav>
|
| 213 |
+
</aside>
|
| 214 |
+
|
| 215 |
+
<!-- Main Content -->
|
| 216 |
+
<main class="main-content">
|
| 217 |
+
<div class="content-header">
|
| 218 |
+
<h1>Speech Assessments</h1>
|
| 219 |
+
<button class="btn btn-primary">Create New Assessment</button>
|
| 220 |
+
</div>
|
| 221 |
+
|
| 222 |
+
<div class="patient-selector">
|
| 223 |
+
<select>
|
| 224 |
+
<option>Select Patient</option>
|
| 225 |
+
<option>Adam Smith - Age 8</option>
|
| 226 |
+
<option>Emma Johnson - Age 6</option>
|
| 227 |
+
<option>Ryan Davis - Age 10</option>
|
| 228 |
+
<option>Sofia Martinez - Age 7</option>
|
| 229 |
+
</select>
|
| 230 |
+
</div>
|
| 231 |
+
|
| 232 |
+
<div class="card">
|
| 233 |
+
<div class="card-header">
|
| 234 |
+
<h2>Available Assessments</h2>
|
| 235 |
+
<p>Select an assessment type to begin evaluation</p>
|
| 236 |
+
</div>
|
| 237 |
+
|
| 238 |
+
<div class="assessment-list">
|
| 239 |
+
<!-- Articulation Assessment -->
|
| 240 |
+
<div class="assessment-card">
|
| 241 |
+
<div class="assessment-card-header">
|
| 242 |
+
<h3>Articulation Assessment</h3>
|
| 243 |
+
</div>
|
| 244 |
+
<div class="assessment-card-body">
|
| 245 |
+
<p>Evaluate production of speech sounds in various positions within words.</p>
|
| 246 |
+
<ul style="margin-top: 1rem; margin-left: 1.5rem;">
|
| 247 |
+
<li>Initial, medial, final positions</li>
|
| 248 |
+
<li>Consonant blends</li>
|
| 249 |
+
<li>Intelligibility rating</li>
|
| 250 |
+
</ul>
|
| 251 |
+
</div>
|
| 252 |
+
<div class="assessment-card-footer">
|
| 253 |
+
<span class="badge badge-success">15-20 min</span>
|
| 254 |
+
<span>40 items</span>
|
| 255 |
+
</div>
|
| 256 |
+
</div>
|
| 257 |
+
|
| 258 |
+
<!-- Language Assessment -->
|
| 259 |
+
<div class="assessment-card">
|
| 260 |
+
<div class="assessment-card-header">
|
| 261 |
+
<h3>Language Assessment</h3>
|
| 262 |
+
</div>
|
| 263 |
+
<div class="assessment-card-body">
|
| 264 |
+
<p>Evaluate receptive and expressive language skills including vocabulary and grammar.</p>
|
| 265 |
+
<ul style="margin-top: 1rem; margin-left: 1.5rem;">
|
| 266 |
+
<li>Vocabulary comprehension</li>
|
| 267 |
+
<li>Following directions</li>
|
| 268 |
+
<li>Sentence formation</li>
|
| 269 |
+
</ul>
|
| 270 |
+
</div>
|
| 271 |
+
<div class="assessment-card-footer">
|
| 272 |
+
<span class="badge badge-warning">25-30 min</span>
|
| 273 |
+
<span>35 items</span>
|
| 274 |
+
</div>
|
| 275 |
+
</div>
|
| 276 |
+
|
| 277 |
+
<!-- Fluency Assessment -->
|
| 278 |
+
<div class="assessment-card">
|
| 279 |
+
<div class="assessment-card-header">
|
| 280 |
+
<h3>Fluency Assessment</h3>
|
| 281 |
+
</div>
|
| 282 |
+
<div class="assessment-card-body">
|
| 283 |
+
<p>Evaluate speech fluency including rate, rhythm, and presence of disfluencies.</p>
|
| 284 |
+
<ul style="margin-top: 1rem; margin-left: 1.5rem;">
|
| 285 |
+
<li>Reading tasks</li>
|
| 286 |
+
<li>Conversation sample</li>
|
| 287 |
+
<li>Picture description</li>
|
| 288 |
+
</ul>
|
| 289 |
+
</div>
|
| 290 |
+
<div class="assessment-card-footer">
|
| 291 |
+
<span class="badge badge-warning">20-25 min</span>
|
| 292 |
+
<span>3 activities</span>
|
| 293 |
+
</div>
|
| 294 |
+
</div>
|
| 295 |
+
|
| 296 |
+
<!-- Voice Assessment -->
|
| 297 |
+
<div class="assessment-card">
|
| 298 |
+
<div class="assessment-card-header">
|
| 299 |
+
<h3>Voice Assessment</h3>
|
| 300 |
+
</div>
|
| 301 |
+
<div class="assessment-card-body">
|
| 302 |
+
<p>Evaluate voice quality, pitch, loudness, and resonance during various speaking tasks.</p>
|
| 303 |
+
<ul style="margin-top: 1rem; margin-left: 1.5rem;">
|
| 304 |
+
<li>Sustained phonation</li>
|
| 305 |
+
<li>Pitch range</li>
|
| 306 |
+
<li>Connected speech</li>
|
| 307 |
+
</ul>
|
| 308 |
+
</div>
|
| 309 |
+
<div class="assessment-card-footer">
|
| 310 |
+
<span class="badge badge-success">15-20 min</span>
|
| 311 |
+
<span>4 activities</span>
|
| 312 |
+
</div>
|
| 313 |
+
</div>
|
| 314 |
+
</div>
|
| 315 |
+
</div>
|
| 316 |
+
</main>
|
| 317 |
+
</div>
|
| 318 |
+
</body>
|
| 319 |
+
</html>
|
| 320 |
+
|
slpapp.js
ADDED
|
@@ -0,0 +1,233 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
// Sample component structure for CASL 2 Web Application
|
| 2 |
+
|
| 3 |
+
// Main App Component
|
| 4 |
+
import React, { useState, useEffect } from 'react';
|
| 5 |
+
import { BrowserRouter as Router, Route, Switch } from 'react-router-dom';
|
| 6 |
+
import { AuthProvider } from './contexts/AuthContext';
|
| 7 |
+
import PrivateRoute from './components/PrivateRoute';
|
| 8 |
+
|
| 9 |
+
// Pages
|
| 10 |
+
import Dashboard from './pages/Dashboard';
|
| 11 |
+
import Login from './pages/Login';
|
| 12 |
+
import PatientProfile from './pages/PatientProfile';
|
| 13 |
+
import AssessmentModule from './pages/AssessmentModule';
|
| 14 |
+
import TherapyExercises from './pages/TherapyExercises';
|
| 15 |
+
import SessionRecording from './pages/SessionRecording';
|
| 16 |
+
import ProgressReports from './pages/ProgressReports';
|
| 17 |
+
import Settings from './pages/Settings';
|
| 18 |
+
|
| 19 |
+
// Core components
|
| 20 |
+
import Navigation from './components/Navigation';
|
| 21 |
+
import Footer from './components/Footer';
|
| 22 |
+
|
| 23 |
+
function App() {
|
| 24 |
+
return (
|
| 25 |
+
<AuthProvider>
|
| 26 |
+
<Router>
|
| 27 |
+
<div className="app-container">
|
| 28 |
+
<Navigation />
|
| 29 |
+
<main className="main-content">
|
| 30 |
+
<Switch>
|
| 31 |
+
<Route exact path="/login" component={Login} />
|
| 32 |
+
<PrivateRoute exact path="/" component={Dashboard} />
|
| 33 |
+
<PrivateRoute path="/patient/:id" component={PatientProfile} />
|
| 34 |
+
<PrivateRoute path="/assessment" component={AssessmentModule} />
|
| 35 |
+
<PrivateRoute path="/exercises" component={TherapyExercises} />
|
| 36 |
+
<PrivateRoute path="/session/:id?" component={SessionRecording} />
|
| 37 |
+
<PrivateRoute path="/progress" component={ProgressReports} />
|
| 38 |
+
<PrivateRoute path="/settings" component={Settings} />
|
| 39 |
+
</Switch>
|
| 40 |
+
</main>
|
| 41 |
+
<Footer />
|
| 42 |
+
</div>
|
| 43 |
+
</Router>
|
| 44 |
+
</AuthProvider>
|
| 45 |
+
);
|
| 46 |
+
}
|
| 47 |
+
|
| 48 |
+
export default App;
|
| 49 |
+
|
| 50 |
+
// Sample Exercise Component
|
| 51 |
+
import React, { useState, useEffect, useRef } from 'react';
|
| 52 |
+
import { useParams } from 'react-router-dom';
|
| 53 |
+
import ExerciseControls from '../components/ExerciseControls';
|
| 54 |
+
import AudioVisualizer from '../components/AudioVisualizer';
|
| 55 |
+
import FeedbackDisplay from '../components/FeedbackDisplay';
|
| 56 |
+
import ProgressIndicator from '../components/ProgressIndicator';
|
| 57 |
+
|
| 58 |
+
function ArticulationExercise() {
|
| 59 |
+
const { exerciseId } = useParams();
|
| 60 |
+
const [exerciseData, setExerciseData] = useState(null);
|
| 61 |
+
const [currentWordIndex, setCurrentWordIndex] = useState(0);
|
| 62 |
+
const [recording, setRecording] = useState(false);
|
| 63 |
+
const [audioData, setAudioData] = useState(null);
|
| 64 |
+
const [results, setResults] = useState([]);
|
| 65 |
+
const audioContext = useRef(null);
|
| 66 |
+
const mediaRecorder = useRef(null);
|
| 67 |
+
|
| 68 |
+
useEffect(() => {
|
| 69 |
+
// Fetch exercise data
|
| 70 |
+
const fetchExercise = async () => {
|
| 71 |
+
try {
|
| 72 |
+
// This would be an API call in a real application
|
| 73 |
+
const response = await fetch(`/api/exercises/${exerciseId}`);
|
| 74 |
+
const data = await response.json();
|
| 75 |
+
setExerciseData(data);
|
| 76 |
+
} catch (error) {
|
| 77 |
+
console.error("Failed to fetch exercise data:", error);
|
| 78 |
+
}
|
| 79 |
+
};
|
| 80 |
+
|
| 81 |
+
fetchExercise();
|
| 82 |
+
|
| 83 |
+
// Initialize audio context
|
| 84 |
+
audioContext.current = new (window.AudioContext || window.webkitAudioContext)();
|
| 85 |
+
|
| 86 |
+
return () => {
|
| 87 |
+
// Cleanup audio resources
|
| 88 |
+
if (audioContext.current) {
|
| 89 |
+
audioContext.current.close();
|
| 90 |
+
}
|
| 91 |
+
};
|
| 92 |
+
}, [exerciseId]);
|
| 93 |
+
|
| 94 |
+
const startRecording = async () => {
|
| 95 |
+
try {
|
| 96 |
+
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
|
| 97 |
+
|
| 98 |
+
mediaRecorder.current = new MediaRecorder(stream);
|
| 99 |
+
const audioChunks = [];
|
| 100 |
+
|
| 101 |
+
mediaRecorder.current.addEventListener("dataavailable", event => {
|
| 102 |
+
audioChunks.push(event.data);
|
| 103 |
+
});
|
| 104 |
+
|
| 105 |
+
mediaRecorder.current.addEventListener("stop", async () => {
|
| 106 |
+
const audioBlob = new Blob(audioChunks, { type: 'audio/webm' });
|
| 107 |
+
setAudioData(audioBlob);
|
| 108 |
+
await processAudio(audioBlob);
|
| 109 |
+
});
|
| 110 |
+
|
| 111 |
+
mediaRecorder.current.start();
|
| 112 |
+
setRecording(true);
|
| 113 |
+
} catch (error) {
|
| 114 |
+
console.error("Error accessing microphone:", error);
|
| 115 |
+
}
|
| 116 |
+
};
|
| 117 |
+
|
| 118 |
+
const stopRecording = () => {
|
| 119 |
+
if (mediaRecorder.current && recording) {
|
| 120 |
+
mediaRecorder.current.stop();
|
| 121 |
+
setRecording(false);
|
| 122 |
+
}
|
| 123 |
+
};
|
| 124 |
+
|
| 125 |
+
const processAudio = async (audioBlob) => {
|
| 126 |
+
try {
|
| 127 |
+
// In a real app, send to speech recognition API
|
| 128 |
+
// For demo, simulate processing
|
| 129 |
+
|
| 130 |
+
// Create form data for API request
|
| 131 |
+
const formData = new FormData();
|
| 132 |
+
formData.append('audio', audioBlob);
|
| 133 |
+
formData.append('targetWord', exerciseData.words[currentWordIndex].word);
|
| 134 |
+
|
| 135 |
+
// This would be an actual API call in production
|
| 136 |
+
// const response = await fetch('/api/analyze-speech', {
|
| 137 |
+
// method: 'POST',
|
| 138 |
+
// body: formData
|
| 139 |
+
// });
|
| 140 |
+
// const analysisResults = await response.json();
|
| 141 |
+
|
| 142 |
+
// Simulate API response
|
| 143 |
+
await new Promise(resolve => setTimeout(resolve, 1500));
|
| 144 |
+
const simulatedAccuracy = Math.random() * 100;
|
| 145 |
+
|
| 146 |
+
const newResult = {
|
| 147 |
+
word: exerciseData.words[currentWordIndex].word,
|
| 148 |
+
accuracy: simulatedAccuracy,
|
| 149 |
+
passed: simulatedAccuracy > 70,
|
| 150 |
+
timestamp: new Date().toISOString()
|
| 151 |
+
};
|
| 152 |
+
|
| 153 |
+
setResults([...results, newResult]);
|
| 154 |
+
|
| 155 |
+
// Move to next word if available
|
| 156 |
+
if (currentWordIndex < exerciseData.words.length - 1) {
|
| 157 |
+
setTimeout(() => setCurrentWordIndex(currentWordIndex + 1), 2000);
|
| 158 |
+
} else {
|
| 159 |
+
// Exercise complete
|
| 160 |
+
await saveExerciseResults([...results, newResult]);
|
| 161 |
+
}
|
| 162 |
+
} catch (error) {
|
| 163 |
+
console.error("Error processing audio:", error);
|
| 164 |
+
}
|
| 165 |
+
};
|
| 166 |
+
|
| 167 |
+
const saveExerciseResults = async (finalResults) => {
|
| 168 |
+
try {
|
| 169 |
+
// In production, save to backend
|
| 170 |
+
console.log("Exercise complete, results:", finalResults);
|
| 171 |
+
|
| 172 |
+
// const response = await fetch('/api/save-results', {
|
| 173 |
+
// method: 'POST',
|
| 174 |
+
// headers: {
|
| 175 |
+
// 'Content-Type': 'application/json'
|
| 176 |
+
// },
|
| 177 |
+
// body: JSON.stringify({
|
| 178 |
+
// exerciseId,
|
| 179 |
+
// results: finalResults,
|
| 180 |
+
// completedAt: new Date().toISOString()
|
| 181 |
+
// })
|
| 182 |
+
// });
|
| 183 |
+
|
| 184 |
+
// Redirect or show summary
|
| 185 |
+
} catch (error) {
|
| 186 |
+
console.error("Failed to save results:", error);
|
| 187 |
+
}
|
| 188 |
+
};
|
| 189 |
+
|
| 190 |
+
if (!exerciseData) return <div>Loading exercise...</div>;
|
| 191 |
+
|
| 192 |
+
const currentWord = exerciseData.words[currentWordIndex];
|
| 193 |
+
|
| 194 |
+
return (
|
| 195 |
+
<div className="exercise-container">
|
| 196 |
+
<h1>{exerciseData.title}</h1>
|
| 197 |
+
<p className="exercise-instructions">{exerciseData.instructions}</p>
|
| 198 |
+
|
| 199 |
+
<div className="word-display">
|
| 200 |
+
<h2>{currentWord.word}</h2>
|
| 201 |
+
<img
|
| 202 |
+
src={currentWord.imageUrl || `/api/placeholder/300/300`}
|
| 203 |
+
alt={currentWord.word}
|
| 204 |
+
className="word-image"
|
| 205 |
+
/>
|
| 206 |
+
</div>
|
| 207 |
+
|
| 208 |
+
<AudioVisualizer
|
| 209 |
+
audioBlob={audioData}
|
| 210 |
+
isRecording={recording}
|
| 211 |
+
/>
|
| 212 |
+
|
| 213 |
+
<ExerciseControls
|
| 214 |
+
isRecording={recording}
|
| 215 |
+
onStartRecording={startRecording}
|
| 216 |
+
onStopRecording={stopRecording}
|
| 217 |
+
disabled={currentWordIndex >= exerciseData.words.length}
|
| 218 |
+
/>
|
| 219 |
+
|
| 220 |
+
{results.length > 0 && results[results.length - 1].word === currentWord.word && (
|
| 221 |
+
<FeedbackDisplay result={results[results.length - 1]} />
|
| 222 |
+
)}
|
| 223 |
+
|
| 224 |
+
<ProgressIndicator
|
| 225 |
+
current={currentWordIndex + 1}
|
| 226 |
+
total={exerciseData.words.length}
|
| 227 |
+
results={results}
|
| 228 |
+
/>
|
| 229 |
+
</div>
|
| 230 |
+
);
|
| 231 |
+
}
|
| 232 |
+
|
| 233 |
+
export default ArticulationExercise;
|
slpapp.py
ADDED
|
@@ -0,0 +1,233 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
// Sample component structure for CASL 2 Web Application
|
| 2 |
+
|
| 3 |
+
// Main App Component
|
| 4 |
+
import React, { useState, useEffect } from 'react';
|
| 5 |
+
import { BrowserRouter as Router, Route, Switch } from 'react-router-dom';
|
| 6 |
+
import { AuthProvider } from './contexts/AuthContext';
|
| 7 |
+
import PrivateRoute from './components/PrivateRoute';
|
| 8 |
+
|
| 9 |
+
// Pages
|
| 10 |
+
import Dashboard from './pages/Dashboard';
|
| 11 |
+
import Login from './pages/Login';
|
| 12 |
+
import PatientProfile from './pages/PatientProfile';
|
| 13 |
+
import AssessmentModule from './pages/AssessmentModule';
|
| 14 |
+
import TherapyExercises from './pages/TherapyExercises';
|
| 15 |
+
import SessionRecording from './pages/SessionRecording';
|
| 16 |
+
import ProgressReports from './pages/ProgressReports';
|
| 17 |
+
import Settings from './pages/Settings';
|
| 18 |
+
|
| 19 |
+
// Core components
|
| 20 |
+
import Navigation from './components/Navigation';
|
| 21 |
+
import Footer from './components/Footer';
|
| 22 |
+
|
| 23 |
+
function App() {
|
| 24 |
+
return (
|
| 25 |
+
<AuthProvider>
|
| 26 |
+
<Router>
|
| 27 |
+
<div className="app-container">
|
| 28 |
+
<Navigation />
|
| 29 |
+
<main className="main-content">
|
| 30 |
+
<Switch>
|
| 31 |
+
<Route exact path="/login" component={Login} />
|
| 32 |
+
<PrivateRoute exact path="/" component={Dashboard} />
|
| 33 |
+
<PrivateRoute path="/patient/:id" component={PatientProfile} />
|
| 34 |
+
<PrivateRoute path="/assessment" component={AssessmentModule} />
|
| 35 |
+
<PrivateRoute path="/exercises" component={TherapyExercises} />
|
| 36 |
+
<PrivateRoute path="/session/:id?" component={SessionRecording} />
|
| 37 |
+
<PrivateRoute path="/progress" component={ProgressReports} />
|
| 38 |
+
<PrivateRoute path="/settings" component={Settings} />
|
| 39 |
+
</Switch>
|
| 40 |
+
</main>
|
| 41 |
+
<Footer />
|
| 42 |
+
</div>
|
| 43 |
+
</Router>
|
| 44 |
+
</AuthProvider>
|
| 45 |
+
);
|
| 46 |
+
}
|
| 47 |
+
|
| 48 |
+
export default App;
|
| 49 |
+
|
| 50 |
+
// Sample Exercise Component
|
| 51 |
+
import React, { useState, useEffect, useRef } from 'react';
|
| 52 |
+
import { useParams } from 'react-router-dom';
|
| 53 |
+
import ExerciseControls from '../components/ExerciseControls';
|
| 54 |
+
import AudioVisualizer from '../components/AudioVisualizer';
|
| 55 |
+
import FeedbackDisplay from '../components/FeedbackDisplay';
|
| 56 |
+
import ProgressIndicator from '../components/ProgressIndicator';
|
| 57 |
+
|
| 58 |
+
function ArticulationExercise() {
|
| 59 |
+
const { exerciseId } = useParams();
|
| 60 |
+
const [exerciseData, setExerciseData] = useState(null);
|
| 61 |
+
const [currentWordIndex, setCurrentWordIndex] = useState(0);
|
| 62 |
+
const [recording, setRecording] = useState(false);
|
| 63 |
+
const [audioData, setAudioData] = useState(null);
|
| 64 |
+
const [results, setResults] = useState([]);
|
| 65 |
+
const audioContext = useRef(null);
|
| 66 |
+
const mediaRecorder = useRef(null);
|
| 67 |
+
|
| 68 |
+
useEffect(() => {
|
| 69 |
+
// Fetch exercise data
|
| 70 |
+
const fetchExercise = async () => {
|
| 71 |
+
try {
|
| 72 |
+
// This would be an API call in a real application
|
| 73 |
+
const response = await fetch(`/api/exercises/${exerciseId}`);
|
| 74 |
+
const data = await response.json();
|
| 75 |
+
setExerciseData(data);
|
| 76 |
+
} catch (error) {
|
| 77 |
+
console.error("Failed to fetch exercise data:", error);
|
| 78 |
+
}
|
| 79 |
+
};
|
| 80 |
+
|
| 81 |
+
fetchExercise();
|
| 82 |
+
|
| 83 |
+
// Initialize audio context
|
| 84 |
+
audioContext.current = new (window.AudioContext || window.webkitAudioContext)();
|
| 85 |
+
|
| 86 |
+
return () => {
|
| 87 |
+
// Cleanup audio resources
|
| 88 |
+
if (audioContext.current) {
|
| 89 |
+
audioContext.current.close();
|
| 90 |
+
}
|
| 91 |
+
};
|
| 92 |
+
}, [exerciseId]);
|
| 93 |
+
|
| 94 |
+
const startRecording = async () => {
|
| 95 |
+
try {
|
| 96 |
+
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
|
| 97 |
+
|
| 98 |
+
mediaRecorder.current = new MediaRecorder(stream);
|
| 99 |
+
const audioChunks = [];
|
| 100 |
+
|
| 101 |
+
mediaRecorder.current.addEventListener("dataavailable", event => {
|
| 102 |
+
audioChunks.push(event.data);
|
| 103 |
+
});
|
| 104 |
+
|
| 105 |
+
mediaRecorder.current.addEventListener("stop", async () => {
|
| 106 |
+
const audioBlob = new Blob(audioChunks, { type: 'audio/webm' });
|
| 107 |
+
setAudioData(audioBlob);
|
| 108 |
+
await processAudio(audioBlob);
|
| 109 |
+
});
|
| 110 |
+
|
| 111 |
+
mediaRecorder.current.start();
|
| 112 |
+
setRecording(true);
|
| 113 |
+
} catch (error) {
|
| 114 |
+
console.error("Error accessing microphone:", error);
|
| 115 |
+
}
|
| 116 |
+
};
|
| 117 |
+
|
| 118 |
+
const stopRecording = () => {
|
| 119 |
+
if (mediaRecorder.current && recording) {
|
| 120 |
+
mediaRecorder.current.stop();
|
| 121 |
+
setRecording(false);
|
| 122 |
+
}
|
| 123 |
+
};
|
| 124 |
+
|
| 125 |
+
const processAudio = async (audioBlob) => {
|
| 126 |
+
try {
|
| 127 |
+
// In a real app, send to speech recognition API
|
| 128 |
+
// For demo, simulate processing
|
| 129 |
+
|
| 130 |
+
// Create form data for API request
|
| 131 |
+
const formData = new FormData();
|
| 132 |
+
formData.append('audio', audioBlob);
|
| 133 |
+
formData.append('targetWord', exerciseData.words[currentWordIndex].word);
|
| 134 |
+
|
| 135 |
+
// This would be an actual API call in production
|
| 136 |
+
// const response = await fetch('/api/analyze-speech', {
|
| 137 |
+
// method: 'POST',
|
| 138 |
+
// body: formData
|
| 139 |
+
// });
|
| 140 |
+
// const analysisResults = await response.json();
|
| 141 |
+
|
| 142 |
+
// Simulate API response
|
| 143 |
+
await new Promise(resolve => setTimeout(resolve, 1500));
|
| 144 |
+
const simulatedAccuracy = Math.random() * 100;
|
| 145 |
+
|
| 146 |
+
const newResult = {
|
| 147 |
+
word: exerciseData.words[currentWordIndex].word,
|
| 148 |
+
accuracy: simulatedAccuracy,
|
| 149 |
+
passed: simulatedAccuracy > 70,
|
| 150 |
+
timestamp: new Date().toISOString()
|
| 151 |
+
};
|
| 152 |
+
|
| 153 |
+
setResults([...results, newResult]);
|
| 154 |
+
|
| 155 |
+
// Move to next word if available
|
| 156 |
+
if (currentWordIndex < exerciseData.words.length - 1) {
|
| 157 |
+
setTimeout(() => setCurrentWordIndex(currentWordIndex + 1), 2000);
|
| 158 |
+
} else {
|
| 159 |
+
// Exercise complete
|
| 160 |
+
await saveExerciseResults([...results, newResult]);
|
| 161 |
+
}
|
| 162 |
+
} catch (error) {
|
| 163 |
+
console.error("Error processing audio:", error);
|
| 164 |
+
}
|
| 165 |
+
};
|
| 166 |
+
|
| 167 |
+
const saveExerciseResults = async (finalResults) => {
|
| 168 |
+
try {
|
| 169 |
+
// In production, save to backend
|
| 170 |
+
console.log("Exercise complete, results:", finalResults);
|
| 171 |
+
|
| 172 |
+
// const response = await fetch('/api/save-results', {
|
| 173 |
+
// method: 'POST',
|
| 174 |
+
// headers: {
|
| 175 |
+
// 'Content-Type': 'application/json'
|
| 176 |
+
// },
|
| 177 |
+
// body: JSON.stringify({
|
| 178 |
+
// exerciseId,
|
| 179 |
+
// results: finalResults,
|
| 180 |
+
// completedAt: new Date().toISOString()
|
| 181 |
+
// })
|
| 182 |
+
// });
|
| 183 |
+
|
| 184 |
+
// Redirect or show summary
|
| 185 |
+
} catch (error) {
|
| 186 |
+
console.error("Failed to save results:", error);
|
| 187 |
+
}
|
| 188 |
+
};
|
| 189 |
+
|
| 190 |
+
if (!exerciseData) return <div>Loading exercise...</div>;
|
| 191 |
+
|
| 192 |
+
const currentWord = exerciseData.words[currentWordIndex];
|
| 193 |
+
|
| 194 |
+
return (
|
| 195 |
+
<div className="exercise-container">
|
| 196 |
+
<h1>{exerciseData.title}</h1>
|
| 197 |
+
<p className="exercise-instructions">{exerciseData.instructions}</p>
|
| 198 |
+
|
| 199 |
+
<div className="word-display">
|
| 200 |
+
<h2>{currentWord.word}</h2>
|
| 201 |
+
<img
|
| 202 |
+
src={currentWord.imageUrl || `/api/placeholder/300/300`}
|
| 203 |
+
alt={currentWord.word}
|
| 204 |
+
className="word-image"
|
| 205 |
+
/>
|
| 206 |
+
</div>
|
| 207 |
+
|
| 208 |
+
<AudioVisualizer
|
| 209 |
+
audioBlob={audioData}
|
| 210 |
+
isRecording={recording}
|
| 211 |
+
/>
|
| 212 |
+
|
| 213 |
+
<ExerciseControls
|
| 214 |
+
isRecording={recording}
|
| 215 |
+
onStartRecording={startRecording}
|
| 216 |
+
onStopRecording={stopRecording}
|
| 217 |
+
disabled={currentWordIndex >= exerciseData.words.length}
|
| 218 |
+
/>
|
| 219 |
+
|
| 220 |
+
{results.length > 0 && results[results.length - 1].word === currentWord.word && (
|
| 221 |
+
<FeedbackDisplay result={results[results.length - 1]} />
|
| 222 |
+
)}
|
| 223 |
+
|
| 224 |
+
<ProgressIndicator
|
| 225 |
+
current={currentWordIndex + 1}
|
| 226 |
+
total={exerciseData.words.length}
|
| 227 |
+
results={results}
|
| 228 |
+
/>
|
| 229 |
+
</div>
|
| 230 |
+
);
|
| 231 |
+
}
|
| 232 |
+
|
| 233 |
+
export default ArticulationExercise;
|