Upload 31 files

README.md

@@ -1,12 +1,123 @@
----
-title: CASLLiveKit
-emoji: 🏢
-colorFrom: red
-colorTo: pink
-sdk: gradio
-sdk_version: 5.23.3
-app_file: app.py
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# CASL Voice Bot
+
+A speech pathology assistant using AI to assess students' speaking abilities based on the CASL-2 framework. This application helps speech-language pathologists (SLPs) with speech assessment in school settings.
+
+## Implementations
+
+This project provides multiple implementations:
+
+1. **LiveKit Implementation** - Uses LiveKit agents with OpenAI's real-time voice API for low-latency, high-quality audio streaming.
+2. **Direct API Implementation** - Uses OpenAI's API directly without LiveKit, for simpler deployment.
+3. **Hugging Face Spaces** - An adaptive implementation that works on Hugging Face Spaces, automatically detecting whether LiveKit is available.
+
+## Features
+
+- Voice-to-voice interaction with AI speech pathologist
+- CASL-2 framework assessment
+- Real-time assessment tracking
+- Session recording and saving
+- Custom note-taking for SLPs
+- Gradio web interface for easy sharing and use in school settings
+
+## CASL-2 Assessment Areas
+
+The AI speech pathologist assesses students in these key areas:
+
+1. **Lexical/Semantic Skills**: Vocabulary knowledge, word meanings, and contextual word use
+2. **Syntactic Skills**: Grammar and sentence structure understanding
+3. **Supralinguistic Skills**: Higher-level language skills beyond literal meanings
+4. **Pragmatic Skills**: Language use in social contexts (less emphasis for younger students)
+
+## Setup Instructions
+
+### Prerequisites
+
+- Python 3.8+
+- OpenAI API key with access to GPT-4o and TTS models
+
+### Installation
+
+1. Clone the repository:
+   ```
+   git clone https://github.com/yourusername/CASLVoiceBot.git
+   cd CASLVoiceBot
+   ```
+
+2. Create a virtual environment and install dependencies:
+   ```
+   python -m venv venv
+   source venv/bin/activate  # On Windows: venv\Scripts\activate
+   pip install -r requirements.txt
+   ```
+
+3. For LiveKit implementation, install LiveKit dependencies:
+   ```
+   # Edit requirements.txt to uncomment the livekit-agents line
+   pip install livekit-agents>=0.7.0
+   ```
+
+4. Set up environment variables:
+   ```
+   cp .env.example .env
+   ```
+   Then edit `.env` to add your OpenAI API key.
+
+### Running the Application
+
+#### LiveKit Implementation (recommended for best performance)
+```
+python run_livekit.py
+```
+
+#### Direct API Implementation (simpler deployment)
+```
+python run_direct.py
+```
+
+#### Command Line Options
+Both implementations support these options:
+- `--share`: Share the app publicly (enabled by default)
+- `--local`: Run the app locally without sharing
+
+## Deployment on Hugging Face Spaces
+
+1. Create a new Space on Hugging Face with the Gradio SDK
+2. Upload the repository contents to the Space
+3. Add your OPENAI_API_KEY as a secret in the Space settings
+
+By default, the Hugging Face Spaces deployment will try to use LiveKit if available, and fall back to direct API if not.
+
+## Project Structure
+
+```
+CASLVoiceBot/
+├── app.py                  # Hugging Face Spaces entry point
+├── run_direct.py           # Direct API implementation runner
+├── run_livekit.py          # LiveKit implementation runner
+├── requirements.txt        # Common dependencies
+├── .env.example            # Environment variables template
+├── implementations/
+│   ├── common/             # Shared utilities
+│   │   ├── casl_utils.py   # CASL-2 assessment utilities
+│   ├── direct/             # Direct API implementation
+│   │   ├── app.py          # Direct OpenAI API app
+│   ├── livekit/            # LiveKit implementation
+│   │   ├── app.py          # LiveKit app
+│   │   ├── livekit_gradio_hf.py # HF-compatible LiveKit app
+├── session_data/           # Saved session data
+```
+
+## Usage
+
+1. Optionally enter a Student ID to track sessions
+2. Select your preferred AI voice
+3. Click "Start Session" to begin a speech assessment
+4. Wait for the AI to introduce itself, then speak when prompted
+5. View real-time assessment in the interface
+6. SLPs can add notes throughout the session
+7. Save the session when finished
+8. Click "Stop Session" to end
+
+## License
+
+[MIT License](LICENSE)

README_livekit.md

@@ -0,0 +1,88 @@
+# CASL Voice Bot with LiveKit
+
+A speech pathology assistant using LiveKit agents with OpenAI's real-time voice capabilities. This application helps speech-language pathologists (SLPs) assess students' speaking abilities based on the CASL-2 framework.
+
+## Features
+
+- Real-time voice interaction with AI speech pathologist using LiveKit
+- OpenAI's GPT-4o for intelligent conversation
+- CASL-2 framework assessment
+- Real-time assessment tracking
+- Session recording and saving
+- Custom note-taking for SLPs
+- Gradio web interface for easy sharing and use in school settings
+
+## CASL-2 Assessment Areas
+
+The AI speech pathologist assesses students in these key areas:
+
+1. **Lexical/Semantic Skills**: Vocabulary knowledge, word meanings, and contextual word use
+2. **Syntactic Skills**: Grammar and sentence structure understanding
+3. **Supralinguistic Skills**: Higher-level language skills beyond literal meanings
+4. **Pragmatic Skills**: Language use in social contexts (less emphasis for younger students)
+
+## Setup Instructions
+
+### Prerequisites
+
+- Python 3.8+
+- OpenAI API key with access to GPT-4o and TTS models
+- Created using the LiveKit multimodal agent template
+
+### Installation
+
+1. Clone the repository:
+   ```
+   git clone https://github.com/yourusername/CASLVoiceBot.git
+   cd CASLVoiceBot
+   ```
+
+2. Create a virtual environment and install dependencies:
+   ```
+   python -m venv venv
+   source venv/bin/activate  # On Windows: venv\Scripts\activate
+   pip install -r livekit_requirements.txt
+   ```
+
+3. Set up environment variables:
+   ```
+   cp .env.example .env
+   ```
+   Then edit `.env` to add your OpenAI API key.
+
+### Running the Application
+
+1. Start the application:
+   ```
+   python run_livekit.py
+   ```
+
+2. Access the application through the URL provided in the terminal.
+
+## Usage
+
+1. Optionally enter a Student ID to track sessions
+2. Select your preferred AI voice
+3. Click "Start Session" to begin a speech assessment
+4. Wait for the AI to introduce itself, then speak when prompted
+5. View real-time assessment in the interface
+6. SLPs can add notes throughout the session
+7. Save the session when finished
+8. Click "Stop Session" to end
+
+## Benefits of Using LiveKit
+
+- **Real-time Audio Processing**: LiveKit provides robust real-time audio streaming capabilities
+- **Low Latency**: Minimizes delay between student speech and AI response
+- **WebRTC Infrastructure**: Built on the same technology used for video calls
+- **Connection Management**: Automatically handles connection issues and reconnections
+- **Scalability**: Can support multiple concurrent sessions if needed
+- **Agent Integration**: LiveKit's agent system is designed specifically for AI assistants
+
+## Deployment on Hugging Face Spaces
+
+For deployment on Hugging Face Spaces, additional configuration may be required due to LiveKit's WebRTC requirements. Please refer to LiveKit documentation for details on setting up appropriate server configurations.
+
+## License
+
+[MIT License](LICENSE)

app.py

@@ -0,0 +1,12 @@
+#!/usr/bin/env python3
+
+"""
+CASL Voice Bot - Hugging Face Spaces entry point
+"""
+
+# Import the adaptive app that works in both LiveKit and direct modes
+from implementations.livekit.livekit_gradio_hf import app
+
+# This is the entry point that Hugging Face Spaces will use
+if __name__ == "__main__":
+    app.launch()

app/.env.example

@@ -0,0 +1,2 @@
+# OpenAI API Key
+OPENAI_API_KEY=your_openai_api_key

app/__init__.py

@@ -0,0 +1,5 @@
+"""
+CASL Voice Bot - AI Speech Therapist
+"""
+
+__version__ = "0.1.0"

app/advanced_features.py

@@ -0,0 +1,169 @@
+#!/usr/bin/env python3
+
+"""
+Advanced features that can be added to the CASL Voice Bot application.
+This module contains extensions that SLPs might want to add to the base system.
+"""
+
+import os
+import pandas as pd
+import datetime
+import json
+from pathlib import Path
+
+class SessionRecorder:
+    """Records session data for later analysis and progress tracking"""
+    
+    def __init__(self, storage_dir="session_data"):
+        self.storage_dir = storage_dir
+        Path(storage_dir).mkdir(exist_ok=True)
+        self.current_session = {
+            "timestamp": datetime.datetime.now().isoformat(),
+            "student_id": None,
+            "transcript": [],
+            "assessment": {}
+        }
+    
+    def set_student_id(self, student_id):
+        """Set the student ID for the current session"""
+        self.current_session["student_id"] = student_id
+    
+    def add_transcript_entry(self, speaker, text):
+        """Add an entry to the transcript"""
+        self.current_session["transcript"].append({
+            "timestamp": datetime.datetime.now().isoformat(),
+            "speaker": speaker,
+            "text": text
+        })
+    
+    def add_assessment_note(self, category, note):
+        """Add an assessment note for a CASL-2 category"""
+        if category not in self.current_session["assessment"]:
+            self.current_session["assessment"][category] = []
+        
+        self.current_session["assessment"][category].append({
+            "timestamp": datetime.datetime.now().isoformat(),
+            "note": note
+        })
+    
+    def save_session(self):
+        """Save the current session to a JSON file"""
+        student_id = self.current_session["student_id"] or "anonymous"
+        timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
+        filename = f"{student_id}_{timestamp}.json"
+        
+        with open(os.path.join(self.storage_dir, filename), 'w') as f:
+            json.dump(self.current_session, f, indent=2)
+        
+        return filename
+
+
+class CASLAnalyzer:
+    """Analyzes transcripts based on CASL-2 framework categories"""
+    
+    def __init__(self):
+        self.categories = {
+            "lexical_semantic": {
+                "description": "Vocabulary knowledge and word meanings",
+                "keywords": ["synonym", "antonym", "vocabulary", "word choice", "meaning"]
+            },
+            "syntactic": {
+                "description": "Grammar and sentence structure",
+                "keywords": ["grammar", "sentence", "verb tense", "agreement", "structure"]
+            },
+            "supralinguistic": {
+                "description": "Higher-level language skills",
+                "keywords": ["inference", "figurative", "metaphor", "context", "implied"]
+            },
+            "pragmatic": {
+                "description": "Social use of language",
+                "keywords": ["conversation", "social", "turn-taking", "appropriate", "context"]
+            }
+        }
+    
+    def categorize_text(self, text):
+        """Categorize text into CASL-2 framework categories"""
+        result = {}
+        
+        for category, info in self.categories.items():
+            score = 0
+            for keyword in info["keywords"]:
+                if keyword.lower() in text.lower():
+                    score += 1
+            
+            if score > 0:
+                result[category] = score
+        
+        return result
+    
+    def generate_summary(self, transcript):
+        """Generate a summary of the transcript based on CASL-2 categories"""
+        all_text = " ".join([entry["text"] for entry in transcript])
+        categorization = self.categorize_text(all_text)
+        
+        summary = {
+            "categories_covered": list(categorization.keys()),
+            "focus_areas": sorted(categorization.items(), key=lambda x: x[1], reverse=True),
+            "recommendations": []
+        }
+        
+        # Generate recommendations based on categories covered
+        for category in self.categories:
+            if category not in categorization:
+                summary["recommendations"].append(
+                    f"Consider adding more {self.categories[category]['description']} exercises"
+                )
+        
+        return summary
+
+
+class VoiceMetricsAnalyzer:
+    """Analyzes voice metrics for speech patterns"""
+    
+    def __init__(self):
+        self.metrics = {
+            "word_count": 0,
+            "unique_words": set(),
+            "sentence_count": 0,
+            "average_words_per_sentence": 0,
+            "hesitations": 0,
+            "speech_rate": 0  # words per minute
+        }
+    
+    def analyze_text(self, text, duration_seconds=None):
+        """Analyze text for speech metrics"""
+        # Count words
+        words = text.split()
+        self.metrics["word_count"] = len(words)
+        self.metrics["unique_words"] = set(word.lower() for word in words)
+        
+        # Count sentences
+        sentences = [s.strip() for s in text.replace("!", ".").replace("?", ".").split(".") if s.strip()]
+        self.metrics["sentence_count"] = len(sentences)
+        
+        # Calculate average words per sentence
+        if self.metrics["sentence_count"] > 0:
+            self.metrics["average_words_per_sentence"] = self.metrics["word_count"] / self.metrics["sentence_count"]
+        
+        # Count hesitations ("um", "uh", "like", etc.)
+        hesitation_markers = ["um", "uh", "er", "like", "you know"]
+        self.metrics["hesitations"] = sum(1 for word in words if word.lower() in hesitation_markers)
+        
+        # Calculate speech rate if duration is provided
+        if duration_seconds:
+            self.metrics["speech_rate"] = (self.metrics["word_count"] / duration_seconds) * 60
+        
+        return self.metrics
+    
+    def get_summary(self):
+        """Get a summary of the voice metrics analysis"""
+        return {
+            "word_count": self.metrics["word_count"],
+            "vocabulary_diversity": len(self.metrics["unique_words"]) / max(1, self.metrics["word_count"]),
+            "average_words_per_sentence": self.metrics["average_words_per_sentence"],
+            "hesitation_frequency": self.metrics["hesitations"] / max(1, self.metrics["word_count"]),
+            "speech_rate": self.metrics["speech_rate"]
+        }
+
+
+# These classes can be imported and used to extend the base CASL Voice Bot with additional features

app/gradio_app.py

@@ -0,0 +1,255 @@
+#!/usr/bin/env python3
+
+import os
+import asyncio
+import gradio as gr
+import logging
+import tempfile
+import queue
+import threading
+import time
+from dotenv import load_dotenv
+from livekit import agents
+from openai import AsyncOpenAI
+
+# Load environment variables
+load_dotenv()
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Initialize OpenAI client
+openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+
+# Speech Pathologist Agent Prompt
+SPEECH_PATHOLOGIST_PROMPT = """
+You are a speech pathologist, a healthcare professional who specializes in evaluating, diagnosing, and treating communication disorders, including speech, language, cognitive-communication, voice, and fluency disorders. Your role is to help patients improve their speech and communication skills through various therapeutic techniques and exercises.
+Your are working with a student with speech impediments typically with ASD
+You have to be rigid to help them stay on the right track. YOu have to start with some sort of intro activity and can not rely on teh student at all to complete your thoughts. You pick a place to start and assess teh speech from there. 
+Each domain from the CASL-2 framework can be analyzed using the sample:
+Lexical/Semantic Skills:
+This category focuses on vocabulary knowledge, word meanings, and the ability to use words contextually. It measures both receptive and expressive language abilities related to word use.
+Key Subtests:
+Antonyms: Identifying words with opposite meanings.
+Synonyms: Identifying words with similar meanings.
+Idiomatic Language: Understanding and interpreting idioms and figurative language.
+Evaluate vocabulary diversity (type-token ratio).
+Note word-finding difficulties, incorrect word choices, or over-reliance on fillers (e.g., "like," "stuff").
+Assess use of specific vs. vague language (e.g., "car" vs. "sedan").
+Syntactic Skills:
+This category evaluates understanding and use of grammar and sentence structure. It focuses on the ability to comprehend and produce grammatically correct sentences.
+Key Subtests:
+Sentence Expression: Producing grammatically correct sentences based on prompts.
+Grammaticality Judgment: Identifying whether a sentence is grammatically correct.
+Examine sentence structure for grammatical accuracy.
+Identify errors in verb tense, subject-verb agreement, or sentence complexity.
+Note the use of clauses, conjunctions, and varied sentence types.
+Supralinguistic Skills:
+This subcategory assesses higher-level language skills that go beyond literal meanings, such as understanding implied meanings, sarcasm, and complex verbal reasoning.
+Key Subtests:
+Inferences: Understanding information that is not explicitly stated.
+Meaning from Context: Deriving meaning from surrounding text or dialogue.
+Nonliteral Language: Interpreting figurative language, such as metaphors or irony
+Look for use or understanding of figurative language, idioms, or humor.
+Assess ability to handle ambiguous or implied meanings in context.
+Identify advanced language use for abstract or hypothetical ideas.
+Pragmatic Skills(focus less on this as it is not typically necessary for the age range you will be dealing with):
+This category measures the ability to use language effectively in social contexts. It evaluates understanding of conversational rules, turn-taking, and adapting communication to different social situations.
+Key Subtests:
+Pragmatic Language Test: Assessing appropriateness of responses in social scenarios.
+
+Begin by introducing yourself as a speech pathologist and start with a simple vocabulary assessment activity. Be encouraging but structured in your approach.
+"""
+
+class GradioInputDevice(agents.InputDevice):
+    """Custom input device that works with Gradio"""
+    
+    def __init__(self):
+        super().__init__()
+        self.audio_queue = queue.Queue()
+        self.is_active = True
+    
+    async def receive(self) -> agents.AudioChunk:
+        """Receive audio data from the queue"""
+        while self.is_active:
+            try:
+                audio_data = self.audio_queue.get(block=True, timeout=0.1)
+                return audio_data
+            except queue.Empty:
+                await asyncio.sleep(0.1)
+        
+        return None
+    
+    def add_audio(self, audio_data):
+        """Add audio data to the queue"""
+        # Convert gradio audio format to AudioChunk
+        sample_rate, audio_array = audio_data
+        audio_chunk = agents.AudioChunk(
+            samples=audio_array,
+            sample_rate=sample_rate,
+            is_last=False
+        )
+        self.audio_queue.put(audio_chunk)
+    
+    def stop(self):
+        """Stop the input device"""
+        self.is_active = False
+
+
+class GradioOutputDevice(agents.OutputDevice):
+    """Custom output device that works with Gradio"""
+    
+    def __init__(self):
+        super().__init__()
+        self.output_queue = queue.Queue()
+    
+    async def transmit(self, audio_chunk: agents.AudioChunk) -> None:
+        """Transmit audio chunk to the queue"""
+        if audio_chunk is not None:
+            self.output_queue.put((audio_chunk.samples, audio_chunk.sample_rate))
+    
+    def get_latest_audio(self):
+        """Get the latest audio from the queue"""
+        try:
+            return self.output_queue.get(block=False)
+        except queue.Empty:
+            return None
+
+
+class SpeechPathologistAssistant:
+    """Speech pathologist assistant using LiveKit agents and Gradio"""
+    
+    def __init__(self):
+        self.input_device = GradioInputDevice()
+        self.output_device = GradioOutputDevice()
+        self.assistant = None
+        self.assistant_task = None
+        self.transcript = []
+        self.is_running = False
+    
+    async def initialize_assistant(self):
+        """Initialize the speech assistant"""
+        self.assistant = agents.VoiceAssistant(
+            openai_client=openai_client,
+            model="gpt-4o",
+            voice="shimmer",  # Using a friendly, professional voice
+            input_device=self.input_device,
+            output_device=self.output_device,
+            initial_message=SPEECH_PATHOLOGIST_PROMPT,
+            real_time=True,  # Enable real-time processing
+        )
+        
+        # Add transcript callback
+        self.assistant.on_transcript = self.on_transcript
+        self.assistant.on_response = self.on_response
+    
+    def on_transcript(self, transcript):
+        """Handle transcript from user"""
+        self.transcript.append(f"Student: {transcript.text}")
+        return True
+    
+    def on_response(self, response):
+        """Handle response from assistant"""
+        self.transcript.append(f"Speech Pathologist: {response.text}")
+        return True
+    
+    async def start_assistant(self):
+        """Start the assistant in a background task"""
+        if not self.assistant:
+            await self.initialize_assistant()
+        
+        self.is_running = True
+        self.assistant_task = asyncio.create_task(self.assistant.run())
+    
+    def stop_assistant(self):
+        """Stop the assistant"""
+        if self.assistant_task and not self.assistant_task.done():
+            self.assistant_task.cancel()
+        
+        self.input_device.stop()
+        self.is_running = False
+    
+    def process_audio(self, audio):
+        """Process audio from Gradio interface"""
+        if not self.is_running or audio is None:
+            return None, self.get_transcript()
+        
+        self.input_device.add_audio(audio)
+        
+        # Check for assistant output
+        output_audio = self.output_device.get_latest_audio()
+        
+        return output_audio, self.get_transcript()
+    
+    def get_transcript(self):
+        """Get the current transcript"""
+        return "\n".join(self.transcript)
+
+
+# Create the assistant instance
+speech_assistant = SpeechPathologistAssistant()
+
+
+def start_session():
+    """Start the speech pathology session"""
+    asyncio.run(speech_assistant.start_assistant())
+    return "Session started. Please speak to begin the assessment."
+
+
+def stop_session():
+    """Stop the speech pathology session"""
+    speech_assistant.stop_assistant()
+    return "Session stopped."
+
+
+def process_audio(audio):
+    """Process audio from microphone"""
+    if audio is None:
+        return None, speech_assistant.get_transcript()
+    
+    return speech_assistant.process_audio(audio)
+
+
+# Create Gradio Interface
+with gr.Blocks(title="CASL-2 Speech Pathology Assistant") as app:
+    gr.Markdown("# Speech Pathology Assistant")
+    gr.Markdown("This tool provides speech therapy assessment based on the CASL-2 framework")
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            start_button = gr.Button("Start Session")
+            stop_button = gr.Button("Stop Session")
+            status = gr.Textbox(label="Status", value="Ready to start")
+        
+        with gr.Column(scale=2):
+            audio_input = gr.Audio(
+                label="Speak",
+                type="microphone", 
+                streaming=True,
+                autoplay=True
+            )
+            audio_output = gr.Audio(label="Response")
+    
+    with gr.Row():
+        transcript = gr.Textbox(label="Transcript", lines=10)
+    
+    # Setup event handlers
+    start_button.click(fn=start_session, outputs=status)
+    stop_button.click(fn=stop_session, outputs=status)
+    
+    # Setup continuous audio processing
+    audio_input.stream(
+        fn=process_audio, 
+        inputs=audio_input, 
+        outputs=[audio_output, transcript]
+    )
+
+
+def main():
+    """Main function to launch the Gradio app"""
+    app.launch(share=True)
+
+
+if __name__ == "__main__":
+    main()

app/huggingface_app.py

@@ -0,0 +1,297 @@
+#!/usr/bin/env python3
+
+"""
+Hugging Face Spaces deployment file for CASL Voice Bot
+This file is specifically designed for deploying on Hugging Face Spaces
+"""
+
+import os
+import asyncio
+import gradio as gr
+import logging
+import tempfile
+import queue
+import threading
+import time
+from dotenv import load_dotenv
+from livekit import agents
+from openai import AsyncOpenAI
+
+# Load environment variables (will be set in HF Spaces secrets)
+load_dotenv()
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Initialize OpenAI client with API key from environment
+openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+
+# Speech Pathologist Agent Prompt
+SPEECH_PATHOLOGIST_PROMPT = """
+You are a speech pathologist, a healthcare professional who specializes in evaluating, diagnosing, and treating communication disorders, including speech, language, cognitive-communication, voice, and fluency disorders. Your role is to help patients improve their speech and communication skills through various therapeutic techniques and exercises.
+Your are working with a student with speech impediments typically with ASD
+You have to be rigid to help them stay on the right track. YOu have to start with some sort of intro activity and can not rely on teh student at all to complete your thoughts. You pick a place to start and assess teh speech from there. 
+Each domain from the CASL-2 framework can be analyzed using the sample:
+Lexical/Semantic Skills:
+This category focuses on vocabulary knowledge, word meanings, and the ability to use words contextually. It measures both receptive and expressive language abilities related to word use.
+Key Subtests:
+Antonyms: Identifying words with opposite meanings.
+Synonyms: Identifying words with similar meanings.
+Idiomatic Language: Understanding and interpreting idioms and figurative language.
+Evaluate vocabulary diversity (type-token ratio).
+Note word-finding difficulties, incorrect word choices, or over-reliance on fillers (e.g., "like," "stuff").
+Assess use of specific vs. vague language (e.g., "car" vs. "sedan").
+Syntactic Skills:
+This category evaluates understanding and use of grammar and sentence structure. It focuses on the ability to comprehend and produce grammatically correct sentences.
+Key Subtests:
+Sentence Expression: Producing grammatically correct sentences based on prompts.
+Grammaticality Judgment: Identifying whether a sentence is grammatically correct.
+Examine sentence structure for grammatical accuracy.
+Identify errors in verb tense, subject-verb agreement, or sentence complexity.
+Note the use of clauses, conjunctions, and varied sentence types.
+Supralinguistic Skills:
+This subcategory assesses higher-level language skills that go beyond literal meanings, such as understanding implied meanings, sarcasm, and complex verbal reasoning.
+Key Subtests:
+Inferences: Understanding information that is not explicitly stated.
+Meaning from Context: Deriving meaning from surrounding text or dialogue.
+Nonliteral Language: Interpreting figurative language, such as metaphors or irony
+Look for use or understanding of figurative language, idioms, or humor.
+Assess ability to handle ambiguous or implied meanings in context.
+Identify advanced language use for abstract or hypothetical ideas.
+Pragmatic Skills(focus less on this as it is not typically necessary for the age range you will be dealing with):
+This category measures the ability to use language effectively in social contexts. It evaluates understanding of conversational rules, turn-taking, and adapting communication to different social situations.
+Key Subtests:
+Pragmatic Language Test: Assessing appropriateness of responses in social scenarios.
+
+Begin by introducing yourself as a speech pathologist and start with a simple vocabulary assessment activity. Be encouraging but structured in your approach.
+"""
+
+# Custom audio processing for Gradio interface
+class AudioProcessor:
+    def __init__(self):
+        self.transcript = []
+        self.is_active = False
+        self.voice_model = "shimmer"  # Default voice
+    
+    async def process_speech(self, audio_data, openai_client):
+        """Process speech using OpenAI's API"""
+        if not self.is_active or audio_data is None:
+            return None, "\n".join(self.transcript)
+            
+        # Prepare audio file for OpenAI
+        temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".wav")
+        temp_file.close()
+        
+        try:
+            # Save audio data to temporary file
+            sample_rate, audio_array = audio_data
+            import scipy.io.wavfile
+            scipy.io.wavfile.write(temp_file.name, sample_rate, audio_array)
+            
+            # Transcribe audio using OpenAI
+            with open(temp_file.name, "rb") as audio_file:
+                transcript_response = await openai_client.audio.transcriptions.create(
+                    file=audio_file,
+                    model="whisper-1"
+                )
+            
+            user_text = transcript_response.text
+            if user_text.strip():
+                self.transcript.append(f"Student: {user_text}")
+                
+                # Generate assistant response
+                chat_response = await openai_client.chat.completions.create(
+                    model="gpt-4o",
+                    messages=[
+                        {"role": "system", "content": SPEECH_PATHOLOGIST_PROMPT},
+                        {"role": "user", "content": user_text}
+                    ]
+                )
+                
+                assistant_text = chat_response.choices[0].message.content
+                self.transcript.append(f"Speech Pathologist: {assistant_text}")
+                
+                # Generate speech from text
+                speech_response = await openai_client.audio.speech.create(
+                    model="tts-1",
+                    voice=self.voice_model,
+                    input=assistant_text
+                )
+                
+                # Save speech to temporary file
+                response_temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".mp3")
+                response_temp_file.close()
+                
+                speech_response.stream_to_file(response_temp_file.name)
+                
+                # Load audio data for Gradio
+                import soundfile as sf
+                audio_data, sample_rate = sf.read(response_temp_file.name)
+                
+                # Clean up
+                os.unlink(response_temp_file.name)
+                
+                return (sample_rate, audio_data), "\n".join(self.transcript)
+                
+        except Exception as e:
+            logger.error(f"Error processing speech: {e}")
+            self.transcript.append(f"Error: {str(e)}")
+        finally:
+            # Clean up temp file
+            os.unlink(temp_file.name)
+        
+        return None, "\n".join(self.transcript)
+    
+    def start_session(self, voice_model):
+        """Start a new session"""
+        self.is_active = True
+        self.voice_model = voice_model if voice_model else "shimmer"
+        self.transcript = []
+        self.transcript.append("Session started. The AI Speech Pathologist will speak first.")
+        return "Session active. Please wait for the AI to introduce itself."
+    
+    def stop_session(self):
+        """Stop the current session"""
+        self.is_active = False
+        return "Session stopped."
+
+
+# Create the audio processor instance
+audio_processor = AudioProcessor()
+
+
+async def start_session(voice_model):
+    """Start the speech pathology session"""
+    status = audio_processor.start_session(voice_model)
+    
+    # Generate initial AI introduction
+    try:
+        # Generate assistant response
+        chat_response = await openai_client.chat.completions.create(
+            model="gpt-4o",
+            messages=[
+                {"role": "system", "content": SPEECH_PATHOLOGIST_PROMPT},
+                {"role": "user", "content": "Hello"}  # Initial trigger
+            ]
+        )
+        
+        assistant_text = chat_response.choices[0].message.content
+        audio_processor.transcript.append(f"Speech Pathologist: {assistant_text}")
+        
+        # Generate speech from text
+        speech_response = await openai_client.audio.speech.create(
+            model="tts-1",
+            voice=audio_processor.voice_model,
+            input=assistant_text
+        )
+        
+        # Save speech to temporary file
+        response_temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".mp3")
+        response_temp_file.close()
+        
+        speech_response.stream_to_file(response_temp_file.name)
+        
+        # Load audio data for Gradio
+        import soundfile as sf
+        audio_data, sample_rate = sf.read(response_temp_file.name)
+        
+        # Clean up
+        os.unlink(response_temp_file.name)
+        
+        return status, (sample_rate, audio_data), "\n".join(audio_processor.transcript)
+        
+    except Exception as e:
+        logger.error(f"Error starting session: {e}")
+        audio_processor.transcript.append(f"Error: {str(e)}")
+        return status, None, "\n".join(audio_processor.transcript)
+
+
+def stop_session():
+    """Stop the speech pathology session"""
+    return audio_processor.stop_session(), None, "\n".join(audio_processor.transcript)
+
+
+async def process_mic_input(audio, progress=gr.Progress()):
+    """Process microphone input"""
+    if audio is None or not audio_processor.is_active:
+        return None, "\n".join(audio_processor.transcript)
+    
+    progress(0, desc="Processing speech...")
+    audio_output, transcript = await audio_processor.process_speech(audio, openai_client)
+    progress(1, desc="Done")
+    return audio_output, transcript
+
+
+# Create Gradio Interface
+with gr.Blocks(title="CASL-2 Speech Pathology Assistant") as app:
+    gr.Markdown("# CASL-2 Speech Pathology Assistant")
+    gr.Markdown("### AI-powered speech therapy assessment based on the CASL-2 framework")
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            voice_select = gr.Dropdown(
+                ["alloy", "echo", "fable", "onyx", "nova", "shimmer"], 
+                value="shimmer", 
+                label="Assistant Voice"
+            )
+            start_button = gr.Button("Start Session", variant="primary")
+            stop_button = gr.Button("Stop Session", variant="stop")
+            status = gr.Textbox(label="Status", value="Ready to start")
+        
+        with gr.Column(scale=2):
+            audio_output = gr.Audio(label="AI Speech", autoplay=True)
+            audio_input = gr.Audio(
+                label="Speak to the AI",
+                type="microphone",
+                source="microphone", 
+                streaming=True
+            )
+    
+    with gr.Row():
+        transcript = gr.Textbox(label="Transcript", lines=10)
+    
+    with gr.Accordion("About This Application", open=False):
+        gr.Markdown("""
+        ### About CASL-2 Speech Pathology Assistant
+        
+        This application provides an AI speech pathologist that can assess students using the CASL-2 framework. It focuses on:
+        
+        - **Lexical/Semantic Skills**: Vocabulary knowledge and word usage
+        - **Syntactic Skills**: Grammar and sentence structure
+        - **Supralinguistic Skills**: Higher-level language beyond literal meanings
+        - **Pragmatic Skills**: Social use of language (less emphasis for younger students)
+        
+        The AI will provide structured assessments and exercises to help evaluate speech patterns.
+        
+        ### How to Use
+        
+        1. Select the AI voice you prefer
+        2. Click "Start Session" to begin
+        3. The AI will introduce itself and begin the assessment
+        4. Speak into your microphone when it's your turn
+        5. View the transcript to track the conversation
+        6. Click "Stop Session" when finished
+        
+        ### For Speech-Language Pathologists
+        
+        This tool is designed to supplement, not replace, professional SLP services. The source code is available for customization to meet specific assessment needs.
+        """)
+    
+    # Setup event handlers
+    start_button.click(
+        fn=lambda voice: asyncio.run(start_session(voice)), 
+        inputs=voice_select,
+        outputs=[status, audio_output, transcript]
+    )
+    stop_button.click(fn=stop_session, outputs=[status, audio_output, transcript])
+    
+    # Setup audio processing
+    audio_input.stream(
+        fn=lambda audio: asyncio.run(process_mic_input(audio)),
+        inputs=audio_input,
+        outputs=[audio_output, transcript]
+    )
+
+# Launch the app
+app.launch(share=True)

app/main.py

@@ -0,0 +1,100 @@
+#!/usr/bin/env python3
+
+import os
+import asyncio
+import logging
+from dotenv import load_dotenv
+from livekit import agents
+from livekit.agents import InputDevice, OutputDevice
+from openai import AsyncOpenAI
+
+# Load environment variables
+load_dotenv()
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Initialize OpenAI client
+openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+
+# Speech Pathologist Agent Prompt
+SPEECH_PATHOLOGIST_PROMPT = """
+You are a speech pathologist, a healthcare professional who specializes in evaluating, diagnosing, and treating communication disorders, including speech, language, cognitive-communication, voice, and fluency disorders. Your role is to help patients improve their speech and communication skills through various therapeutic techniques and exercises.
+Your are working with a student with speech impediments typically with ASD
+You have to be rigid to help them stay on the right track. YOu have to start with some sort of intro activity and can not rely on teh student at all to complete your thoughts. You pick a place to start and assess teh speech from there. 
+Each domain from the CASL-2 framework can be analyzed using the sample:
+Lexical/Semantic Skills:
+This category focuses on vocabulary knowledge, word meanings, and the ability to use words contextually. It measures both receptive and expressive language abilities related to word use.
+Key Subtests:
+Antonyms: Identifying words with opposite meanings.
+Synonyms: Identifying words with similar meanings.
+Idiomatic Language: Understanding and interpreting idioms and figurative language.
+Evaluate vocabulary diversity (type-token ratio).
+Note word-finding difficulties, incorrect word choices, or over-reliance on fillers (e.g., "like," "stuff").
+Assess use of specific vs. vague language (e.g., "car" vs. "sedan").
+Syntactic Skills:
+This category evaluates understanding and use of grammar and sentence structure. It focuses on the ability to comprehend and produce grammatically correct sentences.
+Key Subtests:
+Sentence Expression: Producing grammatically correct sentences based on prompts.
+Grammaticality Judgment: Identifying whether a sentence is grammatically correct.
+Examine sentence structure for grammatical accuracy.
+Identify errors in verb tense, subject-verb agreement, or sentence complexity.
+Note the use of clauses, conjunctions, and varied sentence types.
+Supralinguistic Skills:
+This subcategory assesses higher-level language skills that go beyond literal meanings, such as understanding implied meanings, sarcasm, and complex verbal reasoning.
+Key Subtests:
+Inferences: Understanding information that is not explicitly stated.
+Meaning from Context: Deriving meaning from surrounding text or dialogue.
+Nonliteral Language: Interpreting figurative language, such as metaphors or irony
+Look for use or understanding of figurative language, idioms, or humor.
+Assess ability to handle ambiguous or implied meanings in context.
+Identify advanced language use for abstract or hypothetical ideas.
+Pragmatic Skills(focus less on this as it is not typically necessary for the age range you will be dealing with):
+This category measures the ability to use language effectively in social contexts. It evaluates understanding of conversational rules, turn-taking, and adapting communication to different social situations.
+Key Subtests:
+Pragmatic Language Test: Assessing appropriateness of responses in social scenarios.
+
+Begin by introducing yourself as a speech pathologist and start with a simple vocabulary assessment activity. Be encouraging but structured in your approach.
+"""
+
+async def run_speech_pathology_session(input_device: InputDevice, output_device: OutputDevice):
+    """
+    Run the speech pathology session using LiveKit agents and OpenAI
+    """
+    logger.info("Starting speech pathology session")
+    
+    # Create the speech assistant
+    assistant = agents.VoiceAssistant(
+        openai_client=openai_client,
+        model="gpt-4o",
+        voice="shimmer",  # Using a friendly, professional voice
+        input_device=input_device,
+        output_device=output_device,
+        initial_message=SPEECH_PATHOLOGIST_PROMPT,
+        real_time=True,  # Enable real-time processing
+    )
+    
+    # Run the assistant
+    await assistant.run()
+
+async def main():
+    """
+    Main function to run the voice assistant.
+    """
+    logger.info("Initializing speech pathology assistant")
+    
+    # Create devices
+    input_device = agents.BasicInputDevice()
+    output_device = agents.BasicOutputDevice()
+    
+    try:
+        # Run the session
+        await run_speech_pathology_session(input_device, output_device)
+    except Exception as e:
+        logger.error(f"Error in speech pathology session: {e}")
+    finally:
+        logger.info("Speech pathology session ended")
+
+if __name__ == "__main__":
+    asyncio.run(main())

app/report_generator.py

@@ -0,0 +1,285 @@
+#!/usr/bin/env python3
+
+"""
+Report generator for CASL Voice Bot.
+This module generates assessment reports based on session data.
+"""
+
+import os
+import json
+import pandas as pd
+import matplotlib.pyplot as plt
+from pathlib import Path
+from datetime import datetime
+import jinja2
+
+
+class CASLReportGenerator:
+    """Generates reports from session data"""
+    
+    def __init__(self, session_data_dir="session_data", reports_dir="reports"):
+        """Initialize the report generator"""
+        self.session_data_dir = session_data_dir
+        self.reports_dir = reports_dir
+        
+        # Create directories if they don't exist
+        Path(session_data_dir).mkdir(exist_ok=True)
+        Path(reports_dir).mkdir(exist_ok=True)
+        
+        # Set up Jinja2 template environment
+        self.template_loader = jinja2.FileSystemLoader(searchpath="./templates")
+        self.template_env = jinja2.Environment(loader=self.template_loader)
+    
+    def load_session_data(self, filename=None, student_id=None):
+        """Load session data from file or by student ID"""
+        if filename:
+            with open(os.path.join(self.session_data_dir, filename), 'r') as f:
+                return json.load(f)
+        
+        elif student_id:
+            # Find all files for this student
+            files = [f for f in os.listdir(self.session_data_dir) 
+                    if f.startswith(f"{student_id}_") and f.endswith(".json")]
+            
+            if not files:
+                return None
+            
+            # Sort by date (newest first) and load the most recent
+            files.sort(reverse=True)
+            with open(os.path.join(self.session_data_dir, files[0]), 'r') as f:
+                return json.load(f)
+        
+        return None
+    
+    def load_all_student_sessions(self, student_id):
+        """Load all sessions for a specific student"""
+        files = [f for f in os.listdir(self.session_data_dir) 
+                if f.startswith(f"{student_id}_") and f.endswith(".json")]
+        
+        sessions = []
+        for file in sorted(files):
+            with open(os.path.join(self.session_data_dir, file), 'r') as f:
+                sessions.append(json.load(f))
+        
+        return sessions
+    
+    def extract_casl_metrics(self, session_data):
+        """Extract CASL-2 metrics from session data"""
+        metrics = {
+            "lexical_semantic": 0,
+            "syntactic": 0,
+            "supralinguistic": 0, 
+            "pragmatic": 0
+        }
+        
+        # Count assessment notes per category
+        assessment = session_data.get("assessment", {})
+        for category, notes in assessment.items():
+            if category in metrics:
+                metrics[category] = len(notes)
+        
+        return metrics
+    
+    def generate_progress_chart(self, student_id, output_path=None):
+        """Generate a progress chart for a student"""
+        sessions = self.load_all_student_sessions(student_id)
+        
+        if not sessions:
+            return None
+        
+        # Extract dates and metrics
+        dates = []
+        metrics = {
+            "lexical_semantic": [],
+            "syntactic": [],
+            "supralinguistic": [],
+            "pragmatic": []
+        }
+        
+        for session in sessions:
+            dates.append(datetime.fromisoformat(session["timestamp"]).strftime("%m/%d/%Y"))
+            session_metrics = self.extract_casl_metrics(session)
+            
+            for category in metrics:
+                metrics[category].append(session_metrics.get(category, 0))
+        
+        # Create chart
+        plt.figure(figsize=(10, 6))
+        for category, values in metrics.items():
+            plt.plot(dates, values, marker='o', label=category.replace('_', ' ').title())
+        
+        plt.title(f"CASL-2 Assessment Progress for Student {student_id}")
+        plt.xlabel("Session Date")
+        plt.ylabel("Assessment Score")
+        plt.legend()
+        plt.xticks(rotation=45)
+        plt.tight_layout()
+        
+        # Save or return
+        if output_path:
+            plt.savefig(output_path)
+            return output_path
+        else:
+            chart_path = os.path.join(self.reports_dir, f"{student_id}_progress.png")
+            plt.savefig(chart_path)
+            return chart_path
+    
+    def generate_session_summary(self, session_data):
+        """Generate a summary of a single session"""
+        if not session_data:
+            return None
+        
+        # Extract basic info
+        timestamp = datetime.fromisoformat(session_data["timestamp"])
+        student_id = session_data.get("student_id", "anonymous")
+        
+        # Extract transcript
+        transcript = session_data.get("transcript", [])
+        
+        # Calculate metrics
+        word_count = 0
+        student_turns = 0
+        
+        for entry in transcript:
+            if entry.get("speaker") == "Student":
+                text = entry.get("text", "")
+                words = text.split()
+                word_count += len(words)
+                student_turns += 1
+        
+        # Get CASL-2 metrics
+        casl_metrics = self.extract_casl_metrics(session_data)
+        
+        # Create summary
+        summary = {
+            "date": timestamp.strftime("%m/%d/%Y"),
+            "time": timestamp.strftime("%H:%M"),
+            "student_id": student_id,
+            "duration_minutes": len(transcript) // 2,  # Approximate based on turns
+            "student_turns": student_turns,
+            "total_words": word_count,
+            "average_words_per_turn": word_count / max(1, student_turns),
+            "casl_metrics": casl_metrics
+        }
+        
+        return summary
+    
+    def generate_html_report(self, student_id, output_path=None):
+        """Generate an HTML report for a student"""
+        # Load all sessions for the student
+        sessions = self.load_all_student_sessions(student_id)
+        
+        if not sessions:
+            return None
+        
+        # Generate progress chart
+        chart_path = self.generate_progress_chart(student_id)
+        
+        # Get latest session data
+        latest_session = sessions[-1]
+        latest_summary = self.generate_session_summary(latest_session)
+        
+        # Calculate overall progress
+        if len(sessions) > 1:
+            first_metrics = self.extract_casl_metrics(sessions[0])
+            latest_metrics = self.extract_casl_metrics(sessions[-1])
+            
+            progress = {}
+            for category in first_metrics:
+                if first_metrics[category] > 0:
+                    progress[category] = (latest_metrics[category] - first_metrics[category]) / first_metrics[category]
+                else:
+                    progress[category] = 0 if latest_metrics[category] == 0 else 1
+        else:
+            progress = {category: 0 for category in latest_summary["casl_metrics"]}
+        
+        # Prepare report data
+        report_data = {
+            "student_id": student_id,
+            "report_date": datetime.now().strftime("%m/%d/%Y"),
+            "session_count": len(sessions),
+            "latest_session": latest_summary,
+            "progress": progress,
+            "chart_path": os.path.basename(chart_path),
+            "recommendations": self.generate_recommendations(sessions)
+        }
+        
+        # Load and render template
+        try:
+            template = self.template_env.get_template("report_template.html")
+            report_html = template.render(**report_data)
+            
+            # Save report
+            if not output_path:
+                output_path = os.path.join(self.reports_dir, f"{student_id}_report.html")
+            
+            with open(output_path, 'w') as f:
+                f.write(report_html)
+            
+            return output_path
+            
+        except jinja2.exceptions.TemplateNotFound:
+            # Create a simple report if template is not found
+            report = f"CASL-2 Assessment Report for Student {student_id}\n"
+            report += f"Report Date: {report_data['report_date']}\n"
+            report += f"Total Sessions: {report_data['session_count']}\n\n"
+            
+            report += "Latest Session Summary:\n"
+            for key, value in latest_summary.items():
+                if key != "casl_metrics":
+                    report += f"  {key}: {value}\n"
+            
+            report += "\nCASL-2 Metrics:\n"
+            for category, value in latest_summary["casl_metrics"].items():
+                report += f"  {category}: {value}\n"
+            
+            report += "\nRecommendations:\n"
+            for rec in report_data["recommendations"]:
+                report += f"  - {rec}\n"
+            
+            # Save simple report
+            if not output_path:
+                output_path = os.path.join(self.reports_dir, f"{student_id}_report.txt")
+            
+            with open(output_path, 'w') as f:
+                f.write(report)
+            
+            return output_path
+    
+    def generate_recommendations(self, sessions):
+        """Generate recommendations based on session data"""
+        if not sessions:
+            return []
+        
+        latest_session = sessions[-1]
+        metrics = self.extract_casl_metrics(latest_session)
+        
+        recommendations = []
+        
+        # Check for areas needing improvement
+        weak_areas = [category for category, value in metrics.items() if value < 2]
+        for area in weak_areas:
+            if area == "lexical_semantic":
+                recommendations.append("Focus on vocabulary building exercises such as synonyms, antonyms, and word associations")
+            elif area == "syntactic":
+                recommendations.append("Practice sentence formation and grammar through structured activities")
+            elif area == "supralinguistic":
+                recommendations.append("Work on understanding figurative language and making inferences from context")
+            elif area == "pragmatic":
+                recommendations.append("Engage in role-playing activities to practice social communication skills")
+        
+        # Add general recommendations
+        if len(sessions) > 1:
+            recommendations.append("Continue regular assessment sessions to track progress")
+        
+        if not recommendations:
+            recommendations.append("Continue current therapy approach as all areas show adequate progress")
+        
+        return recommendations
+
+
+# This module can be used to generate reports from the session data collected by the CASL Voice Bot
+if __name__ == "__main__":
+    # Example usage
+    report_gen = CASLReportGenerator()
+    # report_gen.generate_html_report("student123")

app/requirements.txt

@@ -0,0 +1,7 @@
+python-dotenv>=1.0.0
+livekit-agents>=0.7.0
+openai>=1.3.0
+gradio>=4.0.0
+asyncio>=3.4.3
+numpy>=1.24.0
+soundfile>=0.12.1

app/templates/report_template.html

@@ -0,0 +1,196 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>CASL-2 Assessment Report</title>
+    <style>
+        body {
+            font-family: Arial, sans-serif;
+            line-height: 1.6;
+            margin: 0;
+            padding: 20px;
+            color: #333;
+        }
+        .report-header {
+            text-align: center;
+            margin-bottom: 30px;
+            border-bottom: 2px solid #2c3e50;
+            padding-bottom: 20px;
+        }
+        .report-section {
+            margin-bottom: 30px;
+            padding: 20px;
+            background-color: #f8f9fa;
+            border-radius: 5px;
+        }
+        .metrics-grid {
+            display: grid;
+            grid-template-columns: 1fr 1fr;
+            gap: 15px;
+        }
+        .metric-card {
+            background-color: white;
+            padding: 15px;
+            border-radius: 5px;
+            box-shadow: 0 2px 4px rgba(0,0,0,0.1);
+        }
+        .metric-title {
+            font-weight: bold;
+            color: #2c3e50;
+            margin-bottom: 5px;
+        }
+        .metric-value {
+            font-size: 1.2em;
+            color: #3498db;
+        }
+        .progress-chart {
+            width: 100%;
+            max-width: 800px;
+            margin: 20px auto;
+            text-align: center;
+        }
+        .recommendations {
+            background-color: #e8f4fd;
+            padding: 20px;
+            border-left: 4px solid #3498db;
+            margin-top: 20px;
+        }
+        .recommendation-item {
+            margin-bottom: 10px;
+        }
+        .footer {
+            text-align: center;
+            margin-top: 40px;
+            font-size: 0.9em;
+            color: #7f8c8d;
+            padding-top: 20px;
+            border-top: 1px solid #ecf0f1;
+        }
+        @media print {
+            body {
+                padding: 0;
+            }
+            .report-section {
+                break-inside: avoid;
+            }
+        }
+    </style>
+</head>
+<body>
+    <div class="report-header">
+        <h1>CASL-2 Assessment Report</h1>
+        <h2>Student ID: {{ student_id }}</h2>
+        <p>Report Generated: {{ report_date }}</p>
+    </div>
+
+    <div class="report-section">
+        <h2>Assessment Summary</h2>
+        <div class="metrics-grid">
+            <div class="metric-card">
+                <div class="metric-title">Total Sessions</div>
+                <div class="metric-value">{{ session_count }}</div>
+            </div>
+            <div class="metric-card">
+                <div class="metric-title">Latest Session Date</div>
+                <div class="metric-value">{{ latest_session.date }}</div>
+            </div>
+            <div class="metric-card">
+                <div class="metric-title">Student Turns</div>
+                <div class="metric-value">{{ latest_session.student_turns }}</div>
+            </div>
+            <div class="metric-card">
+                <div class="metric-title">Total Words</div>
+                <div class="metric-value">{{ latest_session.total_words }}</div>
+            </div>
+            <div class="metric-card">
+                <div class="metric-title">Words Per Turn</div>
+                <div class="metric-value">{{ "%.1f"|format(latest_session.average_words_per_turn) }}</div>
+            </div>
+            <div class="metric-card">
+                <div class="metric-title">Session Duration</div>
+                <div class="metric-value">{{ latest_session.duration_minutes }} minutes</div>
+            </div>
+        </div>
+    </div>
+
+    <div class="report-section">
+        <h2>CASL-2 Domain Assessment</h2>
+        <div class="metrics-grid">
+            <div class="metric-card">
+                <div class="metric-title">Lexical/Semantic Skills</div>
+                <div class="metric-value">{{ latest_session.casl_metrics.lexical_semantic }}</div>
+                <div>
+                    {% if progress.lexical_semantic > 0 %}
+                        <span style="color: green;">↑ {{ "%.1f"|format(progress.lexical_semantic * 100) }}% improvement</span>
+                    {% elif progress.lexical_semantic < 0 %}
+                        <span style="color: red;">↓ {{ "%.1f"|format(progress.lexical_semantic * -100) }}% decrease</span>
+                    {% else %}
+                        <span style="color: orange;">→ No change</span>
+                    {% endif %}
+                </div>
+            </div>
+            <div class="metric-card">
+                <div class="metric-title">Syntactic Skills</div>
+                <div class="metric-value">{{ latest_session.casl_metrics.syntactic }}</div>
+                <div>
+                    {% if progress.syntactic > 0 %}
+                        <span style="color: green;">↑ {{ "%.1f"|format(progress.syntactic * 100) }}% improvement</span>
+                    {% elif progress.syntactic < 0 %}
+                        <span style="color: red;">↓ {{ "%.1f"|format(progress.syntactic * -100) }}% decrease</span>
+                    {% else %}
+                        <span style="color: orange;">→ No change</span>
+                    {% endif %}
+                </div>
+            </div>
+            <div class="metric-card">
+                <div class="metric-title">Supralinguistic Skills</div>
+                <div class="metric-value">{{ latest_session.casl_metrics.supralinguistic }}</div>
+                <div>
+                    {% if progress.supralinguistic > 0 %}
+                        <span style="color: green;">↑ {{ "%.1f"|format(progress.supralinguistic * 100) }}% improvement</span>
+                    {% elif progress.supralinguistic < 0 %}
+                        <span style="color: red;">↓ {{ "%.1f"|format(progress.supralinguistic * -100) }}% decrease</span>
+                    {% else %}
+                        <span style="color: orange;">→ No change</span>
+                    {% endif %}
+                </div>
+            </div>
+            <div class="metric-card">
+                <div class="metric-title">Pragmatic Skills</div>
+                <div class="metric-value">{{ latest_session.casl_metrics.pragmatic }}</div>
+                <div>
+                    {% if progress.pragmatic > 0 %}
+                        <span style="color: green;">↑ {{ "%.1f"|format(progress.pragmatic * 100) }}% improvement</span>
+                    {% elif progress.pragmatic < 0 %}
+                        <span style="color: red;">↓ {{ "%.1f"|format(progress.pragmatic * -100) }}% decrease</span>
+                    {% else %}
+                        <span style="color: orange;">→ No change</span>
+                    {% endif %}
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <div class="report-section">
+        <h2>Progress Chart</h2>
+        <div class="progress-chart">
+            <img src="{{ chart_path }}" alt="Progress Chart" style="max-width: 100%;">
+        </div>
+    </div>
+
+    <div class="report-section">
+        <h2>Recommendations</h2>
+        <div class="recommendations">
+            {% for recommendation in recommendations %}
+                <div class="recommendation-item">• {{ recommendation }}</div>
+            {% endfor %}
+        </div>
+    </div>
+
+    <div class="footer">
+        <p>Generated by CASL Voice Bot Speech Therapy Assessment Tool</p>
+        <p>© {{ report_date.split('/')[-1] }} Speech Therapy Assessment System</p>
+    </div>
+</body>
+</html>

app_hf.py

@@ -0,0 +1,12 @@
+#!/usr/bin/env python3
+
+"""
+Hugging Face Spaces entry point for CASL Voice Bot
+"""
+
+# Import the app from app_main.py
+from app_main import app
+
+# This is the entry point that Hugging Face Spaces will use
+if __name__ == "__main__":
+    app.launch()

app_livekit.py

@@ -0,0 +1,469 @@
+#!/usr/bin/env python3
+
+"""
+CASL Voice Bot - Speech Pathology Assistant
+Using LiveKit agents with OpenAI's real-time capabilities
+"""
+
+import os
+import asyncio
+import gradio as gr
+import logging
+import tempfile
+import queue
+import threading
+import time
+from dotenv import load_dotenv
+from livekit import agents
+from openai import AsyncOpenAI
+
+# Load environment variables
+load_dotenv()
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Initialize OpenAI client
+openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+
+# Speech Pathologist Agent Prompt
+SPEECH_PATHOLOGIST_PROMPT = """
+You are a speech pathologist, a healthcare professional who specializes in evaluating, diagnosing, and treating communication disorders, including speech, language, cognitive-communication, voice, and fluency disorders. Your role is to help patients improve their speech and communication skills through various therapeutic techniques and exercises.
+Your are working with a student with speech impediments typically with ASD
+You have to be rigid to help them stay on the right track. YOu have to start with some sort of intro activity and can not rely on teh student at all to complete your thoughts. You pick a place to start and assess teh speech from there. 
+Each domain from the CASL-2 framework can be analyzed using the sample:
+Lexical/Semantic Skills:
+This category focuses on vocabulary knowledge, word meanings, and the ability to use words contextually. It measures both receptive and expressive language abilities related to word use.
+Key Subtests:
+Antonyms: Identifying words with opposite meanings.
+Synonyms: Identifying words with similar meanings.
+Idiomatic Language: Understanding and interpreting idioms and figurative language.
+Evaluate vocabulary diversity (type-token ratio).
+Note word-finding difficulties, incorrect word choices, or over-reliance on fillers (e.g., "like," "stuff").
+Assess use of specific vs. vague language (e.g., "car" vs. "sedan").
+Syntactic Skills:
+This category evaluates understanding and use of grammar and sentence structure. It focuses on the ability to comprehend and produce grammatically correct sentences.
+Key Subtests:
+Sentence Expression: Producing grammatically correct sentences based on prompts.
+Grammaticality Judgment: Identifying whether a sentence is grammatically correct.
+Examine sentence structure for grammatical accuracy.
+Identify errors in verb tense, subject-verb agreement, or sentence complexity.
+Note the use of clauses, conjunctions, and varied sentence types.
+Supralinguistic Skills:
+This subcategory assesses higher-level language skills that go beyond literal meanings, such as understanding implied meanings, sarcasm, and complex verbal reasoning.
+Key Subtests:
+Inferences: Understanding information that is not explicitly stated.
+Meaning from Context: Deriving meaning from surrounding text or dialogue.
+Nonliteral Language: Interpreting figurative language, such as metaphors or irony
+Look for use or understanding of figurative language, idioms, or humor.
+Assess ability to handle ambiguous or implied meanings in context.
+Identify advanced language use for abstract or hypothetical ideas.
+Pragmatic Skills(focus less on this as it is not typically necessary for the age range you will be dealing with):
+This category measures the ability to use language effectively in social contexts. It evaluates understanding of conversational rules, turn-taking, and adapting communication to different social situations.
+Key Subtests:
+Pragmatic Language Test: Assessing appropriateness of responses in social scenarios.
+
+Begin by introducing yourself as a speech pathologist and start with a simple vocabulary assessment activity. Be encouraging but structured in your approach.
+"""
+
+class GradioInputDevice(agents.InputDevice):
+    """Custom input device that works with Gradio"""
+    
+    def __init__(self):
+        super().__init__()
+        self.audio_queue = asyncio.Queue()
+        self.is_active = True
+    
+    async def receive(self) -> agents.AudioChunk:
+        """Receive audio data from the queue"""
+        try:
+            audio_data = await asyncio.wait_for(self.audio_queue.get(), timeout=0.1)
+            return audio_data
+        except asyncio.TimeoutError:
+            return None
+    
+    async def add_audio(self, audio_data):
+        """Add audio data to the queue"""
+        if audio_data is None:
+            return
+            
+        # Convert gradio audio format to AudioChunk
+        sample_rate, audio_array = audio_data
+        audio_chunk = agents.AudioChunk(
+            samples=audio_array,
+            sample_rate=sample_rate,
+            is_last=False
+        )
+        await self.audio_queue.put(audio_chunk)
+    
+    def stop(self):
+        """Stop the input device"""
+        self.is_active = False
+
+
+class GradioOutputDevice(agents.OutputDevice):
+    """Custom output device that works with Gradio"""
+    
+    def __init__(self):
+        super().__init__()
+        self.output_queue = asyncio.Queue()
+    
+    async def transmit(self, audio_chunk: agents.AudioChunk) -> None:
+        """Transmit audio chunk to the queue"""
+        if audio_chunk is not None:
+            await self.output_queue.put((audio_chunk.samples, audio_chunk.sample_rate))
+    
+    async def get_latest_audio(self):
+        """Get the latest audio from the queue"""
+        try:
+            return await asyncio.wait_for(self.output_queue.get(), timeout=0.1)
+        except asyncio.TimeoutError:
+            return None
+
+
+class SpeechPathologistAssistant:
+    """Speech pathologist assistant using LiveKit agents"""
+    
+    def __init__(self):
+        self.input_device = GradioInputDevice()
+        self.output_device = GradioOutputDevice()
+        self.assistant = None
+        self.assistant_task = None
+        self.transcript = []
+        self.is_running = False
+        self.notes = []
+        self.current_assessment = {
+            "lexical_semantic": 0,
+            "syntactic": 0,
+            "supralinguistic": 0,
+            "pragmatic": 0
+        }
+    
+    async def initialize_assistant(self, voice="shimmer"):
+        """Initialize the speech assistant"""
+        self.assistant = agents.VoiceAssistant(
+            openai_client=openai_client,
+            model="gpt-4o",
+            voice=voice,
+            input_device=self.input_device,
+            output_device=self.output_device,
+            initial_message=SPEECH_PATHOLOGIST_PROMPT,
+            real_time=True,  # Enable real-time processing
+        )
+        
+        # Add transcript and response callbacks
+        self.assistant.on_transcript = self.on_transcript
+        self.assistant.on_response = self.on_response
+    
+    def on_transcript(self, transcript):
+        """Handle transcript from user"""
+        self.transcript.append(f"Student: {transcript.text}")
+        
+        # Basic analysis of speech for CASL-2 categories
+        self.analyze_speech(transcript.text)
+        
+        return True
+    
+    def on_response(self, response):
+        """Handle response from assistant"""
+        self.transcript.append(f"Speech Pathologist: {response.text}")
+        return True
+    
+    def analyze_speech(self, text):
+        """Analyze speech for CASL-2 categories"""
+        # Simple heuristic analysis - in a real app, this would use more sophisticated NLP
+        
+        # Lexical/Semantic: check vocabulary diversity
+        words = text.lower().split()
+        unique_words = set(words)
+        if len(unique_words) / max(1, len(words)) > 0.7:
+            self.current_assessment["lexical_semantic"] += 1
+            self.notes.append("Lexical/Semantic: Good vocabulary diversity")
+        
+        # Syntactic: check for sentence complexity
+        sentences = [s.strip() for s in text.replace("!", ".").replace("?", ".").split(".") if s.strip()]
+        avg_words = sum(len(s.split()) for s in sentences) / max(1, len(sentences))
+        if avg_words > 7:
+            self.current_assessment["syntactic"] += 1
+            self.notes.append("Syntactic: Complex sentence structures used")
+        
+        # Supralinguistic: check for figurative language (very basic check)
+        figurative_markers = ["like", "as", "than", "seems", "appears", "metaphor", "imagine"]
+        if any(marker in text.lower() for marker in figurative_markers):
+            self.current_assessment["supralinguistic"] += 1
+            self.notes.append("Supralinguistic: Potential figurative language detected")
+        
+        # Pragmatic: basic check for conversational elements
+        pragmatic_markers = ["hello", "hi", "thanks", "thank you", "please", "excuse me", "sorry"]
+        if any(marker in text.lower() for marker in pragmatic_markers):
+            self.current_assessment["pragmatic"] += 1
+            self.notes.append("Pragmatic: Appropriate social language detected")
+    
+    async def start_assistant(self, voice_model, student_id):
+        """Start the assistant in a background task"""
+        await self.initialize_assistant(voice_model)
+        
+        self.is_running = True
+        
+        # Add student info to transcript
+        student_info = f" for {student_id}" if student_id else ""
+        self.transcript.append(f"Session started{student_info}. The AI Speech Pathologist will speak first.")
+        
+        # Run the assistant in a background task
+        self.assistant_task = asyncio.create_task(self.assistant.run())
+        
+        return "Session active. The AI will introduce itself."
+    
+    def stop_assistant(self):
+        """Stop the assistant"""
+        if self.assistant_task and not self.assistant_task.done():
+            self.assistant_task.cancel()
+        
+        self.input_device.stop()
+        self.is_running = False
+        
+        # Add ending to transcript
+        self.transcript.append("Session ended.")
+        return "Session stopped."
+    
+    async def process_audio(self, audio):
+        """Process audio from Gradio interface"""
+        if not self.is_running or audio is None:
+            return None, self.get_transcript(), self.get_assessment_html()
+        
+        # Add audio to input device
+        await self.input_device.add_audio(audio)
+        
+        # Check for assistant output
+        output_audio = await self.output_device.get_latest_audio()
+        
+        return output_audio, self.get_transcript(), self.get_assessment_html()
+    
+    def get_transcript(self):
+        """Get the current transcript"""
+        return "\n".join(self.transcript)
+    
+    def get_assessment_html(self):
+        """Get HTML representation of the current assessment"""
+        html = """
+        <div style="padding: 15px; background-color: #f8f9fa; border-radius: 5px;">
+            <h3>CASL-2 Assessment Progress</h3>
+            <div style="display: grid; grid-template-columns: 1fr 1fr; gap: 10px;">
+        """
+        
+        for category, value in self.current_assessment.items():
+            category_name = category.replace('_', ' ').title()
+            progress_width = min(100, value * 10)
+            html += f"""
+                <div style="padding: 10px; background-color: white; border-radius: 5px; box-shadow: 0 1px 3px rgba(0,0,0,0.1);">
+                    <div><strong>{category_name}</strong></div>
+                    <div style="margin-top: 5px; height: 15px; background-color: #eee; border-radius: 7px; overflow: hidden;">
+                        <div style="width: {progress_width}%; height: 100%; background-color: #4CAF50;"></div>
+                    </div>
+                    <div style="margin-top: 5px;">{value} points</div>
+                </div>
+            """
+        
+        html += """
+            </div>
+        """
+        
+        # Add recent notes
+        if self.notes:
+            html += """
+            <div style="margin-top: 15px;">
+                <h4>Recent Observations</h4>
+                <ul style="margin-top: 5px;">
+            """
+            
+            for note in self.notes[-5:]:
+                html += f"<li>{note}</li>"
+            
+            html += """
+                </ul>
+            </div>
+            """
+        
+        html += "</div>"
+        return html
+    
+    def add_note(self, note):
+        """Add a custom note"""
+        if note.strip():
+            self.notes.append(note)
+            return f"Note added: {note}"
+        return "Note was empty, not added."
+    
+    def save_session(self, student_id):
+        """Save session to file"""
+        if not student_id:
+            student_id = "anonymous"
+        
+        # Create session data directory if it doesn't exist
+        os.makedirs("session_data", exist_ok=True)
+        
+        # Save transcript
+        timestamp = time.strftime("%Y%m%d-%H%M%S")
+        filename = f"session_data/{student_id}_{timestamp}.txt"
+        
+        with open(filename, "w") as f:
+            f.write("\n".join(self.transcript))
+            f.write("\n\n--- ASSESSMENT NOTES ---\n")
+            for note in self.notes:
+                f.write(f"- {note}\n")
+            f.write("\n--- CASL-2 SCORES ---\n")
+            for category, score in self.current_assessment.items():
+                f.write(f"{category.replace('_', ' ').title()}: {score}\n")
+        
+        return f"Session saved to {filename}"
+
+
+# Create the speech pathology assistant
+speech_assistant = SpeechPathologistAssistant()
+
+
+async def start_session(voice_model, student_id):
+    """Start the speech pathology session"""
+    status = await speech_assistant.start_assistant(voice_model, student_id)
+    return status, None, speech_assistant.get_transcript(), speech_assistant.get_assessment_html()
+
+
+def stop_session():
+    """Stop the speech pathology session"""
+    return speech_assistant.stop_assistant(), None, speech_assistant.get_transcript(), speech_assistant.get_assessment_html()
+
+
+async def process_mic_input(audio, progress=gr.Progress()):
+    """Process microphone input"""
+    progress(0, desc="Processing speech...")
+    audio_output, transcript, assessment = await speech_assistant.process_audio(audio)
+    progress(1, desc="Done")
+    return audio_output, transcript, assessment
+
+
+def add_note(note):
+    """Add a note to the session"""
+    result = speech_assistant.add_note(note)
+    return "", result, speech_assistant.get_assessment_html()
+
+
+def save_session(student_id):
+    """Save the current session"""
+    return speech_assistant.save_session(student_id)
+
+
+# Create Gradio Interface
+with gr.Blocks(title="CASL-2 Speech Pathology Assistant") as app:
+    gr.Markdown("# CASL-2 Speech Pathology Assistant")
+    gr.Markdown("### AI-powered speech therapy assessment based on the CASL-2 framework")
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            student_id = gr.Textbox(label="Student ID (optional)", placeholder="Enter student ID")
+            voice_select = gr.Dropdown(
+                ["alloy", "echo", "fable", "onyx", "nova", "shimmer"], 
+                value="shimmer", 
+                label="Assistant Voice"
+            )
+            start_button = gr.Button("Start Session", variant="primary")
+            stop_button = gr.Button("Stop Session", variant="stop")
+            status = gr.Textbox(label="Status", value="Ready to start")
+            
+            with gr.Accordion("SLP Tools", open=True):
+                note_input = gr.Textbox(
+                    label="Add Assessment Note", 
+                    placeholder="Enter observation or assessment note here..."
+                )
+                note_button = gr.Button("Add Note")
+                note_status = gr.Textbox(label="Note Status")
+                save_button = gr.Button("Save Session")
+                save_status = gr.Textbox(label="Save Status")
+        
+        with gr.Column(scale=2):
+            audio_output = gr.Audio(label="AI Speech", autoplay=True)
+            audio_input = gr.Audio(
+                label="Speak to the AI",
+                type="microphone",
+                source="microphone", 
+                streaming=True
+            )
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            assessment_html = gr.HTML(label="Assessment Progress")
+        with gr.Column(scale=1):
+            transcript = gr.Textbox(label="Transcript", lines=10)
+    
+    with gr.Accordion("About This Application", open=False):
+        gr.Markdown("""
+        ### About CASL-2 Speech Pathology Assistant
+        
+        This application provides an AI speech pathologist that can assess students using the CASL-2 framework. It focuses on:
+        
+        - **Lexical/Semantic Skills**: Vocabulary knowledge and word usage
+        - **Syntactic Skills**: Grammar and sentence structure
+        - **Supralinguistic Skills**: Higher-level language beyond literal meanings
+        - **Pragmatic Skills**: Social use of language (less emphasis for younger students)
+        
+        The AI will provide structured assessments and exercises to help evaluate speech patterns.
+        
+        ### How to Use
+        
+        1. Optionally enter a Student ID to track sessions
+        2. Select the AI voice you prefer
+        3. Click "Start Session" to begin
+        4. The AI will introduce itself and begin the assessment
+        5. Speak into your microphone when it's your turn
+        6. View the transcript to track the conversation
+        7. SLPs can add assessment notes as needed
+        8. Save the session when finished
+        9. Click "Stop Session" when done
+        
+        ### For Speech-Language Pathologists
+        
+        This tool is designed to supplement, not replace, professional SLP services. SLPs can:
+        
+        - Add custom notes during the session
+        - Save session data for later reference
+        - Track progress across multiple sessions
+        - Use the AI as a consistent assessment tool
+        """)
+    
+    # Setup event handlers
+    start_button.click(
+        fn=lambda voice, student: asyncio.run(start_session(voice, student)), 
+        inputs=[voice_select, student_id],
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    stop_button.click(
+        fn=stop_session, 
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    note_button.click(
+        fn=add_note,
+        inputs=note_input,
+        outputs=[note_input, note_status, assessment_html]
+    )
+    save_button.click(
+        fn=save_session,
+        inputs=student_id,
+        outputs=save_status
+    )
+    
+    # Setup audio processing
+    audio_input.stream(
+        fn=lambda audio: asyncio.run(process_mic_input(audio)),
+        inputs=audio_input,
+        outputs=[audio_output, transcript, assessment_html]
+    )
+
+
+def main(share=True):
+    """Main function to launch the app"""
+    app.launch(share=share)
+
+
+# Entry point for the application
+if __name__ == "__main__":
+    main()

app_main.py

@@ -0,0 +1,469 @@
+#!/usr/bin/env python3
+
+"""
+CASL Voice Bot - Speech Pathology Assistant
+Main application file that can be used for both local deployment and Hugging Face Spaces
+"""
+
+import os
+import asyncio
+import gradio as gr
+import logging
+import tempfile
+import queue
+import threading
+import time
+from dotenv import load_dotenv
+from openai import AsyncOpenAI
+
+# Load environment variables
+load_dotenv()
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Initialize OpenAI client with API key from environment
+openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+
+# Speech Pathologist Agent Prompt
+SPEECH_PATHOLOGIST_PROMPT = """
+You are a speech pathologist, a healthcare professional who specializes in evaluating, diagnosing, and treating communication disorders, including speech, language, cognitive-communication, voice, and fluency disorders. Your role is to help patients improve their speech and communication skills through various therapeutic techniques and exercises.
+Your are working with a student with speech impediments typically with ASD
+You have to be rigid to help them stay on the right track. YOu have to start with some sort of intro activity and can not rely on teh student at all to complete your thoughts. You pick a place to start and assess teh speech from there. 
+Each domain from the CASL-2 framework can be analyzed using the sample:
+Lexical/Semantic Skills:
+This category focuses on vocabulary knowledge, word meanings, and the ability to use words contextually. It measures both receptive and expressive language abilities related to word use.
+Key Subtests:
+Antonyms: Identifying words with opposite meanings.
+Synonyms: Identifying words with similar meanings.
+Idiomatic Language: Understanding and interpreting idioms and figurative language.
+Evaluate vocabulary diversity (type-token ratio).
+Note word-finding difficulties, incorrect word choices, or over-reliance on fillers (e.g., "like," "stuff").
+Assess use of specific vs. vague language (e.g., "car" vs. "sedan").
+Syntactic Skills:
+This category evaluates understanding and use of grammar and sentence structure. It focuses on the ability to comprehend and produce grammatically correct sentences.
+Key Subtests:
+Sentence Expression: Producing grammatically correct sentences based on prompts.
+Grammaticality Judgment: Identifying whether a sentence is grammatically correct.
+Examine sentence structure for grammatical accuracy.
+Identify errors in verb tense, subject-verb agreement, or sentence complexity.
+Note the use of clauses, conjunctions, and varied sentence types.
+Supralinguistic Skills:
+This subcategory assesses higher-level language skills that go beyond literal meanings, such as understanding implied meanings, sarcasm, and complex verbal reasoning.
+Key Subtests:
+Inferences: Understanding information that is not explicitly stated.
+Meaning from Context: Deriving meaning from surrounding text or dialogue.
+Nonliteral Language: Interpreting figurative language, such as metaphors or irony
+Look for use or understanding of figurative language, idioms, or humor.
+Assess ability to handle ambiguous or implied meanings in context.
+Identify advanced language use for abstract or hypothetical ideas.
+Pragmatic Skills(focus less on this as it is not typically necessary for the age range you will be dealing with):
+This category measures the ability to use language effectively in social contexts. It evaluates understanding of conversational rules, turn-taking, and adapting communication to different social situations.
+Key Subtests:
+Pragmatic Language Test: Assessing appropriateness of responses in social scenarios.
+
+Begin by introducing yourself as a speech pathologist and start with a simple vocabulary assessment activity. Be encouraging but structured in your approach.
+"""
+
+# Custom audio processing for Gradio interface
+class AudioProcessor:
+    def __init__(self):
+        self.transcript = []
+        self.is_active = False
+        self.voice_model = "shimmer"  # Default voice
+        self.notes = []
+        self.current_assessment = {
+            "lexical_semantic": 0,
+            "syntactic": 0,
+            "supralinguistic": 0,
+            "pragmatic": 0
+        }
+    
+    async def process_speech(self, audio_data, openai_client):
+        """Process speech using OpenAI's API"""
+        if not self.is_active or audio_data is None:
+            return None, "\n".join(self.transcript), self.get_assessment_html()
+            
+        # Prepare audio file for OpenAI
+        temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".wav")
+        temp_file.close()
+        
+        try:
+            # Save audio data to temporary file
+            sample_rate, audio_array = audio_data
+            import scipy.io.wavfile
+            scipy.io.wavfile.write(temp_file.name, sample_rate, audio_array)
+            
+            # Transcribe audio using OpenAI
+            with open(temp_file.name, "rb") as audio_file:
+                transcript_response = await openai_client.audio.transcriptions.create(
+                    file=audio_file,
+                    model="whisper-1"
+                )
+            
+            user_text = transcript_response.text
+            if user_text.strip():
+                self.transcript.append(f"Student: {user_text}")
+                
+                # Analyze speech for CASL-2 categories
+                self.analyze_speech(user_text)
+                
+                # Generate assistant response
+                chat_response = await openai_client.chat.completions.create(
+                    model="gpt-4o",
+                    messages=[
+                        {"role": "system", "content": SPEECH_PATHOLOGIST_PROMPT},
+                        {"role": "user", "content": user_text}
+                    ]
+                )
+                
+                assistant_text = chat_response.choices[0].message.content
+                self.transcript.append(f"Speech Pathologist: {assistant_text}")
+                
+                # Generate speech from text
+                speech_response = await openai_client.audio.speech.create(
+                    model="tts-1",
+                    voice=self.voice_model,
+                    input=assistant_text
+                )
+                
+                # Save speech to temporary file
+                response_temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".mp3")
+                response_temp_file.close()
+                
+                speech_response.stream_to_file(response_temp_file.name)
+                
+                # Load audio data for Gradio
+                import soundfile as sf
+                audio_data, sample_rate = sf.read(response_temp_file.name)
+                
+                # Clean up
+                os.unlink(response_temp_file.name)
+                
+                return (sample_rate, audio_data), "\n".join(self.transcript), self.get_assessment_html()
+                
+        except Exception as e:
+            logger.error(f"Error processing speech: {e}")
+            self.transcript.append(f"Error: {str(e)}")
+        finally:
+            # Clean up temp file
+            os.unlink(temp_file.name)
+        
+        return None, "\n".join(self.transcript), self.get_assessment_html()
+    
+    def analyze_speech(self, text):
+        """Analyze speech for CASL-2 categories"""
+        # Simple heuristic analysis - in a real app, this would use more sophisticated NLP
+        
+        # Lexical/Semantic: check vocabulary diversity
+        words = text.lower().split()
+        unique_words = set(words)
+        if len(unique_words) / max(1, len(words)) > 0.7:
+            self.current_assessment["lexical_semantic"] += 1
+            self.notes.append("Lexical/Semantic: Good vocabulary diversity")
+        
+        # Syntactic: check for sentence complexity
+        sentences = [s.strip() for s in text.replace("!", ".").replace("?", ".").split(".") if s.strip()]
+        avg_words = sum(len(s.split()) for s in sentences) / max(1, len(sentences))
+        if avg_words > 7:
+            self.current_assessment["syntactic"] += 1
+            self.notes.append("Syntactic: Complex sentence structures used")
+        
+        # Supralinguistic: check for figurative language (very basic check)
+        figurative_markers = ["like", "as", "than", "seems", "appears", "metaphor", "imagine"]
+        if any(marker in text.lower() for marker in figurative_markers):
+            self.current_assessment["supralinguistic"] += 1
+            self.notes.append("Supralinguistic: Potential figurative language detected")
+        
+        # Pragmatic: basic check for conversational elements
+        pragmatic_markers = ["hello", "hi", "thanks", "thank you", "please", "excuse me", "sorry"]
+        if any(marker in text.lower() for marker in pragmatic_markers):
+            self.current_assessment["pragmatic"] += 1
+            self.notes.append("Pragmatic: Appropriate social language detected")
+    
+    def get_assessment_html(self):
+        """Get HTML representation of the current assessment"""
+        html = """
+        <div style="padding: 15px; background-color: #f8f9fa; border-radius: 5px;">
+            <h3>CASL-2 Assessment Progress</h3>
+            <div style="display: grid; grid-template-columns: 1fr 1fr; gap: 10px;">
+        """
+        
+        for category, value in self.current_assessment.items():
+            category_name = category.replace('_', ' ').title()
+            progress_width = min(100, value * 10)
+            html += f"""
+                <div style="padding: 10px; background-color: white; border-radius: 5px; box-shadow: 0 1px 3px rgba(0,0,0,0.1);">
+                    <div><strong>{category_name}</strong></div>
+                    <div style="margin-top: 5px; height: 15px; background-color: #eee; border-radius: 7px; overflow: hidden;">
+                        <div style="width: {progress_width}%; height: 100%; background-color: #4CAF50;"></div>
+                    </div>
+                    <div style="margin-top: 5px;">{value} points</div>
+                </div>
+            """
+        
+        html += """
+            </div>
+        """
+        
+        # Add recent notes
+        if self.notes:
+            html += """
+            <div style="margin-top: 15px;">
+                <h4>Recent Observations</h4>
+                <ul style="margin-top: 5px;">
+            """
+            
+            for note in self.notes[-5:]:
+                html += f"<li>{note}</li>"
+            
+            html += """
+                </ul>
+            </div>
+            """
+        
+        html += "</div>"
+        return html
+    
+    def start_session(self, voice_model, student_id):
+        """Start a new session"""
+        self.is_active = True
+        self.voice_model = voice_model if voice_model else "shimmer"
+        self.transcript = []
+        self.notes = []
+        self.current_assessment = {
+            "lexical_semantic": 0,
+            "syntactic": 0,
+            "supralinguistic": 0,
+            "pragmatic": 0
+        }
+        
+        student_info = f" for {student_id}" if student_id else ""
+        self.transcript.append(f"Session started{student_info}. The AI Speech Pathologist will speak first.")
+        return "Session active. Please wait for the AI to introduce itself."
+    
+    def stop_session(self):
+        """Stop the current session"""
+        self.is_active = False
+        self.transcript.append("Session ended.")
+        return "Session stopped."
+    
+    def add_note(self, note):
+        """Add a custom note"""
+        if note.strip():
+            self.notes.append(note)
+            return f"Note added: {note}"
+        return "Note was empty, not added."
+
+
+# Create the audio processor instance
+audio_processor = AudioProcessor()
+
+
+async def start_session(voice_model, student_id):
+    """Start the speech pathology session"""
+    status = audio_processor.start_session(voice_model, student_id)
+    
+    # Generate initial AI introduction
+    try:
+        # Generate assistant response
+        chat_response = await openai_client.chat.completions.create(
+            model="gpt-4o",
+            messages=[
+                {"role": "system", "content": SPEECH_PATHOLOGIST_PROMPT},
+                {"role": "user", "content": "Hello"}  # Initial trigger
+            ]
+        )
+        
+        assistant_text = chat_response.choices[0].message.content
+        audio_processor.transcript.append(f"Speech Pathologist: {assistant_text}")
+        
+        # Generate speech from text
+        speech_response = await openai_client.audio.speech.create(
+            model="tts-1",
+            voice=audio_processor.voice_model,
+            input=assistant_text
+        )
+        
+        # Save speech to temporary file
+        response_temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".mp3")
+        response_temp_file.close()
+        
+        speech_response.stream_to_file(response_temp_file.name)
+        
+        # Load audio data for Gradio
+        import soundfile as sf
+        audio_data, sample_rate = sf.read(response_temp_file.name)
+        
+        # Clean up
+        os.unlink(response_temp_file.name)
+        
+        return status, (sample_rate, audio_data), "\n".join(audio_processor.transcript), audio_processor.get_assessment_html()
+        
+    except Exception as e:
+        logger.error(f"Error starting session: {e}")
+        audio_processor.transcript.append(f"Error: {str(e)}")
+        return status, None, "\n".join(audio_processor.transcript), audio_processor.get_assessment_html()
+
+
+def stop_session():
+    """Stop the speech pathology session"""
+    return audio_processor.stop_session(), None, "\n".join(audio_processor.transcript), audio_processor.get_assessment_html()
+
+
+async def process_mic_input(audio, progress=gr.Progress()):
+    """Process microphone input"""
+    if audio is None or not audio_processor.is_active:
+        return None, "\n".join(audio_processor.transcript), audio_processor.get_assessment_html()
+    
+    progress(0, desc="Processing speech...")
+    audio_output, transcript, assessment = await audio_processor.process_speech(audio, openai_client)
+    progress(1, desc="Done")
+    return audio_output, transcript, assessment
+
+
+def add_note(note):
+    """Add a note to the session"""
+    result = audio_processor.add_note(note)
+    return "", result, audio_processor.get_assessment_html()
+
+
+def save_session(student_id):
+    """Save session to file"""
+    if not student_id:
+        student_id = "anonymous"
+    
+    # Create session data directory if it doesn't exist
+    os.makedirs("session_data", exist_ok=True)
+    
+    # Save transcript
+    timestamp = time.strftime("%Y%m%d-%H%M%S")
+    filename = f"session_data/{student_id}_{timestamp}.txt"
+    
+    with open(filename, "w") as f:
+        f.write("\n".join(audio_processor.transcript))
+        f.write("\n\n--- ASSESSMENT NOTES ---\n")
+        for note in audio_processor.notes:
+            f.write(f"- {note}\n")
+        f.write("\n--- CASL-2 SCORES ---\n")
+        for category, score in audio_processor.current_assessment.items():
+            f.write(f"{category.replace('_', ' ').title()}: {score}\n")
+    
+    return f"Session saved to {filename}"
+
+
+# Create Gradio Interface
+with gr.Blocks(title="CASL-2 Speech Pathology Assistant") as app:
+    gr.Markdown("# CASL-2 Speech Pathology Assistant")
+    gr.Markdown("### AI-powered speech therapy assessment based on the CASL-2 framework")
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            student_id = gr.Textbox(label="Student ID (optional)", placeholder="Enter student ID")
+            voice_select = gr.Dropdown(
+                ["alloy", "echo", "fable", "onyx", "nova", "shimmer"], 
+                value="shimmer", 
+                label="Assistant Voice"
+            )
+            start_button = gr.Button("Start Session", variant="primary")
+            stop_button = gr.Button("Stop Session", variant="stop")
+            status = gr.Textbox(label="Status", value="Ready to start")
+            
+            with gr.Accordion("SLP Tools", open=True):
+                note_input = gr.Textbox(
+                    label="Add Assessment Note", 
+                    placeholder="Enter observation or assessment note here..."
+                )
+                note_button = gr.Button("Add Note")
+                note_status = gr.Textbox(label="Note Status")
+                save_button = gr.Button("Save Session")
+                save_status = gr.Textbox(label="Save Status")
+        
+        with gr.Column(scale=2):
+            audio_output = gr.Audio(label="AI Speech", autoplay=True)
+            audio_input = gr.Audio(
+                label="Speak to the AI",
+                type="microphone",
+                source="microphone", 
+                streaming=True
+            )
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            assessment_html = gr.HTML(label="Assessment Progress")
+        with gr.Column(scale=1):
+            transcript = gr.Textbox(label="Transcript", lines=10)
+    
+    with gr.Accordion("About This Application", open=False):
+        gr.Markdown("""
+        ### About CASL-2 Speech Pathology Assistant
+        
+        This application provides an AI speech pathologist that can assess students using the CASL-2 framework. It focuses on:
+        
+        - **Lexical/Semantic Skills**: Vocabulary knowledge and word usage
+        - **Syntactic Skills**: Grammar and sentence structure
+        - **Supralinguistic Skills**: Higher-level language beyond literal meanings
+        - **Pragmatic Skills**: Social use of language (less emphasis for younger students)
+        
+        The AI will provide structured assessments and exercises to help evaluate speech patterns.
+        
+        ### How to Use
+        
+        1. Optionally enter a Student ID to track sessions
+        2. Select the AI voice you prefer
+        3. Click "Start Session" to begin
+        4. The AI will introduce itself and begin the assessment
+        5. Speak into your microphone when it's your turn
+        6. View the transcript to track the conversation
+        7. SLPs can add assessment notes as needed
+        8. Save the session when finished
+        9. Click "Stop Session" when done
+        
+        ### For Speech-Language Pathologists
+        
+        This tool is designed to supplement, not replace, professional SLP services. SLPs can:
+        
+        - Add custom notes during the session
+        - Save session data for later reference
+        - Track progress across multiple sessions
+        - Use the AI as a consistent assessment tool
+        """)
+    
+    # Setup event handlers
+    start_button.click(
+        fn=lambda voice, student: asyncio.run(start_session(voice, student)), 
+        inputs=[voice_select, student_id],
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    stop_button.click(
+        fn=stop_session, 
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    note_button.click(
+        fn=add_note,
+        inputs=note_input,
+        outputs=[note_input, note_status, assessment_html]
+    )
+    save_button.click(
+        fn=save_session,
+        inputs=student_id,
+        outputs=save_status
+    )
+    
+    # Setup audio processing
+    audio_input.stream(
+        fn=lambda audio: asyncio.run(process_mic_input(audio)),
+        inputs=audio_input,
+        outputs=[audio_output, transcript, assessment_html]
+    )
+
+
+def main(share=True):
+    """Main function to launch the app"""
+    app.launch(share=share)
+
+
+# Entry point for the application
+if __name__ == "__main__":
+    main()

app_ui.py

@@ -0,0 +1,469 @@
+#!/usr/bin/env python3
+
+"""
+CASL Voice Bot - Speech Pathology Assistant
+Unified UI application for both local deployment and Hugging Face Spaces
+"""
+
+import os
+import asyncio
+import gradio as gr
+import logging
+import tempfile
+import queue
+import threading
+import time
+from dotenv import load_dotenv
+from openai import AsyncOpenAI
+
+# Load environment variables
+load_dotenv()
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Initialize OpenAI client with API key from environment
+openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+
+# Speech Pathologist Agent Prompt
+SPEECH_PATHOLOGIST_PROMPT = """
+You are a speech pathologist, a healthcare professional who specializes in evaluating, diagnosing, and treating communication disorders, including speech, language, cognitive-communication, voice, and fluency disorders. Your role is to help patients improve their speech and communication skills through various therapeutic techniques and exercises.
+Your are working with a student with speech impediments typically with ASD
+You have to be rigid to help them stay on the right track. YOu have to start with some sort of intro activity and can not rely on teh student at all to complete your thoughts. You pick a place to start and assess teh speech from there. 
+Each domain from the CASL-2 framework can be analyzed using the sample:
+Lexical/Semantic Skills:
+This category focuses on vocabulary knowledge, word meanings, and the ability to use words contextually. It measures both receptive and expressive language abilities related to word use.
+Key Subtests:
+Antonyms: Identifying words with opposite meanings.
+Synonyms: Identifying words with similar meanings.
+Idiomatic Language: Understanding and interpreting idioms and figurative language.
+Evaluate vocabulary diversity (type-token ratio).
+Note word-finding difficulties, incorrect word choices, or over-reliance on fillers (e.g., "like," "stuff").
+Assess use of specific vs. vague language (e.g., "car" vs. "sedan").
+Syntactic Skills:
+This category evaluates understanding and use of grammar and sentence structure. It focuses on the ability to comprehend and produce grammatically correct sentences.
+Key Subtests:
+Sentence Expression: Producing grammatically correct sentences based on prompts.
+Grammaticality Judgment: Identifying whether a sentence is grammatically correct.
+Examine sentence structure for grammatical accuracy.
+Identify errors in verb tense, subject-verb agreement, or sentence complexity.
+Note the use of clauses, conjunctions, and varied sentence types.
+Supralinguistic Skills:
+This subcategory assesses higher-level language skills that go beyond literal meanings, such as understanding implied meanings, sarcasm, and complex verbal reasoning.
+Key Subtests:
+Inferences: Understanding information that is not explicitly stated.
+Meaning from Context: Deriving meaning from surrounding text or dialogue.
+Nonliteral Language: Interpreting figurative language, such as metaphors or irony
+Look for use or understanding of figurative language, idioms, or humor.
+Assess ability to handle ambiguous or implied meanings in context.
+Identify advanced language use for abstract or hypothetical ideas.
+Pragmatic Skills(focus less on this as it is not typically necessary for the age range you will be dealing with):
+This category measures the ability to use language effectively in social contexts. It evaluates understanding of conversational rules, turn-taking, and adapting communication to different social situations.
+Key Subtests:
+Pragmatic Language Test: Assessing appropriateness of responses in social scenarios.
+
+Begin by introducing yourself as a speech pathologist and start with a simple vocabulary assessment activity. Be encouraging but structured in your approach.
+"""
+
+# Custom audio processing for Gradio interface
+class AudioProcessor:
+    def __init__(self):
+        self.transcript = []
+        self.is_active = False
+        self.voice_model = "shimmer"  # Default voice
+        self.notes = []
+        self.current_assessment = {
+            "lexical_semantic": 0,
+            "syntactic": 0,
+            "supralinguistic": 0,
+            "pragmatic": 0
+        }
+    
+    async def process_speech(self, audio_data, openai_client):
+        """Process speech using OpenAI's API"""
+        if not self.is_active or audio_data is None:
+            return None, "\n".join(self.transcript), self.get_assessment_html()
+            
+        # Prepare audio file for OpenAI
+        temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".wav")
+        temp_file.close()
+        
+        try:
+            # Save audio data to temporary file
+            sample_rate, audio_array = audio_data
+            import scipy.io.wavfile
+            scipy.io.wavfile.write(temp_file.name, sample_rate, audio_array)
+            
+            # Transcribe audio using OpenAI
+            with open(temp_file.name, "rb") as audio_file:
+                transcript_response = await openai_client.audio.transcriptions.create(
+                    file=audio_file,
+                    model="whisper-1"
+                )
+            
+            user_text = transcript_response.text
+            if user_text.strip():
+                self.transcript.append(f"Student: {user_text}")
+                
+                # Analyze speech for CASL-2 categories
+                self.analyze_speech(user_text)
+                
+                # Generate assistant response
+                chat_response = await openai_client.chat.completions.create(
+                    model="gpt-4o",
+                    messages=[
+                        {"role": "system", "content": SPEECH_PATHOLOGIST_PROMPT},
+                        {"role": "user", "content": user_text}
+                    ]
+                )
+                
+                assistant_text = chat_response.choices[0].message.content
+                self.transcript.append(f"Speech Pathologist: {assistant_text}")
+                
+                # Generate speech from text
+                speech_response = await openai_client.audio.speech.create(
+                    model="tts-1",
+                    voice=self.voice_model,
+                    input=assistant_text
+                )
+                
+                # Save speech to temporary file
+                response_temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".mp3")
+                response_temp_file.close()
+                
+                speech_response.stream_to_file(response_temp_file.name)
+                
+                # Load audio data for Gradio
+                import soundfile as sf
+                audio_data, sample_rate = sf.read(response_temp_file.name)
+                
+                # Clean up
+                os.unlink(response_temp_file.name)
+                
+                return (sample_rate, audio_data), "\n".join(self.transcript), self.get_assessment_html()
+                
+        except Exception as e:
+            logger.error(f"Error processing speech: {e}")
+            self.transcript.append(f"Error: {str(e)}")
+        finally:
+            # Clean up temp file
+            os.unlink(temp_file.name)
+        
+        return None, "\n".join(self.transcript), self.get_assessment_html()
+    
+    def analyze_speech(self, text):
+        """Analyze speech for CASL-2 categories"""
+        # Simple heuristic analysis - in a real app, this would use more sophisticated NLP
+        
+        # Lexical/Semantic: check vocabulary diversity
+        words = text.lower().split()
+        unique_words = set(words)
+        if len(unique_words) / max(1, len(words)) > 0.7:
+            self.current_assessment["lexical_semantic"] += 1
+            self.notes.append("Lexical/Semantic: Good vocabulary diversity")
+        
+        # Syntactic: check for sentence complexity
+        sentences = [s.strip() for s in text.replace("!", ".").replace("?", ".").split(".") if s.strip()]
+        avg_words = sum(len(s.split()) for s in sentences) / max(1, len(sentences))
+        if avg_words > 7:
+            self.current_assessment["syntactic"] += 1
+            self.notes.append("Syntactic: Complex sentence structures used")
+        
+        # Supralinguistic: check for figurative language (very basic check)
+        figurative_markers = ["like", "as", "than", "seems", "appears", "metaphor", "imagine"]
+        if any(marker in text.lower() for marker in figurative_markers):
+            self.current_assessment["supralinguistic"] += 1
+            self.notes.append("Supralinguistic: Potential figurative language detected")
+        
+        # Pragmatic: basic check for conversational elements
+        pragmatic_markers = ["hello", "hi", "thanks", "thank you", "please", "excuse me", "sorry"]
+        if any(marker in text.lower() for marker in pragmatic_markers):
+            self.current_assessment["pragmatic"] += 1
+            self.notes.append("Pragmatic: Appropriate social language detected")
+    
+    def get_assessment_html(self):
+        """Get HTML representation of the current assessment"""
+        html = """
+        <div style="padding: 15px; background-color: #f8f9fa; border-radius: 5px;">
+            <h3>CASL-2 Assessment Progress</h3>
+            <div style="display: grid; grid-template-columns: 1fr 1fr; gap: 10px;">
+        """
+        
+        for category, value in self.current_assessment.items():
+            category_name = category.replace('_', ' ').title()
+            progress_width = min(100, value * 10)
+            html += f"""
+                <div style="padding: 10px; background-color: white; border-radius: 5px; box-shadow: 0 1px 3px rgba(0,0,0,0.1);">
+                    <div><strong>{category_name}</strong></div>
+                    <div style="margin-top: 5px; height: 15px; background-color: #eee; border-radius: 7px; overflow: hidden;">
+                        <div style="width: {progress_width}%; height: 100%; background-color: #4CAF50;"></div>
+                    </div>
+                    <div style="margin-top: 5px;">{value} points</div>
+                </div>
+            """
+        
+        html += """
+            </div>
+        """
+        
+        # Add recent notes
+        if self.notes:
+            html += """
+            <div style="margin-top: 15px;">
+                <h4>Recent Observations</h4>
+                <ul style="margin-top: 5px;">
+            """
+            
+            for note in self.notes[-5:]:
+                html += f"<li>{note}</li>"
+            
+            html += """
+                </ul>
+            </div>
+            """
+        
+        html += "</div>"
+        return html
+    
+    def start_session(self, voice_model, student_id):
+        """Start a new session"""
+        self.is_active = True
+        self.voice_model = voice_model if voice_model else "shimmer"
+        self.transcript = []
+        self.notes = []
+        self.current_assessment = {
+            "lexical_semantic": 0,
+            "syntactic": 0,
+            "supralinguistic": 0,
+            "pragmatic": 0
+        }
+        
+        student_info = f" for {student_id}" if student_id else ""
+        self.transcript.append(f"Session started{student_info}. The AI Speech Pathologist will speak first.")
+        return "Session active. Please wait for the AI to introduce itself."
+    
+    def stop_session(self):
+        """Stop the current session"""
+        self.is_active = False
+        self.transcript.append("Session ended.")
+        return "Session stopped."
+    
+    def add_note(self, note):
+        """Add a custom note"""
+        if note.strip():
+            self.notes.append(note)
+            return f"Note added: {note}"
+        return "Note was empty, not added."
+
+
+# Create the audio processor instance
+audio_processor = AudioProcessor()
+
+
+async def start_session(voice_model, student_id):
+    """Start the speech pathology session"""
+    status = audio_processor.start_session(voice_model, student_id)
+    
+    # Generate initial AI introduction
+    try:
+        # Generate assistant response
+        chat_response = await openai_client.chat.completions.create(
+            model="gpt-4o",
+            messages=[
+                {"role": "system", "content": SPEECH_PATHOLOGIST_PROMPT},
+                {"role": "user", "content": "Hello"}  # Initial trigger
+            ]
+        )
+        
+        assistant_text = chat_response.choices[0].message.content
+        audio_processor.transcript.append(f"Speech Pathologist: {assistant_text}")
+        
+        # Generate speech from text
+        speech_response = await openai_client.audio.speech.create(
+            model="tts-1",
+            voice=audio_processor.voice_model,
+            input=assistant_text
+        )
+        
+        # Save speech to temporary file
+        response_temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".mp3")
+        response_temp_file.close()
+        
+        speech_response.stream_to_file(response_temp_file.name)
+        
+        # Load audio data for Gradio
+        import soundfile as sf
+        audio_data, sample_rate = sf.read(response_temp_file.name)
+        
+        # Clean up
+        os.unlink(response_temp_file.name)
+        
+        return status, (sample_rate, audio_data), "\n".join(audio_processor.transcript), audio_processor.get_assessment_html()
+        
+    except Exception as e:
+        logger.error(f"Error starting session: {e}")
+        audio_processor.transcript.append(f"Error: {str(e)}")
+        return status, None, "\n".join(audio_processor.transcript), audio_processor.get_assessment_html()
+
+
+def stop_session():
+    """Stop the speech pathology session"""
+    return audio_processor.stop_session(), None, "\n".join(audio_processor.transcript), audio_processor.get_assessment_html()
+
+
+async def process_mic_input(audio, progress=gr.Progress()):
+    """Process microphone input"""
+    if audio is None or not audio_processor.is_active:
+        return None, "\n".join(audio_processor.transcript), audio_processor.get_assessment_html()
+    
+    progress(0, desc="Processing speech...")
+    audio_output, transcript, assessment = await audio_processor.process_speech(audio, openai_client)
+    progress(1, desc="Done")
+    return audio_output, transcript, assessment
+
+
+def add_note(note):
+    """Add a note to the session"""
+    result = audio_processor.add_note(note)
+    return "", result, audio_processor.get_assessment_html()
+
+
+def save_session(student_id):
+    """Save session to file"""
+    if not student_id:
+        student_id = "anonymous"
+    
+    # Create session data directory if it doesn't exist
+    os.makedirs("session_data", exist_ok=True)
+    
+    # Save transcript
+    timestamp = time.strftime("%Y%m%d-%H%M%S")
+    filename = f"session_data/{student_id}_{timestamp}.txt"
+    
+    with open(filename, "w") as f:
+        f.write("\n".join(audio_processor.transcript))
+        f.write("\n\n--- ASSESSMENT NOTES ---\n")
+        for note in audio_processor.notes:
+            f.write(f"- {note}\n")
+        f.write("\n--- CASL-2 SCORES ---\n")
+        for category, score in audio_processor.current_assessment.items():
+            f.write(f"{category.replace('_', ' ').title()}: {score}\n")
+    
+    return f"Session saved to {filename}"
+
+
+# Create Gradio Interface
+with gr.Blocks(title="CASL-2 Speech Pathology Assistant") as app:
+    gr.Markdown("# CASL-2 Speech Pathology Assistant")
+    gr.Markdown("### AI-powered speech therapy assessment based on the CASL-2 framework")
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            student_id = gr.Textbox(label="Student ID (optional)", placeholder="Enter student ID")
+            voice_select = gr.Dropdown(
+                ["alloy", "echo", "fable", "onyx", "nova", "shimmer"], 
+                value="shimmer", 
+                label="Assistant Voice"
+            )
+            start_button = gr.Button("Start Session", variant="primary")
+            stop_button = gr.Button("Stop Session", variant="stop")
+            status = gr.Textbox(label="Status", value="Ready to start")
+            
+            with gr.Accordion("SLP Tools", open=True):
+                note_input = gr.Textbox(
+                    label="Add Assessment Note", 
+                    placeholder="Enter observation or assessment note here..."
+                )
+                note_button = gr.Button("Add Note")
+                note_status = gr.Textbox(label="Note Status")
+                save_button = gr.Button("Save Session")
+                save_status = gr.Textbox(label="Save Status")
+        
+        with gr.Column(scale=2):
+            audio_output = gr.Audio(label="AI Speech", autoplay=True)
+            audio_input = gr.Audio(
+                label="Speak to the AI",
+                type="microphone",
+                source="microphone", 
+                streaming=True
+            )
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            assessment_html = gr.HTML(label="Assessment Progress")
+        with gr.Column(scale=1):
+            transcript = gr.Textbox(label="Transcript", lines=10)
+    
+    with gr.Accordion("About This Application", open=False):
+        gr.Markdown("""
+        ### About CASL-2 Speech Pathology Assistant
+        
+        This application provides an AI speech pathologist that can assess students using the CASL-2 framework. It focuses on:
+        
+        - **Lexical/Semantic Skills**: Vocabulary knowledge and word usage
+        - **Syntactic Skills**: Grammar and sentence structure
+        - **Supralinguistic Skills**: Higher-level language beyond literal meanings
+        - **Pragmatic Skills**: Social use of language (less emphasis for younger students)
+        
+        The AI will provide structured assessments and exercises to help evaluate speech patterns.
+        
+        ### How to Use
+        
+        1. Optionally enter a Student ID to track sessions
+        2. Select the AI voice you prefer
+        3. Click "Start Session" to begin
+        4. The AI will introduce itself and begin the assessment
+        5. Speak into your microphone when it's your turn
+        6. View the transcript to track the conversation
+        7. SLPs can add assessment notes as needed
+        8. Save the session when finished
+        9. Click "Stop Session" when done
+        
+        ### For Speech-Language Pathologists
+        
+        This tool is designed to supplement, not replace, professional SLP services. SLPs can:
+        
+        - Add custom notes during the session
+        - Save session data for later reference
+        - Track progress across multiple sessions
+        - Use the AI as a consistent assessment tool
+        """)
+    
+    # Setup event handlers
+    start_button.click(
+        fn=lambda voice, student: asyncio.run(start_session(voice, student)), 
+        inputs=[voice_select, student_id],
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    stop_button.click(
+        fn=stop_session, 
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    note_button.click(
+        fn=add_note,
+        inputs=note_input,
+        outputs=[note_input, note_status, assessment_html]
+    )
+    save_button.click(
+        fn=save_session,
+        inputs=student_id,
+        outputs=save_status
+    )
+    
+    # Setup audio processing
+    audio_input.stream(
+        fn=lambda audio: asyncio.run(process_mic_input(audio)),
+        inputs=audio_input,
+        outputs=[audio_output, transcript, assessment_html]
+    )
+
+
+def main(share=True):
+    """Main function to launch the app"""
+    app.launch(share=share)
+
+
+# Entry point for the application
+if __name__ == "__main__":
+    main()

huggingface_requirements.txt

@@ -0,0 +1,7 @@
+python-dotenv>=1.0.0
+openai>=1.3.0
+gradio>=4.0.0
+soundfile>=0.12.1
+scipy>=1.10.0
+asyncio>=3.4.3
+numpy>=1.24.0

implementations/common/__init__.py

@@ -0,0 +1,3 @@
+"""
+Common utilities for CASL Voice Bot
+"""

implementations/common/casl_utils.py

@@ -0,0 +1,167 @@
+#!/usr/bin/env python3
+
+"""
+Common utilities for CASL Voice Bot implementations
+"""
+
+import os
+import time
+
+class CASLAssessment:
+    """CASL-2 assessment tracker"""
+    
+    def __init__(self):
+        """Initialize the assessment tracker"""
+        self.notes = []
+        self.current_assessment = {
+            "lexical_semantic": 0,
+            "syntactic": 0,
+            "supralinguistic": 0,
+            "pragmatic": 0
+        }
+    
+    def analyze_speech(self, text):
+        """Analyze speech for CASL-2 categories"""
+        # Simple heuristic analysis - in a real app, this would use more sophisticated NLP
+        
+        # Lexical/Semantic: check vocabulary diversity
+        words = text.lower().split()
+        unique_words = set(words)
+        if len(unique_words) / max(1, len(words)) > 0.7:
+            self.current_assessment["lexical_semantic"] += 1
+            self.notes.append("Lexical/Semantic: Good vocabulary diversity")
+        
+        # Syntactic: check for sentence complexity
+        sentences = [s.strip() for s in text.replace("!", ".").replace("?", ".").split(".") if s.strip()]
+        avg_words = sum(len(s.split()) for s in sentences) / max(1, len(sentences))
+        if avg_words > 7:
+            self.current_assessment["syntactic"] += 1
+            self.notes.append("Syntactic: Complex sentence structures used")
+        
+        # Supralinguistic: check for figurative language (very basic check)
+        figurative_markers = ["like", "as", "than", "seems", "appears", "metaphor", "imagine"]
+        if any(marker in text.lower() for marker in figurative_markers):
+            self.current_assessment["supralinguistic"] += 1
+            self.notes.append("Supralinguistic: Potential figurative language detected")
+        
+        # Pragmatic: basic check for conversational elements
+        pragmatic_markers = ["hello", "hi", "thanks", "thank you", "please", "excuse me", "sorry"]
+        if any(marker in text.lower() for marker in pragmatic_markers):
+            self.current_assessment["pragmatic"] += 1
+            self.notes.append("Pragmatic: Appropriate social language detected")
+    
+    def get_assessment_html(self):
+        """Get HTML representation of the current assessment"""
+        html = """
+        <div style="padding: 15px; background-color: #f8f9fa; border-radius: 5px;">
+            <h3>CASL-2 Assessment Progress</h3>
+            <div style="display: grid; grid-template-columns: 1fr 1fr; gap: 10px;">
+        """
+        
+        for category, value in self.current_assessment.items():
+            category_name = category.replace('_', ' ').title()
+            progress_width = min(100, value * 10)
+            html += f"""
+                <div style="padding: 10px; background-color: white; border-radius: 5px; box-shadow: 0 1px 3px rgba(0,0,0,0.1);">
+                    <div><strong>{category_name}</strong></div>
+                    <div style="margin-top: 5px; height: 15px; background-color: #eee; border-radius: 7px; overflow: hidden;">
+                        <div style="width: {progress_width}%; height: 100%; background-color: #4CAF50;"></div>
+                    </div>
+                    <div style="margin-top: 5px;">{value} points</div>
+                </div>
+            """
+        
+        html += """
+            </div>
+        """
+        
+        # Add recent notes
+        if self.notes:
+            html += """
+            <div style="margin-top: 15px;">
+                <h4>Recent Observations</h4>
+                <ul style="margin-top: 5px;">
+            """
+            
+            for note in self.notes[-5:]:
+                html += f"<li>{note}</li>"
+            
+            html += """
+                </ul>
+            </div>
+            """
+        
+        html += "</div>"
+        return html
+    
+    def add_note(self, note):
+        """Add a custom note"""
+        if note.strip():
+            self.notes.append(note)
+            return f"Note added: {note}"
+        return "Note was empty, not added."
+
+
+def save_session_data(transcript, assessment, student_id=None):
+    """Save session data to a file"""
+    if not student_id:
+        student_id = "anonymous"
+    
+    # Create session data directory if it doesn't exist
+    os.makedirs("session_data", exist_ok=True)
+    
+    # Save transcript
+    timestamp = time.strftime("%Y%m%d-%H%M%S")
+    filename = f"session_data/{student_id}_{timestamp}.txt"
+    
+    with open(filename, "w") as f:
+        f.write("\n".join(transcript))
+        f.write("\n\n--- ASSESSMENT NOTES ---\n")
+        for note in assessment.notes:
+            f.write(f"- {note}\n")
+        f.write("\n--- CASL-2 SCORES ---\n")
+        for category, score in assessment.current_assessment.items():
+            f.write(f"{category.replace('_', ' ').title()}: {score}\n")
+    
+    return f"Session saved to {filename}"
+
+
+# Common prompt used by all implementations
+CASL_PROMPT = """
+You are a speech pathologist, a healthcare professional who specializes in evaluating, diagnosing, and treating communication disorders, including speech, language, cognitive-communication, voice, and fluency disorders. Your role is to help patients improve their speech and communication skills through various therapeutic techniques and exercises.
+Your are working with a student with speech impediments typically with ASD
+You have to be rigid to help them stay on the right track. YOu have to start with some sort of intro activity and can not rely on teh student at all to complete your thoughts. You pick a place to start and assess teh speech from there. 
+Each domain from the CASL-2 framework can be analyzed using the sample:
+Lexical/Semantic Skills:
+This category focuses on vocabulary knowledge, word meanings, and the ability to use words contextually. It measures both receptive and expressive language abilities related to word use.
+Key Subtests:
+Antonyms: Identifying words with opposite meanings.
+Synonyms: Identifying words with similar meanings.
+Idiomatic Language: Understanding and interpreting idioms and figurative language.
+Evaluate vocabulary diversity (type-token ratio).
+Note word-finding difficulties, incorrect word choices, or over-reliance on fillers (e.g., "like," "stuff").
+Assess use of specific vs. vague language (e.g., "car" vs. "sedan").
+Syntactic Skills:
+This category evaluates understanding and use of grammar and sentence structure. It focuses on the ability to comprehend and produce grammatically correct sentences.
+Key Subtests:
+Sentence Expression: Producing grammatically correct sentences based on prompts.
+Grammaticality Judgment: Identifying whether a sentence is grammatically correct.
+Examine sentence structure for grammatical accuracy.
+Identify errors in verb tense, subject-verb agreement, or sentence complexity.
+Note the use of clauses, conjunctions, and varied sentence types.
+Supralinguistic Skills:
+This subcategory assesses higher-level language skills that go beyond literal meanings, such as understanding implied meanings, sarcasm, and complex verbal reasoning.
+Key Subtests:
+Inferences: Understanding information that is not explicitly stated.
+Meaning from Context: Deriving meaning from surrounding text or dialogue.
+Nonliteral Language: Interpreting figurative language, such as metaphors or irony
+Look for use or understanding of figurative language, idioms, or humor.
+Assess ability to handle ambiguous or implied meanings in context.
+Identify advanced language use for abstract or hypothetical ideas.
+Pragmatic Skills(focus less on this as it is not typically necessary for the age range you will be dealing with):
+This category measures the ability to use language effectively in social contexts. It evaluates understanding of conversational rules, turn-taking, and adapting communication to different social situations.
+Key Subtests:
+Pragmatic Language Test: Assessing appropriateness of responses in social scenarios.
+
+Begin by introducing yourself as a speech pathologist and start with a simple vocabulary assessment activity. Be encouraging but structured in your approach.
+"""

implementations/direct/__init__.py

@@ -0,0 +1,3 @@
+"""
+Direct OpenAI API implementation of CASL Voice Bot
+"""

implementations/direct/app.py

@@ -0,0 +1,334 @@
+#!/usr/bin/env python3
+
+"""
+CASL Voice Bot - Speech Pathology Assistant
+Direct OpenAI API implementation (no LiveKit)
+"""
+
+import os
+import asyncio
+import gradio as gr
+import logging
+import sys
+import tempfile
+import time
+from dotenv import load_dotenv
+from openai import AsyncOpenAI
+
+# Add parent directory to path to import common utilities
+sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
+from implementations.common.casl_utils import CASLAssessment, save_session_data, CASL_PROMPT
+
+# Load environment variables
+load_dotenv()
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Initialize OpenAI client
+openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+
+
+class SpeechPathologistAssistant:
+    """Speech pathologist assistant using direct OpenAI API"""
+    
+    def __init__(self):
+        self.transcript = []
+        self.is_running = False
+        self.assessment = CASLAssessment()
+        self.voice_model = "shimmer"
+        self.student_id = None
+    
+    async def start_session(self, voice_model, student_id):
+        """Start a new session"""
+        self.is_running = True
+        self.voice_model = voice_model if voice_model else "shimmer"
+        self.student_id = student_id
+        self.transcript = []
+        self.assessment = CASLAssessment()
+        
+        # Add student info to transcript
+        student_info = f" for {student_id}" if student_id else ""
+        self.transcript.append(f"Session started{student_info}. The AI Speech Pathologist will speak first.")
+        
+        # Generate initial AI message
+        initial_audio = await self.generate_initial_message()
+        
+        return "Session active. The AI will introduce itself.", initial_audio, self.get_transcript(), self.assessment.get_assessment_html()
+    
+    async def generate_initial_message(self):
+        """Generate initial AI message"""
+        # Generate assistant response
+        chat_response = await openai_client.chat.completions.create(
+            model="gpt-4o",
+            messages=[
+                {"role": "system", "content": CASL_PROMPT},
+                {"role": "user", "content": "Hello"}  # Initial trigger
+            ]
+        )
+        
+        assistant_text = chat_response.choices[0].message.content
+        self.transcript.append(f"Speech Pathologist: {assistant_text}")
+        
+        # Generate speech from text
+        speech_response = await openai_client.audio.speech.create(
+            model="tts-1",
+            voice=self.voice_model,
+            input=assistant_text
+        )
+        
+        # Save speech to temporary file
+        response_temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".mp3")
+        response_temp_file.close()
+        
+        speech_response.stream_to_file(response_temp_file.name)
+        
+        # Load audio data for Gradio
+        import soundfile as sf
+        audio_data, sample_rate = sf.read(response_temp_file.name)
+        
+        # Clean up
+        os.unlink(response_temp_file.name)
+        
+        return (sample_rate, audio_data)
+    
+    def stop_session(self):
+        """Stop the current session"""
+        self.is_running = False
+        self.transcript.append("Session ended.")
+        return "Session stopped.", None, self.get_transcript(), self.assessment.get_assessment_html()
+    
+    async def process_audio(self, audio):
+        """Process audio from Gradio interface"""
+        if not self.is_running or audio is None:
+            return None, self.get_transcript(), self.assessment.get_assessment_html()
+        
+        # Prepare audio file for OpenAI
+        temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".wav")
+        temp_file.close()
+        
+        try:
+            # Save audio data to temporary file
+            sample_rate, audio_array = audio
+            import scipy.io.wavfile
+            scipy.io.wavfile.write(temp_file.name, sample_rate, audio_array)
+            
+            # Transcribe audio using OpenAI
+            with open(temp_file.name, "rb") as audio_file:
+                transcript_response = await openai_client.audio.transcriptions.create(
+                    file=audio_file,
+                    model="whisper-1"
+                )
+            
+            user_text = transcript_response.text
+            if user_text.strip():
+                self.transcript.append(f"Student: {user_text}")
+                
+                # Analyze speech for CASL-2 categories
+                self.assessment.analyze_speech(user_text)
+                
+                # Generate assistant response
+                chat_response = await openai_client.chat.completions.create(
+                    model="gpt-4o",
+                    messages=[
+                        {"role": "system", "content": CASL_PROMPT},
+                        {"role": "user", "content": user_text}
+                    ]
+                )
+                
+                assistant_text = chat_response.choices[0].message.content
+                self.transcript.append(f"Speech Pathologist: {assistant_text}")
+                
+                # Generate speech from text
+                speech_response = await openai_client.audio.speech.create(
+                    model="tts-1",
+                    voice=self.voice_model,
+                    input=assistant_text
+                )
+                
+                # Save speech to temporary file
+                response_temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".mp3")
+                response_temp_file.close()
+                
+                speech_response.stream_to_file(response_temp_file.name)
+                
+                # Load audio data for Gradio
+                import soundfile as sf
+                audio_data, sample_rate = sf.read(response_temp_file.name)
+                
+                # Clean up
+                os.unlink(response_temp_file.name)
+                
+                return (sample_rate, audio_data), self.get_transcript(), self.assessment.get_assessment_html()
+                
+        except Exception as e:
+            logger.error(f"Error processing audio: {e}")
+            self.transcript.append(f"Error: {str(e)}")
+        finally:
+            # Clean up temp file
+            os.unlink(temp_file.name)
+        
+        return None, self.get_transcript(), self.assessment.get_assessment_html()
+    
+    def get_transcript(self):
+        """Get the current transcript"""
+        return "\n".join(self.transcript)
+    
+    def add_note(self, note):
+        """Add a custom note"""
+        result = self.assessment.add_note(note)
+        return "", result, self.assessment.get_assessment_html()
+    
+    def save_session(self, student_id=None):
+        """Save session to file"""
+        student_id = student_id or self.student_id
+        return save_session_data(self.transcript, self.assessment, student_id)
+
+
+# Create the speech pathology assistant
+speech_assistant = SpeechPathologistAssistant()
+
+
+async def start_session(voice_model, student_id):
+    """Start the speech pathology session"""
+    return await speech_assistant.start_session(voice_model, student_id)
+
+
+def stop_session():
+    """Stop the speech pathology session"""
+    return speech_assistant.stop_session()
+
+
+async def process_mic_input(audio, progress=gr.Progress()):
+    """Process microphone input"""
+    progress(0, desc="Processing speech...")
+    audio_output, transcript, assessment = await speech_assistant.process_audio(audio)
+    progress(1, desc="Done")
+    return audio_output, transcript, assessment
+
+
+def add_note(note):
+    """Add a note to the session"""
+    return speech_assistant.add_note(note)
+
+
+def save_session(student_id):
+    """Save the current session"""
+    return speech_assistant.save_session(student_id)
+
+
+# Create Gradio Interface
+with gr.Blocks(title="CASL-2 Speech Pathology Assistant") as app:
+    gr.Markdown("# CASL-2 Speech Pathology Assistant")
+    gr.Markdown("### AI-powered speech therapy assessment based on the CASL-2 framework")
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            student_id = gr.Textbox(label="Student ID (optional)", placeholder="Enter student ID")
+            voice_select = gr.Dropdown(
+                ["alloy", "echo", "fable", "onyx", "nova", "shimmer"], 
+                value="shimmer", 
+                label="Assistant Voice"
+            )
+            start_button = gr.Button("Start Session", variant="primary")
+            stop_button = gr.Button("Stop Session", variant="stop")
+            status = gr.Textbox(label="Status", value="Ready to start")
+            
+            with gr.Accordion("SLP Tools", open=True):
+                note_input = gr.Textbox(
+                    label="Add Assessment Note", 
+                    placeholder="Enter observation or assessment note here..."
+                )
+                note_button = gr.Button("Add Note")
+                note_status = gr.Textbox(label="Note Status")
+                save_button = gr.Button("Save Session")
+                save_status = gr.Textbox(label="Save Status")
+        
+        with gr.Column(scale=2):
+            audio_output = gr.Audio(label="AI Speech", autoplay=True)
+            audio_input = gr.Audio(
+                label="Speak to the AI",
+                type="microphone",
+                source="microphone", 
+                streaming=True
+            )
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            assessment_html = gr.HTML(label="Assessment Progress")
+        with gr.Column(scale=1):
+            transcript = gr.Textbox(label="Transcript", lines=10)
+    
+    with gr.Accordion("About This Application", open=False):
+        gr.Markdown("""
+        ### About CASL-2 Speech Pathology Assistant
+        
+        This application provides an AI speech pathologist that can assess students using the CASL-2 framework. It focuses on:
+        
+        - **Lexical/Semantic Skills**: Vocabulary knowledge and word usage
+        - **Syntactic Skills**: Grammar and sentence structure
+        - **Supralinguistic Skills**: Higher-level language beyond literal meanings
+        - **Pragmatic Skills**: Social use of language (less emphasis for younger students)
+        
+        The AI will provide structured assessments and exercises to help evaluate speech patterns.
+        
+        ### How to Use
+        
+        1. Optionally enter a Student ID to track sessions
+        2. Select the AI voice you prefer
+        3. Click "Start Session" to begin
+        4. The AI will introduce itself and begin the assessment
+        5. Speak into your microphone when it's your turn
+        6. View the transcript to track the conversation
+        7. SLPs can add notes throughout the session
+        8. Save the session when finished
+        9. Click "Stop Session" when done
+        
+        ### For Speech-Language Pathologists
+        
+        This tool is designed to supplement, not replace, professional SLP services. SLPs can:
+        
+        - Add custom notes during the session
+        - Save session data for later reference
+        - Track progress across multiple sessions
+        - Use the AI as a consistent assessment tool
+        """)
+    
+    # Setup event handlers
+    start_button.click(
+        fn=lambda voice, student: asyncio.run(start_session(voice, student)), 
+        inputs=[voice_select, student_id],
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    stop_button.click(
+        fn=stop_session, 
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    note_button.click(
+        fn=add_note,
+        inputs=note_input,
+        outputs=[note_input, note_status, assessment_html]
+    )
+    save_button.click(
+        fn=save_session,
+        inputs=student_id,
+        outputs=save_status
+    )
+    
+    # Setup audio processing
+    audio_input.stream(
+        fn=lambda audio: asyncio.run(process_mic_input(audio)),
+        inputs=audio_input,
+        outputs=[audio_output, transcript, assessment_html]
+    )
+
+
+def main(share=True):
+    """Main function to launch the app"""
+    app.launch(share=share)
+
+
+# Entry point for the application
+if __name__ == "__main__":
+    main()

implementations/livekit/__init__.py

@@ -0,0 +1,3 @@
+"""
+LiveKit implementation of CASL Voice Bot
+"""

implementations/livekit/app.py

@@ -0,0 +1,329 @@
+#!/usr/bin/env python3
+
+"""
+CASL Voice Bot - Speech Pathology Assistant
+Using LiveKit agents with OpenAI's real-time capabilities
+"""
+
+import os
+import asyncio
+import gradio as gr
+import logging
+import sys
+import time
+from dotenv import load_dotenv
+from livekit import agents
+from openai import AsyncOpenAI
+
+# Add parent directory to path to import common utilities
+sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
+from implementations.common.casl_utils import CASLAssessment, save_session_data, CASL_PROMPT
+
+# Load environment variables
+load_dotenv()
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Initialize OpenAI client
+openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+
+class GradioInputDevice(agents.InputDevice):
+    """Custom input device that works with Gradio"""
+    
+    def __init__(self):
+        super().__init__()
+        self.audio_queue = asyncio.Queue()
+        self.is_active = True
+    
+    async def receive(self) -> agents.AudioChunk:
+        """Receive audio data from the queue"""
+        try:
+            audio_data = await asyncio.wait_for(self.audio_queue.get(), timeout=0.1)
+            return audio_data
+        except asyncio.TimeoutError:
+            return None
+    
+    async def add_audio(self, audio_data):
+        """Add audio data to the queue"""
+        if audio_data is None:
+            return
+            
+        # Convert gradio audio format to AudioChunk
+        sample_rate, audio_array = audio_data
+        audio_chunk = agents.AudioChunk(
+            samples=audio_array,
+            sample_rate=sample_rate,
+            is_last=False
+        )
+        await self.audio_queue.put(audio_chunk)
+    
+    def stop(self):
+        """Stop the input device"""
+        self.is_active = False
+
+
+class GradioOutputDevice(agents.OutputDevice):
+    """Custom output device that works with Gradio"""
+    
+    def __init__(self):
+        super().__init__()
+        self.output_queue = asyncio.Queue()
+    
+    async def transmit(self, audio_chunk: agents.AudioChunk) -> None:
+        """Transmit audio chunk to the queue"""
+        if audio_chunk is not None:
+            await self.output_queue.put((audio_chunk.samples, audio_chunk.sample_rate))
+    
+    async def get_latest_audio(self):
+        """Get the latest audio from the queue"""
+        try:
+            return await asyncio.wait_for(self.output_queue.get(), timeout=0.1)
+        except asyncio.TimeoutError:
+            return None
+
+
+class SpeechPathologistAssistant:
+    """Speech pathologist assistant using LiveKit agents"""
+    
+    def __init__(self):
+        self.input_device = GradioInputDevice()
+        self.output_device = GradioOutputDevice()
+        self.assistant = None
+        self.assistant_task = None
+        self.transcript = []
+        self.is_running = False
+        self.assessment = CASLAssessment()
+    
+    async def initialize_assistant(self, voice="shimmer"):
+        """Initialize the speech assistant"""
+        self.assistant = agents.VoiceAssistant(
+            openai_client=openai_client,
+            model="gpt-4o",
+            voice=voice,
+            input_device=self.input_device,
+            output_device=self.output_device,
+            initial_message=CASL_PROMPT,
+            real_time=True,  # Enable real-time processing
+        )
+        
+        # Add transcript and response callbacks
+        self.assistant.on_transcript = self.on_transcript
+        self.assistant.on_response = self.on_response
+    
+    def on_transcript(self, transcript):
+        """Handle transcript from user"""
+        self.transcript.append(f"Student: {transcript.text}")
+        
+        # Basic analysis of speech for CASL-2 categories
+        self.assessment.analyze_speech(transcript.text)
+        
+        return True
+    
+    def on_response(self, response):
+        """Handle response from assistant"""
+        self.transcript.append(f"Speech Pathologist: {response.text}")
+        return True
+    
+    async def start_assistant(self, voice_model, student_id):
+        """Start the assistant in a background task"""
+        await self.initialize_assistant(voice_model)
+        
+        self.is_running = True
+        
+        # Add student info to transcript
+        student_info = f" for {student_id}" if student_id else ""
+        self.transcript.append(f"Session started{student_info}. The AI Speech Pathologist will speak first.")
+        
+        # Run the assistant in a background task
+        self.assistant_task = asyncio.create_task(self.assistant.run())
+        
+        return "Session active. The AI will introduce itself."
+    
+    def stop_assistant(self):
+        """Stop the assistant"""
+        if self.assistant_task and not self.assistant_task.done():
+            self.assistant_task.cancel()
+        
+        self.input_device.stop()
+        self.is_running = False
+        
+        # Add ending to transcript
+        self.transcript.append("Session ended.")
+        return "Session stopped."
+    
+    async def process_audio(self, audio):
+        """Process audio from Gradio interface"""
+        if not self.is_running or audio is None:
+            return None, self.get_transcript(), self.assessment.get_assessment_html()
+        
+        # Add audio to input device
+        await self.input_device.add_audio(audio)
+        
+        # Check for assistant output
+        output_audio = await self.output_device.get_latest_audio()
+        
+        return output_audio, self.get_transcript(), self.assessment.get_assessment_html()
+    
+    def get_transcript(self):
+        """Get the current transcript"""
+        return "\n".join(self.transcript)
+    
+    def add_note(self, note):
+        """Add a custom note"""
+        result = self.assessment.add_note(note)
+        return "", result, self.assessment.get_assessment_html()
+    
+    def save_session(self, student_id):
+        """Save session to file"""
+        return save_session_data(self.transcript, self.assessment, student_id)
+
+
+# Create the speech pathology assistant
+speech_assistant = SpeechPathologistAssistant()
+
+
+async def start_session(voice_model, student_id):
+    """Start the speech pathology session"""
+    status = await speech_assistant.start_assistant(voice_model, student_id)
+    return status, None, speech_assistant.get_transcript(), speech_assistant.assessment.get_assessment_html()
+
+
+def stop_session():
+    """Stop the speech pathology session"""
+    return speech_assistant.stop_assistant(), None, speech_assistant.get_transcript(), speech_assistant.assessment.get_assessment_html()
+
+
+async def process_mic_input(audio, progress=gr.Progress()):
+    """Process microphone input"""
+    progress(0, desc="Processing speech...")
+    audio_output, transcript, assessment = await speech_assistant.process_audio(audio)
+    progress(1, desc="Done")
+    return audio_output, transcript, assessment
+
+
+def add_note(note):
+    """Add a note to the session"""
+    return speech_assistant.add_note(note)
+
+
+def save_session(student_id):
+    """Save the current session"""
+    return speech_assistant.save_session(student_id)
+
+
+# Create Gradio Interface
+with gr.Blocks(title="CASL-2 Speech Pathology Assistant") as app:
+    gr.Markdown("# CASL-2 Speech Pathology Assistant")
+    gr.Markdown("### AI-powered speech therapy assessment based on the CASL-2 framework")
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            student_id = gr.Textbox(label="Student ID (optional)", placeholder="Enter student ID")
+            voice_select = gr.Dropdown(
+                ["alloy", "echo", "fable", "onyx", "nova", "shimmer"], 
+                value="shimmer", 
+                label="Assistant Voice"
+            )
+            start_button = gr.Button("Start Session", variant="primary")
+            stop_button = gr.Button("Stop Session", variant="stop")
+            status = gr.Textbox(label="Status", value="Ready to start")
+            
+            with gr.Accordion("SLP Tools", open=True):
+                note_input = gr.Textbox(
+                    label="Add Assessment Note", 
+                    placeholder="Enter observation or assessment note here..."
+                )
+                note_button = gr.Button("Add Note")
+                note_status = gr.Textbox(label="Note Status")
+                save_button = gr.Button("Save Session")
+                save_status = gr.Textbox(label="Save Status")
+        
+        with gr.Column(scale=2):
+            audio_output = gr.Audio(label="AI Speech", autoplay=True)
+            audio_input = gr.Audio(
+                label="Speak to the AI",
+                type="microphone",
+                source="microphone", 
+                streaming=True
+            )
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            assessment_html = gr.HTML(label="Assessment Progress")
+        with gr.Column(scale=1):
+            transcript = gr.Textbox(label="Transcript", lines=10)
+    
+    with gr.Accordion("About This Application", open=False):
+        gr.Markdown("""
+        ### About CASL-2 Speech Pathology Assistant
+        
+        This application provides an AI speech pathologist that can assess students using the CASL-2 framework. It focuses on:
+        
+        - **Lexical/Semantic Skills**: Vocabulary knowledge and word usage
+        - **Syntactic Skills**: Grammar and sentence structure
+        - **Supralinguistic Skills**: Higher-level language beyond literal meanings
+        - **Pragmatic Skills**: Social use of language (less emphasis for younger students)
+        
+        The AI will provide structured assessments and exercises to help evaluate speech patterns.
+        
+        ### How to Use
+        
+        1. Optionally enter a Student ID to track sessions
+        2. Select the AI voice you prefer
+        3. Click "Start Session" to begin
+        4. The AI will introduce itself and begin the assessment
+        5. Speak into your microphone when it's your turn
+        6. View the transcript to track the conversation
+        7. SLPs can add notes throughout the session
+        8. Save the session when finished
+        9. Click "Stop Session" when done
+        
+        ### For Speech-Language Pathologists
+        
+        This tool is designed to supplement, not replace, professional SLP services. SLPs can:
+        
+        - Add custom notes during the session
+        - Save session data for later reference
+        - Track progress across multiple sessions
+        - Use the AI as a consistent assessment tool
+        """)
+    
+    # Setup event handlers
+    start_button.click(
+        fn=lambda voice, student: asyncio.run(start_session(voice, student)), 
+        inputs=[voice_select, student_id],
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    stop_button.click(
+        fn=stop_session, 
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    note_button.click(
+        fn=add_note,
+        inputs=note_input,
+        outputs=[note_input, note_status, assessment_html]
+    )
+    save_button.click(
+        fn=save_session,
+        inputs=student_id,
+        outputs=save_status
+    )
+    
+    # Setup audio processing
+    audio_input.stream(
+        fn=lambda audio: asyncio.run(process_mic_input(audio)),
+        inputs=audio_input,
+        outputs=[audio_output, transcript, assessment_html]
+    )
+
+
+def main(share=True):
+    """Main function to launch the app"""
+    app.launch(share=share)
+
+
+# Entry point for the application
+if __name__ == "__main__":
+    main()

implementations/livekit/livekit_gradio_hf.py

@@ -0,0 +1,490 @@
+#!/usr/bin/env python3
+
+"""
+CASL Voice Bot - Hugging Face Spaces deployment version
+Using LiveKit with Gradio for Hugging Face Spaces compatibility
+
+This is a special version optimized for Hugging Face Spaces deployment
+that works with LiveKit's WebRTC capabilities.
+"""
+
+import os
+import asyncio
+import gradio as gr
+import logging
+import sys
+import time
+import importlib.util
+from dotenv import load_dotenv
+from openai import AsyncOpenAI
+
+# Check if livekit-agents is installed
+try:
+    from livekit import agents
+    LIVEKIT_AVAILABLE = True
+except ImportError:
+    LIVEKIT_AVAILABLE = False
+    # Create dummy classes for type hinting
+    class agents:
+        class InputDevice: pass
+        class OutputDevice: pass
+        class AudioChunk: pass
+        class VoiceAssistant: pass
+
+# Add parent directory to path to import common utilities
+sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
+from implementations.common.casl_utils import CASLAssessment, save_session_data, CASL_PROMPT
+
+# Load environment variables
+load_dotenv()
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Initialize OpenAI client
+openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+
+# Hugging Face Spaces compatibility check
+HF_SPACES = os.environ.get("SPACE_ID") is not None
+logger.info(f"Running in Hugging Face Spaces: {HF_SPACES}")
+logger.info(f"LiveKit available: {LIVEKIT_AVAILABLE}")
+
+class GradioInputDevice(agents.InputDevice if LIVEKIT_AVAILABLE else object):
+    """Custom input device that works with Gradio"""
+    
+    def __init__(self):
+        if LIVEKIT_AVAILABLE:
+            super().__init__()
+        self.audio_queue = asyncio.Queue()
+        self.is_active = True
+    
+    async def receive(self):
+        """Receive audio data from the queue"""
+        try:
+            audio_data = await asyncio.wait_for(self.audio_queue.get(), timeout=0.1)
+            return audio_data
+        except asyncio.TimeoutError:
+            return None
+    
+    async def add_audio(self, audio_data):
+        """Add audio data to the queue"""
+        if audio_data is None:
+            return
+            
+        # Convert gradio audio format to AudioChunk
+        if LIVEKIT_AVAILABLE:
+            sample_rate, audio_array = audio_data
+            audio_chunk = agents.AudioChunk(
+                samples=audio_array,
+                sample_rate=sample_rate,
+                is_last=False
+            )
+            await self.audio_queue.put(audio_chunk)
+        else:
+            # Store raw audio data if LiveKit is not available
+            await self.audio_queue.put(audio_data)
+    
+    def stop(self):
+        """Stop the input device"""
+        self.is_active = False
+
+
+class GradioOutputDevice(agents.OutputDevice if LIVEKIT_AVAILABLE else object):
+    """Custom output device that works with Gradio"""
+    
+    def __init__(self):
+        if LIVEKIT_AVAILABLE:
+            super().__init__()
+        self.output_queue = asyncio.Queue()
+    
+    async def transmit(self, audio_chunk):
+        """Transmit audio chunk to the queue"""
+        if audio_chunk is not None:
+            if LIVEKIT_AVAILABLE:
+                await self.output_queue.put((audio_chunk.samples, audio_chunk.sample_rate))
+            else:
+                # Handle raw audio data if LiveKit is not available
+                await self.output_queue.put(audio_chunk)
+    
+    async def get_latest_audio(self):
+        """Get the latest audio from the queue"""
+        try:
+            return await asyncio.wait_for(self.output_queue.get(), timeout=0.1)
+        except asyncio.TimeoutError:
+            return None
+
+
+class SpeechPathologistAssistant:
+    """Speech pathologist assistant using LiveKit agents or direct OpenAI API"""
+    
+    def __init__(self):
+        self.input_device = GradioInputDevice()
+        self.output_device = GradioOutputDevice()
+        self.assistant = None
+        self.assistant_task = None
+        self.transcript = []
+        self.is_running = False
+        self.assessment = CASLAssessment()
+        self.student_id = None
+        self.voice_model = "shimmer"
+    
+    async def initialize_assistant(self, voice="shimmer"):
+        """Initialize the speech assistant"""
+        self.voice_model = voice
+        
+        if LIVEKIT_AVAILABLE:
+            # Use LiveKit VoiceAssistant if available
+            self.assistant = agents.VoiceAssistant(
+                openai_client=openai_client,
+                model="gpt-4o",
+                voice=voice,
+                input_device=self.input_device,
+                output_device=self.output_device,
+                initial_message=CASL_PROMPT,
+                real_time=True,  # Enable real-time processing
+            )
+            
+            # Add transcript and response callbacks
+            self.assistant.on_transcript = self.on_transcript
+            self.assistant.on_response = self.on_response
+        else:
+            # If LiveKit is not available, we'll use direct OpenAI API
+            logger.info("LiveKit not available, using direct OpenAI API")
+    
+    def on_transcript(self, transcript):
+        """Handle transcript from user (for LiveKit)"""
+        self.transcript.append(f"Student: {transcript.text}")
+        
+        # Basic analysis of speech for CASL-2 categories
+        self.assessment.analyze_speech(transcript.text)
+        
+        return True
+    
+    def on_response(self, response):
+        """Handle response from assistant (for LiveKit)"""
+        self.transcript.append(f"Speech Pathologist: {response.text}")
+        return True
+    
+    async def start_assistant(self, voice_model, student_id):
+        """Start the assistant in a background task"""
+        self.student_id = student_id
+        await self.initialize_assistant(voice_model)
+        
+        self.is_running = True
+        
+        # Add student info to transcript
+        student_info = f" for {student_id}" if student_id else ""
+        self.transcript.append(f"Session started{student_info}. The AI Speech Pathologist will speak first.")
+        
+        if LIVEKIT_AVAILABLE:
+            # Run the LiveKit assistant in a background task
+            self.assistant_task = asyncio.create_task(self.assistant.run())
+        else:
+            # For direct OpenAI API, generate initial message
+            await self.generate_initial_message()
+        
+        return "Session active. The AI will introduce itself."
+    
+    async def generate_initial_message(self):
+        """Generate initial AI message when not using LiveKit"""
+        # Generate assistant response using OpenAI API directly
+        chat_response = await openai_client.chat.completions.create(
+            model="gpt-4o",
+            messages=[
+                {"role": "system", "content": CASL_PROMPT},
+                {"role": "user", "content": "Hello"}  # Initial trigger
+            ]
+        )
+        
+        assistant_text = chat_response.choices[0].message.content
+        self.transcript.append(f"Speech Pathologist: {assistant_text}")
+        
+        # Generate speech from text
+        speech_response = await openai_client.audio.speech.create(
+            model="tts-1",
+            voice=self.voice_model,
+            input=assistant_text
+        )
+        
+        # Save speech to temporary file to get audio data
+        import tempfile
+        import soundfile as sf
+        
+        response_temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".mp3")
+        response_temp_file.close()
+        
+        speech_response.stream_to_file(response_temp_file.name)
+        
+        # Load audio data for Gradio
+        audio_data, sample_rate = sf.read(response_temp_file.name)
+        
+        # Send to output device
+        await self.output_device.transmit((audio_data, sample_rate))
+        
+        # Clean up
+        os.unlink(response_temp_file.name)
+    
+    def stop_assistant(self):
+        """Stop the assistant"""
+        if self.assistant_task and not self.assistant_task.done():
+            self.assistant_task.cancel()
+        
+        self.input_device.stop()
+        self.is_running = False
+        
+        # Add ending to transcript
+        self.transcript.append("Session ended.")
+        return "Session stopped."
+    
+    async def process_audio(self, audio):
+        """Process audio from Gradio interface"""
+        if not self.is_running or audio is None:
+            return None, self.get_transcript(), self.assessment.get_assessment_html()
+        
+        if LIVEKIT_AVAILABLE:
+            # Add audio to input device for LiveKit
+            await self.input_device.add_audio(audio)
+            
+            # Check for assistant output
+            output_audio = await self.output_device.get_latest_audio()
+        else:
+            # For direct OpenAI API
+            output_audio = await self.process_with_direct_api(audio)
+        
+        return output_audio, self.get_transcript(), self.assessment.get_assessment_html()
+    
+    async def process_with_direct_api(self, audio):
+        """Process audio using direct OpenAI API when LiveKit is not available"""
+        # Prepare audio file for OpenAI
+        import tempfile
+        import scipy.io.wavfile
+        
+        temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".wav")
+        temp_file.close()
+        
+        try:
+            # Save audio data to temporary file
+            sample_rate, audio_array = audio
+            scipy.io.wavfile.write(temp_file.name, sample_rate, audio_array)
+            
+            # Transcribe audio using OpenAI
+            with open(temp_file.name, "rb") as audio_file:
+                transcript_response = await openai_client.audio.transcriptions.create(
+                    file=audio_file,
+                    model="whisper-1"
+                )
+            
+            user_text = transcript_response.text
+            if user_text.strip():
+                self.transcript.append(f"Student: {user_text}")
+                
+                # Analyze speech for CASL-2 categories
+                self.assessment.analyze_speech(user_text)
+                
+                # Generate assistant response
+                chat_response = await openai_client.chat.completions.create(
+                    model="gpt-4o",
+                    messages=[
+                        {"role": "system", "content": CASL_PROMPT},
+                        {"role": "user", "content": user_text}
+                    ]
+                )
+                
+                assistant_text = chat_response.choices[0].message.content
+                self.transcript.append(f"Speech Pathologist: {assistant_text}")
+                
+                # Generate speech from text
+                speech_response = await openai_client.audio.speech.create(
+                    model="tts-1",
+                    voice=self.voice_model,
+                    input=assistant_text
+                )
+                
+                # Save speech to temporary file
+                import soundfile as sf
+                
+                response_temp_file = tempfile.NamedTemporaryFile(delete=False, suffix=".mp3")
+                response_temp_file.close()
+                
+                speech_response.stream_to_file(response_temp_file.name)
+                
+                # Load audio data for Gradio
+                audio_data, sample_rate = sf.read(response_temp_file.name)
+                
+                # Clean up
+                os.unlink(response_temp_file.name)
+                
+                return (sample_rate, audio_data)
+        except Exception as e:
+            logger.error(f"Error processing with direct API: {e}")
+        finally:
+            # Clean up temp file
+            os.unlink(temp_file.name)
+        
+        return None
+    
+    def get_transcript(self):
+        """Get the current transcript"""
+        return "\n".join(self.transcript)
+    
+    def add_note(self, note):
+        """Add a custom note"""
+        result = self.assessment.add_note(note)
+        return "", result, self.assessment.get_assessment_html()
+    
+    def save_session(self, student_id=None):
+        """Save session to file"""
+        student_id = student_id or self.student_id
+        return save_session_data(self.transcript, self.assessment, student_id)
+
+
+# Create the speech pathology assistant
+speech_assistant = SpeechPathologistAssistant()
+
+
+async def start_session(voice_model, student_id):
+    """Start the speech pathology session"""
+    status = await speech_assistant.start_assistant(voice_model, student_id)
+    return status, None, speech_assistant.get_transcript(), speech_assistant.assessment.get_assessment_html()
+
+
+def stop_session():
+    """Stop the speech pathology session"""
+    return speech_assistant.stop_assistant(), None, speech_assistant.get_transcript(), speech_assistant.assessment.get_assessment_html()
+
+
+async def process_mic_input(audio, progress=gr.Progress()):
+    """Process microphone input"""
+    progress(0, desc="Processing speech...")
+    audio_output, transcript, assessment = await speech_assistant.process_audio(audio)
+    progress(1, desc="Done")
+    return audio_output, transcript, assessment
+
+
+def add_note(note):
+    """Add a note to the session"""
+    return speech_assistant.add_note(note)
+
+
+def save_session(student_id):
+    """Save the current session"""
+    return speech_assistant.save_session(student_id)
+
+
+# Create Gradio Interface
+with gr.Blocks(title="CASL-2 Speech Pathology Assistant") as app:
+    gr.Markdown("# CASL-2 Speech Pathology Assistant")
+    gr.Markdown("### AI-powered speech therapy assessment based on the CASL-2 framework")
+    
+    # Show LiveKit availability
+    if not LIVEKIT_AVAILABLE:
+        gr.Markdown(
+            "⚠️ **Notice:** Running without LiveKit agents. Using direct OpenAI API instead. "
+            "For best performance, install livekit-agents package."
+        )
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            student_id = gr.Textbox(label="Student ID (optional)", placeholder="Enter student ID")
+            voice_select = gr.Dropdown(
+                ["alloy", "echo", "fable", "onyx", "nova", "shimmer"], 
+                value="shimmer", 
+                label="Assistant Voice"
+            )
+            start_button = gr.Button("Start Session", variant="primary")
+            stop_button = gr.Button("Stop Session", variant="stop")
+            status = gr.Textbox(label="Status", value="Ready to start")
+            
+            with gr.Accordion("SLP Tools", open=True):
+                note_input = gr.Textbox(
+                    label="Add Assessment Note", 
+                    placeholder="Enter observation or assessment note here..."
+                )
+                note_button = gr.Button("Add Note")
+                note_status = gr.Textbox(label="Note Status")
+                save_button = gr.Button("Save Session")
+                save_status = gr.Textbox(label="Save Status")
+        
+        with gr.Column(scale=2):
+            audio_output = gr.Audio(label="AI Speech", autoplay=True)
+            audio_input = gr.Audio(
+                label="Speak to the AI",
+                type="microphone",
+                source="microphone", 
+                streaming=True
+            )
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            assessment_html = gr.HTML(label="Assessment Progress")
+        with gr.Column(scale=1):
+            transcript = gr.Textbox(label="Transcript", lines=10)
+    
+    with gr.Accordion("About This Application", open=False):
+        gr.Markdown("""
+        ### About CASL-2 Speech Pathology Assistant
+        
+        This application provides an AI speech pathologist that can assess students using the CASL-2 framework. It focuses on:
+        
+        - **Lexical/Semantic Skills**: Vocabulary knowledge and word usage
+        - **Syntactic Skills**: Grammar and sentence structure
+        - **Supralinguistic Skills**: Higher-level language beyond literal meanings
+        - **Pragmatic Skills**: Social use of language (less emphasis for younger students)
+        
+        The AI will provide structured assessments and exercises to help evaluate speech patterns.
+        
+        ### How to Use
+        
+        1. Optionally enter a Student ID to track sessions
+        2. Select the AI voice you prefer
+        3. Click "Start Session" to begin
+        4. The AI will introduce itself and begin the assessment
+        5. Speak into your microphone when it's your turn
+        6. View the transcript to track the conversation
+        7. SLPs can add notes throughout the session
+        8. Save the session when finished
+        9. Click "Stop Session" when done
+        
+        ### For Speech-Language Pathologists
+        
+        This tool is designed to supplement, not replace, professional SLP services. SLPs can:
+        
+        - Add custom notes during the session
+        - Save session data for later reference
+        - Track progress across multiple sessions
+        - Use the AI as a consistent assessment tool
+        """)
+    
+    # Setup event handlers
+    start_button.click(
+        fn=lambda voice, student: asyncio.run(start_session(voice, student)), 
+        inputs=[voice_select, student_id],
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    stop_button.click(
+        fn=stop_session, 
+        outputs=[status, audio_output, transcript, assessment_html]
+    )
+    note_button.click(
+        fn=add_note,
+        inputs=note_input,
+        outputs=[note_input, note_status, assessment_html]
+    )
+    save_button.click(
+        fn=save_session,
+        inputs=student_id,
+        outputs=save_status
+    )
+    
+    # Setup audio processing
+    audio_input.stream(
+        fn=lambda audio: asyncio.run(process_mic_input(audio)),
+        inputs=audio_input,
+        outputs=[audio_output, transcript, assessment_html]
+    )
+
+
+# Entry point for the application
+if __name__ == "__main__":
+    app.launch(share=True)

livekit_requirements.txt

@@ -0,0 +1,8 @@
+python-dotenv>=1.0.0
+livekit-agents>=0.7.0
+openai>=1.3.0
+gradio>=4.0.0
+soundfile>=0.12.1
+scipy>=1.10.0
+asyncio>=3.4.3
+numpy>=1.24.0

requirements.txt

@@ -0,0 +1,10 @@
+python-dotenv>=1.0.0
+openai>=1.3.0
+gradio>=4.0.0
+soundfile>=0.12.1
+scipy>=1.10.0
+asyncio>=3.4.3
+numpy>=1.24.0
+
+# Optional: LiveKit integration (uncomment to use)
+# livekit-agents>=0.7.0

run.py

@@ -0,0 +1,16 @@
+#!/usr/bin/env python3
+
+"""
+Main entry point for the CASL Voice Bot application.
+Launches the Gradio web interface.
+"""
+
+import argparse
+from app.gradio_app import main
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="CASL Voice Bot - AI Speech Therapist")
+    parser.add_argument("--share", action="store_true", help="Share the app publicly (for HuggingFace)")
+    args = parser.parse_args()
+    
+    main()

run_app.py

@@ -0,0 +1,17 @@
+#!/usr/bin/env python3
+
+"""
+Main entry point for the CASL Voice Bot application.
+Launches the Gradio web interface.
+"""
+
+import argparse
+from app_main import main
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="CASL Voice Bot - AI Speech Therapist")
+    parser.add_argument("--share", action="store_true", help="Share the app publicly")
+    parser.add_argument("--local", action="store_true", help="Run the app locally without sharing")
+    args = parser.parse_args()
+    
+    main(share=not args.local)

run_direct.py

@@ -0,0 +1,16 @@
+#!/usr/bin/env python3
+
+"""
+Run the CASL Voice Bot using direct OpenAI API (no LiveKit)
+"""
+
+import argparse
+from implementations.direct.app import main
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="CASL Voice Bot - AI Speech Therapist (Direct API)")
+    parser.add_argument("--share", action="store_true", help="Share the app publicly")
+    parser.add_argument("--local", action="store_true", help="Run the app locally without sharing")
+    args = parser.parse_args()
+    
+    main(share=not args.local)

run_livekit.py

@@ -0,0 +1,16 @@
+#!/usr/bin/env python3
+
+"""
+Run the CASL Voice Bot using LiveKit and OpenAI real-time capabilities
+"""
+
+import argparse
+from implementations.livekit.app import main
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="CASL Voice Bot with LiveKit - AI Speech Therapist")
+    parser.add_argument("--share", action="store_true", help="Share the app publicly")
+    parser.add_argument("--local", action="store_true", help="Run the app locally without sharing")
+    args = parser.parse_args()
+    
+    main(share=not args.local)