Amodit commited on
Commit
0fc97b8
·
1 Parent(s): 83fdb7b

Complete project overhaul and feature integration

Browse files
.gitignore CHANGED
@@ -1,3 +1,5 @@
 
 
1
  # Byte-compiled / optimized / DLL files
2
  __pycache__/
3
  *.pyc
@@ -9,4 +11,20 @@ __pycache__/
9
  venv/
10
 
11
  # IDE
12
- .vscode/
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # D:\jan-contract\.gitignore
2
+
3
  # Byte-compiled / optimized / DLL files
4
  __pycache__/
5
  *.pyc
 
11
  venv/
12
 
13
  # IDE
14
+ .vscode/
15
+
16
+ # --- NEW: Files and Folders to Ignore ---
17
+
18
+ # Ignore user-uploaded content directories
19
+ pdfs_demystify/
20
+ video_consents/
21
+
22
+ # Ignore temporary test files
23
+ # Using wildcards (*) to catch all of them
24
+ test_*.py
25
+ minimal_*.py
26
+ simple_*.py
27
+ run_app.py # Assuming this is a local runner, not part of the final project
28
+
29
+ # You can also ignore specific files
30
+ # TROUBLESHOOTING.md # Uncomment this line if you DON'T want to save this file
TROUBLESHOOTING.md ADDED
@@ -0,0 +1,116 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 🎥 Audio/Video Troubleshooting Guide
2
+
3
+ ## Common Issues and Solutions
4
+
5
+ ### 1. Video Recording Issues
6
+
7
+ **Problem:** Video recording creates 0-second files or doesn't work at all.
8
+
9
+ **Solutions:**
10
+ - **Browser Compatibility**: Use Chrome, Firefox, or Edge. Safari may have issues.
11
+ - **Camera Permissions**: Make sure to allow camera access when prompted.
12
+ - **HTTPS Required**: Some browsers require HTTPS for camera access. Use `streamlit run --server.address 0.0.0.0 --server.port 8501` for local testing.
13
+ - **Refresh Page**: If buttons don't respond, try refreshing the page.
14
+
15
+ ### 2. Audio Recording Issues
16
+
17
+ **Problem:** Voice input doesn't work or produces no audio.
18
+
19
+ **Solutions:**
20
+ - **Microphone Permissions**: Allow microphone access when prompted.
21
+ - **Browser Settings**: Check browser settings for microphone permissions.
22
+ - **Clear Browser Cache**: Clear browser cache and cookies.
23
+ - **Try Different Browser**: Some browsers handle WebRTC better than others.
24
+
25
+ ### 3. Dependencies Issues
26
+
27
+ **Problem:** Import errors or missing modules.
28
+
29
+ **Solutions:**
30
+ ```bash
31
+ # Install all dependencies
32
+ pip install -r requirements.txt
33
+
34
+ # If you get errors, try installing individually:
35
+ pip install streamlit-webrtc
36
+ pip install opencv-python-headless
37
+ pip install av
38
+ pip install SpeechRecognition
39
+ pip install gTTS
40
+ pip install PyAudio
41
+ ```
42
+
43
+ ### 4. Windows-Specific Issues
44
+
45
+ **Problem:** PyAudio installation fails on Windows.
46
+
47
+ **Solutions:**
48
+ ```bash
49
+ # Try installing PyAudio with pipwin
50
+ pip install pipwin
51
+ pipwin install pyaudio
52
+
53
+ # Or download from: https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio
54
+ ```
55
+
56
+ ### 5. Performance Issues
57
+
58
+ **Problem:** Slow video/audio processing.
59
+
60
+ **Solutions:**
61
+ - **Reduce Video Quality**: The app uses 640x480 resolution by default.
62
+ - **Close Other Apps**: Close other applications using camera/microphone.
63
+ - **Check System Resources**: Ensure sufficient RAM and CPU available.
64
+
65
+ ## Testing Your Setup
66
+
67
+ Run the test script to verify everything is working:
68
+
69
+ ```bash
70
+ streamlit run test_audio_video.py
71
+ ```
72
+
73
+ This will check:
74
+ - ✅ All dependencies are installed
75
+ - ✅ Directories are writable
76
+ - ✅ Basic functionality
77
+
78
+ ## Browser Requirements
79
+
80
+ - **Chrome**: Best compatibility
81
+ - **Firefox**: Good compatibility
82
+ - **Edge**: Good compatibility
83
+ - **Safari**: Limited compatibility (not recommended)
84
+
85
+ ## Network Requirements
86
+
87
+ - **Local Development**: Works fine on localhost
88
+ - **Production**: HTTPS required for camera/microphone access
89
+ - **Firewall**: Ensure ports 8501 (or your chosen port) is accessible
90
+
91
+ ## Error Messages and Solutions
92
+
93
+ | Error | Solution |
94
+ |-------|----------|
95
+ | "Camera not found" | Check camera permissions and browser settings |
96
+ | "Microphone not found" | Check microphone permissions and browser settings |
97
+ | "WebRTC not supported" | Update browser or try different browser |
98
+ | "Permission denied" | Allow camera/microphone access in browser |
99
+ | "Video file too small" | Record for at least 2-3 seconds |
100
+
101
+ ## Getting Help
102
+
103
+ If you're still having issues:
104
+
105
+ 1. Check the browser console for JavaScript errors
106
+ 2. Run the test script: `streamlit run test_audio_video.py`
107
+ 3. Check if your camera/microphone work in other applications
108
+ 4. Try a different browser
109
+ 5. Restart the Streamlit server
110
+
111
+ ## Development Tips
112
+
113
+ - Use `st.debug()` to add debugging information
114
+ - Check `st.session_state` for state management issues
115
+ - Monitor browser console for WebRTC errors
116
+ - Test on different devices and browsers
agents/general_assistant_agent.py ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # D:\jan-contract\agents\general_assistant_agent.py
2
+
3
+ import os
4
+ import google.generativeai as genai
5
+
6
+ # Configure the API key from the .env file
7
+ try:
8
+ genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))
9
+ # Use a specific, robust model name
10
+ model = genai.GenerativeModel('gemini-1.5-flash')
11
+ except Exception as e:
12
+ print(f"Error configuring Google Generative AI: {e}")
13
+ model = None
14
+
15
+ def ask_gemini(prompt: str) -> str:
16
+ """
17
+ Sends a prompt directly to the Google Gemini API and returns the text response.
18
+ This is the core logic from your script, adapted for our application.
19
+ """
20
+ if model is None:
21
+ return "Error: The Generative AI model is not configured. Please check your API key."
22
+
23
+ try:
24
+ response = model.generate_content(prompt)
25
+ return response.text
26
+ except Exception as e:
27
+ return f"An error occurred while communicating with the Gemini API: {str(e)}"
components/chat_interface.py ADDED
@@ -0,0 +1,227 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # D:\jan-contract\components/chat_interface.py
2
+
3
+ import streamlit as st
4
+ import speech_recognition as sr
5
+ from gtts import gTTS
6
+ import io
7
+ import av
8
+ import queue
9
+ import wave
10
+ import threading
11
+ import time
12
+ import numpy as np
13
+ from typing import Optional
14
+
15
+ from streamlit_webrtc import webrtc_streamer, WebRtcMode
16
+
17
+ # --- Setup ---
18
+ recognizer = sr.Recognizer()
19
+ recognizer.energy_threshold = 300 # Lower threshold for better sensitivity
20
+ recognizer.dynamic_energy_threshold = True
21
+ recognizer.pause_threshold = 0.8
22
+
23
+ def text_to_speech(text: str) -> bytes:
24
+ """Converts text to an in-memory MP3 file bytes."""
25
+ try:
26
+ audio_io = io.BytesIO()
27
+ tts = gTTS(text=text, lang='en', slow=False)
28
+ tts.write_to_fp(audio_io)
29
+ audio_io.seek(0)
30
+ return audio_io.read()
31
+ except Exception as e:
32
+ st.error(f"Error during Text-to-Speech: {e}")
33
+ return None
34
+
35
+ def chat_interface(handler_function, session_state_key: str):
36
+ """
37
+ A reusable component that provides a full Text and Voice chat interface.
38
+
39
+ Args:
40
+ handler_function: The function to call with the user's text input.
41
+ session_state_key (str): A unique key to store chat history AND to use
42
+ as a base for widget keys.
43
+ """
44
+ st.subheader("💬 Chat via Text")
45
+
46
+ if session_state_key not in st.session_state:
47
+ st.session_state[session_state_key] = []
48
+
49
+ for message in st.session_state[session_state_key]:
50
+ with st.chat_message(message["role"]):
51
+ st.markdown(message["content"])
52
+
53
+ if prompt := st.chat_input("Ask a question...", key=f"chat_input_{session_state_key}"):
54
+ st.session_state[session_state_key].append({"role": "user", "content": prompt})
55
+ with st.chat_message("user"):
56
+ st.markdown(prompt)
57
+
58
+ with st.chat_message("assistant"):
59
+ with st.spinner("Thinking..."):
60
+ response = handler_function(prompt)
61
+ st.markdown(response)
62
+
63
+ st.session_state[session_state_key].append({"role": "assistant", "content": response})
64
+
65
+ st.divider()
66
+
67
+ st.subheader("🎙️ Chat via Voice")
68
+ st.info("🎤 **Instructions:** Click START to begin recording, speak your question clearly, then click STOP.")
69
+
70
+ # Initialize session state for voice recording
71
+ voice_key = f"voice_{session_state_key}"
72
+ if f"{voice_key}_frames" not in st.session_state:
73
+ st.session_state[f"{voice_key}_frames"] = []
74
+ if f"{voice_key}_processing" not in st.session_state:
75
+ st.session_state[f"{voice_key}_processing"] = False
76
+ if f"{voice_key}_recording_start" not in st.session_state:
77
+ st.session_state[f"{voice_key}_recording_start"] = None
78
+ if f"{voice_key}_bytes" not in st.session_state:
79
+ st.session_state[f"{voice_key}_bytes"] = 0
80
+ if f"{voice_key}_component_key" not in st.session_state:
81
+ st.session_state[f"{voice_key}_component_key"] = f"voice-chat-{session_state_key}-{int(time.time())}"
82
+
83
+ def audio_frame_callback(frame: av.AudioFrame):
84
+ """Callback to collect audio frames during recording"""
85
+ if st.session_state[f"{voice_key}_processing"]:
86
+ try:
87
+ # Resample every frame to 16kHz mono, 16-bit PCM for SR
88
+ resampled = frame.reformat(format="s16", layout="mono", rate=16000)
89
+ chunk = resampled.planes[0].to_bytes()
90
+ st.session_state[f"{voice_key}_frames"].append(chunk)
91
+ st.session_state[f"{voice_key}_bytes"] += len(chunk)
92
+ except Exception as e:
93
+ st.error(f"Error processing audio frame: {e}")
94
+
95
+ def process_voice_input():
96
+ """Process the collected audio frames and get response"""
97
+ # Short-audio threshold (~0.5s at 16kHz, 16-bit mono)
98
+ total_bytes = st.session_state.get(f"{voice_key}_bytes", 0)
99
+ if total_bytes < int(16000 * 2 * 0.5):
100
+ st.error("❌ No audio captured or recording too short. Please speak for at least 1 second and try again.")
101
+ st.session_state[f"{voice_key}_frames"] = []
102
+ st.session_state[f"{voice_key}_processing"] = False
103
+ st.session_state[f"{voice_key}_bytes"] = 0
104
+ return
105
+
106
+ status_placeholder = st.empty()
107
+ status_placeholder.info("🔄 Processing audio...")
108
+
109
+ try:
110
+ # Combine all audio frames (already PCM s16 mono 16kHz)
111
+ audio_data = b"".join(st.session_state[f"{voice_key}_frames"])
112
+
113
+ # Create WAV file in memory with proper format
114
+ with io.BytesIO() as wav_buffer:
115
+ with wave.open(wav_buffer, 'wb') as wf:
116
+ wf.setnchannels(1) # Mono
117
+ wf.setsampwidth(2) # 16-bit
118
+ wf.setframerate(16000) # 16kHz
119
+ wf.writeframes(audio_data)
120
+ wav_buffer.seek(0)
121
+
122
+ # Use speech recognition with better error handling
123
+ with sr.AudioFile(wav_buffer) as source:
124
+ # Adjust for ambient noise quickly; avoid long pauses
125
+ recognizer.adjust_for_ambient_noise(source, duration=0.1)
126
+ audio = recognizer.record(source)
127
+
128
+ # Recognize speech with multiple fallbacks
129
+ try:
130
+ user_input = recognizer.recognize_google(audio, language="en-US")
131
+ except sr.UnknownValueError:
132
+ try:
133
+ user_input = recognizer.recognize_google(audio, language="en-GB")
134
+ except sr.UnknownValueError:
135
+ st.error("❌ Could not understand the audio. Please speak more clearly and try again.")
136
+ return
137
+
138
+ if not user_input.strip():
139
+ st.error("❌ No speech detected. Please try again.")
140
+ return
141
+
142
+ st.write(f"🎤 **You said:** *{user_input}*")
143
+
144
+ # Get response from handler
145
+ with st.spinner("🤔 Getting response..."):
146
+ response_text = handler_function(user_input)
147
+
148
+ st.write(f"🤖 **Assistant says:** *{response_text}*")
149
+
150
+ # Generate audio response
151
+ with st.spinner("🔊 Generating audio response..."):
152
+ audio_response = text_to_speech(response_text)
153
+ if audio_response:
154
+ st.audio(audio_response, format="audio/mp3", start_time=0)
155
+ st.success("✅ Audio response generated!")
156
+
157
+ # Add to chat history
158
+ st.session_state[session_state_key].append({"role": "user", "content": user_input})
159
+ st.session_state[session_state_key].append({"role": "assistant", "content": response_text})
160
+
161
+ except sr.RequestError as e:
162
+ st.error(f"❌ Speech recognition service error: {e}")
163
+ except Exception as e:
164
+ st.error(f"❌ Error processing audio: {str(e)}")
165
+ finally:
166
+ # Clear the audio frames
167
+ st.session_state[f"{voice_key}_frames"] = []
168
+ st.session_state[f"{voice_key}_processing"] = False
169
+ st.session_state[f"{voice_key}_bytes"] = 0
170
+ status_placeholder.empty()
171
+
172
+ # Create a unique key for each component instance to avoid registration issues
173
+ component_key = st.session_state[f"{voice_key}_component_key"]
174
+
175
+ # WebRTC streamer with proper error handling and component lifecycle
176
+ try:
177
+ ctx = webrtc_streamer(
178
+ key=component_key,
179
+ mode=WebRtcMode.SENDONLY,
180
+ rtc_configuration={
181
+ "iceServers": [
182
+ {"urls": ["stun:stun.l.google.com:19302"]},
183
+ {"urls": ["stun:stun1.l.google.com:19302"]}
184
+ ]
185
+ },
186
+ audio_frame_callback=audio_frame_callback,
187
+ media_stream_constraints={
188
+ "video": False,
189
+ "audio": {
190
+ "echoCancellation": True,
191
+ "noiseSuppression": True,
192
+ "autoGainControl": True
193
+ }
194
+ },
195
+ async_processing=True,
196
+ on_change=lambda: None, # Prevent component registration issues
197
+ )
198
+
199
+ # Handle recording state with better feedback
200
+ bytes_captured = st.session_state.get(f"{voice_key}_bytes", 0)
201
+
202
+ if ctx.state.playing and not st.session_state.get(f"{voice_key}_processing", False):
203
+ st.session_state[f"{voice_key}_processing"] = True
204
+ st.session_state[f"{voice_key}_recording_start"] = time.time()
205
+ st.session_state[f"{voice_key}_frames"] = []
206
+ st.session_state[f"{voice_key}_bytes"] = 0
207
+ st.success("🔴 **Recording started!** Speak your question now...")
208
+
209
+ elif ctx.state.playing and st.session_state.get(f"{voice_key}_processing", False):
210
+ # Show recording progress
211
+ if st.session_state.get(f"{voice_key}_recording_start"):
212
+ elapsed = time.time() - st.session_state[f"{voice_key}_recording_start"]
213
+ approx_seconds = bytes_captured / (16000 * 2) if bytes_captured else 0
214
+ st.caption(f"🎤 Recording... ~{approx_seconds:.1f}s captured")
215
+
216
+ # Process audio when recording stops
217
+ if not ctx.state.playing and st.session_state.get(f"{voice_key}_processing", False):
218
+ process_voice_input()
219
+
220
+ except Exception as e:
221
+ st.error(f"❌ WebRTC Error: {str(e)}")
222
+ st.info("💡 Try refreshing the page or using a different browser (Chrome recommended).")
223
+
224
+ # Fallback: manual audio input
225
+ st.subheader("🔄 Fallback: Manual Audio Input")
226
+ if st.button("Try Alternative Audio Method", key=f"fallback_{voice_key}"):
227
+ st.info("This feature requires WebRTC support. Please ensure your browser supports WebRTC and try again.")
components/video_recorder.py CHANGED
@@ -4,6 +4,8 @@ import os
4
  import streamlit as st
5
  import datetime
6
  import av
 
 
7
 
8
  from streamlit_webrtc import webrtc_streamer, WebRtcMode
9
 
@@ -12,75 +14,127 @@ os.makedirs(VIDEO_CONSENT_DIR, exist_ok=True)
12
 
13
  def record_consent_video():
14
  """
15
- Encapsulates the video recording logic using the component's internal state.
16
-
17
- The video is automatically saved when the user clicks the "STOP" button
18
- on the webrtc component.
19
 
20
  Returns:
21
  str | None: The file path of the saved video, or None if not saved yet.
22
  """
23
- # Instructions for the new, more intuitive workflow
24
- st.info("Instructions: Click START, record your consent, then click STOP to finalize.")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
 
 
26
  webrtc_ctx = webrtc_streamer(
27
  key="video-consent-recorder",
28
- mode=WebRtcMode.SENDRECV, # SENDRECV mode is needed for the stop-button-triggered callback
29
- media_stream_constraints={"video": True, "audio": True},
30
- video_receiver_size=256,
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  async_processing=True,
32
  )
33
 
34
- # This block executes ONLY when the component is running (after START is clicked)
35
- if webrtc_ctx.state.playing and webrtc_ctx.video_receiver:
36
- # Inform the user that recording is in progress
37
- st.success("🔴 Recording in progress...")
38
-
39
- # If the 'frames_buffer' is not in session state, initialize it
40
- if "frames_buffer" not in st.session_state:
41
- st.session_state.frames_buffer = []
42
 
43
- # Append each new frame to our session state buffer
44
- while True:
45
- try:
46
- frame = webrtc_ctx.video_receiver.get_frame(timeout=1)
47
- st.session_state.frames_buffer.append(frame)
48
- except av.error.TimeoutError:
49
- break # Break the loop when the stream ends (user clicks STOP)
50
 
51
- # This block executes after the user clicks STOP
52
- if not webrtc_ctx.state.playing and st.session_state.get("frames_buffer"):
53
- with st.spinner("Saving your recording..."):
 
 
 
54
  try:
55
- video_frames = st.session_state.frames_buffer
 
 
 
 
 
 
56
 
57
- # Generate a unique filename
58
  timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
59
  video_filename = os.path.join(VIDEO_CONSENT_DIR, f"consent_{timestamp}.mp4")
60
-
61
- # Use the av library to write the buffered frames to a video file
62
- with av.open(video_filename, mode="w") as container:
63
- stream = container.add_stream("libx264", rate=24)
64
- stream.width = video_frames[0].width
65
- stream.height = video_frames[0].height
66
- stream.pix_fmt = "yuv420p"
67
-
68
- for frame in video_frames:
69
- packet = stream.encode(frame)
70
- container.mux(packet)
71
-
72
- # Flush the stream
73
- packet = stream.encode()
74
- container.mux(packet)
75
 
76
- # Clear the buffer from session state and return the path
77
- st.session_state.frames_buffer = []
78
- st.session_state.video_filename = video_filename
79
- return video_filename
80
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
81
  except Exception as e:
82
- st.error(f"An error occurred while saving the video: {e}")
83
- st.session_state.frames_buffer = [] # Clear buffer on error
84
  return None
85
 
 
 
 
 
86
  return None
 
4
  import streamlit as st
5
  import datetime
6
  import av
7
+ import numpy as np
8
+ from typing import Optional
9
 
10
  from streamlit_webrtc import webrtc_streamer, WebRtcMode
11
 
 
14
 
15
  def record_consent_video():
16
  """
17
+ Improved video recording component with better error handling and reliability.
 
 
 
18
 
19
  Returns:
20
  str | None: The file path of the saved video, or None if not saved yet.
21
  """
22
+ st.info("🎥 **Instructions:** Click START to begin recording, speak your consent, then click STOP to save.")
23
+
24
+ # Initialize session state for video recording
25
+ if "video_frames_buffer" not in st.session_state:
26
+ st.session_state.video_frames_buffer = []
27
+ if "video_recording" not in st.session_state:
28
+ st.session_state.video_recording = False
29
+ if "video_processed" not in st.session_state:
30
+ st.session_state.video_processed = False
31
+ if "recording_start_time" not in st.session_state:
32
+ st.session_state.recording_start_time = None
33
+
34
+ def video_frame_callback(frame: av.VideoFrame):
35
+ """Callback to collect video frames during recording"""
36
+ if st.session_state.video_recording:
37
+ try:
38
+ # Convert frame to numpy array for easier handling
39
+ img = frame.to_ndarray(format="bgr24")
40
+ st.session_state.video_frames_buffer.append(img)
41
+ except Exception as e:
42
+ st.error(f"Error processing video frame: {e}")
43
 
44
+ # WebRTC streamer configuration
45
  webrtc_ctx = webrtc_streamer(
46
  key="video-consent-recorder",
47
+ mode=WebRtcMode.SENDONLY,
48
+ rtc_configuration={
49
+ "iceServers": [
50
+ {"urls": ["stun:stun.l.google.com:19302"]},
51
+ {"urls": ["stun:stun1.l.google.com:19302"]}
52
+ ]
53
+ },
54
+ media_stream_constraints={
55
+ "video": {
56
+ "width": {"ideal": 640},
57
+ "height": {"ideal": 480},
58
+ "frameRate": {"ideal": 30}
59
+ },
60
+ "audio": False
61
+ },
62
+ video_frame_callback=video_frame_callback,
63
  async_processing=True,
64
  )
65
 
66
+ # Handle recording state
67
+ if webrtc_ctx.state.playing and not st.session_state.video_recording:
68
+ st.session_state.video_recording = True
69
+ st.session_state.video_processed = False
70
+ st.session_state.recording_start_time = datetime.datetime.now()
71
+ st.session_state.video_frames_buffer = [] # Clear previous buffer
72
+ st.success("🔴 **Recording started!** Speak your consent now...")
 
73
 
74
+ elif webrtc_ctx.state.playing and st.session_state.video_recording:
75
+ # Show recording progress
76
+ frames_captured = len(st.session_state.video_frames_buffer)
77
+ if st.session_state.recording_start_time:
78
+ elapsed = (datetime.datetime.now() - st.session_state.recording_start_time).total_seconds()
79
+ st.caption(f"📹 Recording... Frames: {frames_captured} | Duration: {elapsed:.1f}s")
 
80
 
81
+ # Process video when recording stops
82
+ if not webrtc_ctx.state.playing and st.session_state.video_recording and not st.session_state.video_processed:
83
+ st.session_state.video_recording = False
84
+ st.session_state.video_processed = True
85
+
86
+ with st.spinner("💾 Processing and saving your recording..."):
87
  try:
88
+ video_frames = st.session_state.video_frames_buffer.copy()
89
+
90
+ # Enhanced validation
91
+ if len(video_frames) < 30: # At least 1 second at 30fps
92
+ st.warning(f"⚠️ Recording too short ({len(video_frames)} frames). Please record for at least 2-3 seconds.")
93
+ st.session_state.video_frames_buffer = []
94
+ return None
95
 
96
+ # Generate unique filename
97
  timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
98
  video_filename = os.path.join(VIDEO_CONSENT_DIR, f"consent_{timestamp}.mp4")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99
 
100
+ # Get video dimensions from first frame
101
+ height, width = video_frames[0].shape[:2]
102
+ fps = 30
103
+
104
+ # Use OpenCV for more reliable video writing
105
+ import cv2
106
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v')
107
+ out = cv2.VideoWriter(video_filename, fourcc, fps, (width, height))
108
+
109
+ # Write frames
110
+ for frame in video_frames:
111
+ out.write(frame)
112
+
113
+ out.release()
114
+
115
+ # Verify the video was created successfully
116
+ if os.path.exists(video_filename) and os.path.getsize(video_filename) > 1000:
117
+ # Clear the buffer
118
+ st.session_state.video_frames_buffer = []
119
+ st.session_state.video_filename = video_filename
120
+
121
+ # Calculate duration
122
+ duration = len(video_frames) / fps
123
+ st.success(f"✅ **Video saved successfully!**")
124
+ st.caption(f"📊 Duration: {duration:.1f}s | Frames: {len(video_frames)} | Size: {os.path.getsize(video_filename)/1024:.1f}KB")
125
+
126
+ return video_filename
127
+ else:
128
+ st.error("❌ Failed to save video file properly.")
129
+ return None
130
+
131
  except Exception as e:
132
+ st.error(f" Error saving video: {str(e)}")
133
+ st.session_state.video_frames_buffer = []
134
  return None
135
 
136
+ # Show recording status
137
+ if st.session_state.video_recording:
138
+ st.info("🎥 **Recording in progress...** Click STOP when finished.")
139
+
140
  return None
main_streamlit.py CHANGED
@@ -4,11 +4,13 @@ import os
4
  import streamlit as st
5
  from dotenv import load_dotenv
6
 
 
7
  from agents.demystifier_agent import process_document_for_demystification
8
  from components.video_recorder import record_consent_video
9
  from utils.pdf_generator import generate_formatted_pdf
10
-
11
- # --- Initial Setup ---
 
12
  load_dotenv()
13
  st.set_page_config(layout="wide", page_title="Jan-Contract Unified Assistant")
14
  st.title("Jan-Contract: Your Digital Workforce Assistant")
@@ -16,11 +18,12 @@ st.title("Jan-Contract: Your Digital Workforce Assistant")
16
  PDF_UPLOAD_DIR = "pdfs_demystify"
17
  os.makedirs(PDF_UPLOAD_DIR, exist_ok=True)
18
 
19
- # --- Tabs ---
20
- tab1, tab2, tab3 = st.tabs([
21
- " **Contract Generator**",
22
- " **Scheme Finder**",
23
- " **Document Demystifier & Chat**"
 
24
  ])
25
 
26
  # --- TAB 1: Contract Generator ---
@@ -31,7 +34,8 @@ with tab1:
31
  st.subheader("Step 1: Describe and Generate Your Agreement")
32
  user_request = st.text_area("Describe the agreement...", height=120, key="contract_request")
33
 
34
- if st.button("Generate Document & Get Legal Info", type="primary"):
 
35
  if user_request:
36
  with st.spinner("Generating document..."):
37
  from agents.legal_agent import legal_agent
@@ -41,7 +45,7 @@ with tab1:
41
  if 'video_path_from_component' in st.session_state:
42
  del st.session_state['video_path_from_component']
43
  if 'frames_buffer' in st.session_state:
44
- del st.session_state['frames_buffer'] # Clear old frames
45
  else:
46
  st.error("Please describe the agreement.")
47
 
@@ -57,11 +61,22 @@ with tab1:
57
 
58
  with col2:
59
  st.subheader("Relevant Legal Trivia")
60
- # ... [Trivia display logic] ...
 
 
 
 
 
 
 
61
 
62
  st.divider()
63
 
64
  st.subheader("Step 2: Record Video Consent for this Agreement")
 
 
 
 
65
  saved_video_path = record_consent_video()
66
 
67
  if saved_video_path:
@@ -71,6 +86,9 @@ with tab1:
71
  st.success("✅ Your consent has been recorded and saved!")
72
  st.video(st.session_state.video_path_from_component)
73
  st.info("This video is now linked to your generated agreement.")
 
 
 
74
  # --- TAB 2: Scheme Finder (Unchanged) ---
75
  with tab2:
76
  st.header("Find Relevant Government Schemes")
@@ -81,7 +99,6 @@ with tab2:
81
  if st.button("Find Schemes", type="primary", key="b2"):
82
  if user_profile:
83
  with st.spinner("Initializing models and searching for schemes..."):
84
- # Lazy import the agent
85
  from agents.scheme_chatbot import scheme_chatbot
86
  response = scheme_chatbot.invoke({"user_profile": user_profile})
87
  st.session_state.scheme_response = response
@@ -98,67 +115,68 @@ with tab2:
98
  st.write(f"**Description:** {scheme.description}")
99
  st.link_button("Go to Official Page ➡️", scheme.official_link)
100
 
101
- # --- TAB 3: Demystifier & Chat (RESTORED to original functionality) ---
102
  with tab3:
103
- st.header("Simplify & Chat With Your Legal Document")
104
- st.markdown("Get a plain-English summary of your document, then ask specific follow-up questions.")
105
 
106
  uploaded_file = st.file_uploader("Choose a PDF document", type="pdf", key="demystify_uploader")
107
 
 
108
  if uploaded_file and st.button("Analyze Document", type="primary"):
109
  with st.spinner("Performing deep analysis and preparing for chat..."):
110
- # Save the file to a persistent location
111
  temp_file_path = os.path.join(PDF_UPLOAD_DIR, uploaded_file.name)
112
  with open(temp_file_path, "wb") as f:
113
  f.write(uploaded_file.getbuffer())
114
 
115
- # Single call to the backend agent logic
116
  analysis_result = process_document_for_demystification(temp_file_path)
117
 
118
- # Store the results returned by the agent
119
- st.session_state.demystify_report = analysis_result["report"]
120
  st.session_state.rag_chain = analysis_result["rag_chain"]
121
- st.session_state.messages = [] # Initialize chat history
122
 
123
- # This part of the UI only displays after the analysis is complete
124
- if 'demystify_report' in st.session_state:
125
- # Step 1: Display Report
126
- report = st.session_state.demystify_report
127
  st.divider()
128
  st.header("Step 1: Automated Document Analysis")
 
129
  with st.container(border=True):
130
  st.subheader("📄 Document Summary")
131
  st.write(report.summary)
132
  st.divider()
 
133
  st.subheader("🔑 Key Terms Explained")
134
  for term in report.key_terms:
135
  with st.expander(f"**{term.term}**"):
136
  st.write(term.explanation)
137
  st.markdown(f"[Learn More Here]({term.resource_link})")
138
  st.divider()
 
139
  st.success(f"**Overall Advice:** {report.overall_advice}")
 
140
  st.divider()
141
 
142
- # Step 2: Display Chat
143
  st.header("Step 2: Ask Follow-up Questions")
144
- st.info("The document is now ready for your questions. Chat with it below.")
145
-
146
- for message in st.session_state.get("messages", []):
147
- with st.chat_message(message["role"]):
148
- st.markdown(message["content"])
149
-
150
- if prompt := st.chat_input("Ask a specific question about the document..."):
151
- st.session_state.messages.append({"role": "user", "content": prompt})
152
- with st.chat_message("user"):
153
- st.markdown(prompt)
154
-
155
- with st.chat_message("assistant"):
156
- with st.spinner("Searching the document..."):
157
- rag_chain = st.session_state.rag_chain
158
- response = rag_chain.invoke(prompt)
159
- st.markdown(response)
160
 
161
- st.session_state.messages.append({"role": "assistant", "content": response})
162
-
163
  elif not uploaded_file:
164
- st.info("Upload a PDF document to begin the analysis.")
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  import streamlit as st
5
  from dotenv import load_dotenv
6
 
7
+ # --- Agent and Component Imports (Cleaned up) ---
8
  from agents.demystifier_agent import process_document_for_demystification
9
  from components.video_recorder import record_consent_video
10
  from utils.pdf_generator import generate_formatted_pdf
11
+ from components.chat_interface import chat_interface
12
+ from agents.general_assistant_agent import ask_gemini
13
+ # --- 1. Initial Setup ---
14
  load_dotenv()
15
  st.set_page_config(layout="wide", page_title="Jan-Contract Unified Assistant")
16
  st.title("Jan-Contract: Your Digital Workforce Assistant")
 
18
  PDF_UPLOAD_DIR = "pdfs_demystify"
19
  os.makedirs(PDF_UPLOAD_DIR, exist_ok=True)
20
 
21
+ # --- 2. Streamlit UI with Tabs ---
22
+ tab1, tab2, tab3, tab4 = st.tabs([
23
+ "📝 **Contract Generator**",
24
+ "🏦 **Scheme Finder**",
25
+ "📜 **Document Demystifier & Chat**",
26
+ "🤖 **General Assistant**"
27
  ])
28
 
29
  # --- TAB 1: Contract Generator ---
 
34
  st.subheader("Step 1: Describe and Generate Your Agreement")
35
  user_request = st.text_area("Describe the agreement...", height=120, key="contract_request")
36
 
37
+ # --- FIX: Added a unique key="b1" for consistency ---
38
+ if st.button("Generate Document & Get Legal Info", type="primary", key="b1"):
39
  if user_request:
40
  with st.spinner("Generating document..."):
41
  from agents.legal_agent import legal_agent
 
45
  if 'video_path_from_component' in st.session_state:
46
  del st.session_state['video_path_from_component']
47
  if 'frames_buffer' in st.session_state:
48
+ del st.session_state['frames_buffer']
49
  else:
50
  st.error("Please describe the agreement.")
51
 
 
61
 
62
  with col2:
63
  st.subheader("Relevant Legal Trivia")
64
+ # --- FIX: Restored the missing trivia display logic ---
65
+ if result.get('legal_trivia') and result['legal_trivia'].trivia:
66
+ for item in result['legal_trivia'].trivia:
67
+ st.markdown(f"- **{item.point}**")
68
+ st.caption(item.explanation)
69
+ st.markdown(f"[Source Link]({item.source_url})")
70
+ else:
71
+ st.write("Could not retrieve structured legal trivia.")
72
 
73
  st.divider()
74
 
75
  st.subheader("Step 2: Record Video Consent for this Agreement")
76
+
77
+ # Browser compatibility check
78
+ st.info("🌐 **Browser Requirements:** This feature works best in Chrome, Firefox, or Edge. Make sure to allow camera access when prompted.")
79
+
80
  saved_video_path = record_consent_video()
81
 
82
  if saved_video_path:
 
86
  st.success("✅ Your consent has been recorded and saved!")
87
  st.video(st.session_state.video_path_from_component)
88
  st.info("This video is now linked to your generated agreement.")
89
+ else:
90
+ st.info("💡 **Tip:** If video recording isn't working, try refreshing the page and allowing camera permissions.")
91
+
92
  # --- TAB 2: Scheme Finder (Unchanged) ---
93
  with tab2:
94
  st.header("Find Relevant Government Schemes")
 
99
  if st.button("Find Schemes", type="primary", key="b2"):
100
  if user_profile:
101
  with st.spinner("Initializing models and searching for schemes..."):
 
102
  from agents.scheme_chatbot import scheme_chatbot
103
  response = scheme_chatbot.invoke({"user_profile": user_profile})
104
  st.session_state.scheme_response = response
 
115
  st.write(f"**Description:** {scheme.description}")
116
  st.link_button("Go to Official Page ➡️", scheme.official_link)
117
 
118
+ # --- TAB 3: Demystifier & Chat ---
119
  with tab3:
120
+ st.header("📜 Simplify & Chat With Your Legal Document")
121
+ st.markdown("Get a plain-English summary of your document, then ask questions using text or your voice.")
122
 
123
  uploaded_file = st.file_uploader("Choose a PDF document", type="pdf", key="demystify_uploader")
124
 
125
+ # This button triggers the one-time analysis and embedding process
126
  if uploaded_file and st.button("Analyze Document", type="primary"):
127
  with st.spinner("Performing deep analysis and preparing for chat..."):
128
+ # Save the uploaded file to a temporary location for processing
129
  temp_file_path = os.path.join(PDF_UPLOAD_DIR, uploaded_file.name)
130
  with open(temp_file_path, "wb") as f:
131
  f.write(uploaded_file.getbuffer())
132
 
133
+ # Call the master controller function from the agent
134
  analysis_result = process_document_for_demystification(temp_file_path)
135
 
136
+ # Store the two key results in the session state
137
+ st.session_state.demystifier_report = analysis_result["report"]
138
  st.session_state.rag_chain = analysis_result["rag_chain"]
 
139
 
140
+ # This UI section only appears after a document has been successfully analyzed
141
+ if 'demystifier_report' in st.session_state:
 
 
142
  st.divider()
143
  st.header("Step 1: Automated Document Analysis")
144
+ report = st.session_state.demystifier_report
145
  with st.container(border=True):
146
  st.subheader("📄 Document Summary")
147
  st.write(report.summary)
148
  st.divider()
149
+
150
  st.subheader("🔑 Key Terms Explained")
151
  for term in report.key_terms:
152
  with st.expander(f"**{term.term}**"):
153
  st.write(term.explanation)
154
  st.markdown(f"[Learn More Here]({term.resource_link})")
155
  st.divider()
156
+
157
  st.success(f"**Overall Advice:** {report.overall_advice}")
158
+
159
  st.divider()
160
 
 
161
  st.header("Step 2: Ask Follow-up Questions")
162
+ # Call our reusable chat component, passing the RAG chain specific to this document.
163
+ # The RAG chain's .invoke method is the handler function.
164
+ chat_interface(
165
+ handler_function=st.session_state.rag_chain.invoke,
166
+ session_state_key="doc_chat_history" # Use a unique key for this chat's history
167
+ )
 
 
 
 
 
 
 
 
 
 
168
 
 
 
169
  elif not uploaded_file:
170
+ st.info("Upload a PDF document to begin analysis and enable chat.")
171
+
172
+ # --- TAB 4: General Assistant (Complete) ---
173
+ with tab4:
174
+ st.header("🤖 General Assistant")
175
+ st.markdown("Ask a general question and get a response directly from the Gemini AI model. You can use text or your voice.")
176
+
177
+ # Call our reusable chat component.
178
+ # This time, we pass the simple `ask_gemini` function as the handler.
179
+ chat_interface(
180
+ handler_function=ask_gemini,
181
+ session_state_key="general_chat_history" # Use a different key for this chat's history
182
+ )
requirements.txt CHANGED
@@ -1,33 +1,42 @@
1
  # D:\jan-contract\requirements.txt
2
 
3
  # Core LangChain libraries
4
- langchain-core
5
- langchain
6
- langchain-community
7
- langgraph
8
 
9
  # LLM Integrations
10
- langchain_google_genai
11
- langchain-groq
 
12
 
13
  # Tooling
14
- tavily-python
15
- pypdf
16
- pymupdf
17
- fastembed
18
- faiss-cpu
19
- python-multipart
 
20
  # Web Frameworks
21
- fastapi
22
- uvicorn
23
- streamlit
 
 
 
 
 
 
 
 
24
 
25
  # Utilities
26
- python-dotenv
27
- pydantic
28
- fpdf2
 
29
 
30
- # --- NEW: For Video Recording ---
31
- streamlit-webrtc
32
- opencv-python-headless
33
- av
 
1
  # D:\jan-contract\requirements.txt
2
 
3
  # Core LangChain libraries
4
+ langchain-core>=0.2.0
5
+ langchain>=0.2.0
6
+ langchain-community>=0.2.0
7
+ langgraph>=0.2.0
8
 
9
  # LLM Integrations
10
+ langchain_google_genai>=0.1.0
11
+ langchain-groq>=0.1.0
12
+ google-generativeai>=0.8.0
13
 
14
  # Tooling
15
+ tavily-python>=0.4.0
16
+ pypdf>=4.0.0
17
+ pymupdf>=1.24.0
18
+ fastembed>=0.2.0
19
+ faiss-cpu>=1.7.0
20
+ python-multipart>=0.0.6
21
+
22
  # Web Frameworks
23
+ fastapi>=0.104.0
24
+ uvicorn>=0.24.0
25
+ streamlit>=1.28.0
26
+
27
+ # Video and Audio Processing
28
+ streamlit-webrtc>=0.63.4
29
+ opencv-python-headless>=4.8.0
30
+ av>=14.0.0
31
+ SpeechRecognition>=3.10.0
32
+ gTTS>=2.4.0
33
+ PyAudio>=0.2.11
34
 
35
  # Utilities
36
+ python-dotenv>=1.0.0
37
+ pydantic>=2.5.0
38
+ fpdf2>=2.7.0
39
+ numpy>=1.24.0
40
 
41
+ # Additional dependencies for better audio/video handling
42
+ # Note: webrtc-streamer is not needed as streamlit-webrtc handles this
 
 
run_app.py ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ Jan-Contract App Launcher
4
+ This script helps you run the Streamlit app with proper configuration.
5
+ """
6
+
7
+ import os
8
+ import sys
9
+ import subprocess
10
+ import webbrowser
11
+ import time
12
+
13
+
14
+ def check_dependencies():
15
+ """Check if all required dependencies are installed"""
16
+ # Map human/package names to actual importable module names
17
+ required_modules = [
18
+ ("streamlit", "streamlit"),
19
+ ("streamlit-webrtc", "streamlit_webrtc"),
20
+ ("opencv-python-headless", "cv2"), # import cv2, not opencv_python_headless
21
+ ("av", "av"),
22
+ ("SpeechRecognition", "speech_recognition"),
23
+ ("gTTS", "gtts"),
24
+ ("numpy", "numpy"),
25
+ ]
26
+
27
+ missing = []
28
+ for package_label, module_name in required_modules:
29
+ try:
30
+ __import__(module_name)
31
+ except ImportError:
32
+ missing.append(package_label)
33
+
34
+ if missing:
35
+ print("❌ Missing dependencies:")
36
+ for package in missing:
37
+ print(f" - {package}")
38
+ print("\n💡 Install missing packages with:")
39
+ print(" pip install -r requirements.txt")
40
+ return False
41
+
42
+ print("✅ All dependencies are installed!")
43
+ return True
44
+
45
+
46
+ def check_directories():
47
+ """Check if required directories exist"""
48
+ required_dirs = ['video_consents', 'pdfs_demystify']
49
+
50
+ for dir_name in required_dirs:
51
+ if not os.path.exists(dir_name):
52
+ os.makedirs(dir_name, exist_ok=True)
53
+ print(f"📁 Created directory: {dir_name}")
54
+
55
+ print("✅ All directories are ready!")
56
+
57
+
58
+ def main():
59
+ print("🚀 Jan-Contract App Launcher")
60
+ print("=" * 40)
61
+
62
+ # Check dependencies
63
+ if not check_dependencies():
64
+ print("\n❌ Please install missing dependencies before running the app.")
65
+ return
66
+
67
+ # Check directories
68
+ check_directories()
69
+
70
+ print("\n🌐 Starting Streamlit app...")
71
+ print("💡 The app will open in your default browser.")
72
+ print("💡 If it doesn't open automatically, go to: http://localhost:8501")
73
+ print("\n📋 Tips for best experience:")
74
+ print(" - Use Chrome, Firefox, or Edge")
75
+ print(" - Allow camera and microphone permissions")
76
+ print(" - Record videos for at least 2-3 seconds")
77
+ print(" - Speak clearly for voice input")
78
+
79
+ # Start the Streamlit app using `python -m streamlit` so PATH is not required
80
+ try:
81
+ # Open browser after a short delay
82
+ def open_browser():
83
+ time.sleep(3)
84
+ webbrowser.open('http://localhost:8501')
85
+
86
+ import threading
87
+ browser_thread = threading.Thread(target=open_browser)
88
+ browser_thread.daemon = True
89
+ browser_thread.start()
90
+
91
+ # Run Streamlit
92
+ subprocess.run([
93
+ sys.executable, '-m', 'streamlit', 'run', 'main_streamlit.py',
94
+ '--server.port', '8501',
95
+ '--server.address', 'localhost'
96
+ ])
97
+
98
+ except KeyboardInterrupt:
99
+ print("\n👋 App stopped by user.")
100
+ except Exception as e:
101
+ print(f"\n❌ Error starting app: {e}")
102
+ print("💡 Try running manually: python -m streamlit run main_streamlit.py")
103
+
104
+
105
+ if __name__ == "__main__":
106
+ main()