- Add cancel_event checks in the streamer loop to enable true cancellation during response generation. :contentReference[oaicite:1]{index=1}
- Launch DuckDuckGo web search in a background thread to prevent blocking the streaming pipeline. :contentReference[oaicite:2]{index=2}
- Implement dtype fallback (bfloat16 → float16 → float32) for broader hardware compatibility. :contentReference[oaicite:3]{index=3}
- Suppress repeated debug messages after the first token to avoid UI flooding. :contentReference[oaicite:4]{index=4}
- Remove unused imports and streamline load_pipeline caching logic for cleaner code. :contentReference[oaicite:5]{index=5}
bugfixc: not using pipeline for response generation
- Run web search lookup in a background thread to decouple it from prompt construction.
- Add a 2-second timeout using thread.join(timeout=2) to prevent delaying response generation.
- Fallback to using only the system prompt and user query if web search results aren't retrieved in time.
- Update sidebar UI to show the fetched context or a "No context found" message.
- Retain current streaming and chat history functionalities for model inference.
