Spaces:
Sleeping
Sleeping
Commit History
-
Track metrics during streaming, remove redundant generation re-runs
Florian valade- Add StreamingResult dataclass to hold metrics from streaming generation - Update StreamEvent with 'complete' event type and result field - Extract _format_and_encode_prompt() helper to reduce code duplication - Update generate_streaming() and generate_full_model_streaming() to yield final 'complete' event with accumulated metrics - Refactor app.py to use streaming results instead of re-running generation - Remove unused code: set_full_cache(), create_cache_from_model(), to_json() methods, get_threshold() - Add jagged_cache.py and test files for KV cache operations This fixes the bug where output would change after streaming completed and metrics would appear after a delay.33efa44