Spaces:
Sleeping
Sleeping
Commit History
-
Verify early-exit EOS before terminating output
Verified signature5419a23 -
Use Qwen non-thinking prompt for short DSSD responses
Verified signaturee2f59ad
-
Fix transformers compatibility: pin versions and rename past_key_value to past_key_values
Florian valade687049b -
Track metrics during streaming, remove redundant generation re-runs
Florian valade- Add StreamingResult dataclass to hold metrics from streaming generation - Update StreamEvent with 'complete' event type and result field - Extract _format_and_encode_prompt() helper to reduce code duplication - Update generate_streaming() and generate_full_model_streaming() to yield final 'complete' event with accumulated metrics - Refactor app.py to use streaming results instead of re-running generation - Remove unused code: set_full_cache(), create_cache_from_model(), to_json() methods, get_threshold() - Add jagged_cache.py and test files for KV cache operations This fixes the bug where output would change after streaming completed and metrics would appear after a delay.33efa44 -
Fix early exit inference loop to eliminate redundant computation
Florian valadeKey fixes: - _draft_single_token now always returns a token (never None) - When no early exit head is confident, continues to lm_head instead of returning None and triggering a redundant full model pass - Extracts 'bonus token' from verification pass when all drafts accepted - Same fixes applied to generate_streaming method This eliminates the double computation bug where layers were processed twice when no head was confident, and adds the bonus token optimization that extracts an extra token from each verification pass. Adds comprehensive tests in tests/test_inference_loop.pya781577
-
fix cache position for newer version of transformers
Florian valade7848d77
-
Initial commit of standalone DSSD demo for HF Spaces
Florian valade72b2f6d