Spaces:
Running
Context compression model?
Your task-specific models (like SupraTitle, reasoning summarizer, and SupraRouter) have been really good at their narrow task, while remaining runnable on iGPUs.
I was wondering if you would be willing to develop a similar model for context compression. If done, it should accept the existing context and an output token or character budget, then return a summary fitting in that size. I am not sure if you could get the context window large enough for this, but if so, it would extremely useful.
Most existing context compression pipelines combine a low-quality deterministic compressor with an expensive cloud model, getting mediocre results with large cost and latency.
Hey there!
This sounds like an amazing idea.
We'll definitely look into it!
🔥
@python-processing-unit , we will work on it, but this would be computationally expensive since we need to train a model at above 64k context length, but we will see what we can do!