API Queue Issue
Ahoy Jane Street team,
I've signed up for the dormant-puzzle challenge. Posting here on dormant-model-1 since it's the only repo with discussions enabled, but this applies to all three competition models. They're all 671B DeepSeek V3 in FP8, so anyone using the jsinfer batch API is likely hitting the same issue.
Batches are sitting in "in_progress" with 0 tokens processed for 60+ minutes before inference even begins. The queue delay dwarfs the actual compute time, which makes iterating on strategies really slow. This is especially critical as a one-person team with no way to parallelize around it.
Two suggestions that could help all participants:
- Expose queue position or ETA in the batch status response. Right now "in_progress" doesn't distinguish queued from running. I can't tell if my job hasn't started or is just slow.
- Stream partial results so completed items within a batch are available before the entire batch finishes.
Appreciate the infrastructure you've put together for this challenge. These changes would make the dev loop a lot tighter for everyone.
I am also facing the same issue, yesterday i was getting response within few minutes, now it's been more than 35 minutes and still waiting.
Yes, me too. Yesterday was fine, around 1-2 minutes. Now it is queued for over half an hour.
Sorry to hear y'all are going though the same situation, we're in a pressure cooker situation!
I added a timeout to my polling with partial result retrieval so I could grab what was completed. I recommend doing the same, just to keep up with the idea of some progress is good progress.
Hi everyone, thanks for the reports! We investigated some issues yesterday and upgraded the infra. Hopefully it's better now, but please keep us updated!