Hey, I made an ONNX version, cause I was so surprised how great this model is!
There you go - https://huggingface.co/MarcinEU/finegrain-box-segmenter-ONNX
I've been working on some tooling and discovered the Finegrain Box Segmenter - IMHO it's a strong candidate for a SOTA model in the background-removal space. There is no one be-all-end-all model, of course, but... what you achieved here is easily among the best for cutting out images.
It's been 2 years, I know, but I'm still looking forward for the release of the version that you can prompt for an object! Love you all!
Hey, thanks!
We don't have a specific model that you can prompt by text, we just use a pre-processor to detect the object and output the box or rough mask, e.g. Grounding Dino in the space. We do have a slightly different version in production but it has the same interface (box segmenter).
We focus on our image editing models that run on (mobile) device for now so open sourcing more segmenter-related things is not something we plan short-term.
Ohhh.... so you first do object detection, crop that, and only then run it through the finegrain-box-segmenter. I actually built a prototype of that for the MCP I'm working on, but just abandoned the topic, hoping that perhaps there would be something like "Finegrain Box Segmenter v0.2" with prompting built in. I did it "the stupid way" - asked large multimodal model (Opus 4.8 🙃) to set me the bounding box, and then just cropped it with Sharp before feeding to the Segmenter. It worked well, but the moment I started testing SLMs (in a flow Prompt+Image -> cropped, background-removed object) it all fell apart as I gave up before finding any model that could accurately spot secondary/background objects. If you have any recommendations - I would love to hear them.
We focus on our image editing models that run on (mobile) device
I seen that! You guys do amazing work. The demo of the lamps turning on was particularly fascinating
I forgot that's what this version does - yes it crops (with padding). Our internal segmenter takes a second input which is the box, but it's the same thing.
For detection we don't use a SLM here, in the space we use Grounding Dino (you can check the source code, it's open).