mradermacher

/

model_requests

like 132

Do you really want to make HuggingFace run out of money? Team mradermacher currently has 39,245 models using far over 5 PB of storage. Currently storing this on AWS S3 likely costs them around $100000/month just in storage fees. Storing this all on XET would probably be extremely expensive for them. They have unfortunately not communicated their XET storage cost so it could also be cheaper especially once everyone migrated to XET and deduplication would remove the need to store all duplicate static quants between us and other quanters but it’s not really something I would feel comfortable to decide as I do not have enough information. It instead makes sense to let the HuggingFace staff decide if and when they migrate team mradermacher repository to XET so they can do whatever doesn't ruin them financially. For now, we are just glad that they have not implemented the 1 TB storage limit last December or it would be over for us. We highly appreciate that HuggingFace offers us all this storage for free and really don’t want to ask for more. I'm honestly quite skeptical for how much longer they can be so generous. It's also worth mentioning that I once calculated that their bandwidth cost to serve all our models must far exceed any storage cost due to the massive number of downloads of our models.

Beside that financial reasons migrating to XET poses a risk for our operations. Currently everything works well regarding uploading our models to S3 but there are no guarantee XET would not cause any issues. Given the massive complexity of our systems despite XET being a drop-in replacement there almost certainly will be some issues. I remember last summer when we had almost half of our uploads randomly fail and we wasted like half of our upload bandwidth retrying failed uploads. I myself switched to XET for my personal account and so far, made great experience with it. I have super-fast up- and downloads and so far had zero issues. I'm for sure willing to switch to XET once the HuggingFace staff decides to do so and help resolve any potential issues this causes.

I'm honestly quite skeptical for how much longer they can be so generous

Once they feel they totally control the market and have enough people hooked... then tighten the screws. Pretty SoP. ( i trust no one... )

(There won't be any deduplication between us and other quanters, unfortunately, as we have unique kv entries in our quants - unless they do some really nifty deduplication based on understanding gguf files.)

then tighten the screws

I agree. There is no such thing as a free lunch.

There won't be any deduplication between us and other quanters, unfortunately, as we have unique kv entries in our quants - unless they do some really nifty deduplication based on understanding gguf files.)

There will be massive amounts of deduplication. It even manages to dedublicate between finetunes of a specific model. The way Xet does so is by using content-defined chunking (CDC). It is such an amazing technique and will be easily able to deal with our custom metadata: I highly recommend you read https://huggingface.co/blog/xet-on-the-hub and https://huggingface.co/blog/from-files-to-chunks - this technology is really exciting.

Ah, so they pretty much do what rsync/gzip/etc. are doing - very good :)

@nicoboss well, it's happening: https://huggingface.co/posts/reach-vb/41517305778191 - seems we are a guinea pig.

I just hope it doesn't cause more I/O on the client side, or anything :)

@yamatazen hf asked us to join the waitlist, which we have done