Hugging Face's logo
Aleksandr Sobolev's picture
2 41

Aleksandr Sobolev

alexsobolev
robtacconelli's profile picture shtefcs's profile picture
Ā·

AI & ML interests

None yet

Recent Activity

liked a model 16 days ago
mrfakename/EmoAct-MiMo
reacted to robtacconelli's post with šŸ”„ 16 days ago
šŸ† Nacrith: a 135M model that out-compresses everything on natural language What if a tiny LM could compress english text better than _every_ compressor out there ā€” classical or neural, small or large? Nacrith pairs SmolLM2-135M with an ensemble of online predictors and high-precision arithmetic coding. What's inside The standard LLM+arithmetic coding approach wastes ~75% of CDF precision on large vocabularies. Our CDF-24 fix alone recovers 0.5 bpb. On top: a token N-gram that skips the GPU on predictable tokens, an adaptive bias head, llama.cpp backend (7Ɨ faster than PyTorch), multi-GPU parallel compression, and a binary file format (NC06) ā€” the first LLM-based binary compressor we know of. Runs on a GTX 1050 Ti. ~500 MB weights, ~1.2 GB VRAM per worker. šŸ’» Code: https://github.com/robtacconelli/Nacrith-GPU ā­ Space: https://huggingface.co/spaces/robtacconelli/Nacrith-GPU šŸ“„ Paper: https://huggingface.co/papers/2602.19626 Try it, break it, share your results ā€” all feedback welcome. ā­ on the repo appreciated! Results across all systems we tested: - alice29.txt ā†’ 0.918 bpb (āˆ’44% vs CMIX, āˆ’20% vs ts_zip) ā€” below the 2nd-order Shannon entropy bound - enwik8 (100 MB) ā†’ 0.9389 bpb (āˆ’8% vs FineZip/LLMZip's 8B model, āˆ’15% vs ts_zip) - Unseen text ā†’ 0.723 bpb on a doc published after training cutoff ā€” no memorization, 26% better than FineZip/LLMZip on the same model SmolLM2-135M by https://huggingface.co/HuggingFaceTB
reacted to robtacconelli's post with šŸš€ 16 days ago
šŸ† Nacrith: a 135M model that out-compresses everything on natural language What if a tiny LM could compress english text better than _every_ compressor out there ā€” classical or neural, small or large? Nacrith pairs SmolLM2-135M with an ensemble of online predictors and high-precision arithmetic coding. What's inside The standard LLM+arithmetic coding approach wastes ~75% of CDF precision on large vocabularies. Our CDF-24 fix alone recovers 0.5 bpb. On top: a token N-gram that skips the GPU on predictable tokens, an adaptive bias head, llama.cpp backend (7Ɨ faster than PyTorch), multi-GPU parallel compression, and a binary file format (NC06) ā€” the first LLM-based binary compressor we know of. Runs on a GTX 1050 Ti. ~500 MB weights, ~1.2 GB VRAM per worker. šŸ’» Code: https://github.com/robtacconelli/Nacrith-GPU ā­ Space: https://huggingface.co/spaces/robtacconelli/Nacrith-GPU šŸ“„ Paper: https://huggingface.co/papers/2602.19626 Try it, break it, share your results ā€” all feedback welcome. ā­ on the repo appreciated! Results across all systems we tested: - alice29.txt ā†’ 0.918 bpb (āˆ’44% vs CMIX, āˆ’20% vs ts_zip) ā€” below the 2nd-order Shannon entropy bound - enwik8 (100 MB) ā†’ 0.9389 bpb (āˆ’8% vs FineZip/LLMZip's 8B model, āˆ’15% vs ts_zip) - Unseen text ā†’ 0.723 bpb on a doc published after training cutoff ā€” no memorization, 26% better than FineZip/LLMZip on the same model SmolLM2-135M by https://huggingface.co/HuggingFaceTB
View all activity

Organizations

None yet