🤝 Open to Collab

Ivan PRO

aufklarer

·

https://blog.ivan.digital

AI & ML interests

GenAI

Recent Activity

updated a dataset about 22 hours ago

aufklarer/central-bank-communications

reacted to theirpost with 🤗 1 day ago

⚡ GLiNER2.5-Decide now runs natively on Apple Silicon, in MLX Swift Fastino's 340M decision model scores every label you give it, or finds entity spans, in a single forward pass. Nothing is generated. I ported it to speech-swift and published three MLX conversions: - https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX-8bit (default) - https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX-fp16 - https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX Measured on an idle M5 Pro, full request including tokenization: - INT8: 7.6 ms routing, 8.9 ms extraction, 0.85 GB peak - FP16: 8.8 / 10.0 ms, 1.58 GB - FP32: 11.1 / 12.6 ms, 2.55 GB (Python gliner2-mlx: 13.7 / 15.0 ms) Every precision returns the same labels, spans and offsets as the PyTorch original on 24 reference cases. Most of the speedup came from one change: DeBERTa's relative-position projections depend only on the weights, so they are computed once at load instead of on every request. I also compared it with TypeSafe's hosted Jev. Worth knowing: the published 60.2% vs 57.6% result is against JevK5, an open reproduction, not Jev itself. 📝 Write-up with animated walkthroughs: https://soniqo.audio/blog/gliner-decide-vs-jev 💻 Code: https://github.com/soniqo/speech-swift

reacted to theirpost with 🔥 1 day ago

⚡ GLiNER2.5-Decide now runs natively on Apple Silicon, in MLX Swift Fastino's 340M decision model scores every label you give it, or finds entity spans, in a single forward pass. Nothing is generated. I ported it to speech-swift and published three MLX conversions: - https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX-8bit (default) - https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX-fp16 - https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX Measured on an idle M5 Pro, full request including tokenization: - INT8: 7.6 ms routing, 8.9 ms extraction, 0.85 GB peak - FP16: 8.8 / 10.0 ms, 1.58 GB - FP32: 11.1 / 12.6 ms, 2.55 GB (Python gliner2-mlx: 13.7 / 15.0 ms) Every precision returns the same labels, spans and offsets as the PyTorch original on 24 reference cases. Most of the speedup came from one change: DeBERTa's relative-position projections depend only on the weights, so they are computed once at load instead of on every request. I also compared it with TypeSafe's hosted Jev. Worth knowing: the published 60.2% vs 57.6% result is against JevK5, an open reproduction, not Jev itself. 📝 Write-up with animated walkthroughs: https://soniqo.audio/blog/gliner-decide-vs-jev 💻 Code: https://github.com/soniqo/speech-swift

View all activity

Organizations

aufklarer 's collections 2