aufklarer/central-bank-communications
Viewer ā¢ Updated ā¢ 263k ā¢ 835 ā¢ 5
Ivan PRO
aufklarer
AI & ML interests
GenAI
Recent Activity
updated a dataset about 3 hours ago
aufklarer/central-bank-communications reacted to theirpost with š¤ 2 days ago
ā” GLiNER2.5-Decide now runs natively on Apple Silicon, in MLX Swift
Fastino's 340M decision model scores every label you give it, or finds entity spans, in a single forward pass. Nothing is generated. I ported it to speech-swift and published three MLX conversions:
- https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX-8bit (default)
- https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX-fp16
- https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX
Measured on an idle M5 Pro, full request including tokenization:
- INT8: 7.6 ms routing, 8.9 ms extraction, 0.85 GB peak
- FP16: 8.8 / 10.0 ms, 1.58 GB
- FP32: 11.1 / 12.6 ms, 2.55 GB (Python gliner2-mlx: 13.7 / 15.0 ms)
Every precision returns the same labels, spans and offsets as the PyTorch original on 24 reference cases. Most of the speedup came from one change: DeBERTa's relative-position projections depend only on the weights, so they are computed once at load instead of on every request.
I also compared it with TypeSafe's hosted Jev. Worth knowing: the published 60.2% vs 57.6% result is against JevK5, an open reproduction, not Jev itself.
š Write-up with animated walkthroughs: https://soniqo.audio/blog/gliner-decide-vs-jev
š» Code: https://github.com/soniqo/speech-swift reacted to theirpost with š„ 2 days ago
ā” GLiNER2.5-Decide now runs natively on Apple Silicon, in MLX Swift
Fastino's 340M decision model scores every label you give it, or finds entity spans, in a single forward pass. Nothing is generated. I ported it to speech-swift and published three MLX conversions:
- https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX-8bit (default)
- https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX-fp16
- https://huggingface.co/aufklarer/GLiNER2.5-Decide-340M-MLX
Measured on an idle M5 Pro, full request including tokenization:
- INT8: 7.6 ms routing, 8.9 ms extraction, 0.85 GB peak
- FP16: 8.8 / 10.0 ms, 1.58 GB
- FP32: 11.1 / 12.6 ms, 2.55 GB (Python gliner2-mlx: 13.7 / 15.0 ms)
Every precision returns the same labels, spans and offsets as the PyTorch original on 24 reference cases. Most of the speedup came from one change: DeBERTa's relative-position projections depend only on the weights, so they are computed once at load instead of on every request.
I also compared it with TypeSafe's hosted Jev. Worth knowing: the published 60.2% vs 57.6% result is against JevK5, an open reproduction, not Jev itself.
š Write-up with animated walkthroughs: https://soniqo.audio/blog/gliner-decide-vs-jev
š» Code: https://github.com/soniqo/speech-swiftOrganizations
updated a
dataset about 3 hours ago
reacted to their post with š¤š„š 2 days ago
Post
89
ā” GLiNER2.5-Decide now runs natively on Apple Silicon, in MLX Swift
Fastino's 340M decision model scores every label you give it, or finds entity spans, in a single forward pass. Nothing is generated. I ported it to speech-swift and published three MLX conversions:
- aufklarer/GLiNER2.5-Decide-340M-MLX-8bit (default)
- aufklarer/GLiNER2.5-Decide-340M-MLX-fp16
- aufklarer/GLiNER2.5-Decide-340M-MLX
Measured on an idle M5 Pro, full request including tokenization:
- INT8: 7.6 ms routing, 8.9 ms extraction, 0.85 GB peak
- FP16: 8.8 / 10.0 ms, 1.58 GB
- FP32: 11.1 / 12.6 ms, 2.55 GB (Python gliner2-mlx: 13.7 / 15.0 ms)
Every precision returns the same labels, spans and offsets as the PyTorch original on 24 reference cases. Most of the speedup came from one change: DeBERTa's relative-position projections depend only on the weights, so they are computed once at load instead of on every request.
I also compared it with TypeSafe's hosted Jev. Worth knowing: the published 60.2% vs 57.6% result is against JevK5, an open reproduction, not Jev itself.
š Write-up with animated walkthroughs: https://soniqo.audio/blog/gliner-decide-vs-jev
š» Code: https://github.com/soniqo/speech-swift
Fastino's 340M decision model scores every label you give it, or finds entity spans, in a single forward pass. Nothing is generated. I ported it to speech-swift and published three MLX conversions:
- aufklarer/GLiNER2.5-Decide-340M-MLX-8bit (default)
- aufklarer/GLiNER2.5-Decide-340M-MLX-fp16
- aufklarer/GLiNER2.5-Decide-340M-MLX
Measured on an idle M5 Pro, full request including tokenization:
- INT8: 7.6 ms routing, 8.9 ms extraction, 0.85 GB peak
- FP16: 8.8 / 10.0 ms, 1.58 GB
- FP32: 11.1 / 12.6 ms, 2.55 GB (Python gliner2-mlx: 13.7 / 15.0 ms)
Every precision returns the same labels, spans and offsets as the PyTorch original on 24 reference cases. Most of the speedup came from one change: DeBERTa's relative-position projections depend only on the weights, so they are computed once at load instead of on every request.
I also compared it with TypeSafe's hosted Jev. Worth knowing: the published 60.2% vs 57.6% result is against JevK5, an open reproduction, not Jev itself.
š Write-up with animated walkthroughs: https://soniqo.audio/blog/gliner-decide-vs-jev
š» Code: https://github.com/soniqo/speech-swift
posted an update 2 days ago
Post
89
ā” GLiNER2.5-Decide now runs natively on Apple Silicon, in MLX Swift
Fastino's 340M decision model scores every label you give it, or finds entity spans, in a single forward pass. Nothing is generated. I ported it to speech-swift and published three MLX conversions:
- aufklarer/GLiNER2.5-Decide-340M-MLX-8bit (default)
- aufklarer/GLiNER2.5-Decide-340M-MLX-fp16
- aufklarer/GLiNER2.5-Decide-340M-MLX
Measured on an idle M5 Pro, full request including tokenization:
- INT8: 7.6 ms routing, 8.9 ms extraction, 0.85 GB peak
- FP16: 8.8 / 10.0 ms, 1.58 GB
- FP32: 11.1 / 12.6 ms, 2.55 GB (Python gliner2-mlx: 13.7 / 15.0 ms)
Every precision returns the same labels, spans and offsets as the PyTorch original on 24 reference cases. Most of the speedup came from one change: DeBERTa's relative-position projections depend only on the weights, so they are computed once at load instead of on every request.
I also compared it with TypeSafe's hosted Jev. Worth knowing: the published 60.2% vs 57.6% result is against JevK5, an open reproduction, not Jev itself.
š Write-up with animated walkthroughs: https://soniqo.audio/blog/gliner-decide-vs-jev
š» Code: https://github.com/soniqo/speech-swift
Fastino's 340M decision model scores every label you give it, or finds entity spans, in a single forward pass. Nothing is generated. I ported it to speech-swift and published three MLX conversions:
- aufklarer/GLiNER2.5-Decide-340M-MLX-8bit (default)
- aufklarer/GLiNER2.5-Decide-340M-MLX-fp16
- aufklarer/GLiNER2.5-Decide-340M-MLX
Measured on an idle M5 Pro, full request including tokenization:
- INT8: 7.6 ms routing, 8.9 ms extraction, 0.85 GB peak
- FP16: 8.8 / 10.0 ms, 1.58 GB
- FP32: 11.1 / 12.6 ms, 2.55 GB (Python gliner2-mlx: 13.7 / 15.0 ms)
Every precision returns the same labels, spans and offsets as the PyTorch original on 24 reference cases. Most of the speedup came from one change: DeBERTa's relative-position projections depend only on the weights, so they are computed once at load instead of on every request.
I also compared it with TypeSafe's hosted Jev. Worth knowing: the published 60.2% vs 57.6% result is against JevK5, an open reproduction, not Jev itself.
š Write-up with animated walkthroughs: https://soniqo.audio/blog/gliner-decide-vs-jev
š» Code: https://github.com/soniqo/speech-swift
updated a
model 2 days ago
published a
model 2 days ago
updated a
model 2 days ago
published a
model 2 days ago
updated a
model 2 days ago
published a
model 2 days ago
updated a
model 2 days ago
updated a
collection 6 days ago
updated a
model 6 days ago
published a
model 6 days ago
updated a
collection 6 days ago
updated a
model 6 days ago
published a
model 6 days ago
published a
model 9 days ago
updated a
collection 9 days ago