evan108108/pulpie-orange-small-coreml

pulpie-orange-small — Core ML

Core ML (.mlpackage) conversion of feyninc/pulpie-orange-small, a 210M-parameter EuroBERT token classifier that labels HTML blocks as main (content) or other (boilerplate).

All modelling credit belongs to Feyn Labs / feyninc. This repo contains only a format conversion — no retraining, no fine-tuning. The shipping artifact keeps the original float32 weights; a float16 variant was also produced and evaluated (see below) but is not recommended. Licensed Apache-2.0, same as the source model.

• Upstream model: https://huggingface.co/feyninc/pulpie-orange-small
• pulpie library: https://pypi.org/project/pulpie/
• Write-up: https://usefeyn.com/blog/pulpie-pareto-optimal-models-for-cleaning-the-web

What it does

pulpie simplifies a raw HTML page into a sequence of blocks, each tagged with an _item_id, then packs those blocks into one sequence separated by a special <|sep|> token:

[BOS] block_0 <|sep|> block_1 <|sep|> ... block_N <|sep|> [EOS]

A single encoder forward pass classifies every block at once. The prediction for block i is read from the logits at the position of the <|sep|> token that follows it — not from a pooled sequence output.

Which file to use

This bundle contains fp32 only — it is the shipping artifact.

variant	size	block-label parity	Wikipedia page latency
pulpie-orange-small-fp32.mlpackage	~808 MB	920/920 — 100%	8.9 s

fp16 was converted and evaluated but is not shipped: it loses exactness (912/920 — 99.13%) and runs ~3× slower (27.8 s vs 8.9 s), because the fp16 graph lands on an accelerated path that handles this model poorly (see Shape, below). Its only advantage is size, which is not worth an inexact and slower model. It is retained offline in the combined tarball (see Download) for reproducibility, and is not published here.

Download

One tarball is published here — the fp32-only shipping artifact:

https://huggingface.co/evan108108/pulpie-orange-small-coreml/resolve/main/pulpie-orange-small-coreml-fp32.tar.gz

contains	fp32 .mlpackage + conversion_meta.json + tokenizer sidecars
bytes	403,179,468
sha256	116aeb6ca92b647c5b6536b2cd226c3d5961f82ec44bef6e6864627d93c14921

It unpacks to a single top-level pulpie-orange-small-coreml/ directory.

A combined fp32+fp16 tarball (732,209,538 bytes, sha256 2c15388f59fb9df764c2ab93b9a26eb4fa375e6ed7d78c299c377dcfb8ab29be) was also produced during conversion and retained offline for reproducibility, but is not published here — dropping fp16 cuts the first-run download by 329,030,070 bytes (44.93% smaller) and fp16 is not recommended for use. The fp32 .mlpackage is byte-identical between the two (weight.bin sha256 995acb528764c21f6af2c3a7b2a7f7f4b8220ded2329f2a47e34313dcaa85576) — the fp32-only tarball is a repackaging, not a re-conversion.

Do not replace this file in place. Consumers pin its sha256 and byte count, and tar output is not reproducible across runs, so re-cutting an "identical" tarball breaks every pinned client. Publish changes under a new filename.

There is no config.json in either bundle. The fields a runtime needs (sep_token_id, bos/eos/pad, sequence_length, label mapping) live in conversion_meta.json.

Interface

	name	shape	dtype	notes
in	input_ids	[1, 8192]	int32	right-pad with 128001
in	attention_mask	[1, 8192]	int32	1 real, 0 padding
out	logits	[1, 8192, 2]	float32	index at `<

argmax over the last axis: 1 = main, 0 = other.

Key token ids (also in conversion_meta.json):

token	id
`<	sep
BOS `<	begin_of_text
EOS `<	end_of_text
pad (config.pad_token_id)	128001

<|sep|> is already in the tokenizer vocab (len(tokenizer) == vocab_size == 128257), so no embedding resize happens — do not add it as a new token.

Tokenizer sidecars are included: tokenizer.json, tokenizer_config.json, special_tokens_map.json. Load them with swift-transformers or tokenizers; the .mlpackage does no tokenization itself.

Deployment target: macOS 15 / iOS 18.

Parity with the Python reference

Verified against a live pulpie==0.0.2 service (torch 2.13.0, transformers 4.57.6) on a 4-page corpus of rendered HTML. The service's /classify endpoint labels externally supplied simplified HTML, so both pipelines are compared on byte-identical input with identical chunking (pulpie's own pack_chunks builds the CoreML side).

page	blocks	fp32	fp16
usaspending.gov — DOD FY2025	195	195/195	195/195
data.cityofchicago.org — Building Permits	193	193/193	192/193
en.wikipedia.org — Retrieval-augmented generation	483	483/483	477/483
anthropic.com — Newsroom	49	49/49	48/49
total	920	920/920 — 100%	912/920 — 99.13%

fp32 reproduces the reference exactly — every block label, and the emitted markdown byte-for-byte identical to the service's on all four pages.

For fp16, every one of the 8 disagreements is a near-tie: the reference's own decision margin |logit_main − logit_other| at those blocks is 0.05–0.15, against a median block margin of 2.7–7.2. Blocks the reference is confident about agree universally.

Those 8 flips are attributable to the execution path, not to fp16 storage: the same fp16 .mlpackage run under CPU_ONLY scores 49/49 on the Anthropic page (vs 48/49 on ComputeUnit.ALL), with markdown matching the service. A full CPU_ONLY sweep is not obtainable here — it segfaults partway through the corpus at seq 8192 (see Limitations).

Run on Apple M1 Max, 64 GB, macOS 15.3, coremltools 9.0.

Conversion recipe

python3.12 -m venv .venv
.venv/bin/pip install coremltools==9.0 torch==2.13.0 transformers==4.57.6 \
                      huggingface_hub safetensors

Load exactly as pulpie.model_utils.load_model_and_tokenizer does — including its fix_rotary_embeddings step, since EuroBERT registers inv_freq as a non-persistent buffer that does not survive from_pretrained — then apply three trace-time patches, torch.jit.trace, and convert.

Each patch below is asserted equivalent to the original in float32 before being applied; none of them changes the model's math.

1. rotate_half — bind the split point. It slices at x.shape[-1] // 2; under trace that shape read becomes a tensor and emits aten::floor_divide → aten::Int (48 of them), which coremltools cannot constant-fold. The last dim is always head_dim=64, so the split is a compile-time constant (32).

2. Attention mask — avoid .expand() and finfo.min. HF's _expand_mask calls .expand() with traced shape scalars (more aten::Int), and masks with finfo(float32).min ≈ -3.4e38, which overflows to -inf in fp16 — a fully-masked padding row then softmaxes to NaN, and because padded positions remain keys/values for real queries, the NaN propagates into every real token. Replaced with pure broadcasting and a mask value of -1e4: still far below any real attention logit (masked weights underflow to exactly 0) but finite in fp16.

3. RoPE — precompute cos/sin as a table. EuroBertRotaryEmbedding computes inv_freq @ position_ids at runtime and forces float32 with autocast(enabled=False), because at seq 8192 the angles reach ~8192 where fp16's resolution is 4.0. compute_precision=FLOAT16 overrides that guard. Building the table at conversion time moves the large-angle arithmetic offline; the stored values are cos/sin outputs in [-1, 1], where fp16 is precise.

Precision: fp16 weights, fp32 RMSNorm reduction. A blanket compute_precision=FLOAT16 produces garbage (measured logit correlation 0.07 vs the reference). EuroBertRMSNorm computes mean(x²) over hidden_size=768 and the sum peaks near 84,000 in the deeper layers, past fp16's 65,504 ceiling → inf → rsqrt → 0, zeroing the normalized activations from layer 4 on. The reference avoids this by upcasting to fp32 inside RMSNorm, which FLOAT16 overrides. Pinning just reduce_mean and rsqrt to fp32 fixes it at no size cost — those ops carry no weights, while every weight-bearing op stays fp16:

from coremltools.converters.mil.mil.passes.defs.quantization import FP16ComputePrecision
precision = FP16ComputePrecision(op_selector=lambda op: op.op_type not in {"reduce_mean", "rsqrt"})

Shape: fixed [1, 8192], deliberately not enumerated. EnumeratedShapes buckets convert fine and are correct on CPU_ONLY, but on the accelerated path (ComputeUnit.ALL, i.e. ANE/GPU) the fp16 model returns garbage — logits uncorrelated with the reference, accompanied by mpsgraph bytecode errors, and in some runs a hard EXC_BAD_ACCESS crash in com.apple.coreml.MLE5ExecutionStream. Fixed shapes are correct on every compute unit, so callers pad each chunk to 8192 and mask. (This is a runtime backend bug on macOS 15.3 / coremltools 9.0, not a conversion error — the same .mlpackage is exact under CPU_ONLY.)

mlmodel = ct.convert(
    traced,
    inputs=[ct.TensorType(name="input_ids", shape=ct.Shape((1, 8192)), dtype=np.int32),
            ct.TensorType(name="attention_mask", shape=ct.Shape((1, 8192)), dtype=np.int32)],
    outputs=[ct.TensorType(name="logits", dtype=np.float32)],
    minimum_deployment_target=ct.target.macOS15,   # <15 allows only ONE enumerated-shape input
    compute_precision=precision,
    convert_to="mlprogram",
)

Usage sketch

import numpy as np, coremltools as ct
from pulpie.chunker import extract_blocks, pack_chunks, tokenize_blocks
from pulpie.simplify import simplify

SEQ, SEP, PAD = 8192, 128256, 128001
ml = ct.models.MLModel("pulpie-orange-small-fp32.mlpackage")

simplified, map_html = simplify(raw_html)
blocks = extract_blocks(simplified)
preds = [0] * len(blocks)

for chunk_ids, block_indices in pack_chunks(
        tokenize_blocks(blocks, tok), max_tokens=SEQ, sep_token_id=SEP,
        bos_token_id=tok.bos_token_id, eos_token_id=tok.eos_token_id):
    n = len(chunk_ids)
    ids = np.full((1, SEQ), PAD, np.int32); ids[0, :n] = chunk_ids
    am = np.zeros((1, SEQ), np.int32);      am[0, :n] = 1
    logits = np.asarray(ml.predict({"input_ids": ids, "attention_mask": am})["logits"])[0]
    for i, bi in enumerate(block_indices):          # SEP positions, in order
        preds[bi] = int(logits[np.flatnonzero(ids[0, :n] == SEP)][i].argmax())

labels = {item_id: "main" if p == 1 else "other" for item_id, p in zip(item_ids, preds)}

Limitations

• Fixed 8192 sequence: short pages pay full-length attention. Chunks must be padded, not truncated.
• 8192 is also the chunking budget the reference uses; changing it changes chunk boundaries and therefore the labels a block sees context from.
• fp16 only: near-tie blocks (reference margin < ~0.2) may flip. Use the fp32 variant if you need exact agreement.
• Core ML runtime stability on this model is imperfect at seq 8192. Both CPU_ONLY and enumerated-shape fp16 configurations have produced hard EXC_BAD_ACCESS crashes in com.apple.coreml.MLE5ExecutionStream.resetQueue on macOS 15.3. The shipped configuration (fixed shape, ComputeUnit.ALL) completed every run, but host apps should treat prediction as fallible and not assume a crash-free process.
• Not verified on iOS or on Intel Macs.