FDS-Iterations/third-pass-feed-ranker

Third-Pass Feed Ranker (job-title × feed-post relevance)

A lightweight cross-encoder that scores how relevant an enterprise social-feed post is to a viewer, given only the viewer's job title and the post text. It is a third-pass reranker: it re-scores a small candidate slate (~20 items) from earlier passes to surface a genuinely job-relevant post that was buried below the top slot.

• Input: job_title (query) + post_text (passage) → single relevance score (higher = more relevant)
• Base: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 (multilingual MiniLM-L12, ~117M params)
• Runtime: server-class CPU at scale (~120 pairs/sec on 8 CPU threads; INT8/ONNX 3–6× more)
• Trained with a listwise ranking objective (optimizes which item wins the slate), not just pointwise regression — so it is markedly better at putting the buried relevant post at #1.

It ranks by MEANING — and the role keyword is neither a crutch nor a trap

Earlier iterations keyed on whether a post named the viewer's role. This model is trained so the role keyword carries zero relevance information: during training the role name is injected into a random half of all posts regardless of their true relevance. As a result:

• Naming the viewer's role changes a relevant post's score by ≈0.00 (it neither helps nor hurts).
• Masking role words in the text does not change the ranking (0% keyword-dependence).

Relevance is decided by substance (e.g. "patient-monitoring escalation protocol" → nurse), which is exactly what a keyword matcher cannot do.

Usage — scoring + the gate

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch
name = "FDS-Iterations/third-pass-feed-ranker"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name).eval()

def scores(title, posts):
    enc = tok([title]*len(posts), posts, truncation=True, max_length=160, padding=True, return_tensors="pt")
    with torch.no_grad():
        return model(**enc).logits.squeeze(-1).tolist()

TAU = 3.0    # promotion margin; calibrate per deployment (see below)
def third_pass(title, slate):
    s = scores(title, slate)
    challenger = max(range(1, len(slate)), key=lambda i: s[i])
    return ("promote", challenger) if s[challenger] - s[0] > TAU else ("no_change", None)

The model only scores. The gate (which item to move to slot 1, and the τ threshold) is your logic — none of it is in the weights. Calibrate τ on your own feeds to trade recovery vs. false-promotion. τ is distribution-sensitive. This model was trained with a listwise objective that widens the score range, so its natural margins are large: τ≈3.0 holds false-promotion around 10% on realistic feeds — much higher than the τ≈0.15 that suited the older pointwise model. Re-calibrate on a sample of your own no-relevance feeds.

Evaluation (held-out roles AND posts; synthetic)

metric	value
realistic feed: buried relevant post surfaced to #1	0.84
role-keyword effect on a relevant post's score (target ≈ 0)	≈ 0.00
keyword-dependence (masked vs normal ranking)	0%
gated recovery on adversarial worst-case slates (τ=3.0)	0.08–0.26
protection of high-value org announcements	~0.81–0.92

On a realistic feed — the relevant post competing against ordinary filler — the model puts it first 84% of the time. On a deliberately adversarial worst-case slate (~9 simultaneous strong competitors, including posts genuinely relevant to other roles), exact-#1 recovery drops to ~0.29 (top-3 ~0.59); that is a stress bound, not a realistic-feed number.

Announcement protection is a deliberate trade. Because the listwise objective makes a strongly relevant post outscore an incumbent announcement more often, must-see-announcement protection is ~0.81–0.92 (down from ~1.0 in the pointwise model). This model favors recovering a buried role-relevant post over protecting every announcement. If you need stricter announcement protection, raise τ or add an explicit announcement guard in your decision logic.

Intended use & limitations

• Re-ranking short enterprise-feed candidate slates by job-title relevance; abstains on role-less titles.
• Trained entirely on SYNTHETIC data (LLM-generated posts + synthetic slates) — it has not seen real feed content. Validate on your own data before production.
• Being a small CPU model, it does not perfectly resolve many simultaneous strong competitors (e.g. distinguishing this role's post from several other-role-relevant posts at once); it excels when the relevant item competes against ordinary filler.
• Cross-lingual mixing (non-English title vs English-only feed) is weaker than same-language feeds.
• Relevance is title-driven; recency/importance beyond the relevance signal must be added in your decision logic.

Training & method

O*NET-derived job titles; LLM-generated posts where relevance is by substance; adversarial 20-item slates (gem / abstain / announcement types) with held-out roles+posts. Two things make this version different from a plain pointwise regressor:

1. Role-keyword rebalance — the viewer's role name is injected into a random half of all posts during training (independent of the label), so the keyword is decorrelated from relevance and the model cannot use it as either a shortcut or a penalty.
2. Listwise ranking loss — training batches whole slates and adds a softmax cross-entropy term that pushes the truly-relevant post to rank #1, on top of a pointwise anchor. Inference stays pointwise (one title×post score at a time), so CPU cost is unchanged. Headline metric is role-masked recovery so a keyword shortcut cannot inflate it.

License & attribution

Apache-2.0. Inherits from cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 — verify its license carries through. Training posts were generated with a Qwen model; review the applicable terms.