zlm-v1-signal-extract

A 77M-parameter FLAN-T5 model that reads a short text and returns its keywords and the user intent behind it — in one generative pass, in ~136 ms on CPU.

Given an ad headline, search query, or product blurb, the model generates two signals that normally require two separate systems (a keyword extractor and an intent classifier):

• keywords — the terms that carry the text's meaning, and
• user intent — a free-text intent phrase plus one of three standard search-intent classes: informational, transactional, or navigational.

On an independent 1,000-example benchmark it beats a frontier-LLM baseline (GPT-5-nano) on both jobs — +0.10 semantic keyword F1 and +0.32 intent-category accuracy — while running ~9× faster. It is built for high-volume, latency-sensitive enrichment (contextual ad targeting, query understanding, analytics) where an LLM call per request is too slow and too expensive.

How it works

The model is a fine-tuned google/flan-t5-small (encoder-decoder, 77M parameters). Both signals come out as a single short sequence:

Input:   extract keywords and user intent: <your text>
Output:  keyword 1, keyword 2, ... [SEP] <intent name> [SEP] <intent category>

Example:

Input:   extract keywords and user intent: Best price on wireless noise-cancelling headphones
Output:  wireless headphones, noise cancelling, best price [SEP] buy headphones [SEP] transactional

It reads up to 256 input tokens and generates up to 48. Input is expected in English — in multilingual deployments, translate to English first.

Files

Standard Optimum split-ONNX layout, mixed precision for CPU serving:

File	Precision	Role
encoder_model.onnx	fp32	Runs once per input
decoder_model_quantized.onnx	int8 (dynamic)	First generation step
decoder_with_past_model_quantized.onnx	int8 (dynamic)	Subsequent steps (KV-cache)

Why mixed precision: quantizing the encoder collapses output quality (measured keyword F1: fp32 encoder 0.451 → full-int8 0.044), because every generated token conditions on the encoder's hidden states. The encoder runs once, so fp32 there is cheap; the decoders run per output token, so int8 there is where the speed comes from.

Usage

With optimum[onnxruntime]:

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

repo = "ZeroGPU/zlm-v1-signal-extract"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = ORTModelForSeq2SeqLM.from_pretrained(
    repo,
    encoder_file_name="encoder_model.onnx",
    decoder_file_name="decoder_model_quantized.onnx",
    decoder_with_past_file_name="decoder_with_past_model_quantized.onnx",
)

def extract(text: str):
    inputs = tokenizer(f"extract keywords and user intent: {text}", return_tensors="pt", truncation=True, max_length=256)
    out = model.generate(**inputs, max_new_tokens=48)
    decoded = tokenizer.decode(out[0], skip_special_tokens=True)
    parts = [p.strip() for p in decoded.split("[SEP]")]
    keywords = [k.strip() for k in (parts[0] if parts else "").split(",") if k.strip()]
    return {
        "keywords": list(dict.fromkeys(keywords))[:8],  # dedupe + cap, as in production
        "intent_name": parts[1] if len(parts) > 1 else "",
        "intent_category": parts[2] if len(parts) > 2 else "",
    }

print(extract("Best price on wireless noise-cancelling headphones"))

Notes:

• [SEP] is registered as a non-special token, so it survives skip_special_tokens=True decoding — that is what makes the three-field output parseable.
• The same files run in Node.js with onnxruntime-node (greedy loop over the split decoders); generative_enrichment_metadata.json carries the serving defaults (prompt template, token limits, file roles).

Evaluation

Two complementary benchmarks: a balanced independent held-out set, and a harder real-world multilingual stress test.

Independent 1,000-example benchmark (vs GPT-5-nano)

Metric	zlm-v1-signal-extract	GPT-5-nano
Keyword F1 (semantic match)	0.566	0.467
Keyword F1 (exact match)	0.381	0.235
Intent category accuracy	0.915	0.594
Intent name ROUGE-1	0.327	0.358
Latency p50 (CPU)	136 ms	1,218 ms

Per-class intent accuracy: informational 0.895, transactional 0.887, navigational 0.975 — the model discriminates all three classes rather than defaulting to the majority one.

5,000 real-world multilingual prompts (vs GPT-5.4-nano, GPT-5.5 gold)

Keyword F1	zlm-v1-signal-extract	GPT-5.4-nano
All 5,000 prompts	0.465	0.415
English	0.479	0.609
Non-English (translated)	0.456	0.286

The model's keyword quality is essentially language-stable (0.479 vs 0.456), while the LLM baseline loses more than half its accuracy outside English. On this production-skewed set (~69% informational) the model's intent calibration is its honest weak spot: category macro-F1 0.381 vs the LLM's 0.628, driven by over-predicting the rare navigational class. Trained class-balanced, it is excellent on balanced data (0.915 above) and miscalibrated under heavy skew — recalibrating class priors to the deployment distribution is the known next improvement.

Training

• Fine-tuned from google/flan-t5-small on a teacher-labelled corpus of short real-world texts; keywords and intents were relabelled by a stronger teacher LLM, with the rare navigational class oversampled (from under 1% to roughly a third of examples) so all three intent classes are learned.
• Trained in fp32: FLAN-T5 was pretrained in bf16 and overflows to NaN loss under fp16 — a practical caveat for anyone reproducing the fine-tune on pre-Ampere GPUs.
• Exported to split ONNX (encoder / decoder / decoder-with-past) with dynamic int8 quantization on the decoders only, for the reasons in Files.

The training and evaluation datasets are not published.

Limitations

• English input only — quality degrades on untranslated non-English text.
• Intent calibration under skew — see Evaluation; apply your own class priors or thresholds if your traffic is heavily informational.
• The intent name is free-form generated text; treat it as a descriptive hint, not a controlled vocabulary.
• Benchmark gold labels are LLM judgements (GPT-5.5 / independent labelling), not human annotation — read absolute numbers as relative comparisons between systems.
• Output is capped at 48 tokens: roughly 6–10 keywords plus the intent fields.

License and attribution

• Weights: Apache-2.0 (same as the base model).
• Built on FLAN-T5 (Google).

Citation

@misc{zerogpu2026signalextract,
  title  = {zlm-v1-signal-extract: single-pass keyword and user-intent extraction with a fine-tuned FLAN-T5-small},
  author = {ZeroGPU},
  year   = {2026},
  url    = {https://huggingface.co/ZeroGPU/zlm-v1-signal-extract}
}