WhissleAI/arc-tagger

Arc Tagger (10M), v0.3.0

Arc Tagger is a text-only streaming tagger for live speech transcripts. It reads the word stream of any ASR (Deepgram, Speechmatics, Whisper, NeMo, your own ASR, or plain text). As each word arrives, it emits:

• entity and number spans;
• PII spans for redaction;
• profanity and toxicity;
• discourse acts, intents and safety flags, including prompt-injection / jailbreak attempts;
• speaker roles;
• events, such as speaker change, turn yield, sports, legal and meeting events;
• content state;
• affect: emotion, arousal, valence and intensity (experimental, text only).

The model has about 10 M parameters and runs on CPU. Interim ASR hypotheses can be revised: only the revised words are re-processed.

Usage terms

• Inference use only. This repository provides the model and inference code. No training or fine-tuning recipes are provided.
• Non-commercial. The weights and code are licensed under CC BY-NC 4.0 plus the Whissle terms (see LICENSE).
• Commercial licensing: contact WhissleAI (ksingla@whissle.ai).

Tag families

284 tags across 10 read-outs (365 tag@readout columns). ontology/ holds the full definitions, and config.json["columns"] lists every column.

family	tags (examples)	read-out
Entities	entity.mention.{person,organization,team,place,event_name,legal_ref,term,...} (10)	word spans
Numbers	entity.number.{date,time_of_day,duration,count,money,percent,measure,score,clock,ordinal,id_pii,...} (16), entity.numeral	word spans
PII / redaction	safety.pii.* (11): {private_name,phone_number,email,street_address,card_account_number,gov_id,date_of_birth,url_handle,public_contact} spans; safety.pii.{request,leak} per utterance	spans and utterance
Profanity / toxicity	safety.profanity.{mild,strong}, safety.toxicity.* (6)	spans and utterance
Safety	safety.jailbreak.attempt (experimental), safety.scam.* (6), safety.crisis.self_harm, safety.threat.escalation, safety.child.* (2), safety.solicitation.dating_romantic	utterance
Discourse act	discourse.act.* (7: statement, question, command_request, ...), discourse.answer, discourse.backchannel, discourse.assistant.* (2)	utterance
Intent	26 cross-domain intent.* (request_information, schedule_arrange, complain_criticize, ...), 60 assistant discourse.intent.* (alarm_set, weather_query, ...)	utterance
Speaker role	speaker.role.* (19, including contact-centre agent and customer)	per speaker (needs speaker ids on the words)
Events	speaker.change, discourse.turn_yield, discourse.interruption, event.sports.* (18), event.legal.* (3), event.meeting.{decision,action_item}	per word
Content state	content.mode (8), content.subject (12), content.game_state (6), content.legal_phase (9), content.meeting_phase (6), content.sports_focus (7)	per word
Affect (experimental)	affect.emotion.* (5), affect.arousal.{low,medium,high}, affect.valence.{neg,neu,pos,mixed}, affect.intensity.{1,2,3}	utterance

Quickstart

pip install huggingface_hub
huggingface-cli download WhissleAI/arc-tagger --local-dir arc-tagger
cd arc-tagger
pip install -r requirements.txt     # CPU torch is enough: pip install torch --index-url https://download.pytorch.org/whl/cpu
python example.py

Offline

import sys; sys.path.insert(0, "arc-tagger")           # the arctag package ships in the repo
from arctag import ArcTagger

tg = ArcTagger.from_pretrained("arc-tagger")            # a local dir, or the repo id "WhissleAI/arc-tagger" (the default)
out = tg.tag("Hi, this is Maria from Northwind Bank. Can I get your phone number?", domain="talk")
print(out["redacted_text"])      # Hi, this is [NAME] from Northwind Bank. ...
print(out["spans"])              # entity / number / PII / profanity spans with probabilities
print(out["utterances"])         # per-utterance intent, act, safety, role, state, affect

Streaming

Push words as the ASR emits them. An interim hypothesis is replaced with revise_from=k (it replaces words[k:]).

st = tg.stream(domain="assistant")
st.push_words(["set", "an", "alarm", "for"], final=False)                                   # interim
up = st.push_words(["set", "an", "alarm", "for", "seven", "thirty."], revise_from=0, final=True)
up = st.endpoint()           # end of utterance: utterance read-outs arrive in up["utterances"]
res = st.finalize()          # the full result (same schema as tag())

Live ASR scripts (low latency)

Two scripts connect a live ASR to the tagger. Each takes a wav file or the microphone, and prints the live tags and redacted text as JSON lines. The API key is read only from an environment variable.

pip install websockets          # plus sounddevice for --mic

export DEEPGRAM_API_KEY=...
python stream_deepgram.py --wav call.wav --domain talk
python stream_deepgram.py --mic --domain assistant --seconds 30

export SPEECHMATICS_API_KEY=...
python stream_speechmatics.py --wav call.wav --domain talk
python stream_speechmatics.py --mic --domain meeting --url wss://eu2.rt.speechmatics.com/v2

How provider messages map to the stream:

provider message	tagger call
Deepgram Results, is_final=false	push_words(words, final=False, revise_from=n_final)
Deepgram Results, is_final=true	push_words(words, final=True, revise_from=n_final)
Deepgram speech_final=true or UtteranceEnd	endpoint()
Speechmatics AddPartialTranscript	push_words(words, final=False, revise_from=n_final)
Speechmatics AddTranscript	push_words(words, final=True, revise_from=n_final)
Speechmatics EndOfUtterance	endpoint()

• n_final is the number of words the provider has already finalised, so interim words never rewrite final ones.
• An end of utterance that arrives while interim words are pending is applied right after those words become final.
• Speaker ids from diarization (Deepgram diarize=true, Speechmatics diarization: "speaker") are passed on as spk. They enable speaker roles.
• Useful options:
  • --no-interim: print only final updates.
  • --result out.json: write the full result at the end.
  • --tier T1: delayed, more stable labels.
• Each line is {"type": "update", "source": "interim" | "final" | "endpoint", "spans", "events", "utterances", "running", "redacted_text"}. The last line is {"type": "done", ...}.

Status.

• Deepgram: the adapter is tested offline against recorded-format message sequences.
• Speechmatics: tested against the documented message format; not yet tested live.

The fixtures and tests are in tests/ (python tests/test_adapters.py). The adapters themselves (arctag/asr.py) have no network code, so you can reuse them with any client.

Input

tag(x, domain="auto", tier="T0")

x can be any of:

• a str;
• a list of words;
• a list of word dicts {"w": str, "t0": s, "t1": s, "conf": float, "spk": str|int, "final": bool}. Every field except w is optional.

Offline, utterances end after words ending in . ? !. In streaming, call endpoint() at the ASR's end of utterance.

domain is one of auto | talk | meeting | court | assistant | sports | news.

• It selects the per-domain behaviour and thresholds.
• talk covers contact centre, podcasts and chat.
• In sports and court, the specific league or court type is detected from the words.
• auto is weaker. Set the domain when you know it.

Streaming: push_words(words, final=True, revise_from=None, endpoint=False), endpoint(), finalize().

Output (JSON)

{
 "meta": {"model": "arc-tagger", "version": "0.3.0", "domain": "talk", "fine_domain": "talk", "tier": "T0",
          "thresholds": "display + events + default"},
 "words": ["Hi,", "this", "is", "Maria", ...],
 "spans": [                                     // word indices [start, end)
   {"tag": "safety.pii.private_name", "family": "pii", "start": 3, "end": 4, "text": "Maria", "p": 0.97, "thr": 0.42,
    "alt": {"tag": "...", "p": 0.01},           // runner-up type
    "ver": 0.99,                                // span verifier probability
    "src": "head"}
 ],
 "entities": [...], "pii": [...], "profanity_toxicity": [...],   // spans split by family
 "redacted_text": "Hi, this is [NAME] from ...",                  // PII, profanity and toxicity spans masked (public_contact is not)
 "utterances": [
   {"start": 0, "end": 8, "text": "...",
    "intent": ["intent.request_information"], "act": ["discourse.act.question"], "safety": ["safety.pii.request"],
    "role": [], "role_note": "speaker role needs diarization (no speaker id on words)",
    "item": [], "state": {"content.mode": ["content.mode.call_in"]},
    "emotion": [{"tag": "affect.emotion.neutral", "p": 0.93, "thr": 0.5, "on": true},   // top tag of each affect group:
                {"tag": "affect.arousal.low", ...}, {"tag": "affect.valence.neu", ...}, {"tag": "affect.intensity.1", ...}],
    "scores": {"intent": [{"tag": "...", "p": 0.0, "thr": 0.0, "on": false}], "act": [...], ...}}   // top-k with probabilities
 ],
 "events": [{"tag": "speaker.change", "at": 15, "word": "Yeah,", "p": 0.94, "thr": 0.3, "tier": "T0", "read": "ev_rec"}]
}

• The label lists (intent, act, safety, ...) hold the tags at or above their threshold.
• scores keeps the top-k with probabilities.
• emotion holds the most likely tag of each affect group (emotion, arousal, valence, intensity) with its probability.

Each streaming update (from push_words or endpoint) contains:

• n_words and changed_from;
• spans: the re-decoded window (replace your spans that end after its start);
• events: new events only;
• utterances: the utterances that just ended;
• running: the live intent, act, safety and state of the open utterance.

Latency tiers

tier	lookahead	use
T0	0 words	live display and zero-lag redaction (default)
T1	up to 3 words	revisions a few words later
T2	to the utterance end	final utterance labels

All three tiers are read from the same pass, so they add no compute. Utterance-level labels (intent, act, safety, role, affect) are read at the endpoint.

Measured CPU latency (1 thread, one word per call, on a heavily shared server CPU; expect lower on an idle core):

	p50	p95
model step per word	24.56 ms	50.02 ms
full push_words() call per word (span decoding and read-outs included)	35.2 ms	72.4 ms

Evaluation

All numbers are on the Whissle internal held-out test set (multi-domain: contact centre, assistant, meetings, news, talk, sports and courts; ASR word streams). The labels are internal, and the numbers are not comparable to public leaderboards.

Whissle internal held-out test set, offline pass (609 sessions, 249,866 words)

metric	value
primary score (spans, AP, events, exclusive groups, PII leak)	0.7584
span exact F1 (families, mean over tiers T0 to T2)	0.7089
macro AP over tags (mean over tiers)	0.7326
macro AP over tags, T0 / T1 / T2	0.722 / 0.7396 / 0.7361
macro AUC over tags, T0 / T1 / T2	0.9737 / 0.974 / 0.974
event F1 (±2 words, T0, events with at least 20 positives)	0.8854
exclusive-group macro F1 (T0)	0.6012
PII words shown unmasked at first display (T0)	1.23 % (non-PII words masked: 1.557 %)

family	span exact F1, T0	span exact F1, T2	AP, T0
PII (safety.pii)	0.7304	0.7751	0.8609
Profanity (safety.profanity)	0.8614	0.8557	0.9686
Numbers (entity.number)	0.6807	0.7383	0.7568
Entities (entity.mention)	0.506	0.5116	0.5716
Speaker change (event)	F1 0.9674		0.9951
Turn yield (event)			0.7747
Speaker role (speaker.role)			0.8013
Assistant intent (discourse.intent)	top-1 0.5953, top-3 0.7921		0.6276
Content mode (content.mode)			0.9947

Span F1 is for the span-head decode, which the package uses for every span family.

Whissle internal held-out test set, streaming first display (35 sessions, 36,662 words)

The sessions are replayed through the streaming engine as a live-ASR message sequence (interim words, revisions, finals, endpoints). Each label is scored as first shown to the user (T0), with the default thresholds.

read-out (streaming, T0 first display)	precision	recall	F1	positives
spans (word level, all span families)	0.761	0.694	0.726	3,516
intents (utterance)	0.742	0.524	0.615	368
turn events (speaker change, turn yield, ...)	0.939	0.375	0.536	1,668
speaker roles	0.560	0.667	0.609	21
utterance safety flags	0.818	1.000	0.900	9
discourse acts (utterance)	0.380	0.475	0.422	40

span family (streaming, span exact F1)	T0	T1	T2
Numbers (entity.number)	0.5726	0.6912	0.6909
Entities (entity.mention)	0.4163	0.4444	0.4542

The speaker-role, act and safety rows have few positives (21, 40 and 9).

ONNX / C runtime

onnx/ holds the same model exported for the T0 read-outs: arctag.onnx (fp32) and arctag_int8.onnx (int8), with arctag_meta.json (read-out columns, thresholds, domain profiles), tokenizer.json and SHA256SUMS. A dependency-light C runtime that loads these files is at github.com/WhissleAI/arc-tagger-c.

	fp32	int8
file size	34.0 MB	11.5 MB
resident memory (RSS)	110 MB	76 MB
span F1 of int8 against fp32 output		0.962
span F1 on the Whissle internal held-out test set	0.468	0.465

Files

file	what
model.safetensors	weights (fp32, 45 MB)
config.json	read-out columns, ontology id and default thresholds
tok/bpe8k/	tokenizer
ontology/	tag ontology, text-tag table, the ontology extension (contact-centre roles, jailbreak, affect valence and intensity), tag-definition embeddings
thresholds/display.json	display thresholds per tag, read-out and domain (tags without an entry use the default thresholds in config.json)
thresholds/events.json	per-event display tier, thresholds and hidden events
thresholds/span_head.json	span decoding threshold per family and tier
engine_settings.json	the engine's display floors and domain mapping (for reference)
arctag/	Python package: ArcTagger, driver.py (session to JSON), asr.py (live ASR adapters), live.py (audio input), _core/ (streaming engine)
stream_deepgram.py, stream_speechmatics.py	live ASR scripts
tests/	adapter tests and provider message fixtures
example.py	quickstart
LICENSE, NOTICE	licence (CC BY-NC 4.0 + Whissle terms) and copyright notice
onnx/	fp32 and int8 ONNX exports, their metadata, tokenizer and checksums (see ONNX / C runtime)
MANIFEST.sha256	checksums

Limitations

• English only. Non-English input gives meaningless output.
• Text only, with no audio cues. The model cannot detect speaker identity, emotion in the voice or vocal events. Speaker roles need speaker ids (spk) on the words, for example from ASR diarization.
• The internal test set above has no toxicity spans, so this card reports no toxicity number. Validate toxicity detection on your own data before relying on it.
• Entity mention spans (entity.mention) are the weakest span family (exact F1 about 0.51 on the test set).
• Spans shown at first display (T0, often on interim ASR words) are less accurate than the same spans a few words later (T1) or at the utterance end (T2). Use tier="T1" where a delay of up to 3 words is acceptable.
• Utterance-level safety, jailbreak and scam flags can fire on short or out-of-domain utterances, and on utterances that follow a flagged one. For anything consequential, use scores (p) and your own policy.
• Affect (emotion, arousal, valence, intensity) comes from the words only and is experimental; this card reports no affect evaluation.
• The auto domain is weaker than setting the domain. Some sports events fire loosely outside sports.
• Redaction masks the decoded spans. It is not a compliance guarantee: review it for your use case.
• The default thresholds favour precision. Tune them for your traffic if you can.
• The Python package runs fp32 on CPU; the ONNX exports cover the T0 read-outs only.

Licence

The weights and the code, including all inference code: CC BY-NC 4.0 plus the Whissle terms in LICENSE.

• Non-commercial use only.
• Commercial use needs a separate written licence from Whissle Inc.: ksingla@whissle.ai.

This covers every file in the repository, including arctag/_core/ and onnx/ (see NOTICE).