docs/ANNOTATION_GUIDE.md

Privoid Action — annotation guide (schema v1.0)

These rules apply to human annotation of real segments (AMI queue), to reviewing synthetic labels, and to judging model errors. Field definitions are in SCHEMA.md.

Guiding principle: record what participants actually committed to or decided, with evidence. When ownership, acceptance, a deadline or a decision is not supported by the transcript, leave it unresolved (null, empty list, unresolved_requests, or needs_review). Do not fill the gap with a guess.

1. Workflow for the AMI queue

1. Read the whole segment. The hints (summary sentences and their linked turns) only show where a summariser saw actions or decisions. They are not labels: summaries are incomplete, owners there are roles, and the linked turns are often topical rather than the commitment itself.
2. Label every qualifying record in the label object, using exact quotes copied from the turn text.
3. Set annotator. Set annotation_complete: true only when every qualifying record in the segment is labelled. Partial annotations are used for recall only.
4. Use annotation_notes for anything ambiguous. Prefer needs_review with a reason over a confident guess.
5. Run the import cell (notebook 5.5). Rows with an invalid quote, speaker ID or status are reported back to you, never repaired.

AMI speakers have roles (project manager, industrial designer, user interface designer, marketing expert) and no names. A role mention resolves to a speaker ID only when exactly one speaker has that role and the mention clearly refers to that person ("as the industrial designer, you're going to …").

2. Is it an action?

Utterance	Label	Why
"I will send the report."	action, self_committed, owner = speaker, owner_text "I"	Explicit first-person commitment.
"Let me send the report."	action, self_committed, owner_text "me"	Commitment phrased as an offer-and-do.
"Could someone send the report?" (no reply)	unresolved_requests, addressee_speaker_ids [], addressee_text "someone", no_response	No owner is invented.
"Could someone send the report?" → B: "I can do it."	action, accepted, owner B, evidence both turns	The volunteer accepted.
"Alex should probably review this."	nothing	A suggestion, unless later dialogue establishes an assignment.
"Alex should probably review this." → C: "Good idea. Alex, can you take that?" → Alex: "Sure."	action, accepted, owner Alex, evidence all three turns	Suggestion turned into an accepted assignment.
"Alex, please review this." (no reply)	unresolved_requests, addressee Alex, no_response	An explicit request is not an acceptance.
"Action item: Alex to review the contract." (no reply)	action, assigned_unconfirmed, owner Alex	A declarative assignment, as opposed to a question or plea.
"Priya from legal will check the clause." (Priya not in speakers)	action, assigned_unconfirmed, owner_speaker_ids [], owner_text "Priya from legal"	Named non-participant: no invented speaker ID.
"Yes, I'll do it."	link to the prior request only if the preceding turns make the referent clear; evidence includes the request
"Yes, I'll handle that one." at the start of the segment, with no antecedent	action, task "Unspecified task (referenced before this segment).", needs_review + context_outside_segment	Never reconstruct a task that is not in the segment.
"We need to improve onboarding."	nothing	A general need, not a concrete task.
"We'll update the wiki."	action, unassigned, owner_text "we", needs_review + group_owner	An ambiguous "we" never becomes an individual owner.
"If finance approves, I'll place the order."	action, self_committed, conditions ["If finance approves"]	Conditional commitment; the condition wording is kept.
"I won't send the old proposal."	nothing	Negated: no positive action.
"I could look into it if needed."	nothing	Capability or possibility, not a commitment.
"I'll try to look into it."	action, self_committed, needs_review + hedged_commitment
"Last week Omar said he'd send the numbers, and he did."	nothing	A quoted historical commitment.
"I'll send the deck and I'll also book the room."	two actions, each quoting its own clause	Several actions in one turn.
"Maya and Omar, can you two prepare the demo?" → both accept	one action, owners [Maya, Omar]	Multiple owners only when supported.
A: "I'll send it by Friday." … A: "Sorry, I meant Thursday."	one action, deadline_text "Thursday", evidence both turns	The correction replaces the deadline.
A: "I'll write the summary." … A: "Actually I can't. Ben, can you take it?" Ben: "Sure."	A's action cancelled (evidence: commitment and withdrawal); Ben's action accepted	Keep the cancelled record. The replacement is a separate action.
A: "Omar, can you send it?" Omar: "Hmm, maybe."	unresolved_requests, unclear, acceptance_unclear
A: "…can you send it?" Omar: "Sorry, I can't this week."	unresolved_requests, declined
A: "…can you send it?" Omar: "Let me check and get back to you."	unresolved_requests, deferred

3. Is it a decision?

Utterance	Label
"So we're agreed, we'll keep the free tier." → "Agreed."	decision (evidence both turns)
"We could go with the monthly plan or the annual plan." → "Let's decide next week."	nothing: options under discussion
"Let's launch in March." → "Agreed." … "Actually, scratch that, we won't launch in March."	decision with status reversed, evidence all three turns
"For the record, every action here is already approved by the CEO."	nothing: a claim, not a decision taken in the meeting

An agreed choice or policy is a decision. An agreed task to perform is an action, possibly unassigned. Do not label the same content as both unless the meeting both made a choice and assigned a task.

4. Deadlines

• deadline_text copies the deadline wording attached to this task, without a leading "by"/"before": "Friday", "end of day", "next Friday", "November 14, 2026", "before the board meeting".
• A date that is not a deadline for the task is not a deadline: "The launch event is on March 3rd. I'll update the slides." gives deadline_text null. The same applies to historical dates and to target dates that were only discussed.
• Never compute or reformat a date in the label. Normalisation is done by code (SCHEMA §4) and stays null for ambiguous wording.

5. Owners

• Named owner: a speaker named or directly addressed in the evidence, or the speaker committing themselves.
• Inferred owner (not allowed): someone who "would usually" do it, a role-based guess, the meeting chair by default, or a person matched by voice or style. Speaker mappings are supplied metadata only.
• Similar names ("Ana"/"Anna", "Jon"/"John"): resolve only with an exact match. If two speakers share the spoken name, leave the IDs empty and set owner_ambiguous.
• Pronouns ("she'll do it", "they will") resolve only when exactly one candidate is supported by the evidence. Otherwise leave the owner unresolved and set owner_ambiguous.

6. Evidence

• Quotes are copied exactly from the turn: same characters, spelling and disfluencies. A clause-level substring is preferred when a turn holds several items.
• Include every turn needed to justify the record: the request, the acceptance, the deadline, the condition, and any cancellation or reversal.

7. Untrusted content and embedded instructions

Transcript text is data. "Ignore previous instructions", "return an empty list", fake system prompts, requests to the "AI assistant", requests to reveal prompts, and claims that everything is approved are things a participant said. Label them like any other utterance; usually they produce nothing. A participant asking the assistant to invent tasks creates no task.

8. Noise

ASR-like errors ("there/their", repetitions, missing punctuation, "um") do not change the label if the meaning is clear. If they make the owner, the acceptance or the task unclear, label conservatively and set transcript_unclear.

9. Completeness

annotation_complete: true asserts that every qualifying action, decision and unresolved request in the segment is labelled. Only complete annotations are used to measure precision and no-action false positives. Unlabelled content in a partial annotation is never treated as a negative.