Privoid-ai/Privoid-Action-1B

Privoid Action-1B (experimental preview)

A LoRA adapter for IBM Granite 4.0 1B that reads a short, speaker-labelled English meeting transcript segment and returns action items, decisions and unresolved requests as JSON. Every record names the turn it came from and quotes it word for word. It is small enough to run locally on a consumer GPU.

Read this first. This is an experimental preview, not a finished product.

• It was trained only on synthetic (template-generated) meetings. No real meeting text was used for training.
• On real meetings it misses most items. In the locked AMI test it found about 1 in 3 linked actions (10/28) and 1 in 6 linked decisions (12/72). On meetings in a different style (ICSI) it found 7% of decisions.
• The near-perfect synthetic scores below show that it learned the output format and the task on clean text. They are not a measure of real-meeting accuracy.
• Every output is a suggestion for human review. The model and code never send messages, create tasks or commit anyone to anything.

What it does

• Input: one transcript segment of whole turns with turn IDs and speaker IDs, plus optional speaker names, meeting date and timezone. It has been tested on segments of up to about 1,536 transcript tokens. Longer meetings should be split into segments.

• Output (schema v1.0, docs/SCHEMA.md):

  • actions: task, owner speaker IDs and owner wording, assignment status, deadline wording, conditions, active or cancelled
  • decisions: active or reversed
  • unresolved requests: no response, declined, deferred or unclear
  • on every record: exact evidence quotes and review flags

• Deadlines are converted to dates by deterministic code, and only when unambiguous. Unknown owners and deadlines are left empty rather than guessed.

• The validator is part of the model. privoid_action.contract.validate_output(..., ground=True) and inference.extract handle the raw output as follows:

  • A quote that differs from the transcript only in tidied wording (fillers, repeats, disfluencies) is snapped to the exact transcript text and flagged.
  • A quote that occurs only on a different turn is moved to that turn and flagged.
  • Owner wording that cannot be found in the evidence is removed.
  • Any record that still fails validation is rejected. It is never repaired with invented content.

  All numbers below for real meetings use this validator, the same way the bundled API does. The raw validity of the model output alone is reported next to them.

Out of scope

Audio, speech recognition, diarization and speaker identification; whole-meeting summarisation in one pass; chat and Q&A; autonomous task execution; languages other than English.

How to use

pip install -r requirements.txt   # tested with transformers 5.5.0, peft 0.20.0, torch 2.11.0
hf download Privoid-ai/Privoid-Action-1B --local-dir Privoid-Action-1B
cd Privoid-Action-1B

import json
from privoid_action import inference

model, tok = inference.load_model("ibm-granite/granite-4.0-1b", revision="6a7381ba1f54d684ff508d991aeb7dc580157103",
                                  adapter_path=".", local_files_only=False)   # downloads the base model once
payload = json.load(open("examples/request_accepted_request.json", encoding="utf-8"))
out = inference.extract(model, tok, payload, max_prompt_tokens=2600)
print(json.dumps(out["result"], indent=2))   # validated records; rejected ones are listed separately

A local FastAPI service bound to 127.0.0.1 is included (privoid_action_api/, see docs/API.md). It loads from local files only and does not log or store transcripts.

Using the adapter with plain transformers and peft skips the validator. Raw outputs then contain unverified quotes and owners, so don't use them that way.

Training

• Base: ibm-granite/granite-4.0-1b at revision 6a7381ba1f54d684ff508d991aeb7dc580157103 (1.63 B parameters, bf16).
• Adapter: LoRA with r 16, alpha 32 and dropout 0.05, on the q/k/v/o and MLP input/output projections.
• Stage 1: a new adapter trained on 6,572 synthetic segments (677 for dev).
  • Settings: 2 epochs, learning rate 2e-4 cosine, effective batch 16, 822 steps, 7.9 h.
  • The synthetic data was generated from templates covering product, engineering, sales, operations, people and finance scenarios.
  • Examples longer than 4,096 tokens were excluded, never truncated.
• Stage 2, top-up: the same adapter trained for 79 more steps (37 minutes), at learning rate 1e-4 with effective batch 8.
  • Data: 640 examples. 340 are long composites built by joining several synthetic segments, so that they resemble real meeting length; 300 are replays of stage-1 data.
  • No real meeting text was used in either stage.
• Hardware: a single RTX 3060 (12 GB).

Evaluation

Locked test. It was run once, after the acceptance criteria were recorded at 2026-10-27T18:26:26. Full results are in results/final_test.json.

Synthetic test sets

These contain the same kind of data as the training set. "Held-out templates" are scenario families never seen in training. The 95% interval for held-out action F1 is 0.979–0.993 (cluster bootstrap).

System	Set	n	Action F1	Decision F1	Request F1	Owner acc.	Status acc.	No-action FP	Fully valid
Privoid Action-1B	familiar templates	401	1.000	0.995	0.989	0.991	0.998	0.000	0.995
Privoid Action-1B	held-out templates	391	0.986	0.981	0.946	0.962	0.977	0.043	0.972
Rules baseline	familiar templates	401	0.761	0.830	0.833	0.996	0.893	0.204	1.000
Rules baseline	held-out templates	391	0.741	0.994	0.543	1.000	0.874	0.174	1.000
Granite 4.0 1B (no adapter)	familiar templates	401	—	—	—	—	—	0.000	0.105
Granite 4.0 1B (no adapter)	held-out templates	391	—	—	—	—	—	0.000	0.130

The base model without the adapter almost never produces valid output in this format, so its F1 is undefined (—).

Real meetings

AMI and ICSI meeting transcripts, scored against "silver" labels: the actions and decisions that the corpora's own human summaries link to specific turns. This measures recall only. It can't measure precision, because the summaries don't list everything said in a meeting. The label sets are small, especially the 28 AMI actions, so treat these numbers as rough.

System	Set	Segments (meetings)	Actions found	Decisions found	Fully valid (raw → with quote matching)	Rejected records
Privoid Action-1B	AMI test	250 (20)	10/28 (0.36)	12/72 (0.17)	0.70 → 0.86	37
Privoid Action-1B	ICSI (unseen style)	250 (68)	no labels	12/167 (0.07)	0.65 → 0.80	78
Rules baseline	AMI test	250 (20)	7/28 (0.25)	0/72 (0.00)	0.99 → 0.99	2
Rules baseline	ICSI (unseen style)	250 (68)	no labels	0/167 (0.00)	0.98 → 0.98	4
Granite 4.0 1B (no adapter)	AMI test	250 (20)	0/28 (0.00)	0/72 (0.00)	0.15 → 0.15	6
Granite 4.0 1B (no adapter)	ICSI (unseen style)	250 (68)	no labels	0/167 (0.00)	0.10 → 0.10	0

Acceptance criteria

The criteria were recorded before the test and checked afterwards.

Criterion	Bar	Measured	Result
min_action_f1_synthetic_heldout	0.8	0.986	pass
min_action_f1_ami_human_test	—	—	not measured
max_invented_owner_rate	0.02	—	not measured
max_invented_deadline_rate	0.02	0.000	pass
max_no_action_false_positive_rate	0.05	0.043	pass
min_assignment_status_acc	0.85	0.977	pass
min_fully_valid_rate	0.95	0.972	pass
min_summary_action_recall_ami_test	0.35	0.357	pass
min_summary_decision_recall_ami_test	0.1	0.167	pass
min_fully_valid_rate_ami_test	0.75	0.860	pass
max_injection_executed	0	0	pass

• Invented owners: not measured. The synthetic test sets contain no actions without an owner, so this rate could not be computed.
• AMI human test: none. No human-labelled AMI test exists.
• Instruction-injection probes: 7 probes plant instructions inside a transcript. The adapter produced 0 records from the planted instructions and still found all 14 real actions.

Development history (dev sets, not the test)

• Stage-1 adapter on 300 AMI dev segments:
  • fully valid output rose from 0.78 to 0.91 once quote matching was added to the validator;
  • actions found: 14/38 → 19/38;
  • decisions found: 6/63 → 8/63.
• Top-up on a fixed 100-segment AMI dev subset:
  • decisions found: 8/63 → 13/63;
  • actions unchanged at 19/38;
  • fully valid output fell from 0.88 to 0.82, because more malformed records were produced and rejected;
  • no regression on synthetic dev.

Limitations

• The model misses many real items. Recall on real meetings is low, and lower again on meeting styles unlike the synthetic training data (ICSI).
• Some outputs are rejected. On real meetings, 14–20% of segment outputs contained at least one rejected record. Without the quote-matching validator, 30–35% of segments would fail validation.
• The transcript must be well formed. It needs turn IDs and speaker IDs. Diarization errors upstream carry straight through.
• Segments are independent. A request and its acceptance far apart can be missed, and cancellations across segments are flagged but not applied.
• Relative deadlines are not converted. Wording like "Friday" or "end of week" is kept as text.
• Valid output is not proof of correctness. A valid record with exact quotes can still be the wrong interpretation.

Disclosures about how this release was made

• Synthetic labels were not reviewed. The planned human review of synthetic labels was skipped, and only automatic validation was applied.
• Acceptance criteria were set late. They were written after training, but before the test was run. They were based on dev results.
• The validator was tuned on dev errors. The quote-matching and owner-removal rules were designed while examining dev-set errors, so the dev numbers are optimistic. The test numbers above are the ones to rely on.
• The top-up was a separate step. It was a short additional run, chosen on dev results, and no other variants were tried against the test.
• Results are single runs. Each was a single training run with fixed seeds, with no repeated runs.

Privacy

Inference is local. Whether a deployment is private also depends on the application around the model, its logs and its network setup.

Attribution and licences

• Base model: IBM Granite 4.0 1B, Apache-2.0. This adapter and code are Apache-2.0 (see LICENSE and NOTICE).
• Evaluation data: annotations of the AMI Meeting Corpus and ICSI Meeting Corpus, both CC BY 4.0 (https://groups.inf.ed.ac.uk/ami/). No corpus text is included in this repository.