Privoid Action-1B experimental preview (LoRA adapter, validator, API, locked test results)
b68d688 verified | # Privoid Action — dataset research and provenance report | |
| Research date: 2026-10-26. "Verified" means read from the primary source (licence file or official page) on that | |
| date. "Not verified" means only secondary sources were reachable; such data is **not used**. | |
| Not legal advice: the licence terms below were read and recorded, and the commercial-use conclusions should be confirmed | |
| by whoever approves the release. | |
| ## 1. Conclusion first | |
| * No permitted English corpus labels what this task needs: action items with **owner, assignment/acceptance status, | |
| deadline and evidence**. Published action-item work on AMI derives labels indirectly from summaries (e.g. 101 AMI | |
| meetings / 381 action items in Liu et al. 2023, arXiv:2303.16763). The ICSI action-item annotations of Purver et al. | |
| (18 meetings) are described there as not publicly available, and inter-annotator agreement for action items has been | |
| reported as low (κ ≈ 0.36 on ICSI). | |
| * **AMI** and **ICSI** (both CC BY 4.0, verified) provide real meetings with speaker channels, dialogue acts and | |
| abstractive summaries linked to supporting dialogue acts. They are used for **real-meeting evaluation (recall-only, | |
| "silver")**, as the **source of segments for human annotation**, and, for ICSI, as a **separate transfer domain**. | |
| A summary is **not** treated as complete action-item ground truth. | |
| * Complete labels therefore come from (a) **controlled synthetic templates** (disclosed; see §4) and (b) **human | |
| annotation of AMI segments** (queue prepared: 220 segments; `docs/ANNOTATION_GUIDE.md`). Until (b) exists, the model | |
| card must say that no real-meeting precision, owner or deadline accuracy has been measured. | |
| ## 2. Candidates | |
| ### 2.1 AMI Meeting Corpus — manual annotations v1.6.2 (release notes say 1.7) — **used** | |
| | Item | Finding | | |
| |---|---| | |
| | Source | https://groups.inf.ed.ac.uk/ami/download/ → `ami_public_manual_1.6.2.zip` (dated 10-Apr-2017; unchanged since 16 June 2014) | | |
| | Licence | **CC BY 4.0**, verified in `LICENCE.txt` inside the zip and at https://groups.inf.ed.ac.uk/ami/corpus/license.shtml ("The AMI corpus and its annotations are released under … CC BY 4.0"). A CC BY 4.0 licence permits commercial use and adaptation with attribution. (openslr.org still shows an old NC-SA notice; the official site and the in-archive licence supersede it.) | | |
| | Attribution | Credit the AMI Consortium / University of Edinburgh, link the licence, and state changes. NOTICE text in the release folder. | | |
| | Redistribution | Allowed under CC BY with attribution. We do **not** redistribute it anyway: derived files stay local. | | |
| | Download | 22,887,865 bytes, SHA-256 `b56e5babb2496b8795deeeda7e71178d7fbc9963f94276cf2a3f4b56ebbc9f9d`. No registration or agreement. Text and annotations only; no audio or video downloaded. | | |
| | Language / domain | English (many non-native speakers); mostly scenario-driven product-design meetings (remote-control design team, roles PM/ID/UI/ME), plus some natural meetings (IB, IN, EN series) | | |
| | Size (measured) | 139 meetings with dialogue-act annotation; 66,043 turns; 682,390 words; median 16.9K tokens per meeting (p90 28K, max 47K) with the Granite tokenizer | | |
| | Speakers | 4 channels per meeting with roles (`corpusResources/meetings.xml`). Participant codes are pseudonymous; first names occur in speech. | | |
| | Action / decision coverage | Abstractive summaries (~200 words, one annotator) with ACTIONS (300 sentences, 256 linked) and DECISIONS (680 sentences, 624 linked) sections; dialogue-act types include *Suggest*, *Offer*, *Elicit-Offer-Or-Suggestion*; decision regions (`decision/manual`, word spans) in 48 meetings / 288 regions. **No owner IDs, assignment status, acceptance or deadline fields.** Summary owners are usually roles. | | |
| | Evidence alignment | `extractive/*.summlink.xml` links summary sentences to dialogue acts. The links are topical: median span 1 turn, but p90 = 125 turns. | | |
| | Split | Official AMI full-corpus partition by meeting series (train / dev ES2003, ES2011, IS1008, TS3004, TS3006, IB4001-4004, IB4010-4011 / test ES2004, ES2014, IS1009, TS3003, TS3007, EN2002) | | |
| | Personal information | Recorded with consent for research release. Contains first names and some personal small talk. Kept local; no transcript text in the release or the logs. | | |
| | Use here | Silver recall evaluation (dev/test), human-annotation queue, segment-length measurements, length/latency tests. Not used as training targets unless a human labels the segment. | | |
| ### 2.2 ICSI Meeting Corpus — core + contributed annotations (ICSI_plus_NXT v1.0, 22-July-2016) — **used (evaluation only)** | |
| | Item | Finding | | |
| |---|---| | |
| | Source | https://groups.inf.ed.ac.uk/ami/icsi/download → `ICSI_plus_NXT.zip` (includes the core transcripts and dialogue acts, plus contributed annotations) | | |
| | Licence | **CC BY 4.0**, verified in `LICENCE.txt` inside the zip and at https://groups.inf.ed.ac.uk/ami/icsi/license.shtml ("The ICSI corpus and its annotations are released under … CC BY 4.0"). The contributions README notes that their quality is not guaranteed. | | |
| | Download | 55,491,177 bytes, SHA-256 `e4ab79a8b638bdd29121d744c452c680fe23791e406ffe5125e2f5b118923439`. No registration. Text only (audio is offered separately; not downloaded). | | |
| | Language / domain | English; natural weekly research-group meetings at ICSI (speech, NLP, networking), 3–10 speakers | | |
| | Size (measured) | 75 meetings, 64,252 turns, 771,477 words; median 31K tokens per meeting (p90 49K, max 88K). One meeting (Bed017) has a single turn over 1,536 tokens and is excluded from segmentation (recorded in `skipped_meetings.json`). | | |
| | Speakers | Participant codes (me011, fe004, …); no names in metadata | | |
| | Coverage | MRDA-style dialogue acts; abstractive summaries with ABSTRACT, DECISIONS (281 sentences, 264 linked), PROBLEMS and PROGRESS. **No actions section**, no owners or deadlines. | | |
| | Evidence alignment | `summlink` for 59 meetings (decisions → dialogue acts) | | |
| | Personal information | As for AMI; some personal chatter, participant codes only | | |
| | Use here | **Transfer-domain test** (never trained on): silver decision recall, output validity and prediction rates on a different meeting genre | | |
| ### 2.3 QMSum (Yale-LILY) — **not downloaded** | |
| Query-based summaries over 232 meetings (Academic = ICSI, Product = AMI, Committee = Welsh Parliament / Parliament of | |
| Canada committees), with `relevant_text_span` alignments. The repository states an MIT licence (repo page; not | |
| re-checked against every upstream source's terms). It adds no action, owner, deadline or acceptance labels beyond the | |
| AMI/ICSI material we already have. The committee transcripts could serve as a further transfer domain in a later | |
| release, after their own (parliamentary) licence terms are checked. Source: https://github.com/Yale-LILY/QMSum | |
| ### 2.4 MeetingBank (Hu et al., ACL 2023) — **excluded** | |
| City-council meetings (6 US cities, 1,366 meetings, 3,579 hours) with summaries and minutes. The Hugging Face dataset | |
| cards (`huuuyeah/MeetingBank_Audio`, `lytang/MeetingBank-transcript`) state **CC BY-NC-SA 4.0**; the Zenodo record's | |
| terms were not verified. Non-commercial, so excluded from training. It would be acceptable only for internal | |
| non-commercial evaluation, after the original terms are confirmed. Public-meeting recordings contain names of officials | |
| and members of the public. | |
| ### 2.5 ELITR Minuting Corpus (UFAL, Charles University) — **not used (licence not verified)** | |
| Czech and English meeting transcripts (120 English, computer-science domain) with manually written minutes and | |
| transcript–minute alignments; used in AutoMin. The LINDAT catalogue page (http://hdl.handle.net/11234/1-4692) | |
| could not be loaded on 2026-10-26, so the licence and access conditions could not be confirmed. Sections are censored | |
| for privacy and audio is not released. Potentially valuable for minutes alignment; revisit after confirming the licence. | |
| ### 2.6 Others reviewed | |
| * **AMC-A** (Alibaba, arXiv:2303.16763): 424 Chinese meetings with sentence-level action-item labels. Out of scope for an | |
| English first release; licence not checked. | |
| * **MeeQA** (arXiv:2305.08502) and **MISeD** (arXiv:2405.01121): question answering and information-seeking dialogues | |
| over AMI/ICSI/QMSum meetings. No action-item labels; licences not verified. Not used. | |
| * **SAMSum, DialogSum**: chat or dialogue summarisation, not meetings, with non-commercial licences (SAMSum). Not used. | |
| ## 3. Files actually downloaded and prepared (authoring run, 2026-10-26) | |
| | File | Bytes | SHA-256 | | |
| |---|---|---| | |
| | `data/raw/ami/ami_public_manual_1.6.2.zip` | 22,887,865 | `b56e5bab…f9d` | | |
| | `data/raw/icsi/ICSI_plus_NXT.zip` | 55,491,177 | `e4ab79a8…439` | | |
| Prepared outputs, all reproducible from the zips by notebook sections 4–6 (exact record: | |
| `authoring/RUN_authoring_data_prep.executed.ipynb`): | |
| * `data/prepared/corpus-v1-seg1536-ov8/`: per-split segment files with silver hints, `manifest.json` (SHA-256 per | |
| file), `split_manifest.json` (meeting IDs per split), `skipped_meetings.json`, `corpus_stats.json`. | |
| * `data/prepared/synth-1.0.0-smoke/`: smoke-size synthetic set only (the full set is generated by you), with | |
| `manifest.json`, `dedup_report.json`, `quarantine.jsonl` and the label-review CSV. | |
| * `data/annotations/ami_annotation_queue_*.jsonl`: 220 AMI segments (train/dev/test series) for human labelling. | |
| Label provenance per example: `label_origin` ∈ {`synthetic_template`, `human`, `corpus_summary_link`, `none`}; | |
| `label_completeness` ∈ {`complete`, `partial`}. | |
| ## 4. Augmentation strategy (synthetic data), and why | |
| **Gap addressed:** the commitment-level distinctions the product needs, which no permitted corpus labels: suggestions | |
| vs commitments, requests with and without acceptance, acceptance in a later turn, multiple actions in one turn, shared | |
| and missing owners, non-participant owners, relative and absolute deadlines, corrections and withdrawn commitments, | |
| conditional tasks, decisions with and without actions, options, quoted historical commitments, interruptions and | |
| disfluency, ASR-like errors, similar speaker names, and embedded instructions. | |
| **How:** a deterministic local template generator (`privoid_action/synth.py`, version 1.0.0). It uses no paid API and | |
| no external model; an optional paraphrase pass with the local Granite model rewrites only non-evidence filler turns. | |
| Gold labels are built together with the text, re-validated with the same strict contract as model output, and | |
| quarantined if they fail. Expected deadline normalisation is specified per phrase, independently of the normaliser | |
| (0 mismatches over 4,865 generated examples during authoring). Exact and near-duplicate removal (MinHash, Jaccard ≥ 0.85) | |
| keeps the evaluation copy. Every phenomenon is sampled into a human label-review CSV, and full training requires a | |
| recorded sign-off. | |
| **Disclosure:** synthetic proportion of training data is recorded per run (100% unless human AMI labels are added). | |
| Results on template data (`test_iid`, `test_heldout`) are reported separately and are **never** presented as | |
| real-meeting performance. During authoring, a simple rules baseline already reached action F1 ≈ 0.81 on synthetic dev | |
| (52 examples) but recalled only 1 of 16 summary-linked actions (0.06) in a 30-segment real AMI dev sample. That gap is | |
| why the human-labelled AMI test matters. | |
| ## 5. Sources | |
| * AMI download page: https://groups.inf.ed.ac.uk/ami/download/ | |
| * AMI licence: https://groups.inf.ed.ac.uk/ami/corpus/license.shtml | |
| * AMI annotation overview: https://groups.inf.ed.ac.uk/ami/corpus/annotation.shtml | |
| * AMI partitions: https://groups.inf.ed.ac.uk/ami/corpus/datasets.shtml | |
| * ICSI download: https://groups.inf.ed.ac.uk/ami/icsi/download | |
| * ICSI licence: https://groups.inf.ed.ac.uk/ami/icsi/license.shtml | |
| * QMSum: https://github.com/Yale-LILY/QMSum | |
| * MeetingBank cards: https://huggingface.co/datasets/huuuyeah/MeetingBank_Audio ; Zenodo record https://zenodo.org/records/7983743 | |
| * ELITR Minuting Corpus: https://lindat.mff.cuni.cz/services/catalog/view/http:__hdl_handle_net_11234_1-4692 | |
| * Liu et al. 2023, Meeting Action Item Detection with Regularized Context Modeling: https://arxiv.org/abs/2303.16763 | |
| * MeeQA: https://arxiv.org/abs/2305.08502 ; MISeD: https://arxiv.org/abs/2405.01121 | |