The reader's data
What lab.reader trains, checks and scores the tone reader with, each file at its key on the tone-reader-data
volume. It holds no third-party text:
- 49 files that are ours, as they are: rc1's and rc2's runs (weights, results, each export and 16-bit file), the five r2 teachers' labels, the move pool, the label sets whose lines are all ours, and rc2's support-call and contrast sets: their committee labels and each line's call as it was written.
- 8 stand-ins, in
rebuild/standins/at their files' keys: files that mix our lines with others'. Each mirrors its file line for line in the same JSON style. Our rows are as they are; each value from a third-party source is a hole that says how to make it again from the source at its pin (holesin the manifest). - The 14 open sets are not here: they are rebuilt from their sources' archives.
rebuild/manifest.json holds every file's sha256, the sources' pins, and the sha256 of each archive file the open
sets are read from.
Rebuilding
lab.reader.pipeline.pull copies this folder to the volume. lab.reader.pipeline.fetch downloads each source at
its pin, rebuilds the open sets and the stand-ins' files, and writes a file only if its sha256 is the manifest's;
it also checks the files pull copied. Both run as pipecat-tone's
src/lab/README.md says.
Sources
| Source | At | Licence | Credit |
|---|---|---|---|
| natcs | amazon-science/dstc11-track2-intent-induction@969b95a | CDLA-Permissive-2.0 | NatCS (Gung et al., 2023; Amazon, DSTC11 Track 2), CDLA-Permissive-2.0 |
| taskmaster1 | google-research-datasets/Taskmaster@d92cb6a | CC-BY-4.0 | Taskmaster-1 (Byrne et al., 2019; Google), CC BY 4.0 |
| taskmaster2 | google-research-datasets/Taskmaster@d92cb6a | CC-BY-4.0 | Taskmaster-2 (Byrne et al., 2020; Google), CC BY 4.0 |
| taskmaster3 | google-research-datasets/Taskmaster@d92cb6a | CC-BY-4.0 | Taskmaster-3 (Byrne et al., 2020; Google), CC BY 4.0 |
| abcd | asappresearch/abcd@6b8700c | MIT | ABCD, the Action-Based Conversations Dataset (Chen et al., 2021; ASAPP), MIT |
| multiwoz22 | budzianowski/multiwoz@fe0c8e6 | MIT | MultiWOZ 2.2 (Zang et al., 2020; Budzianowski et al., 2018), MIT |
| star | RasaHQ/STAR@3058975 | MIT | STAR (Mosig et al., 2020; Rasa), MIT |
| ccpem | google-research-datasets/ccpe@2c9cd30 | CC-BY-4.0 | CCPE-M (Radlinski et al., 2019; Google), CC BY 4.0 |
| harpervalley | cricketclub/gridspace-stanford-harper-valley@0bd721e | CC-BY-4.0 | Harper Valley Bank (Wu et al., 2020; Gridspace and Stanford), CC BY 4.0 |
| ami | ami_public_manual_1.6.2.zip | CC-BY-4.0 | AMI Meeting Corpus manual annotations (Carletta et al., 2005; the AMI Consortium), CC BY 4.0 |
| dipco | Zenodo 8122551 | CDLA-Permissive-1.0 | DiPCo, the Dinner Party Corpus (Van Segbroeck et al., 2020; Amazon), CDLA-Permissive-1.0 |
| dynasent_r2 | cgpotts/dynasent@707b620 | CC-BY-4.0 | DynaSent round 2 (Potts et al., 2021), CC BY 4.0 |
| isarcasmeval | iabufarha/iSarcasmEval@dfc708b | MIT | iSarcasmEval, SemEval-2022 Task 6 (Abu Farha et al., 2022), MIT |
| topicalchat_test | alexa/Topical-Chat@7c93922 | CDLA-Sharing-1.0 | Topical-Chat (Gopalakrishnan et al., 2019; Amazon), CDLA-Sharing-1.0; its test splits only, for evaluation |
| goemotions | google-research-datasets/go_emotions@add4922 | Apache-2.0 | GoEmotions (Demszky et al., 2020; Google), Apache 2.0 |
| civil_comments | google/civil_comments@f2970eb | CC0-1.0 | Civil Comments (Borkan et al., 2019; Jigsaw), CC0 1.0 |
| gutenberg | willwade/Gutenberg-dialog-en@f3a0a1d | MIT; the novels are public domain (Project Gutenberg) | The Gutenberg Dialogue Dataset (Csaky and Recski, 2021), English, from public-domain Project Gutenberg books, MIT |
Topical-Chat is share-alike: its test splits are read for evaluation only and never trained on. The rest of this folder is ours.