Spaces:
Sleeping
Sleeping
| title: Listening and Responsiveness Study | |
| emoji: 🎧 | |
| colorFrom: green | |
| colorTo: yellow | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: false | |
| # Repair-aware modality-revision annotation | |
| This Space runs a randomized human study of whether synchronized audio changes | |
| judgments of answer responsiveness in earnings-call Q&A. | |
| ## Design | |
| - 600 items across 226 calls. | |
| - 400 development and 200 call-disjoint locked-confirmatory items. | |
| - Six tasks per item: three `text_only`, three `text_audio`. | |
| - Dynamic task allocation permits more than six annotators while preventing | |
| any participant from seeing both conditions for the same item. | |
| - Response-opportunity gate precedes responsiveness judgment. | |
| - Primary measure: continuous responsiveness (0–100). | |
| - Secondary measures: Rasiah category, requested information supplied, | |
| confidence, delivery impressions, audio revision, audible event, rationale, | |
| response time, complete-audio compliance, and audio-play time. | |
| - Mandatory synthetic practice checks the response-opportunity gate before any | |
| real item is assigned; attempts are included as quality-control metadata. | |
| Twenty-three malformed, non-question, social-only, or dangling-continuation | |
| items (including two unvalidated reconstructed-repair candidates) were replaced | |
| with label- and partition-matched candidates before packaging. Provisional | |
| model labels and source audio paths are not included in the Space manifest. | |
| ## Required Space secrets | |
| - `ANNOTATOR_SECRET`: stable cookie-signing secret. | |
| - `ANNOTATOR_ACCESS_CODE`: shared enrollment code supplied only to recruited | |
| participants. | |
| - `ANNOTATOR_ADMIN_TOKEN`: gates export, status, and media upload. | |
| - `ANNOTATOR_EXPORT_SALT`: pseudonymizes participant IDs in exports. | |
| Persistent storage must be mounted at `/data`. The database lives at | |
| `/data/annotator.db`; protected MP3 clips live at `/data/study_clips`. | |
| ## Study phases | |
| `STUDY_PHASE=development` serves only the 400 development items. Do not change | |
| to `confirmatory` until instructions, exclusions, and analysis code are frozen. | |
| The confirmatory phase serves only the 200 call-disjoint locked items. | |
| ## Safety and validity | |
| - The public source repository contains no study MP3s. | |
| - Audio is served only to the authenticated participant currently assigned the | |
| corresponding `text_audio` task. | |
| - Text-only participants cannot access the media route for their task. | |
| - Deploying a new manifest is additive and does not delete existing annotations. | |
| - Admin exports use salted participant hashes rather than participant IDs. | |
| - Existing pilot rows remain in the database and export schema. | |
| The study must not open to participants until the research team confirms | |
| ethics, source-audio terms, access secrets, the remaining content-review queue, | |
| and a successful database backup/migration. | |