README.md

metadata

title: Listening and Responsiveness Study
emoji: 🎧
colorFrom: green
colorTo: yellow
sdk: docker
app_port: 7860
pinned: false

Repair-aware modality-revision annotation

This Space runs a randomized human study of whether synchronized audio changes judgments of answer responsiveness in earnings-call Q&A.

Design

• 600 items across 226 calls.
• 400 development and 200 call-disjoint locked-confirmatory items.
• Six tasks per item: three text_only, three text_audio.
• Dynamic task allocation permits more than six annotators while preventing any participant from seeing both conditions for the same item.
• Response-opportunity gate precedes responsiveness judgment.
• Primary measure: continuous responsiveness (0–100).
• Secondary measures: Rasiah category, requested information supplied, confidence, delivery impressions, audio revision, audible event, rationale, response time, complete-audio compliance, and audio-play time.
• Mandatory synthetic practice checks the response-opportunity gate before any real item is assigned; attempts are included as quality-control metadata.

Twenty-three malformed, non-question, social-only, or dangling-continuation items (including two unvalidated reconstructed-repair candidates) were replaced with label- and partition-matched candidates before packaging. Provisional model labels and source audio paths are not included in the Space manifest.

Required Space secrets

• ANNOTATOR_SECRET: stable cookie-signing secret.
• ANNOTATOR_ACCESS_CODE: shared enrollment code supplied only to recruited participants.
• ANNOTATOR_ADMIN_TOKEN: gates export, status, and media upload.
• ANNOTATOR_EXPORT_SALT: pseudonymizes participant IDs in exports.

Persistent storage must be mounted at /data. The database lives at /data/annotator.db; protected MP3 clips live at /data/study_clips.

Study phases

STUDY_PHASE=development serves only the 400 development items. Do not change to confirmatory until instructions, exclusions, and analysis code are frozen. The confirmatory phase serves only the 200 call-disjoint locked items.

Safety and validity

• The public source repository contains no study MP3s.
• Audio is served only to the authenticated participant currently assigned the corresponding text_audio task.
• Text-only participants cannot access the media route for their task.
• Deploying a new manifest is additive and does not delete existing annotations.
• Admin exports use salted participant hashes rather than participant IDs.
• Existing pilot rows remain in the database and export schema.

The study must not open to participants until the research team confirms ethics, source-audio terms, access secrets, the remaining content-review queue, and a successful database backup/migration.