reports/validation_predictions.csv

id,user_request,true_intent,predicted_intent,correct,predicted_probability
NW-AIO-0002,"Map an automation that moves a verified record from the demonstration service portal to the mock policy administration ledger for a mutual insurance provider. It must prevent duplicate work, preserve the original submission, notify the policy servicing supervisor of failures, and require explicit approval before changing cover, beneficiary details or payment instructions.",workflow_automation,workflow_automation,True,0.713526856158
NW-AIO-0004,"Design a controlled workflow for routing learner-support requests at a vocational training provider. When a synthetic learner-support request arriving through the help desk, capture an audit event, assign the next step to the learner support co-ordinator, and pause before making a safeguarding decision or changing an academic outcome. Include retry, expiry and manual override rules.",workflow_automation,workflow_automation,True,0.611165537085
NW-AIO-0006,"Create a repeatable operating sequence for tracking mentor follow-ups that shows the trigger, validation checks, responsible roles, exception route and completion evidence. The sequence must support outreach programme staff without allowing the system to perform giving personal counselling or changing a learner record on its own.",workflow_automation,workflow_automation,True,0.669125551161
NW-AIO-0012,"Create a repeatable operating sequence for reviewing field observations that shows the trigger, validation checks, responsible roles, exception route and completion evidence. The sequence must support monitoring and programme staff without allowing the system to perform publishing precise habitat locations or asserting an unverified impact on its own.",workflow_automation,workflow_automation,True,0.677034273982
NW-AIO-0025,"Process a batch of entirely synthetic claim-intake forms for a regional insurer. Extract incident date, cover type, document checklist and contact preference, return a confidence score for each field, preserve page references, and send unreadable or conflicting values to the claims team leader rather than guessing.",document_processing,document_processing,True,0.553288154772
NW-AIO-0028,"Process a batch of entirely synthetic learner-support request forms for a vocational training provider. Extract course code, request category, urgency and preferred support channel, return a confidence score for each field, preserve page references, and send unreadable or conflicting values to the learner support co-ordinator rather than guessing.",document_processing,document_processing,True,0.545678870199
NW-AIO-0035,"Define a document-ingestion job for synthetic distribution request summaries received through the demonstration request register. Produce structured JSON and a review queue, detect duplicates, redact unnecessary free text, and block downstream use when a required field is uncertain.",document_processing,document_processing,True,0.753265661390
NW-AIO-0040,"Process a batch of entirely synthetic stock count sheets for a co-operative retail network. Extract item category, quantity band, count date and verification status, return a confidence score for each field, preserve page references, and send unreadable or conflicting values to the operations co-ordinator rather than guessing.",document_processing,document_processing,True,0.501356935824
NW-AIO-0052,"Analyse synthetic learner-support cases for a vocational training provider by course, request category, response time and status. Show volumes, median turnaround, ageing distribution and data-completeness warnings, but do not infer causes or rank individuals. Give the learner support co-ordinator the filters and definitions used.",analytics,analytics,True,0.691371873893
NW-AIO-0053,"Build a decision-support analysis from synthetic workshop registration records that compares the current period with the previous one across workshop, location, attendance status and capacity band. Separate observed figures from interpretations, expose small-sample warnings, and route any consequential conclusion to the programme administrator.",analytics,analytics,True,0.700583334944
NW-AIO-0055,"Analyse synthetic production task records for an independent media studio by format, production stage, deadline risk and responsible role. Show volumes, median turnaround, ageing distribution and data-completeness warnings, but do not infer causes or rank individuals. Give the production manager the filters and definitions used.",analytics,analytics,True,0.679777216806
NW-AIO-0063,"Investigate where operational delays appear in synthetic support performance records for service desk staff. Return reproducible aggregates, missing-data rates and outlier records for review; do not use the result to automate changing production access or promising a restoration time.",analytics,analytics,True,0.719453524507
NW-AIO-0083,"Create a retrieval workflow for distribution programme staff covering distribution criteria, data minimisation and referral routes. Rank passages from authorised guidance, quote only the minimum useful excerpt, attach document version and date, and refuse to invent a procedure when no source supports it.",knowledge_retrieval,knowledge_retrieval,True,0.759952551846
NW-AIO-0085,"Answer staff questions about enquiry qualification, conflicts checks and response ownership using only an approved, versioned knowledge base for a small professional-services firm. Return the relevant section titles and source links, state when the material is silent, and refer unresolved cases to the client services manager.",knowledge_retrieval,knowledge_retrieval,True,0.705347277074
NW-AIO-0086,"Create a retrieval workflow for project delivery staff covering change control, evidence standards and approval responsibilities. Rank passages from authorised guidance, quote only the minimum useful excerpt, attach document version and date, and refuse to invent a procedure when no source supports it.",knowledge_retrieval,knowledge_retrieval,True,0.764980733188
NW-AIO-0091,"Answer staff questions about inspection evidence, maintenance ownership and safety escalation using only an approved, versioned knowledge base for a mining equipment support contractor. Return the relevant section titles and source links, state when the material is silent, and refer unresolved cases to the maintenance support supervisor.",knowledge_retrieval,knowledge_retrieval,True,0.692505201553
NW-AIO-0098,"Design an exception path for co-ordinating policy-servicing hand-offs when the requested change lacks an authorised approval or conflicts with the effective date. It should classify severity, avoid duplicate alerts, preserve prior states, set an accountable owner and record how the case was resolved without silently changing source data.",exception_handling,exception_handling,True,0.710398427444
NW-AIO-0104,"Design an exception path for reviewing story pitches when a source cannot be verified or a pitch includes potentially defamatory wording. It should classify severity, avoid duplicate alerts, preserve prior states, set an accountable owner and record how the case was resolved without silently changing source data.",exception_handling,exception_handling,True,0.725576686670
NW-AIO-0108,"Handle this operational failure safely: the observation cannot be corroborated or contains a sensitive habitat location. Quarantine only the affected synthetic record, continue unaffected work, give the monitoring and evaluation lead a diagnostic summary and require a documented decision before resuming any step that could involve publishing precise habitat locations or asserting an unverified impact.",exception_handling,exception_handling,True,0.558617281674
NW-AIO-0120,"Handle this operational failure safely: a claim lacks an authoritative source or could be interpreted as individual medical advice. Quarantine only the affected synthetic record, continue unaffected work, give the health content editor a diagnostic summary and require a documented decision before resuming any step that could involve giving medical advice or publishing an unsupported health claim.",exception_handling,exception_handling,True,0.549695525223
NW-AIO-0122,"Create a human-review gate for co-ordinating policy-servicing hand-offs. The gate must show the triggering condition, source evidence, system confidence and possible consequences, give the policy servicing supervisor approve, return and reject options, and record their reason.",human_escalation,human_escalation,True,0.644035313107
NW-AIO-0125,"Create a human-review gate for managing workshop registrations. The gate must show the triggering condition, source evidence, system confidence and possible consequences, give the programme administrator approve, return and reject options, and record their reason.",human_escalation,human_escalation,True,0.673540453102
NW-AIO-0131,"Create a human-review gate for scheduling community distributions. The gate must show the triggering condition, source evidence, system confidence and possible consequences, give the distribution programme lead approve, return and reject options, and record their reason.",human_escalation,human_escalation,True,0.665498648093
NW-AIO-0133,"Detect when a request at a small professional-services firm could involve accepting contractual terms or promising an outcome. Prepare a neutral case summary with evidence gaps, urgency and the applicable guidance, then escalate it to the client services manager; do not recommend or execute the final decision.",human_escalation,human_escalation,True,0.626079703550
NW-AIO-0146,"Specify a read-only API integration that gives policy servicing staff current status from the demonstration service portal. Validate response types, cache only non-sensitive fields, log correlation identifiers rather than payloads, and alert the policy servicing supervisor when the upstream contract changes.",api_integration,api_integration,True,0.724042453063
NW-AIO-0148,"Synchronise synthetic records from the mock learner portal to the sandbox support register for a vocational training provider. Use an idempotency key, schema validation, least-privilege access, bounded retries and a dead-letter queue; never trigger making a safeguarding decision or changing an academic outcome from an unverified response.",api_integration,api_integration,True,0.681649894175
NW-AIO-0163,"Synchronise synthetic records from the mock inspection register to the sandbox maintenance queue for a mining equipment support contractor. Use an idempotency key, schema validation, least-privilege access, bounded retries and a dead-letter queue; never trigger returning equipment to service or overriding a safety hold from an unverified response.",api_integration,api_integration,True,0.647437538904
NW-AIO-0164,"Specify a read-only API integration that gives logistics control staff current status from the demonstration logistics register. Validate response types, cache only non-sensitive fields, log correlation identifiers rather than payloads, and alert the logistics control supervisor when the upstream contract changes.",api_integration,api_integration,True,0.700198610581
NW-AIO-0173,"Prepare an auditable monthly summary of synthetic workshop registration records for programme administration staff, grouped by workshop, location, attendance status and capacity band. Clearly label synthetic figures, show the extraction time and coverage, link back to source record identifiers, and separate facts from commentary.",reporting,reporting,True,0.657573025613
NW-AIO-0180,"Create a concise management report on reviewing field observations from synthetic conservation activity records. Show progress, overdue work, exception severity and follow-up ownership, but suppress small groups and require the monitoring and evaluation lead to approve any external distribution.",reporting,reporting,True,0.709600755454
NW-AIO-0186,"Create a concise management report on scheduling approved repair work from synthetic workshop job records. Show progress, overdue work, exception severity and follow-up ownership, but suppress small groups and require the workshop manager to approve any external distribution.",reporting,reporting,True,0.700777765564
NW-AIO-0189,"Create a concise management report on co-ordinating competency reviews from synthetic technical training records. Show progress, overdue work, exception severity and follow-up ownership, but suppress small groups and require the technical training manager to approve any external distribution.",reporting,reporting,True,0.690692097221