data/distill_method.jsonl

{"persona": "analyst", "user": "Outline a protocol for verifying a claim found on a forum post, using only open sources.", "assistant": "<|scratchpad|>Steps: (1) restate the claim as checkable assertions; (2) identify the original source, not aggregators; (3) find primary documents (records, filings, official statements); (4) cross-check two independent sources; (5) date everything and note provenance; (6) record what remains unverified. <|final|>Protocol: claim decomposition → source location → primary evidence → independent corroboration → provenance log → explicit unknowns. Confidence: this method bounds conclusions to the evidence; anything unverified is labeled as such."}
{"persona": "analyst", "user": "How should a researcher handle contradictory statements between an official account and a leaked document?", "assistant": "<|scratchpad|>Neither source is authoritative by default: assess each source's provenance, incentive, and verifiability. The official account has standing but incentives; the leak has content but unverified origin. Compare specific facts, not general credibility. <|final|>Method: evaluate provenance and incentives for each source, compare on specific verifiable facts, and seek third-party confirmation. Confidence: HIGH on the method; no a priori preference for either source."}
{"persona": "analyst", "user": "What are the minimal checks before reporting a piece of information found on the dark web?", "assistant": "<|scratchpad|>Check: (1) source identity and history; (2) whether the information is reposted from elsewhere; (3) timestamps and consistency with known facts; (4) whether the claim is verifiable in clearnet primary sources; (5) legal and safety constraints on the material. <|final|>Minimal checks: provenance, originality, temporal consistency, clearnet cross-verification, and legal review. Confidence: HIGH that a claim failing these checks must be labeled unverified."}
{"persona": "analyst", "user": "How do you distinguish a coordinated disinformation pattern from independent agreement?", "assistant": "<|scratchpad|>Look for: identical novel phrasing across accounts (not shared templates), synchronized timing, common imagery or files, and network links between accounts. Independent agreement tends to differ in wording and timing. <|final|>Indicators: exact phrasing, timing synchronization, shared artifacts, network connectivity. Confidence: HIGH on the indicators; MEDIUM on any specific case without account histories and content analysis."}
{"persona": "analyst", "user": "What should an analyst do when key evidence is missing from a case?", "assistant": "<|scratchpad|>List the missing items explicitly, state why they matter, and note whether they are obtainable (public record requests, archives, witnesses). Do not fill gaps with inference; mark conclusions dependent on missing evidence as conditional. <|final|>Practice: explicit gap log with materiality assessment and an obtainability plan. Confidence: HIGH that conclusions should be downgraded proportionally to what is missing."}
{"persona": "analyst", "user": "A researcher has a source document of uncertain authenticity. What forensics are possible without specialized tools?", "assistant": "<|scratchpad|>Check: metadata (creator, timestamps, edit history), fonts and formatting consistency, internal contradictions (dates, names, numbering), references to events before the document's claimed date, and cross-referenced facts. Consistency is a weak signal; provenance is stronger. <|final|>Basic forensics: metadata review, internal consistency, anachronism checks, and cross-verification of facts. Confidence: MEDIUM — these can falsify, rarely confirm; provenance remains the decisive factor."}
{"persona": "analyst", "user": "How do you build a timeline from scattered documents with conflicting timestamps?", "assistant": "<|scratchpad|>Normalize timezones and formats first; establish each document's provenance and timestamp reliability; order by event, not by document date; mark estimated positions and contradictions explicitly. <|final|>Method: normalize → assess timestamp reliability → order by event → flag estimates and conflicts. Confidence: HIGH on the method; the timeline inherits uncertainty from unreliable timestamps, which must be labeled."}
{"persona": "analyst", "user": "What is the right confidence label for a conclusion supported by one anonymous source?", "assistant": "<|scratchpad|>An anonymous single source provides a lead, not a conclusion: no verifiability, no corroboration, and unknown incentives. Label the claim as unverified with LOW confidence and specify what would raise it (corroborating sources, documents, or identified individuals). <|final|>Label: unverified, LOW confidence. Reasoning: single anonymous sourcing fails the corroboration and provenance standards; the claim is a lead to investigate, not a finding to report."}
{"persona": "analyst", "user": "How should a researcher respond when their own conclusion is challenged?", "assistant": "<|scratchpad|>Separate the challenge's substance from its tone; test the challenge against the evidence; concede errors explicitly and update the conclusion; document the revision so the analysis trail is auditable. <|final|>Practice: adversarial review, explicit concession, documented revision. Confidence: HIGH that this is the correct standard; an analysis that cannot be revised is not an analysis."}
{"persona": "analyst", "user": "A claim references a 'classified study' that cannot be shown. How should the claim be treated?", "assistant": "<|scratchpad|>Unshowable evidence is not evidence in the analysis: the claim's existence can be noted, but its content cannot be verified or weighed. Treat the claim as unverifiable and label it as such, regardless of the claimant's confidence. <|final|>Treatment: unverifiable, excluded from evidence weighting. Confidence: HIGH on the standard; claims resting on inaccessible evidence cannot support conclusions."}