src/prompts/scoring.py

"""
LLM prompt templates for probability scoring.

These prompts take the structured match analysis and produce
calibrated probability estimates with reasoning.
"""

PROBABILITY_SCORING_PROMPT = """You are a hiring outcome prediction system. You think like a hiring data scientist,
not a recruiter. You are calibrated to be CONSERVATIVE.

MATCH ANALYSIS:
{match_analysis}

CALIBRATION RULES (YOU MUST FOLLOW THESE):
1. Average candidates score between 35% and 55% overall
2. Only truly exceptional candidates exceed 75%
3. Missing critical skills cap shortlist probability at 40%
4. Missing 2+ critical skills cap shortlist at 25%
5. Compensation mismatch caps offer acceptance at 35%
6. Average tenure under 12 months caps retention at 40%
7. No candidate gets above 92% on any dimension
8. No candidate gets below 5% on any dimension (unless hard disqualified)
9. These are PROBABILITIES of outcomes, not quality scores

BASE RATES (anchor your estimates here):
- Only ~15% of applicants get shortlisted
- Only ~25% of interviewed candidates pass
- ~70% of candidates who receive offers accept
- ~85% of hires stay past 6 months
- Overall P(hire) for a random applicant is ~3%

SCORING FRAMEWORK:

For SHORTLIST probability, weight these:
- Skill coverage (30%): Do they have the must-have skills?
- Experience depth (25%): Do they have enough relevant experience?
- Seniority alignment (20%): Right level for the role?
- Impact evidence (15%): Have they demonstrated results?
- Domain relevance (10%): Industry/domain knowledge?

For OFFER ACCEPTANCE probability, weight these:
- Compensation alignment (30%): Will the comp work?
- Career trajectory fit (25%): Is this a logical next step?
- Company stage fit (20%): Are they drawn to this type of company?
- Location fit (15%): Does the location/remote setup work?
- Role scope appeal (10%): Is the scope interesting for them?

For RETENTION (6-month) probability, weight these:
- Tenure history (25%): Do they tend to stay?
- Growth room (25%): Can they grow in this role?
- Scope alignment (20%): Is the scope right (not too big or small)?
- Company stage fit (15%): Will they thrive in this environment?
- Overqualification risk (15%): Will they get bored?

For OVERALL HIRE probability:
P(hire) = P(shortlist) × P(interview_pass | shortlist) × P(offer_accept | offer)
Where P(interview_pass | shortlist) is estimated from skill depth and impact evidence.

OUTPUT THIS EXACT JSON:

{{
  "shortlist_probability": {{
    "value": number (5-92),
    "component_scores": {{
      "skill_coverage": number (0-100),
      "experience_depth": number (0-100),
      "seniority_alignment": number (0-100),
      "impact_evidence": number (0-100),
      "domain_relevance": number (0-100)
    }},
    "primary_driver": "string explaining main factor",
    "hard_caps_applied": ["list of cap rules triggered, if any"]
  }},
  "interview_pass_estimate": {{
    "value": number (10-80),
    "reasoning": "string"
  }},
  "offer_acceptance_probability": {{
    "value": number (5-92),
    "component_scores": {{
      "compensation_alignment": number (0-100),
      "career_trajectory_fit": number (0-100),
      "company_stage_fit": number (0-100),
      "location_fit": number (0-100),
      "role_scope_appeal": number (0-100)
    }},
    "primary_driver": "string",
    "hard_caps_applied": []
  }},
  "retention_6m_probability": {{
    "value": number (5-92),
    "component_scores": {{
      "tenure_history": number (0-100),
      "growth_room": number (0-100),
      "scope_alignment": number (0-100),
      "company_stage_fit": number (0-100),
      "overqualification_risk": number (0-100)
    }},
    "primary_driver": "string",
    "hard_caps_applied": []
  }},
  "overall_hire_probability": {{
    "value": number (5-92),
    "formula_inputs": {{
      "p_shortlist": number,
      "p_interview_pass": number,
      "p_offer_accept": number
    }},
    "explanation": "string"
  }},
  "confidence_level": "low | medium | high",
  "confidence_reasoning": "string explaining confidence assessment"
}}

IMPORTANT:
- Show your work: the component_scores should be traceable to match_analysis data
- Apply hard caps BEFORE computing final values
- State which caps were triggered
- P(overall) must be mathematically derivable from its components
- Think about what ACTUALLY predicts hiring outcomes, not what looks good on paper
"""

EXPLANATION_PROMPT = """Given the scoring results and match analysis, produce a concise
human-readable explanation of the hiring probability assessment.

SCORING RESULTS:
{scoring_results}

MATCH ANALYSIS:
{match_analysis}

Produce JSON:

{{
  "reasoning_summary": "2-3 sentence summary of the overall assessment",
  "positive_signals": [
    "Each signal as a clear, evidence-backed statement (max 6)"
  ],
  "risk_signals": [
    "Each risk as a clear, evidence-backed statement (max 6)"
  ],
  "missing_signals": [
    "Important information that was unavailable for scoring (max 4)"
  ],
  "recommendation": "strong_pass | pass | borderline | no_pass | strong_no_pass",
  "key_interview_questions": [
    "3-5 specific questions to validate uncertain signals"
  ]
}}

Rules:
- Every signal must reference specific evidence from the data
- Do not mention age, gender, ethnicity, university prestige, or personal characteristics
- Be specific, not generic (bad: "good experience" / good: "4 years leading distributed systems teams of 8+")
- Missing signals should be things that would materially change the assessment
"""