Spaces:
Sleeping
Sleeping
Commit History
-
feat: add canonical demo URLs and update root endpoint response
34a93bb
-
feat(training): outcome-based RLVR reward + clean Colab T4 notebook
Replace heuristic-match (distillation) GRPO reward with terminal $-PnL from the multi-round Issuer + arbitration outcome: - training/outcome_reward.py: compute_outcome_reward() runs the model action at the captured state then rolls the heuristic forward; reward is normalized merchant_net_pnl in [-1, +1]. compute_format_reward() provides dense parse-validity shaping for early training. - tests/test_outcome_reward.py: 6 tests pinning bounds, parse-fail, default state_step, length mismatch, format reward, and reward variance across diverse first-actions. - notebooks/train_merchant_agent.ipynb: full rewrite. - Pin downgrade to transformers 4.51.3 + trl 0.21.0 + peft 0.14.0 + tokenizers 0.21.4 + huggingface-hub 0.26.5 (combo tested against Colab + Kaggle pre-install drift, with hard asserts). - Strip ChoiceConstrainedGRPOTrainer subclass, LogitsProcessor monkey patch, A/B/C/D candidate-choice prompts, choice-SFT warmup. Vanilla GRPOTrainer with reward_funcs=[o…1f49d52 -
feat: Implement wait_for_updates action for handling delayed cases and evidence
- Added support for the "wait_for_updates" action in baseline_runner and benchmark_runner to manage scenarios with no open cases. - Enhanced the ChargebackOpsEnvironment to handle pending issuer reviews and delayed evidence, allowing agents to wait for updates when necessary. - Introduced a new marathon task scenario with wave arrivals, simulating long-horizon backlog management. - Updated tests to validate the behavior of the new wait_for_updates action and its integration with delayed evidence and issuer reviews.2dedffd
-
feat(training): SFT dataset + stall detection in eval rollout
Two permanent fixes for the failure mode that produced eval=0 across all GRPO checkpoints in the prior Colab run: 1. SFT dataset module (training/sft_dataset.py) — heuristic rollouts captured as (prompt, oracle_completion) pairs for supervised pre-training. Two-phase RLHF (SFT then GRPO) is the proven recipe for teaching a base model the JSON schema and per-state action variety before sparse RL reward kicks in. Pure GRPO from a base model collapses to 'always emit select_case' because the model never sees varied action_types in the prompt-only training data. 2. Stall detection in run_episode_with_text_policy — the dominant failure was a checkpoint emitting select_case at every state. The env silently no-ops the duplicate select_case (returns -0.02 reward, advances step_count), so the rollout burns its entire step budget without flipping done. New code: - Hard-coded predicate _predicted_noop catches the dominant case (select_case when a case is al…02a6a9f
-
fix(training): per-action reward scoring vs heuristic oracle
Prior compute_reward fell back to the scripted heuristic on parse-fail, which trained the model that emitting garbage was optimal: the heuristic played the whole episode and earned ~0.96 reward regardless of model output. Group reward variance collapsed to std~=0.005 across 6 wildly different completions, GRPO advantage went to 0, loss collapsed to 0. This rewrites compute_reward as a pure per-action scorer against the heuristic oracle at the dataset's recorded env state: - parse-fail -> 0.0 - action_type not in available_actions at this state -> 0.1 - valid action_type, different than oracle -> 0.4 - right action_type, wrong target (case_id/system) -> 0.7 - exact match on action_type + targets -> 1.0 Variance across the same 6 diverse completions: std=0.39 (70x lift), distinct values {0.0, 0.1, 0.7, 1.0}. GRPO now has real gradient. Adds build_state_action_dataset() to roll the heuristic and capture (state, oracle_action) pairs, so trainin…243aa68 -
feat: enhance completion parsing to handle truncated JSON and `<think>` blocks
71f1fe0 -
feat: add per-family evaluation and plotting for training curves
a79d430
-
feat: tighten EscalationROI, add ambiguous medium case, LLM note judge wrapper
- Add pre_arb_recovery_medium headline case to raise round-2 fire rate - Tighten EscalationROIRubric concede penalty on positive-EV contestable cases - Add disjoint required/helpful evidence in headline + generator templates - Drop dispute_complexity multiplier - Heuristic switches to EV-based escalation (P(win)*amount vs $250 fee) - New LLMNoteJudgeRubric (opt-in via USE_LLM_NOTE_JUDGE) with provider fallback - Pin gemini-2.5-flash across llm_softening + baseline docs - Refresh demo_ui with multi-round panels (issuer rationale, arb ruling, P&L) - Update README architecture, AGENT.md scoring tables, RESULTS.md numbers - 86/86 tests green; headline heuristic 0.8254, escalate_all 0.7713e32a33b -
feat: Add training curve evaluation and plotting utilities with unit tests
8fe3b35 -
Add training notebook and benchmark runner for ChargebackOps
- Introduced `train_merchant_agent.ipynb` for end-to-end GRPO training of the merchant-side chargeback agent, including environment setup, model loading, training prompt dataset creation, and evaluation. - Created `benchmark_runner.py` to implement scripted policies for benchmarking against the trained agent, including heuristic, escalate_all, concede_all, and naive policies. - Updated `issuer_model.py` to refine rationale messages for issuer decisions. - Added unit tests for the benchmark runner and training adapter to ensure policy validity and reward computation accuracy. - Implemented training helpers in `training/__init__.py`, `env_adapter.py`, and `reward_adapter.py` to facilitate interaction with the GRPO trainer.bd00c06 -
feat: Add LLM softening support to IssuerAgent and implement related tests
06abe10 -
feat: Implement multi-round dispute lifecycle with arbitration scoring and related tests
b7aa1f0 -
feat: Implement Issuer agent for multi-round dispute lifecycle
- Added `issuer_model.py` to define the Issuer agent's decision-making process based on evidence strength scores. - Introduced `IssuerDecision` enum for decision types: ACCEPT, REQUEST_MORE_EVIDENCE, and ESCALATE_TO_ARBITRATION. - Created `IssuerReview` dataclass to encapsulate the Issuer's response to representment submissions. - Implemented evidence scoring logic in `evidence_strength_score` function to evaluate attached evidence. - Updated `CaseProgress` in `simulation.py` to track round number and issuer decisions. - Integrated the Issuer agent into `ChargebackOpsEnvironment` to handle case resolutions based on Issuer decisions. - Added unit tests in `test_issuer.py` to validate the Issuer agent's decision-making logic across different scenarios.b105545
-
refactor: build grading on OpenEnv Rubric system
Each of the seven scoring dimensions is now an openenv.core.rubrics.Rubric subclass in evaluation/rubrics.py, composed via WeightedSum inside a per-case CaseRubric and an episode-level ChargebackOpsEpisodeRubric that is wired into ChargebackOpsEnvironment.rubric. The legacy evaluation/grading.py functions remain as thin adapters so every call site (env step, episode store, API, audit tooling, tests) keeps its current contract and outputs are bit-for-bit identical to the previous implementation. Wins: env.rubric.named_rubrics() exposes the full grader tree, every dimension gets register_forward_hook / last_score / state_dict for free, and swapping a dimension (e.g. LLMJudge for the note rubric) is a one-line change. Docs (README, AGENT.md, OPENENV.md) and the grader test are updated to reflect the rubric integration.c8ebaee
-
feat: add Gradio demo UI, realistic metadata, info dict, and metrics
- Add /demo Gradio tab with step-by-step episode runner and task catalog - Add realistic case metadata: transaction IDs, ISO timestamps, MCC codes, merchant names, masked card numbers - Add info dict to observations: deadline warnings, unqueried systems, missing evidence, harmful evidence attached - Add episode-level metrics: evidence coverage, deadline pressure, triage efficiency - Support curriculum reset with difficulty parameter9ae9432 -
feat: add adversarial evidence, nightmare difficulty, and benchmark splits
- Add adversarial evidence templates where titles sound helpful but content is harmful (delivery verification, account review, etc.) - Add nightmare difficulty tier: 5-6 cases, ~2.4 steps/case budget, adversarial templates mandatory - Structure tasks into showcase/holdout/replay benchmark splits - Wire ISO replay cases into list_tasks() when data is available - Difficulty curve: easy 0.96, medium 0.80, hard 0.70, nightmare 0.549e6686d
-
feat: harden grading, expand task catalog, add episode persistence
- Add root inference.py re-export (submission contract requires it) - Expand list_tasks() from 3 to 8 tasks (3 built-in + 5 generated) - Tighten grading: acceptable fallback 0.55->0.35, abandoned contest evidence 0.3->0.15, outcome fallback 0.6->0.4 - Add JSONL file persistence to episode_store - Add /results endpoint for episode history inspection - Remove circular grader-gaming comments from baseline_runner - Update README project layout and commands for new structure - Update AGENT.md file map for new directory paths87c40c2
-
refactor: update all imports for new directory structure
Update import paths in server/, tests/, and root __init__.py to reference modules in their new locations (core/, evaluation/, runners/, scenarios/).95f11da
-
feat: Add parametric case generator for ChargebackOps
- Implemented a case generator in `case_generator.py` to create reproducible chargeback cases from reason-code templates using a seeded RNG. - Introduced various case templates and evidence blueprints to support different chargeback scenarios. - Added a new endpoint `/generate` in `app.py` to generate tasks based on specified difficulty levels and seed values. - Created tests in `test_agent_audit.py` and `test_env.py` to validate the functionality and reproducibility of generated tasks. - Updated assertions in `test_requirements.py` to accommodate changes in ambiguous state limits.a6b0c55
-
test: add audits docs and regression coverage
379f291