Reinforcement Learning
Transformers
English
post-training
distillation
agentic-coding
composer-2.5
cursor
kimi-k2
grpo
dapo
diloco
openenv
trl
verl
research
methodology
Instructions to use Codeseys/composer-replication-framework with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Codeseys/composer-replication-framework with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Codeseys/composer-replication-framework", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Wave 3: close the HIGH review findings (kill-switch wiring, HeldoutSplit, EKS entrypoint bug)
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Phase-7 reconciliation of the concurrent review team's findings (research/review-*.json). HIGH: - R1: wire HeldOutGuard into ComposerReplicationTrainer — OPTIONAL + OFF by default (no heldout_guard => byte-identical legacy behavior). When configured, _maybe_update_killswitch folds in-loop reward + injected heldout_eval_fn() + token-mean KL into the guard at the logging cadence; fires a hard (CollapseStopError) or soft (control.should_training_stop) halt. The #2 collapse safeguard now actually fires instead of being dead code. + test_killswitch_integration.py. - R2: build composer_replication/safety/holdout.py — HeldoutSplit disjointness enforcer (id-set + optional content-hash; .split()/.validate()/.assert_disjoint; raises HeldoutOverlapError listing the leak). The un-built second half of C1 that keeps the guard's proxy-real gap signal meaningful. +10 tests. Re-exported from safety/__init__ (resolves the dangling refs). - R3: EKS entrypoint contract bug — replica_e…bd0c358 -
Wave 2: 4 new modules (kill-switch, EKS/SageMaker executors, DockerSandbox) + B4/B7 completion
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Built by the parallel execution team (worktree-isolated), integrated + tested here. New modules (all CPU-testable via mock/lazy-import; optional deps gated): - composer_replication/safety/ — HeldOutGuard run-level collapse kill-switch (held-out-declines-while-reward-rises streak + KL-to-init hard-stop 0.08 + proxy-real hacking-gap; EMA-denoised, latched-fire, CollapseStopError). The documented #2 safeguard for the self-evolving flywheel. 23 tests. - composer_replication/diloco/serverless/eks.py — EKSExecutor satisfying the ServerlessExecutor Protocol via a SINGLE Kubernetes Indexed Job → N rank-ordered ReplicaHandles, gang-cancel (Background propagation), REPLICA_RANK via the downward API, S3 rendezvous (IRSA). 28 tests (mock BatchV1/CoreV1). - composer_replication/diloco/serverless/sagemaker.py — SageMakerExecutor (boto3 create_training_job, same S3 rendezvous, status mapping). +13-test module written during integration (the build agent shipped it test-less). - compos…7a55e1e