README.md

---
  license: apache-2.0
  pipeline_tag: reinforcement-learning
  tags:
    - reinforcement-learning
    - meta-learning
    - pytorch
  ---

  Pretrained weights for the Disco103 meta-network from [Discovering State-of-the-art Reinforcement Learning
  Algorithms](https://doi.org/10.1038/s41586-025-09761-x) (Nature, 2025).

  ## What is this?

  A small LSTM neural network (754,778 parameters) that generates loss targets for RL agents. Instead of hand-crafted
  loss functions like PPO or GRPO, Disco103 observes an agent's rollout — policy logits, rewards, advantages, auxiliary
  predictions — and outputs target distributions the agent should match.

  Meta-trained by DeepMind across 103 complex environments (Atari, ProcGen, DMLab-30). Originally in JAX, this is a
  PyTorch port.

  ## Quick Start

  ```python
  from disco_torch import DiscoTrainer, collect_rollout

  agent = YourAgent(obs_dim=64, num_actions=3).to(device)
  trainer = DiscoTrainer(agent, device=device)  # auto-downloads weights

  env = YourEnv(num_envs=2)
  obs = env.obs()
  lstm_state = agent.init_lstm_state(env.num_envs, device)

  def step_fn(actions):
      rewards, dones = env.step(actions)
      return env.obs(), rewards, dones

  for step in range(1000):
      rollout, obs, lstm_state = collect_rollout(
          agent, step_fn, obs, lstm_state, rollout_len=29, device=device,
      )
      logs = trainer.step(rollout)  # replay buffer, gradient loop, target updates — all handled

  DiscoTrainer encapsulates the full training loop: replay buffer, 32x inner gradient steps, per-element gradient
  clipping, Polyak target network updates, and meta-state management. See
  https://github.com/asystemoffields/disco-torch/blob/main/examples/catch_disco.py for a complete working example that
  reaches 99% catch rate in 1000 steps.

  Advanced: Low-level API

  from disco_torch import DiscoUpdateRule, load_disco103_weights

  rule = DiscoUpdateRule()
  load_disco103_weights(rule)  # auto-downloads from this repo

  # Generate loss targets from a rollout
  meta_out, new_state = rule.unroll_meta_net(
      rollout, agent_params, meta_state, unroll_fn, hyper_params
  )
  loss, logs = rule.agent_loss(rollout, meta_out, hyper_params)

  File

  disco_103.npz — NumPy archive with 42 parameters (754,778 values total), converted from the original JAX checkpoint.

  PyTorch Port

  See https://github.com/asystemoffields/disco-torch for the full PyTorch implementation, examples, and experiment
  results.

  Citation

  @article{oh2025disco,
    title={Discovering State-of-the-art Reinforcement Learning Algorithms},
    author={Oh, Junhyuk and Farquhar, Greg and Kemaev, Iurii and Calian, Dan A. and Hessel, Matteo and Zintgraf, Luisa
  and Singh, Satinder and van Hasselt, Hado and Silver, David},
    journal={Nature},
    volume={648},
    pages={312--319},
    year={2025},
    doi={10.1038/s41586-025-09761-x}
  }

  License

  Apache 2.0 — same as the original https://github.com/google-deepmind/disco_rl.