diff --git "a/Copy_of_unit8_part1.ipynb" "b/Copy_of_unit8_part1.ipynb" new file mode 100644--- /dev/null +++ "b/Copy_of_unit8_part1.ipynb" @@ -0,0 +1,5631 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "-cf5-oDPjwf8" + }, + "source": [ + "# Unit 8: Proximal Policy Gradient (PPO) with PyTorch πŸ€–\n", + "\n", + "\"Unit\n", + "\n", + "\n", + "In this notebook, you'll learn to **code your PPO agent from scratch with PyTorch using CleanRL implementation as model**.\n", + "\n", + "To test its robustness, we're going to train it in:\n", + "\n", + "- [LunarLander-v2 πŸš€](https://www.gymlibrary.dev/environments/box2d/lunar_lander/)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2Fl6Rxt0lc0O" + }, + "source": [ + "⬇️ Here is an example of what you will achieve. ⬇️" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "id": "DbKfCj5ilgqT", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 421 + }, + "outputId": "47d08bd6-53a1-406d-f526-a6d6ab284718" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n" + ] + }, + "metadata": {} + } + ], + "source": [ + "%%html\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YcOFdWpnlxNf" + }, + "source": [ + "We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the GitHub Repo](https://github.com/huggingface/deep-rl-class/issues)." + ] + }, + { + "cell_type": "markdown", + "source": [ + "## Objectives of this notebook πŸ†\n", + "\n", + "At the end of the notebook, you will:\n", + "\n", + "- Be able to **code your PPO agent from scratch using PyTorch**.\n", + "- Be able to **push your trained agent and the code to the Hub** with a nice video replay and an evaluation score πŸ”₯.\n", + "\n", + "\n" + ], + "metadata": { + "id": "T6lIPYFghhYL" + } + }, + { + "cell_type": "markdown", + "source": [ + "## This notebook is from the Deep Reinforcement Learning Course\n", + "\"Deep\n", + "\n", + "In this free course, you will:\n", + "\n", + "- πŸ“– Study Deep Reinforcement Learning in **theory and practice**.\n", + "- πŸ§‘β€πŸ’» Learn to **use famous Deep RL libraries** such as Stable Baselines3, RL Baselines3 Zoo, CleanRL and Sample Factory 2.0.\n", + "- πŸ€– Train **agents in unique environments**\n", + "\n", + "Don’t forget to **sign up to the course** (we are collecting your email to be able toΒ **send you the links when each Unit is published and give you information about the challenges and updates).**\n", + "\n", + "\n", + "The best way to keep in touch is to join our discord server to exchange with the community and with us πŸ‘‰πŸ» https://discord.gg/ydHrjt3WP5" + ], + "metadata": { + "id": "Wp-rD6Fuhq31" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Prerequisites πŸ—οΈ\n", + "Before diving into the notebook, you need to:\n", + "\n", + "πŸ”² πŸ“š Study [PPO by reading Unit 8](https://huggingface.co/deep-rl-course/unit8/introduction) πŸ€— " + ], + "metadata": { + "id": "rasqqGQlhujA" + } + }, + { + "cell_type": "markdown", + "source": [ + "To validate this hands-on for the [certification process](https://huggingface.co/deep-rl-course/en/unit0/introduction#certification-process), you need to push one model, we don't ask for a minimal result but we **advise you to try different hyperparameters settings to get better results**.\n", + "\n", + "If you don't find your model, **go to the bottom of the page and click on the refresh button**\n", + "\n", + "For more information about the certification process, check this section πŸ‘‰ https://huggingface.co/deep-rl-course/en/unit0/introduction#certification-process" + ], + "metadata": { + "id": "PUFfMGOih3CW" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Set the GPU πŸ’ͺ\n", + "- To **accelerate the agent's training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n", + "\n", + "\"GPU" + ], + "metadata": { + "id": "PU4FVzaoM6fC" + } + }, + { + "cell_type": "markdown", + "source": [ + "- `Hardware Accelerator > GPU`\n", + "\n", + "\"GPU" + ], + "metadata": { + "id": "KV0NyFdQM9ZG" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Create a virtual display πŸ”½\n", + "\n", + "During the notebook, we'll need to generate a replay video. To do so, with colab, **we need to have a virtual screen to be able to render the environment** (and thus record the frames).\n", + "\n", + "Hence the following cell will install the librairies and create and run a virtual screen πŸ–₯" + ], + "metadata": { + "id": "bTpYcVZVMzUI" + } + }, + { + "cell_type": "code", + "source": [ + "!pip install setuptools==65.5.0" + ], + "metadata": { + "id": "Fd731S8-NuJA", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "30c4500c-ea95-45b5-bc32-1549d901b76b" + }, + "execution_count": 2, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Requirement already satisfied: setuptools==65.5.0 in /usr/local/lib/python3.11/dist-packages (65.5.0)\n" + ] + } + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "jV6wjQ7Be7p5" + }, + "outputs": [], + "source": [ + "%%capture\n", + "!apt install python-opengl\n", + "!apt install ffmpeg\n", + "!apt install xvfb\n", + "!apt install swig cmake\n", + "!pip install pyglet==1.5\n", + "!pip3 install pyvirtualdisplay" + ] + }, + { + "cell_type": "code", + "source": [ + "# Virtual display\n", + "from pyvirtualdisplay import Display\n", + "\n", + "virtual_display = Display(visible=0, size=(1400, 900))\n", + "virtual_display.start()" + ], + "metadata": { + "id": "ww5PQH1gNLI4", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "1725a243-ef22-41b6-af27-6049a463154a" + }, + "execution_count": 4, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ] + }, + "metadata": {}, + "execution_count": 4 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ncIgfNf3mOtc" + }, + "source": [ + "## Install dependencies πŸ”½\n", + "For this exercise, we use `gym==0.22`." + ] + }, + { + "cell_type": "code", + "source": [], + "metadata": { + "id": "1l_hvNxgf8Ln" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "!pip install gym\n", + "!pip install imageio-ffmpeg\n", + "!pip install huggingface_hub\n", + "!pip install pygame\n", + "!pip install gym[box2d]" + ], + "metadata": { + "id": "9xZQFTPcsKUK", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "22b7dc54-8b1c-461a-beae-e192e70e4a76" + }, + "execution_count": 14, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Requirement already satisfied: gym in /usr/local/lib/python3.11/dist-packages (0.22.0)\n", + "Requirement already satisfied: numpy>=1.18.0 in /usr/local/lib/python3.11/dist-packages (from gym) (2.0.2)\n", + "Requirement already satisfied: cloudpickle>=1.2.0 in /usr/local/lib/python3.11/dist-packages (from gym) (3.1.1)\n", + "Requirement already satisfied: gym_notices>=0.0.4 in /usr/local/lib/python3.11/dist-packages (from gym) (0.0.8)\n", + "Requirement already satisfied: imageio-ffmpeg in /usr/local/lib/python3.11/dist-packages (0.6.0)\n", + "Requirement already satisfied: huggingface_hub in /usr/local/lib/python3.11/dist-packages (0.30.2)\n", + "Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from huggingface_hub) (3.18.0)\n", + "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.11/dist-packages (from huggingface_hub) (2025.3.2)\n", + "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.11/dist-packages (from huggingface_hub) (24.2)\n", + "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.11/dist-packages (from huggingface_hub) (6.0.2)\n", + "Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from huggingface_hub) (2.32.3)\n", + "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.11/dist-packages (from huggingface_hub) (4.67.1)\n", + "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.11/dist-packages (from huggingface_hub) (4.13.2)\n", + "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface_hub) (3.4.1)\n", + "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface_hub) (3.10)\n", + "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface_hub) (2.4.0)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface_hub) (2025.1.31)\n", + "Requirement already satisfied: pygame in /usr/local/lib/python3.11/dist-packages (2.6.1)\n", + "Requirement already satisfied: gym[box2d] in /usr/local/lib/python3.11/dist-packages (0.22.0)\n", + "Requirement already satisfied: numpy>=1.18.0 in /usr/local/lib/python3.11/dist-packages (from gym[box2d]) (2.0.2)\n", + "Requirement already satisfied: cloudpickle>=1.2.0 in /usr/local/lib/python3.11/dist-packages (from gym[box2d]) (3.1.1)\n", + "Requirement already satisfied: gym_notices>=0.0.4 in /usr/local/lib/python3.11/dist-packages (from gym[box2d]) (0.0.8)\n", + "Collecting box2d-py==2.3.5 (from gym[box2d])\n", + " Using cached box2d-py-2.3.5.tar.gz (374 kB)\n", + " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Collecting pygame==2.1.0 (from gym[box2d])\n", + " Using cached pygame-2.1.0.tar.gz (5.8 MB)\n", + " \u001b[1;31merror\u001b[0m: \u001b[1msubprocess-exited-with-error\u001b[0m\n", + " \n", + " \u001b[31mΓ—\u001b[0m \u001b[32mpython setup.py egg_info\u001b[0m did not run successfully.\n", + " \u001b[31mβ”‚\u001b[0m exit code: \u001b[1;36m1\u001b[0m\n", + " \u001b[31m╰─>\u001b[0m See above for output.\n", + " \n", + " \u001b[1;35mnote\u001b[0m: This error originates from a subprocess, and is likely not a problem with pip.\n", + " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25herror\n", + "\u001b[1;31merror\u001b[0m: \u001b[1mmetadata-generation-failed\u001b[0m\n", + "\n", + "\u001b[31mΓ—\u001b[0m Encountered error while generating package metadata.\n", + "\u001b[31m╰─>\u001b[0m See above for output.\n", + "\n", + "\u001b[1;35mnote\u001b[0m: This is an issue with the package mentioned above, not pip.\n", + "\u001b[1;36mhint\u001b[0m: See above for details.\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "oDkUufewmq6v" + }, + "source": [ + "## Let's code PPO from scratch with Costa Huang tutorial\n", + "- For the core implementation of PPO we're going to use the excellent [Costa Huang](https://costa.sh/) tutorial.\n", + "- In addition to the tutorial, to go deeper you can read the 37 core implementation details: https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/\n", + "\n", + "πŸ‘‰ The video tutorial: https://youtu.be/MEt6rrxH8W4" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "id": "aNgEL1_uvhaq", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 371 + }, + "outputId": "506fd98e-1627-447a-cb30-2a3c8a02a8fd" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.11/dist-packages/IPython/core/display.py:724: UserWarning: Consider using IPython.display.IFrame instead\n", + " warnings.warn(\"Consider using IPython.display.IFrame instead\")\n" + ] + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "" + ] + }, + "metadata": {}, + "execution_count": 6 + } + ], + "source": [ + "from IPython.display import HTML\n", + "\n", + "HTML('')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "f34ILn7AvTbt" + }, + "source": [ + "- The best is to code first on the cell below, this way, if you kill the machine **you don't loose the implementation**." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "id": "_bE708C6mhE7", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "42ffaa41-88e8-46ee-e6fc-9a8a71070167" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "\n" + ] + } + ], + "source": [ + "### Your code here:" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "mk-a9CmNuS2W" + }, + "source": [ + "## Add the Hugging Face Integration πŸ€—\n", + "- In order to push our model to the Hub, we need to define a function `package_to_hub`" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "TPi1Nme-oGWd" + }, + "source": [ + "- Add dependencies we need to push our model to the Hub" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": { + "id": "Sj8bz-AmoNVj" + }, + "outputs": [], + "source": [ + "from huggingface_hub import HfApi, upload_folder\n", + "from huggingface_hub.repocard import metadata_eval_result, metadata_save\n", + "\n", + "from pathlib import Path\n", + "import datetime\n", + "import tempfile\n", + "import json\n", + "import shutil\n", + "import imageio\n", + "import gym\n", + "\n", + "from wasabi import Printer\n", + "msg = Printer()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "5rDr8-lWn0zi" + }, + "source": [ + "- Add new argument in `parse_args()` function to define the repo-id where we want to push the model." + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "id": "iHQiqQEFn0QH", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 159 + }, + "outputId": "e7644918-94e6-4429-bd69-bc762468e312" + }, + "outputs": [ + { + "output_type": "error", + "ename": "NameError", + "evalue": "name 'parser' is not defined", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m# Adding HuggingFace argument\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mparser\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd_argument\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"--repo-id\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdefault\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"ThomasSimonini/ppo-CartPole-v1\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhelp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"id of the model repository from the Hugging Face Hub {username/repo_name}\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[0;31mNameError\u001b[0m: name 'parser' is not defined" + ] + } + ], + "source": [ + "# Adding HuggingFace argument\n", + "parser.add_argument(\"--repo-id\", type=str, default=\"ThomasSimonini/ppo-CartPole-v1\", help=\"id of the model repository from the Hugging Face Hub {username/repo_name}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "blLZMiBAoUVT" + }, + "source": [ + "- Next, we add the methods needed to push the model to the Hub\n", + "\n", + "- These methods will:\n", + " - `_evalutate_agent()`: evaluate the agent.\n", + " - `_generate_model_card()`: generate the model card of your agent.\n", + " - `_record_video()`: record a video of your agent." + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": { + "id": "WlLcz4L9odXs" + }, + "outputs": [], + "source": [ + "def package_to_hub(repo_id,\n", + " model,\n", + " hyperparameters,\n", + " eval_env,\n", + " video_fps=30,\n", + " commit_message=\"Push agent to the Hub\",\n", + " token= None,\n", + " logs=None\n", + " ):\n", + " \"\"\"\n", + " Evaluate, Generate a video and Upload a model to Hugging Face Hub.\n", + " This method does the complete pipeline:\n", + " - It evaluates the model\n", + " - It generates the model card\n", + " - It generates a replay video of the agent\n", + " - It pushes everything to the hub\n", + " :param repo_id: id of the model repository from the Hugging Face Hub\n", + " :param model: trained model\n", + " :param eval_env: environment used to evaluate the agent\n", + " :param fps: number of fps for rendering the video\n", + " :param commit_message: commit message\n", + " :param logs: directory on local machine of tensorboard logs you'd like to upload\n", + " \"\"\"\n", + " msg.info(\n", + " \"This function will save, evaluate, generate a video of your agent, \"\n", + " \"create a model card and push everything to the hub. \"\n", + " \"It might take up to 1min. \\n \"\n", + " \"This is a work in progress: if you encounter a bug, please open an issue.\"\n", + " )\n", + " # Step 1: Clone or create the repo\n", + " repo_url = HfApi().create_repo(\n", + " repo_id=repo_id,\n", + " token=token,\n", + " private=False,\n", + " exist_ok=True,\n", + " )\n", + "\n", + " with tempfile.TemporaryDirectory() as tmpdirname:\n", + " tmpdirname = Path(tmpdirname)\n", + "\n", + " # Step 2: Save the model\n", + " torch.save(model.state_dict(), tmpdirname / \"model.pt\")\n", + "\n", + " # Step 3: Evaluate the model and build JSON\n", + " mean_reward, std_reward = _evaluate_agent(eval_env,\n", + " 10,\n", + " model)\n", + "\n", + " # First get datetime\n", + " eval_datetime = datetime.datetime.now()\n", + " eval_form_datetime = eval_datetime.isoformat()\n", + "\n", + " evaluate_data = {\n", + " \"env_id\": hyperparameters.env_id,\n", + " \"mean_reward\": mean_reward,\n", + " \"std_reward\": std_reward,\n", + " \"n_evaluation_episodes\": 10,\n", + " \"eval_datetime\": eval_form_datetime,\n", + " }\n", + "\n", + " # Write a JSON file\n", + " with open(tmpdirname / \"results.json\", \"w\") as outfile:\n", + " json.dump(evaluate_data, outfile)\n", + "\n", + " # Step 4: Generate a video\n", + " video_path = tmpdirname / \"replay.mp4\"\n", + " record_video(eval_env, model, video_path, video_fps)\n", + "\n", + " # Step 5: Generate the model card\n", + " generated_model_card, metadata = _generate_model_card(\"PPO\", hyperparameters.env_id, mean_reward, std_reward, hyperparameters)\n", + " _save_model_card(tmpdirname, generated_model_card, metadata)\n", + "\n", + " # Step 6: Add logs if needed\n", + " if logs:\n", + " _add_logdir(tmpdirname, Path(logs))\n", + "\n", + " msg.info(f\"Pushing repo {repo_id} to the Hugging Face Hub\")\n", + "\n", + " repo_url = upload_folder(\n", + " repo_id=repo_id,\n", + " folder_path=tmpdirname,\n", + " path_in_repo=\"\",\n", + " commit_message=commit_message,\n", + " token=token,\n", + " )\n", + "\n", + " msg.info(f\"Your model is pushed to the Hub. You can view your model here: {repo_url}\")\n", + " return repo_url\n", + "\n", + "\n", + "def _evaluate_agent(env, n_eval_episodes, policy):\n", + " \"\"\"\n", + " Evaluate the agent for ``n_eval_episodes`` episodes and returns average reward and std of reward.\n", + " :param env: The evaluation environment\n", + " :param n_eval_episodes: Number of episode to evaluate the agent\n", + " :param policy: The agent\n", + " \"\"\"\n", + " episode_rewards = []\n", + " for episode in range(n_eval_episodes):\n", + " state = env.reset()\n", + " step = 0\n", + " done = False\n", + " total_rewards_ep = 0\n", + "\n", + " while done is False:\n", + " state = torch.Tensor(state).to(device)\n", + " action, _, _, _ = policy.get_action_and_value(state)\n", + " new_state, reward, done, info = env.step(action.cpu().numpy())\n", + " total_rewards_ep += reward\n", + " if done:\n", + " break\n", + " state = new_state\n", + " episode_rewards.append(total_rewards_ep)\n", + " mean_reward = np.mean(episode_rewards)\n", + " std_reward = np.std(episode_rewards)\n", + "\n", + " return mean_reward, std_reward\n", + "\n", + "\n", + "def record_video(env, policy, out_directory, fps=30):\n", + " images = []\n", + " done = False\n", + " state = env.reset()\n", + " img = env.render(mode='rgb_array')\n", + " images.append(img)\n", + " while not done:\n", + " state = torch.Tensor(state).to(device)\n", + " # Take the action (index) that have the maximum expected future reward given that state\n", + " action, _, _, _ = policy.get_action_and_value(state)\n", + " state, reward, done, info = env.step(action.cpu().numpy()) # We directly put next_state = state for recording logic\n", + " img = env.render(mode='rgb_array')\n", + " images.append(img)\n", + " imageio.mimsave(out_directory, [np.array(img) for i, img in enumerate(images)], fps=fps)\n", + "\n", + "\n", + "def _generate_model_card(model_name, env_id, mean_reward, std_reward, hyperparameters):\n", + " \"\"\"\n", + " Generate the model card for the Hub\n", + " :param model_name: name of the model\n", + " :env_id: name of the environment\n", + " :mean_reward: mean reward of the agent\n", + " :std_reward: standard deviation of the mean reward of the agent\n", + " :hyperparameters: training arguments\n", + " \"\"\"\n", + " # Step 1: Select the tags\n", + " metadata = generate_metadata(model_name, env_id, mean_reward, std_reward)\n", + "\n", + " # Transform the hyperparams namespace to string\n", + " converted_dict = vars(hyperparameters)\n", + " converted_str = str(converted_dict)\n", + " converted_str = converted_str.split(\", \")\n", + " converted_str = '\\n'.join(converted_str)\n", + "\n", + " # Step 2: Generate the model card\n", + " model_card = f\"\"\"\n", + " # PPO Agent Playing {env_id}\n", + "\n", + " This is a trained model of a PPO agent playing {env_id}.\n", + "\n", + " # Hyperparameters\n", + " ```python\n", + " {converted_str}\n", + " ```\n", + " \"\"\"\n", + " return model_card, metadata\n", + "\n", + "\n", + "def generate_metadata(model_name, env_id, mean_reward, std_reward):\n", + " \"\"\"\n", + " Define the tags for the model card\n", + " :param model_name: name of the model\n", + " :param env_id: name of the environment\n", + " :mean_reward: mean reward of the agent\n", + " :std_reward: standard deviation of the mean reward of the agent\n", + " \"\"\"\n", + " metadata = {}\n", + " metadata[\"tags\"] = [\n", + " env_id,\n", + " \"ppo\",\n", + " \"deep-reinforcement-learning\",\n", + " \"reinforcement-learning\",\n", + " \"custom-implementation\",\n", + " \"deep-rl-course\"\n", + " ]\n", + "\n", + " # Add metrics\n", + " eval = metadata_eval_result(\n", + " model_pretty_name=model_name,\n", + " task_pretty_name=\"reinforcement-learning\",\n", + " task_id=\"reinforcement-learning\",\n", + " metrics_pretty_name=\"mean_reward\",\n", + " metrics_id=\"mean_reward\",\n", + " metrics_value=f\"{mean_reward:.2f} +/- {std_reward:.2f}\",\n", + " dataset_pretty_name=env_id,\n", + " dataset_id=env_id,\n", + " )\n", + "\n", + " # Merges both dictionaries\n", + " metadata = {**metadata, **eval}\n", + "\n", + " return metadata\n", + "\n", + "\n", + "def _save_model_card(local_path, generated_model_card, metadata):\n", + " \"\"\"Saves a model card for the repository.\n", + " :param local_path: repository directory\n", + " :param generated_model_card: model card generated by _generate_model_card()\n", + " :param metadata: metadata\n", + " \"\"\"\n", + " readme_path = local_path / \"README.md\"\n", + " readme = \"\"\n", + " if readme_path.exists():\n", + " with readme_path.open(\"r\", encoding=\"utf8\") as f:\n", + " readme = f.read()\n", + " else:\n", + " readme = generated_model_card\n", + "\n", + " with readme_path.open(\"w\", encoding=\"utf-8\") as f:\n", + " f.write(readme)\n", + "\n", + " # Save our metrics to Readme metadata\n", + " metadata_save(readme_path, metadata)\n", + "\n", + "\n", + "def _add_logdir(local_path: Path, logdir: Path):\n", + " \"\"\"Adds a logdir to the repository.\n", + " :param local_path: repository directory\n", + " :param logdir: logdir directory\n", + " \"\"\"\n", + " if logdir.exists() and logdir.is_dir():\n", + " # Add the logdir to the repository under new dir called logs\n", + " repo_logdir = local_path / \"logs\"\n", + "\n", + " # Delete current logs if they exist\n", + " if repo_logdir.exists():\n", + " shutil.rmtree(repo_logdir)\n", + "\n", + " # Copy logdir into repo logdir\n", + " shutil.copytree(logdir, repo_logdir)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "TqX8z8_rooD6" + }, + "source": [ + "- Finally, we call this function at the end of the PPO training" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": { + "id": "I8V1vNiTo2hL", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 356 + }, + "outputId": "3bfc9249-49b2-4817-bc59-334bdf022bb0" + }, + "outputs": [ + { + "output_type": "error", + "ename": "AttributeError", + "evalue": "module 'gym.envs.box2d' has no attribute 'LunarLander'", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m# Create the evaluation environment\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0meval_env\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgym\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmake\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0menv_id\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m package_to_hub(repo_id = args.repo_id,\n\u001b[1;32m 5\u001b[0m \u001b[0mmodel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0magent\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;31m# The model we want to save\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/gym/envs/registration.py\u001b[0m in \u001b[0;36mmake\u001b[0;34m(id, **kwargs)\u001b[0m\n\u001b[1;32m 674\u001b[0m \u001b[0;31m# fmt: on\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 675\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mmake\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mid\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0;34m\"Env\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 676\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mregistry\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmake\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mid\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 677\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 678\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/gym/envs/registration.py\u001b[0m in \u001b[0;36mmake\u001b[0;34m(self, path, **kwargs)\u001b[0m\n\u001b[1;32m 518\u001b[0m \u001b[0mspec\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mspec\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 519\u001b[0m \u001b[0;31m# Construct the environment\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 520\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mspec\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmake\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 521\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 522\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/gym/envs/registration.py\u001b[0m in \u001b[0;36mmake\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m 137\u001b[0m \u001b[0menv\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mentry_point\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0m_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 138\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 139\u001b[0;31m \u001b[0mcls\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mload\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mentry_point\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 140\u001b[0m \u001b[0menv\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcls\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0m_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 141\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/gym/envs/registration.py\u001b[0m in \u001b[0;36mload\u001b[0;34m(name)\u001b[0m\n\u001b[1;32m 54\u001b[0m \u001b[0mmod_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mattr_name\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\":\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 55\u001b[0m \u001b[0mmod\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mimportlib\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mimport_module\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmod_name\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 56\u001b[0;31m \u001b[0mfn\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmod\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mattr_name\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 57\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 58\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mAttributeError\u001b[0m: module 'gym.envs.box2d' has no attribute 'LunarLander'" + ] + } + ], + "source": [ + "# Create the evaluation environment\n", + "eval_env = gym.make(args.env_id)\n", + "\n", + "package_to_hub(repo_id = args.repo_id,\n", + " model = agent, # The model we want to save\n", + " hyperparameters = args,\n", + " eval_env = gym.make(args.env_id),\n", + " logs= f\"runs/{run_name}\",\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "muCCzed4o5TC" + }, + "source": [ + "- Here's what look the ppo.py final file" + ] + }, + { + "cell_type": "code", + "source": [ + "!pip install gym[box2d]\n", + "!pip install box2d-py==2.3.5\n", + "!pip install gymnasium[box2d]\n", + "import gymnasium as gym" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "M__2qnjXkj4f", + "outputId": "584341e5-0eb9-4a0c-bd2d-cfc0f95d849d" + }, + "execution_count": 37, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Requirement already satisfied: gym[box2d] in /usr/local/lib/python3.11/dist-packages (0.22.0)\n", + "Requirement already satisfied: numpy>=1.18.0 in /usr/local/lib/python3.11/dist-packages (from gym[box2d]) (2.0.2)\n", + "Requirement already satisfied: cloudpickle>=1.2.0 in /usr/local/lib/python3.11/dist-packages (from gym[box2d]) (3.1.1)\n", + "Requirement already satisfied: gym_notices>=0.0.4 in /usr/local/lib/python3.11/dist-packages (from gym[box2d]) (0.0.8)\n", + "Collecting box2d-py==2.3.5 (from gym[box2d])\n", + " Using cached box2d-py-2.3.5.tar.gz (374 kB)\n", + " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Collecting pygame==2.1.0 (from gym[box2d])\n", + " Using cached pygame-2.1.0.tar.gz (5.8 MB)\n", + " \u001b[1;31merror\u001b[0m: \u001b[1msubprocess-exited-with-error\u001b[0m\n", + " \n", + " \u001b[31mΓ—\u001b[0m \u001b[32mpython setup.py egg_info\u001b[0m did not run successfully.\n", + " \u001b[31mβ”‚\u001b[0m exit code: \u001b[1;36m1\u001b[0m\n", + " \u001b[31m╰─>\u001b[0m See above for output.\n", + " \n", + " \u001b[1;35mnote\u001b[0m: This error originates from a subprocess, and is likely not a problem with pip.\n", + " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25herror\n", + "\u001b[1;31merror\u001b[0m: \u001b[1mmetadata-generation-failed\u001b[0m\n", + "\n", + "\u001b[31mΓ—\u001b[0m Encountered error while generating package metadata.\n", + "\u001b[31m╰─>\u001b[0m See above for output.\n", + "\n", + "\u001b[1;35mnote\u001b[0m: This is an issue with the package mentioned above, not pip.\n", + "\u001b[1;36mhint\u001b[0m: See above for details.\n", + "Collecting box2d-py==2.3.5\n", + " Using cached box2d-py-2.3.5.tar.gz (374 kB)\n", + " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Building wheels for collected packages: box2d-py\n", + " Building wheel for box2d-py (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for box2d-py: filename=box2d_py-2.3.5-cp311-cp311-linux_x86_64.whl size=2351180 sha256=ca2fda1e15951ec1b12c7bc1023b31f12ecedd604e69d3be70aa8b58d084d0d3\n", + " Stored in directory: /root/.cache/pip/wheels/ab/f1/0c/d56f4a2bdd12bae0a0693ec33f2f0daadb5eb9753c78fa5308\n", + "Successfully built box2d-py\n", + "Installing collected packages: box2d-py\n", + "Successfully installed box2d-py-2.3.5\n", + "Requirement already satisfied: gymnasium[box2d] in /usr/local/lib/python3.11/dist-packages (1.1.1)\n", + "Requirement already satisfied: numpy>=1.21.0 in /usr/local/lib/python3.11/dist-packages (from gymnasium[box2d]) (2.0.2)\n", + "Requirement already satisfied: cloudpickle>=1.2.0 in /usr/local/lib/python3.11/dist-packages (from gymnasium[box2d]) (3.1.1)\n", + "Requirement already satisfied: typing-extensions>=4.3.0 in /usr/local/lib/python3.11/dist-packages (from gymnasium[box2d]) (4.13.2)\n", + "Requirement already satisfied: farama-notifications>=0.0.1 in /usr/local/lib/python3.11/dist-packages (from gymnasium[box2d]) (0.0.4)\n", + "Requirement already satisfied: box2d-py==2.3.5 in /usr/local/lib/python3.11/dist-packages (from gymnasium[box2d]) (2.3.5)\n", + "Requirement already satisfied: pygame>=2.1.3 in /usr/local/lib/python3.11/dist-packages (from gymnasium[box2d]) (2.6.1)\n", + "Collecting swig==4.* (from gymnasium[box2d])\n", + " Downloading swig-4.3.1-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (3.5 kB)\n", + "Downloading swig-4.3.1-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.9 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m37.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hInstalling collected packages: swig\n", + "Successfully installed swig-4.3.1\n" + ] + } + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": { + "id": "LviRdtXgo7kF", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000, + "referenced_widgets": [ + "1bc55520f7224951be56724f2f8c56eb", + "d19388de6acb4619b09d4134c883bc12", + "e97db22db95b4737939291ebdc027542", + "e5d1de9ee5804a79af2de598ac189d23", + "1222af9728444699878d6eff3eadb422", + "da290322509e40a38b773e29203beb08", + "76691d2225a140be84cb039bfa5ddc66", + "e2494ca3eaa94f878910f0fb3294124e", + "ef94b647379e425c91b30a41d319bf81", + "5f043d406d0b4d4e8d70f20142605b9e", + "868e6b0273bc4c0b970fea9d3ad8c6a3", + "17f1ac1f06f145bc8b94eb7c3c110205", + "b73bf3572a0345c19f35575e522b7781", + "a749f78e8bd7444fb45c4c2d0c3ccfd0", + "b1a7db22d14a46f8a645f688cb34f8a5", + "dff438dd40884b3b8eb30355d41ce00d", + "d7e5a99e71a94a68a932354faa110c12", + "8509fab4cc404b159b14e51b04c90b25", + "a3133154031f45df9a0f2acb50c5251e", + "03164def3ed94865b5ba32e4b050322a", + "0e991537a7c0444b83fd37b9eeefbc0c", + "22aba18c4ffb42dfac6ad66f2cbd55aa", + "6c57350f8d0a41f5a46b0dacabb0f644", + "5014d74eaa114bbd8b691521125e981b", + "500437911cc845b4b2a0036cf37ee7cf", + "b0dc84fd87464ec88b3233c2ce93974b", + "946f8a4fcd4a490a900d1cefdd9f89dd", + "ab5415ee8f8940e894e48ad5f0f0b564", + "1af9913925b0426c92769f0c031f20ed", + "58e9c158205f43cb9d6323e3dbffce87", + "66cb3f30d59140508a100b82389f1a2c", + "04e290bcbd7448a1b2df9b0624f01b67", + "d3453f32a0fe4cf6b9518892d6c7b633" + ] + }, + "outputId": "32228287-22d9-4231-c79f-96d12fed853d" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "global_step=264, episodic_return=0.0\n", + "global_step=264, episodic_return=-120.59577606841899\n", + "global_step=264, episodic_return=0.0\n", + "global_step=264, episodic_return=0.0\n", + "global_step=296, episodic_return=0.0\n", + "global_step=296, episodic_return=0.0\n", + "global_step=296, episodic_return=0.0\n", + "global_step=296, episodic_return=-142.44558042494307\n", + "global_step=400, episodic_return=0.0\n", + "global_step=400, episodic_return=0.0\n", + "global_step=400, episodic_return=-272.89851048990147\n", + "global_step=400, episodic_return=0.0\n", + "SPS: 1544\n", + "global_step=568, episodic_return=-243.25644108023786\n", + "global_step=568, episodic_return=0.0\n", + "global_step=568, episodic_return=0.0\n", + "global_step=568, episodic_return=0.0\n", + "global_step=596, episodic_return=0.0\n", + "global_step=596, episodic_return=0.0\n", + "global_step=596, episodic_return=0.0\n", + "global_step=596, episodic_return=-105.19169251121353\n", + "global_step=720, episodic_return=0.0\n", + "global_step=720, episodic_return=0.0\n", + "global_step=720, episodic_return=-117.75900795480652\n", + "global_step=720, episodic_return=0.0\n", + "global_step=768, episodic_return=0.0\n", + "global_step=768, episodic_return=-446.94387602732155\n", + "global_step=768, episodic_return=0.0\n", + "global_step=768, episodic_return=0.0\n", + "global_step=832, episodic_return=-137.4714416185058\n", + "global_step=832, episodic_return=0.0\n", + "global_step=832, episodic_return=0.0\n", + "global_step=832, episodic_return=0.0\n", + "SPS: 1585\n", + "global_step=1080, episodic_return=0.0\n", + "global_step=1080, episodic_return=0.0\n", + "global_step=1080, episodic_return=0.0\n", + "global_step=1080, episodic_return=-422.9722815668329\n", + "global_step=1108, episodic_return=0.0\n", + "global_step=1108, episodic_return=-303.62654406394904\n", + "global_step=1108, episodic_return=0.0\n", + "global_step=1108, episodic_return=0.0\n", + "global_step=1164, episodic_return=0.0\n", + "global_step=1164, episodic_return=0.0\n", + "global_step=1164, episodic_return=-276.7134131326226\n", + "global_step=1164, episodic_return=0.0\n", + "global_step=1236, episodic_return=-385.1534278637485\n", + "global_step=1236, episodic_return=0.0\n", + "global_step=1236, episodic_return=0.0\n", + "global_step=1236, episodic_return=0.0\n", + "global_step=1400, episodic_return=0.0\n", + "global_step=1400, episodic_return=-159.98395358213668\n", + "global_step=1400, episodic_return=0.0\n", + "global_step=1400, episodic_return=0.0\n", + "global_step=1444, episodic_return=0.0\n", + "global_step=1444, episodic_return=0.0\n", + "global_step=1444, episodic_return=0.0\n", + "global_step=1444, episodic_return=-113.13635990067539\n", + "SPS: 1583\n", + "global_step=1588, episodic_return=-100.83826451839066\n", + "global_step=1588, episodic_return=0.0\n", + "global_step=1588, episodic_return=0.0\n", + "global_step=1588, episodic_return=0.0\n", + "global_step=1616, episodic_return=0.0\n", + "global_step=1616, episodic_return=0.0\n", + "global_step=1616, episodic_return=-206.66326590779892\n", + "global_step=1616, episodic_return=0.0\n", + "global_step=1848, episodic_return=0.0\n", + "global_step=1848, episodic_return=0.0\n", + "global_step=1848, episodic_return=0.0\n", + "global_step=1848, episodic_return=-75.66484859220114\n", + "global_step=1956, episodic_return=0.0\n", + "global_step=1956, episodic_return=-514.8964304182357\n", + "global_step=1956, episodic_return=-74.65810589261875\n", + "global_step=1956, episodic_return=0.0\n", + "global_step=1980, episodic_return=-395.26290575607675\n", + "global_step=1980, episodic_return=0.0\n", + "global_step=1980, episodic_return=0.0\n", + "global_step=1980, episodic_return=0.0\n", + "SPS: 1574\n", + "global_step=2072, episodic_return=0.0\n", + "global_step=2072, episodic_return=0.0\n", + "global_step=2072, episodic_return=0.0\n", + "global_step=2072, episodic_return=-82.65847474275112\n", + "global_step=2344, episodic_return=-110.28046679095004\n", + "global_step=2344, episodic_return=0.0\n", + "global_step=2344, episodic_return=0.0\n", + "global_step=2344, episodic_return=0.0\n", + "global_step=2360, episodic_return=0.0\n", + "global_step=2360, episodic_return=-129.32686251761936\n", + "global_step=2360, episodic_return=0.0\n", + "global_step=2360, episodic_return=0.0\n", + "global_step=2432, episodic_return=0.0\n", + "global_step=2432, episodic_return=0.0\n", + "global_step=2432, episodic_return=-62.28844832178548\n", + "global_step=2432, episodic_return=0.0\n", + "global_step=2464, episodic_return=0.0\n", + "global_step=2464, episodic_return=0.0\n", + "global_step=2464, episodic_return=0.0\n", + "global_step=2464, episodic_return=-44.17787427249507\n", + "SPS: 1587\n", + "global_step=2708, episodic_return=0.0\n", + "global_step=2708, episodic_return=0.0\n", + "global_step=2708, episodic_return=-165.33298362205517\n", + "global_step=2708, episodic_return=0.0\n", + "global_step=2764, episodic_return=-120.80270677632771\n", + "global_step=2764, episodic_return=0.0\n", + "global_step=2764, episodic_return=0.0\n", + "global_step=2764, episodic_return=0.0\n", + "global_step=2796, episodic_return=0.0\n", + "global_step=2796, episodic_return=0.0\n", + "global_step=2796, episodic_return=0.0\n", + "global_step=2796, episodic_return=-113.57930158991111\n", + "global_step=2872, episodic_return=0.0\n", + "global_step=2872, episodic_return=-233.41917257280468\n", + "global_step=2872, episodic_return=0.0\n", + "global_step=2872, episodic_return=0.0\n", + "global_step=3044, episodic_return=0.0\n", + "global_step=3044, episodic_return=0.0\n", + "global_step=3044, episodic_return=-91.5171186245687\n", + "global_step=3044, episodic_return=0.0\n", + "SPS: 1588\n", + "global_step=3164, episodic_return=-250.09999683552473\n", + "global_step=3164, episodic_return=0.0\n", + "global_step=3164, episodic_return=0.0\n", + "global_step=3164, episodic_return=0.0\n", + "global_step=3172, episodic_return=0.0\n", + "global_step=3172, episodic_return=0.0\n", + "global_step=3172, episodic_return=0.0\n", + "global_step=3172, episodic_return=-106.38718194805801\n", + "global_step=3240, episodic_return=0.0\n", + "global_step=3240, episodic_return=-97.79544436367601\n", + "global_step=3240, episodic_return=0.0\n", + "global_step=3240, episodic_return=0.0\n", + "global_step=3396, episodic_return=0.0\n", + "global_step=3396, episodic_return=0.0\n", + "global_step=3396, episodic_return=-225.6343207683618\n", + "global_step=3396, episodic_return=0.0\n", + "global_step=3428, episodic_return=-127.21012918614178\n", + "global_step=3428, episodic_return=0.0\n", + "global_step=3428, episodic_return=0.0\n", + "global_step=3428, episodic_return=0.0\n", + "global_step=3544, episodic_return=0.0\n", + "global_step=3544, episodic_return=0.0\n", + "global_step=3544, episodic_return=0.0\n", + "global_step=3544, episodic_return=-126.68497802167201\n", + "SPS: 1510\n", + "global_step=3692, episodic_return=0.0\n", + "global_step=3692, episodic_return=-299.79840806454354\n", + "global_step=3692, episodic_return=0.0\n", + "global_step=3692, episodic_return=0.0\n", + "global_step=3696, episodic_return=0.0\n", + "global_step=3696, episodic_return=0.0\n", + "global_step=3696, episodic_return=16.05301603605467\n", + "global_step=3696, episodic_return=0.0\n", + "global_step=3828, episodic_return=0.0\n", + "global_step=3828, episodic_return=0.0\n", + "global_step=3828, episodic_return=0.0\n", + "global_step=3828, episodic_return=-368.61129896367163\n", + "global_step=3924, episodic_return=-253.34859796449797\n", + "global_step=3924, episodic_return=0.0\n", + "global_step=3924, episodic_return=0.0\n", + "global_step=3924, episodic_return=0.0\n", + "global_step=3940, episodic_return=0.0\n", + "global_step=3940, episodic_return=-74.10404002404859\n", + "global_step=3940, episodic_return=0.0\n", + "global_step=3940, episodic_return=0.0\n", + "global_step=4072, episodic_return=0.0\n", + "global_step=4072, episodic_return=0.0\n", + "global_step=4072, episodic_return=-201.60489772062988\n", + "global_step=4072, episodic_return=0.0\n", + "SPS: 1475\n", + "global_step=4184, episodic_return=0.0\n", + "global_step=4184, episodic_return=-75.26798177824165\n", + "global_step=4184, episodic_return=0.0\n", + "global_step=4184, episodic_return=0.0\n", + "global_step=4276, episodic_return=0.0\n", + "global_step=4276, episodic_return=0.0\n", + "global_step=4276, episodic_return=0.0\n", + "global_step=4276, episodic_return=-115.43323375073096\n", + "global_step=4324, episodic_return=-360.598061182735\n", + "global_step=4324, episodic_return=0.0\n", + "global_step=4324, episodic_return=0.0\n", + "global_step=4324, episodic_return=0.0\n", + "global_step=4464, episodic_return=0.0\n", + "global_step=4464, episodic_return=0.0\n", + "global_step=4464, episodic_return=-172.68997479140637\n", + "global_step=4464, episodic_return=0.0\n", + "global_step=4536, episodic_return=0.0\n", + "global_step=4536, episodic_return=-272.19219885966857\n", + "global_step=4536, episodic_return=0.0\n", + "global_step=4536, episodic_return=0.0\n", + "global_step=4552, episodic_return=0.0\n", + "global_step=4552, episodic_return=0.0\n", + "global_step=4552, episodic_return=0.0\n", + "global_step=4552, episodic_return=-89.57761770227117\n", + "SPS: 1455\n", + "global_step=4736, episodic_return=-158.13986542266952\n", + "global_step=4736, episodic_return=0.0\n", + "global_step=4736, episodic_return=0.0\n", + "global_step=4736, episodic_return=0.0\n", + "global_step=4852, episodic_return=0.0\n", + "global_step=4852, episodic_return=-380.2850360902736\n", + "global_step=4852, episodic_return=0.0\n", + "global_step=4852, episodic_return=0.0\n", + "global_step=4932, episodic_return=0.0\n", + "global_step=4932, episodic_return=0.0\n", + "global_step=4932, episodic_return=-281.56207975445955\n", + "global_step=4932, episodic_return=0.0\n", + "global_step=4984, episodic_return=0.0\n", + "global_step=4984, episodic_return=0.0\n", + "global_step=4984, episodic_return=0.0\n", + "global_step=4984, episodic_return=-141.47273581507866\n", + "SPS: 1414\n", + "global_step=5196, episodic_return=-192.82454920554238\n", + "global_step=5196, episodic_return=0.0\n", + "global_step=5196, episodic_return=0.0\n", + "global_step=5196, episodic_return=0.0\n", + "global_step=5380, episodic_return=0.0\n", + "global_step=5380, episodic_return=0.0\n", + "global_step=5380, episodic_return=0.0\n", + "global_step=5380, episodic_return=-159.84491954272747\n", + "global_step=5384, episodic_return=0.0\n", + "global_step=5384, episodic_return=-117.33367404637231\n", + "global_step=5384, episodic_return=0.0\n", + "global_step=5384, episodic_return=0.0\n", + "global_step=5452, episodic_return=0.0\n", + "global_step=5452, episodic_return=0.0\n", + "global_step=5452, episodic_return=-317.2781243919893\n", + "global_step=5452, episodic_return=0.0\n", + "SPS: 1380\n", + "global_step=5636, episodic_return=-284.53517701125327\n", + "global_step=5636, episodic_return=0.0\n", + "global_step=5636, episodic_return=0.0\n", + "global_step=5636, episodic_return=0.0\n", + "global_step=5668, episodic_return=0.0\n", + "global_step=5668, episodic_return=0.0\n", + "global_step=5668, episodic_return=0.0\n", + "global_step=5668, episodic_return=-119.47246121146613\n", + "global_step=5672, episodic_return=0.0\n", + "global_step=5672, episodic_return=-156.17783100129878\n", + "global_step=5672, episodic_return=0.0\n", + "global_step=5672, episodic_return=0.0\n", + "global_step=5856, episodic_return=0.0\n", + "global_step=5856, episodic_return=0.0\n", + "global_step=5856, episodic_return=-101.67713297530631\n", + "global_step=5856, episodic_return=0.0\n", + "global_step=5948, episodic_return=-94.74972035926143\n", + "global_step=5948, episodic_return=0.0\n", + "global_step=5948, episodic_return=0.0\n", + "global_step=5948, episodic_return=0.0\n", + "global_step=6040, episodic_return=0.0\n", + "global_step=6040, episodic_return=-157.4612016768381\n", + "global_step=6040, episodic_return=0.0\n", + "global_step=6040, episodic_return=0.0\n", + "global_step=6056, episodic_return=0.0\n", + "global_step=6056, episodic_return=0.0\n", + "global_step=6056, episodic_return=0.0\n", + "global_step=6056, episodic_return=-2.6447879404137353\n", + "SPS: 1385\n", + "global_step=6268, episodic_return=0.0\n", + "global_step=6268, episodic_return=0.0\n", + "global_step=6268, episodic_return=-174.21207353204173\n", + "global_step=6268, episodic_return=0.0\n", + "global_step=6356, episodic_return=-394.22703100888907\n", + "global_step=6356, episodic_return=0.0\n", + "global_step=6356, episodic_return=0.0\n", + "global_step=6356, episodic_return=0.0\n", + "global_step=6452, episodic_return=0.0\n", + "global_step=6452, episodic_return=0.0\n", + "global_step=6452, episodic_return=0.0\n", + "global_step=6452, episodic_return=-243.65948916778223\n", + "global_step=6592, episodic_return=0.0\n", + "global_step=6592, episodic_return=-182.33360974915996\n", + "global_step=6592, episodic_return=0.0\n", + "global_step=6592, episodic_return=0.0\n", + "global_step=6648, episodic_return=0.0\n", + "global_step=6648, episodic_return=0.0\n", + "global_step=6648, episodic_return=-147.0634860218737\n", + "global_step=6648, episodic_return=0.0\n", + "SPS: 1400\n", + "global_step=6828, episodic_return=0.0\n", + "global_step=6828, episodic_return=0.0\n", + "global_step=6828, episodic_return=0.0\n", + "global_step=6828, episodic_return=-370.40438730621304\n", + "global_step=6848, episodic_return=-165.9151882794846\n", + "global_step=6848, episodic_return=0.0\n", + "global_step=6848, episodic_return=0.0\n", + "global_step=6848, episodic_return=0.0\n", + "global_step=6884, episodic_return=0.0\n", + "global_step=6884, episodic_return=-63.919365604661394\n", + "global_step=6884, episodic_return=0.0\n", + "global_step=6884, episodic_return=0.0\n", + "global_step=7032, episodic_return=0.0\n", + "global_step=7032, episodic_return=0.0\n", + "global_step=7032, episodic_return=-444.8038042497371\n", + "global_step=7032, episodic_return=0.0\n", + "global_step=7100, episodic_return=-135.18542020119173\n", + "global_step=7100, episodic_return=0.0\n", + "global_step=7100, episodic_return=0.0\n", + "global_step=7100, episodic_return=0.0\n", + "global_step=7140, episodic_return=0.0\n", + "global_step=7140, episodic_return=0.0\n", + "global_step=7140, episodic_return=0.0\n", + "global_step=7140, episodic_return=-121.16320037686047\n", + "SPS: 1416\n", + "global_step=7432, episodic_return=0.0\n", + "global_step=7432, episodic_return=0.0\n", + "global_step=7432, episodic_return=-388.94941415526847\n", + "global_step=7432, episodic_return=0.0\n", + "global_step=7436, episodic_return=0.0\n", + "global_step=7436, episodic_return=-140.36295176657902\n", + "global_step=7436, episodic_return=0.0\n", + "global_step=7436, episodic_return=0.0\n", + "global_step=7476, episodic_return=-178.96979695603883\n", + "global_step=7476, episodic_return=0.0\n", + "global_step=7476, episodic_return=0.0\n", + "global_step=7476, episodic_return=0.0\n", + "global_step=7596, episodic_return=0.0\n", + "global_step=7596, episodic_return=0.0\n", + "global_step=7596, episodic_return=0.0\n", + "global_step=7596, episodic_return=-202.2559847858841\n", + "SPS: 1429\n", + "global_step=7780, episodic_return=-181.1272905763647\n", + "global_step=7780, episodic_return=0.0\n", + "global_step=7780, episodic_return=0.0\n", + "global_step=7780, episodic_return=0.0\n", + "global_step=7848, episodic_return=0.0\n", + "global_step=7848, episodic_return=0.0\n", + "global_step=7848, episodic_return=-156.4082192422158\n", + "global_step=7848, episodic_return=0.0\n", + "global_step=7896, episodic_return=0.0\n", + "global_step=7896, episodic_return=0.0\n", + "global_step=7896, episodic_return=0.0\n", + "global_step=7896, episodic_return=-110.2698546846485\n", + "global_step=7912, episodic_return=0.0\n", + "global_step=7912, episodic_return=-313.92400958007005\n", + "global_step=7912, episodic_return=0.0\n", + "global_step=7912, episodic_return=0.0\n", + "global_step=8172, episodic_return=-103.56891109935187\n", + "global_step=8172, episodic_return=0.0\n", + "global_step=8172, episodic_return=0.0\n", + "global_step=8172, episodic_return=0.0\n", + "SPS: 1438\n", + "global_step=8236, episodic_return=0.0\n", + "global_step=8236, episodic_return=0.0\n", + "global_step=8236, episodic_return=-199.19561164192476\n", + "global_step=8236, episodic_return=0.0\n", + "global_step=8240, episodic_return=0.0\n", + "global_step=8240, episodic_return=-184.84821183622228\n", + "global_step=8240, episodic_return=0.0\n", + "global_step=8240, episodic_return=0.0\n", + "global_step=8316, episodic_return=0.0\n", + "global_step=8316, episodic_return=0.0\n", + "global_step=8316, episodic_return=0.0\n", + "global_step=8316, episodic_return=-129.6803267931211\n", + "global_step=8436, episodic_return=-74.43744280426776\n", + "global_step=8436, episodic_return=0.0\n", + "global_step=8436, episodic_return=0.0\n", + "global_step=8436, episodic_return=0.0\n", + "global_step=8572, episodic_return=0.0\n", + "global_step=8572, episodic_return=0.0\n", + "global_step=8572, episodic_return=-233.1699862394074\n", + "global_step=8572, episodic_return=0.0\n", + "global_step=8604, episodic_return=0.0\n", + "global_step=8604, episodic_return=-127.30669591701518\n", + "global_step=8604, episodic_return=0.0\n", + "global_step=8604, episodic_return=0.0\n", + "global_step=8660, episodic_return=0.0\n", + "global_step=8660, episodic_return=0.0\n", + "global_step=8660, episodic_return=0.0\n", + "global_step=8660, episodic_return=-227.51911370753072\n", + "SPS: 1445\n", + "global_step=8804, episodic_return=-82.36400068398375\n", + "global_step=8804, episodic_return=0.0\n", + "global_step=8804, episodic_return=0.0\n", + "global_step=8804, episodic_return=0.0\n", + "global_step=8896, episodic_return=0.0\n", + "global_step=8896, episodic_return=-240.83568357415353\n", + "global_step=8896, episodic_return=0.0\n", + "global_step=8896, episodic_return=0.0\n", + "global_step=8904, episodic_return=0.0\n", + "global_step=8904, episodic_return=0.0\n", + "global_step=8904, episodic_return=-50.46679475657642\n", + "global_step=8904, episodic_return=0.0\n", + "global_step=8936, episodic_return=0.0\n", + "global_step=8936, episodic_return=0.0\n", + "global_step=8936, episodic_return=0.0\n", + "global_step=8936, episodic_return=-96.39948307958602\n", + "global_step=9108, episodic_return=-123.49802528911147\n", + "global_step=9108, episodic_return=0.0\n", + "global_step=9108, episodic_return=0.0\n", + "global_step=9108, episodic_return=0.0\n", + "global_step=9160, episodic_return=0.0\n", + "global_step=9160, episodic_return=-95.1139561739227\n", + "global_step=9160, episodic_return=0.0\n", + "global_step=9160, episodic_return=0.0\n", + "SPS: 1454\n", + "global_step=9232, episodic_return=0.0\n", + "global_step=9232, episodic_return=0.0\n", + "global_step=9232, episodic_return=-112.45428052316244\n", + "global_step=9232, episodic_return=0.0\n", + "global_step=9248, episodic_return=0.0\n", + "global_step=9248, episodic_return=0.0\n", + "global_step=9248, episodic_return=0.0\n", + "global_step=9248, episodic_return=-146.14212789645293\n", + "global_step=9408, episodic_return=-224.97830240990282\n", + "global_step=9408, episodic_return=0.0\n", + "global_step=9408, episodic_return=0.0\n", + "global_step=9408, episodic_return=0.0\n", + "global_step=9524, episodic_return=0.0\n", + "global_step=9524, episodic_return=0.0\n", + "global_step=9524, episodic_return=-109.2787448331092\n", + "global_step=9524, episodic_return=0.0\n", + "global_step=9596, episodic_return=0.0\n", + "global_step=9596, episodic_return=0.0\n", + "global_step=9596, episodic_return=0.0\n", + "global_step=9596, episodic_return=-84.14375288194863\n", + "global_step=9604, episodic_return=0.0\n", + "global_step=9604, episodic_return=-144.66632141433618\n", + "global_step=9604, episodic_return=0.0\n", + "global_step=9604, episodic_return=0.0\n", + "global_step=9712, episodic_return=-149.96591247024247\n", + "global_step=9712, episodic_return=0.0\n", + "global_step=9712, episodic_return=0.0\n", + "global_step=9712, episodic_return=0.0\n", + "SPS: 1459\n", + "global_step=9948, episodic_return=0.0\n", + "global_step=9948, episodic_return=0.0\n", + "global_step=9948, episodic_return=-131.78972893730634\n", + "global_step=9948, episodic_return=0.0\n", + "global_step=9976, episodic_return=0.0\n", + "global_step=9976, episodic_return=-234.62931138967454\n", + "global_step=9976, episodic_return=0.0\n", + "global_step=9976, episodic_return=0.0\n", + "global_step=10008, episodic_return=0.0\n", + "global_step=10008, episodic_return=0.0\n", + "global_step=10008, episodic_return=0.0\n", + "global_step=10008, episodic_return=-351.05778217569264\n", + "global_step=10080, episodic_return=-211.66272154553207\n", + "global_step=10080, episodic_return=0.0\n", + "global_step=10080, episodic_return=0.0\n", + "global_step=10080, episodic_return=0.0\n", + "SPS: 1465\n", + "global_step=10408, episodic_return=0.0\n", + "global_step=10408, episodic_return=-131.6731332497025\n", + "global_step=10408, episodic_return=0.0\n", + "global_step=10408, episodic_return=0.0\n", + "global_step=10448, episodic_return=0.0\n", + "global_step=10448, episodic_return=0.0\n", + "global_step=10448, episodic_return=-89.74858205824421\n", + "global_step=10448, episodic_return=0.0\n", + "global_step=10456, episodic_return=0.0\n", + "global_step=10456, episodic_return=0.0\n", + "global_step=10456, episodic_return=0.0\n", + "global_step=10456, episodic_return=-165.49158889525944\n", + "global_step=10548, episodic_return=-171.4040441938651\n", + "global_step=10548, episodic_return=0.0\n", + "global_step=10548, episodic_return=0.0\n", + "global_step=10548, episodic_return=0.0\n", + "SPS: 1473\n", + "global_step=10812, episodic_return=0.0\n", + "global_step=10812, episodic_return=-262.820394299633\n", + "global_step=10812, episodic_return=0.0\n", + "global_step=10812, episodic_return=0.0\n", + "global_step=10832, episodic_return=0.0\n", + "global_step=10832, episodic_return=0.0\n", + "global_step=10832, episodic_return=-201.65987340999015\n", + "global_step=10832, episodic_return=0.0\n", + "global_step=10848, episodic_return=0.0\n", + "global_step=10848, episodic_return=0.0\n", + "global_step=10848, episodic_return=0.0\n", + "global_step=10848, episodic_return=1.2972815955988608\n", + "global_step=10896, episodic_return=-317.742328682201\n", + "global_step=10896, episodic_return=0.0\n", + "global_step=10896, episodic_return=0.0\n", + "global_step=10896, episodic_return=0.0\n", + "global_step=11160, episodic_return=0.0\n", + "global_step=11160, episodic_return=0.0\n", + "global_step=11160, episodic_return=-100.3115188419056\n", + "global_step=11160, episodic_return=0.0\n", + "global_step=11168, episodic_return=0.0\n", + "global_step=11168, episodic_return=-116.70650730476918\n", + "global_step=11168, episodic_return=0.0\n", + "global_step=11168, episodic_return=0.0\n", + "global_step=11220, episodic_return=0.0\n", + "global_step=11220, episodic_return=0.0\n", + "global_step=11220, episodic_return=0.0\n", + "global_step=11220, episodic_return=-64.17375904480899\n", + "SPS: 1476\n", + "global_step=11308, episodic_return=-82.51359014798476\n", + "global_step=11308, episodic_return=0.0\n", + "global_step=11308, episodic_return=0.0\n", + "global_step=11308, episodic_return=0.0\n", + "global_step=11500, episodic_return=0.0\n", + "global_step=11500, episodic_return=-228.71957106191184\n", + "global_step=11500, episodic_return=0.0\n", + "global_step=11500, episodic_return=0.0\n", + "global_step=11600, episodic_return=0.0\n", + "global_step=11600, episodic_return=0.0\n", + "global_step=11600, episodic_return=0.0\n", + "global_step=11600, episodic_return=-0.4709329568038072\n", + "global_step=11624, episodic_return=0.0\n", + "global_step=11624, episodic_return=0.0\n", + "global_step=11624, episodic_return=-111.4660514550657\n", + "global_step=11624, episodic_return=0.0\n", + "global_step=11732, episodic_return=-217.64628251241783\n", + "global_step=11732, episodic_return=0.0\n", + "global_step=11732, episodic_return=0.0\n", + "global_step=11732, episodic_return=0.0\n", + "global_step=11776, episodic_return=0.0\n", + "global_step=11776, episodic_return=-163.61985650329365\n", + "global_step=11776, episodic_return=0.0\n", + "global_step=11776, episodic_return=0.0\n", + "SPS: 1481\n", + "global_step=11868, episodic_return=0.0\n", + "global_step=11868, episodic_return=0.0\n", + "global_step=11868, episodic_return=-92.65234097242711\n", + "global_step=11868, episodic_return=0.0\n", + "global_step=12008, episodic_return=0.0\n", + "global_step=12008, episodic_return=0.0\n", + "global_step=12008, episodic_return=0.0\n", + "global_step=12008, episodic_return=-108.97451749048963\n", + "global_step=12072, episodic_return=-88.54588480311826\n", + "global_step=12072, episodic_return=0.0\n", + "global_step=12072, episodic_return=0.0\n", + "global_step=12072, episodic_return=0.0\n", + "global_step=12212, episodic_return=0.0\n", + "global_step=12212, episodic_return=0.0\n", + "global_step=12212, episodic_return=-79.62641043591177\n", + "global_step=12212, episodic_return=0.0\n", + "global_step=12244, episodic_return=0.0\n", + "global_step=12244, episodic_return=0.0\n", + "global_step=12244, episodic_return=0.0\n", + "global_step=12244, episodic_return=-61.73156279542036\n", + "global_step=12280, episodic_return=0.0\n", + "global_step=12280, episodic_return=-372.7246048302052\n", + "global_step=12280, episodic_return=0.0\n", + "global_step=12280, episodic_return=0.0\n", + "SPS: 1488\n", + "global_step=12508, episodic_return=-193.82281827322635\n", + "global_step=12508, episodic_return=0.0\n", + "global_step=12508, episodic_return=0.0\n", + "global_step=12508, episodic_return=0.0\n", + "global_step=12564, episodic_return=0.0\n", + "global_step=12564, episodic_return=0.0\n", + "global_step=12564, episodic_return=0.0\n", + "global_step=12564, episodic_return=-84.57915057541324\n", + "global_step=12660, episodic_return=0.0\n", + "global_step=12660, episodic_return=0.0\n", + "global_step=12660, episodic_return=-188.19006358928908\n", + "global_step=12660, episodic_return=0.0\n", + "global_step=12720, episodic_return=0.0\n", + "global_step=12720, episodic_return=-266.8588979171453\n", + "global_step=12720, episodic_return=0.0\n", + "global_step=12720, episodic_return=0.0\n", + "SPS: 1490\n", + "global_step=12820, episodic_return=0.0\n", + "global_step=12820, episodic_return=0.0\n", + "global_step=12820, episodic_return=0.0\n", + "global_step=12820, episodic_return=-107.64402742342112\n", + "global_step=12828, episodic_return=-117.3277642691033\n", + "global_step=12828, episodic_return=0.0\n", + "global_step=12828, episodic_return=0.0\n", + "global_step=12828, episodic_return=0.0\n", + "global_step=12928, episodic_return=0.0\n", + "global_step=12928, episodic_return=0.0\n", + "global_step=12928, episodic_return=-125.87004999961647\n", + "global_step=12928, episodic_return=0.0\n", + "global_step=13060, episodic_return=0.0\n", + "global_step=13060, episodic_return=-327.2781107318136\n", + "global_step=13060, episodic_return=0.0\n", + "global_step=13060, episodic_return=0.0\n", + "global_step=13192, episodic_return=0.0\n", + "global_step=13192, episodic_return=0.0\n", + "global_step=13192, episodic_return=0.0\n", + "global_step=13192, episodic_return=-250.42132742611437\n", + "global_step=13240, episodic_return=-212.84988008206477\n", + "global_step=13240, episodic_return=0.0\n", + "global_step=13240, episodic_return=0.0\n", + "global_step=13240, episodic_return=0.0\n", + "global_step=13244, episodic_return=0.0\n", + "global_step=13244, episodic_return=0.0\n", + "global_step=13244, episodic_return=-132.5560295585287\n", + "global_step=13244, episodic_return=0.0\n", + "SPS: 1494\n", + "global_step=13496, episodic_return=0.0\n", + "global_step=13496, episodic_return=0.0\n", + "global_step=13496, episodic_return=-103.76001132273157\n", + "global_step=13496, episodic_return=0.0\n", + "global_step=13508, episodic_return=0.0\n", + "global_step=13508, episodic_return=0.0\n", + "global_step=13508, episodic_return=0.0\n", + "global_step=13508, episodic_return=-95.02991901825543\n", + "global_step=13552, episodic_return=0.0\n", + "global_step=13552, episodic_return=-275.70423009711635\n", + "global_step=13552, episodic_return=0.0\n", + "global_step=13552, episodic_return=0.0\n", + "global_step=13556, episodic_return=-102.95719553590986\n", + "global_step=13556, episodic_return=0.0\n", + "global_step=13556, episodic_return=0.0\n", + "global_step=13556, episodic_return=0.0\n", + "SPS: 1499\n", + "global_step=13856, episodic_return=-112.60993178805576\n", + "global_step=13856, episodic_return=0.0\n", + "global_step=13856, episodic_return=0.0\n", + "global_step=13856, episodic_return=-125.17734211351183\n", + "global_step=13868, episodic_return=0.0\n", + "global_step=13868, episodic_return=-117.70093205596383\n", + "global_step=13868, episodic_return=0.0\n", + "global_step=13868, episodic_return=0.0\n", + "global_step=14004, episodic_return=0.0\n", + "global_step=14004, episodic_return=0.0\n", + "global_step=14004, episodic_return=-119.40614758444455\n", + "global_step=14004, episodic_return=0.0\n", + "global_step=14136, episodic_return=0.0\n", + "global_step=14136, episodic_return=0.0\n", + "global_step=14136, episodic_return=0.0\n", + "global_step=14136, episodic_return=-76.85800483979162\n", + "global_step=14164, episodic_return=-321.80018363765544\n", + "global_step=14164, episodic_return=0.0\n", + "global_step=14164, episodic_return=0.0\n", + "global_step=14164, episodic_return=0.0\n", + "SPS: 1500\n", + "global_step=14372, episodic_return=0.0\n", + "global_step=14372, episodic_return=-305.9218686939164\n", + "global_step=14372, episodic_return=0.0\n", + "global_step=14372, episodic_return=0.0\n", + "global_step=14592, episodic_return=0.0\n", + "global_step=14592, episodic_return=0.0\n", + "global_step=14592, episodic_return=-130.40500195547983\n", + "global_step=14592, episodic_return=0.0\n", + "global_step=14620, episodic_return=-184.3281880581295\n", + "global_step=14620, episodic_return=0.0\n", + "global_step=14620, episodic_return=0.0\n", + "global_step=14620, episodic_return=0.0\n", + "global_step=14628, episodic_return=0.0\n", + "global_step=14628, episodic_return=0.0\n", + "global_step=14628, episodic_return=0.0\n", + "global_step=14628, episodic_return=-145.53341298037725\n", + "global_step=14780, episodic_return=0.0\n", + "global_step=14780, episodic_return=-282.2832475503245\n", + "global_step=14780, episodic_return=0.0\n", + "global_step=14780, episodic_return=0.0\n", + "SPS: 1504\n", + "global_step=14968, episodic_return=-155.81718373206854\n", + "global_step=14968, episodic_return=0.0\n", + "global_step=14968, episodic_return=0.0\n", + "global_step=14968, episodic_return=0.0\n", + "global_step=15120, episodic_return=0.0\n", + "global_step=15120, episodic_return=0.0\n", + "global_step=15120, episodic_return=0.0\n", + "global_step=15120, episodic_return=-158.3813296592036\n", + "global_step=15164, episodic_return=0.0\n", + "global_step=15164, episodic_return=0.0\n", + "global_step=15164, episodic_return=-105.976956173705\n", + "global_step=15164, episodic_return=0.0\n", + "global_step=15212, episodic_return=0.0\n", + "global_step=15212, episodic_return=-229.9558635927603\n", + "global_step=15212, episodic_return=0.0\n", + "global_step=15212, episodic_return=0.0\n", + "SPS: 1508\n", + "global_step=15408, episodic_return=-245.69238525633108\n", + "global_step=15408, episodic_return=0.0\n", + "global_step=15408, episodic_return=0.0\n", + "global_step=15408, episodic_return=0.0\n", + "global_step=15464, episodic_return=0.0\n", + "global_step=15464, episodic_return=0.0\n", + "global_step=15464, episodic_return=0.0\n", + "global_step=15464, episodic_return=-149.45386497945356\n", + "global_step=15564, episodic_return=0.0\n", + "global_step=15564, episodic_return=0.0\n", + "global_step=15564, episodic_return=-144.92977027743598\n", + "global_step=15564, episodic_return=0.0\n", + "global_step=15704, episodic_return=0.0\n", + "global_step=15704, episodic_return=-220.8850397081519\n", + "global_step=15704, episodic_return=0.0\n", + "global_step=15704, episodic_return=0.0\n", + "global_step=15856, episodic_return=-150.75940613515678\n", + "global_step=15856, episodic_return=0.0\n", + "global_step=15856, episodic_return=0.0\n", + "global_step=15856, episodic_return=0.0\n", + "SPS: 1512\n", + "global_step=15880, episodic_return=0.0\n", + "global_step=15880, episodic_return=0.0\n", + "global_step=15880, episodic_return=-83.33848936761459\n", + "global_step=15880, episodic_return=0.0\n", + "global_step=15968, episodic_return=0.0\n", + "global_step=15968, episodic_return=0.0\n", + "global_step=15968, episodic_return=0.0\n", + "global_step=15968, episodic_return=-477.0088605828854\n", + "global_step=16156, episodic_return=0.0\n", + "global_step=16156, episodic_return=-159.32268805286031\n", + "global_step=16156, episodic_return=0.0\n", + "global_step=16156, episodic_return=0.0\n", + "global_step=16180, episodic_return=0.0\n", + "global_step=16180, episodic_return=0.0\n", + "global_step=16180, episodic_return=-67.03224958832126\n", + "global_step=16180, episodic_return=0.0\n", + "global_step=16296, episodic_return=-186.6820901623518\n", + "global_step=16296, episodic_return=0.0\n", + "global_step=16296, episodic_return=0.0\n", + "global_step=16296, episodic_return=0.0\n", + "global_step=16356, episodic_return=0.0\n", + "global_step=16356, episodic_return=0.0\n", + "global_step=16356, episodic_return=0.0\n", + "global_step=16356, episodic_return=-190.1293710897196\n", + "SPS: 1513\n", + "global_step=16492, episodic_return=0.0\n", + "global_step=16492, episodic_return=0.0\n", + "global_step=16492, episodic_return=-250.11832669317565\n", + "global_step=16492, episodic_return=0.0\n", + "global_step=16536, episodic_return=0.0\n", + "global_step=16536, episodic_return=-108.8047254857897\n", + "global_step=16536, episodic_return=0.0\n", + "global_step=16536, episodic_return=0.0\n", + "global_step=16600, episodic_return=-66.66774241921598\n", + "global_step=16600, episodic_return=0.0\n", + "global_step=16600, episodic_return=0.0\n", + "global_step=16600, episodic_return=0.0\n", + "global_step=16628, episodic_return=0.0\n", + "global_step=16628, episodic_return=0.0\n", + "global_step=16628, episodic_return=0.0\n", + "global_step=16628, episodic_return=-149.42225268116408\n", + "global_step=16848, episodic_return=0.0\n", + "global_step=16848, episodic_return=-95.25977721575917\n", + "global_step=16848, episodic_return=0.0\n", + "global_step=16848, episodic_return=0.0\n", + "SPS: 1517\n", + "global_step=16964, episodic_return=0.0\n", + "global_step=16964, episodic_return=0.0\n", + "global_step=16964, episodic_return=0.0\n", + "global_step=16964, episodic_return=-131.47511387132988\n", + "global_step=17064, episodic_return=-106.67139734822153\n", + "global_step=17064, episodic_return=0.0\n", + "global_step=17064, episodic_return=-93.48740280999831\n", + "global_step=17064, episodic_return=0.0\n", + "global_step=17268, episodic_return=0.0\n", + "global_step=17268, episodic_return=-5.80295379442633\n", + "global_step=17268, episodic_return=0.0\n", + "global_step=17268, episodic_return=0.0\n", + "global_step=17376, episodic_return=0.0\n", + "global_step=17376, episodic_return=0.0\n", + "global_step=17376, episodic_return=0.0\n", + "global_step=17376, episodic_return=-115.82357112276296\n", + "global_step=17400, episodic_return=-105.5795615602694\n", + "global_step=17400, episodic_return=0.0\n", + "global_step=17400, episodic_return=0.0\n", + "global_step=17400, episodic_return=0.0\n", + "SPS: 1519\n", + "global_step=17540, episodic_return=0.0\n", + "global_step=17540, episodic_return=0.0\n", + "global_step=17540, episodic_return=-227.03742398940767\n", + "global_step=17540, episodic_return=0.0\n", + "global_step=17624, episodic_return=0.0\n", + "global_step=17624, episodic_return=-127.86954731049516\n", + "global_step=17624, episodic_return=0.0\n", + "global_step=17624, episodic_return=0.0\n", + "global_step=17676, episodic_return=-90.32690853371248\n", + "global_step=17676, episodic_return=0.0\n", + "global_step=17676, episodic_return=0.0\n", + "global_step=17676, episodic_return=0.0\n", + "global_step=17884, episodic_return=0.0\n", + "global_step=17884, episodic_return=-100.80894059093855\n", + "global_step=17884, episodic_return=0.0\n", + "global_step=17884, episodic_return=0.0\n", + "global_step=17904, episodic_return=0.0\n", + "global_step=17904, episodic_return=0.0\n", + "global_step=17904, episodic_return=0.0\n", + "global_step=17904, episodic_return=-137.64909179041808\n", + "SPS: 1520\n", + "global_step=18008, episodic_return=0.0\n", + "global_step=18008, episodic_return=0.0\n", + "global_step=18008, episodic_return=-218.020368613073\n", + "global_step=18008, episodic_return=0.0\n", + "global_step=18100, episodic_return=-176.26523260257767\n", + "global_step=18100, episodic_return=0.0\n", + "global_step=18100, episodic_return=0.0\n", + "global_step=18100, episodic_return=0.0\n", + "global_step=18260, episodic_return=0.0\n", + "global_step=18260, episodic_return=-97.9725738907\n", + "global_step=18260, episodic_return=0.0\n", + "global_step=18260, episodic_return=0.0\n", + "SPS: 1523\n", + "global_step=18468, episodic_return=0.0\n", + "global_step=18468, episodic_return=0.0\n", + "global_step=18468, episodic_return=0.0\n", + "global_step=18468, episodic_return=-49.19289779885543\n", + "global_step=18528, episodic_return=0.0\n", + "global_step=18528, episodic_return=-88.39671355863237\n", + "global_step=18528, episodic_return=0.0\n", + "global_step=18528, episodic_return=0.0\n", + "global_step=18576, episodic_return=0.0\n", + "global_step=18576, episodic_return=0.0\n", + "global_step=18576, episodic_return=-139.99008737315353\n", + "global_step=18576, episodic_return=0.0\n", + "global_step=18608, episodic_return=-167.47578937123208\n", + "global_step=18608, episodic_return=0.0\n", + "global_step=18608, episodic_return=0.0\n", + "global_step=18608, episodic_return=0.0\n", + "global_step=18772, episodic_return=0.0\n", + "global_step=18772, episodic_return=0.0\n", + "global_step=18772, episodic_return=0.0\n", + "global_step=18772, episodic_return=-103.46212981579501\n", + "global_step=18836, episodic_return=0.0\n", + "global_step=18836, episodic_return=-96.39579493385206\n", + "global_step=18836, episodic_return=0.0\n", + "global_step=18836, episodic_return=0.0\n", + "global_step=18884, episodic_return=0.0\n", + "global_step=18884, episodic_return=0.0\n", + "global_step=18884, episodic_return=-118.07340979863736\n", + "global_step=18884, episodic_return=0.0\n", + "SPS: 1526\n", + "global_step=18948, episodic_return=-119.95846964630735\n", + "global_step=18948, episodic_return=0.0\n", + "global_step=18948, episodic_return=0.0\n", + "global_step=18948, episodic_return=0.0\n", + "global_step=19028, episodic_return=0.0\n", + "global_step=19028, episodic_return=0.0\n", + "global_step=19028, episodic_return=0.0\n", + "global_step=19028, episodic_return=-92.60292537064349\n", + "global_step=19164, episodic_return=0.0\n", + "global_step=19164, episodic_return=-141.3970894586315\n", + "global_step=19164, episodic_return=0.0\n", + "global_step=19164, episodic_return=0.0\n", + "global_step=19196, episodic_return=0.0\n", + "global_step=19196, episodic_return=0.0\n", + "global_step=19196, episodic_return=-108.30290844581086\n", + "global_step=19196, episodic_return=0.0\n", + "global_step=19240, episodic_return=-100.4055173290944\n", + "global_step=19240, episodic_return=0.0\n", + "global_step=19240, episodic_return=0.0\n", + "global_step=19240, episodic_return=0.0\n", + "global_step=19348, episodic_return=0.0\n", + "global_step=19348, episodic_return=0.0\n", + "global_step=19348, episodic_return=0.0\n", + "global_step=19348, episodic_return=54.398932860393444\n", + "SPS: 1526\n", + "global_step=19488, episodic_return=0.0\n", + "global_step=19488, episodic_return=-149.92714832912418\n", + "global_step=19488, episodic_return=0.0\n", + "global_step=19488, episodic_return=0.0\n", + "global_step=19576, episodic_return=0.0\n", + "global_step=19576, episodic_return=0.0\n", + "global_step=19576, episodic_return=-146.07270595212458\n", + "global_step=19576, episodic_return=0.0\n", + "global_step=19624, episodic_return=0.0\n", + "global_step=19624, episodic_return=0.0\n", + "global_step=19624, episodic_return=0.0\n", + "global_step=19624, episodic_return=-76.39276935455356\n", + "global_step=19756, episodic_return=-152.3121975463605\n", + "global_step=19756, episodic_return=0.0\n", + "global_step=19756, episodic_return=0.0\n", + "global_step=19756, episodic_return=0.0\n", + "global_step=19804, episodic_return=0.0\n", + "global_step=19804, episodic_return=-122.21320939942282\n", + "global_step=19804, episodic_return=0.0\n", + "global_step=19804, episodic_return=0.0\n", + "global_step=19884, episodic_return=0.0\n", + "global_step=19884, episodic_return=0.0\n", + "global_step=19884, episodic_return=-2.38544630333638\n", + "global_step=19884, episodic_return=0.0\n", + "SPS: 1528\n", + "global_step=20040, episodic_return=0.0\n", + "global_step=20040, episodic_return=0.0\n", + "global_step=20040, episodic_return=0.0\n", + "global_step=20040, episodic_return=-250.23002965177818\n", + "global_step=20176, episodic_return=-282.86220576576136\n", + "global_step=20176, episodic_return=0.0\n", + "global_step=20176, episodic_return=0.0\n", + "global_step=20176, episodic_return=0.0\n", + "global_step=20224, episodic_return=0.0\n", + "global_step=20224, episodic_return=-270.58110063739696\n", + "global_step=20224, episodic_return=0.0\n", + "global_step=20224, episodic_return=0.0\n", + "global_step=20264, episodic_return=0.0\n", + "global_step=20264, episodic_return=0.0\n", + "global_step=20264, episodic_return=-86.29211511230301\n", + "global_step=20264, episodic_return=0.0\n", + "global_step=20396, episodic_return=0.0\n", + "global_step=20396, episodic_return=0.0\n", + "global_step=20396, episodic_return=0.0\n", + "global_step=20396, episodic_return=-133.96777906886055\n", + "SPS: 1531\n", + "global_step=20616, episodic_return=-128.82656904517353\n", + "global_step=20616, episodic_return=0.0\n", + "global_step=20616, episodic_return=0.0\n", + "global_step=20616, episodic_return=0.0\n", + "global_step=20632, episodic_return=0.0\n", + "global_step=20632, episodic_return=-204.87470920200587\n", + "global_step=20632, episodic_return=0.0\n", + "global_step=20632, episodic_return=0.0\n", + "global_step=20636, episodic_return=0.0\n", + "global_step=20636, episodic_return=0.0\n", + "global_step=20636, episodic_return=-137.92244912553252\n", + "global_step=20636, episodic_return=0.0\n", + "global_step=20768, episodic_return=0.0\n", + "global_step=20768, episodic_return=0.0\n", + "global_step=20768, episodic_return=0.0\n", + "global_step=20768, episodic_return=-99.50431025579887\n", + "global_step=20924, episodic_return=0.0\n", + "global_step=20924, episodic_return=0.0\n", + "global_step=20924, episodic_return=-87.04807827464691\n", + "global_step=20924, episodic_return=0.0\n", + "global_step=20940, episodic_return=-128.35232544824345\n", + "global_step=20940, episodic_return=0.0\n", + "global_step=20940, episodic_return=0.0\n", + "global_step=20940, episodic_return=0.0\n", + "SPS: 1531\n", + "global_step=21092, episodic_return=0.0\n", + "global_step=21092, episodic_return=-209.68903915930332\n", + "global_step=21092, episodic_return=0.0\n", + "global_step=21092, episodic_return=0.0\n", + "global_step=21136, episodic_return=0.0\n", + "global_step=21136, episodic_return=0.0\n", + "global_step=21136, episodic_return=0.0\n", + "global_step=21136, episodic_return=-81.10173488519084\n", + "global_step=21352, episodic_return=0.0\n", + "global_step=21352, episodic_return=0.0\n", + "global_step=21352, episodic_return=-28.611182830054446\n", + "global_step=21352, episodic_return=0.0\n", + "global_step=21368, episodic_return=-378.31172276699874\n", + "global_step=21368, episodic_return=0.0\n", + "global_step=21368, episodic_return=0.0\n", + "global_step=21368, episodic_return=0.0\n", + "global_step=21460, episodic_return=0.0\n", + "global_step=21460, episodic_return=-179.22419858721776\n", + "global_step=21460, episodic_return=0.0\n", + "global_step=21460, episodic_return=0.0\n", + "global_step=21468, episodic_return=0.0\n", + "global_step=21468, episodic_return=0.0\n", + "global_step=21468, episodic_return=0.0\n", + "global_step=21468, episodic_return=-212.11101438682547\n", + "SPS: 1533\n", + "global_step=21632, episodic_return=0.0\n", + "global_step=21632, episodic_return=0.0\n", + "global_step=21632, episodic_return=-64.43987329959064\n", + "global_step=21632, episodic_return=0.0\n", + "global_step=21748, episodic_return=-232.95674196551616\n", + "global_step=21748, episodic_return=0.0\n", + "global_step=21748, episodic_return=0.0\n", + "global_step=21748, episodic_return=0.0\n", + "global_step=21768, episodic_return=0.0\n", + "global_step=21768, episodic_return=0.0\n", + "global_step=21768, episodic_return=0.0\n", + "global_step=21768, episodic_return=-78.31875335084237\n", + "global_step=21912, episodic_return=0.0\n", + "global_step=21912, episodic_return=0.0\n", + "global_step=21912, episodic_return=-67.29481108464066\n", + "global_step=21912, episodic_return=0.0\n", + "global_step=21976, episodic_return=0.0\n", + "global_step=21976, episodic_return=-128.05148553298378\n", + "global_step=21976, episodic_return=0.0\n", + "global_step=21976, episodic_return=0.0\n", + "SPS: 1528\n", + "global_step=22048, episodic_return=0.0\n", + "global_step=22048, episodic_return=0.0\n", + "global_step=22048, episodic_return=0.0\n", + "global_step=22048, episodic_return=-114.11833197202508\n", + "global_step=22196, episodic_return=-359.4509953895788\n", + "global_step=22196, episodic_return=0.0\n", + "global_step=22196, episodic_return=0.0\n", + "global_step=22196, episodic_return=0.0\n", + "global_step=22260, episodic_return=0.0\n", + "global_step=22260, episodic_return=-48.336536031388206\n", + "global_step=22260, episodic_return=0.0\n", + "global_step=22260, episodic_return=0.0\n", + "global_step=22408, episodic_return=0.0\n", + "global_step=22408, episodic_return=0.0\n", + "global_step=22408, episodic_return=-213.3721975595982\n", + "global_step=22408, episodic_return=0.0\n", + "global_step=22476, episodic_return=0.0\n", + "global_step=22476, episodic_return=0.0\n", + "global_step=22476, episodic_return=0.0\n", + "global_step=22476, episodic_return=-141.88847655602586\n", + "SPS: 1519\n", + "global_step=22580, episodic_return=0.0\n", + "global_step=22580, episodic_return=-302.6635805242829\n", + "global_step=22580, episodic_return=0.0\n", + "global_step=22580, episodic_return=0.0\n", + "global_step=22748, episodic_return=0.0\n", + "global_step=22748, episodic_return=0.0\n", + "global_step=22748, episodic_return=0.0\n", + "global_step=22748, episodic_return=-108.83438766764839\n", + "global_step=22824, episodic_return=-395.0693275775275\n", + "global_step=22824, episodic_return=0.0\n", + "global_step=22824, episodic_return=0.0\n", + "global_step=22824, episodic_return=0.0\n", + "global_step=22828, episodic_return=0.0\n", + "global_step=22828, episodic_return=0.0\n", + "global_step=22828, episodic_return=-268.9780164033224\n", + "global_step=22828, episodic_return=0.0\n", + "global_step=22940, episodic_return=0.0\n", + "global_step=22940, episodic_return=-193.56700414836223\n", + "global_step=22940, episodic_return=0.0\n", + "global_step=22940, episodic_return=0.0\n", + "SPS: 1513\n", + "global_step=23096, episodic_return=0.0\n", + "global_step=23096, episodic_return=0.0\n", + "global_step=23096, episodic_return=0.0\n", + "global_step=23096, episodic_return=-83.9613740223362\n", + "global_step=23160, episodic_return=0.0\n", + "global_step=23160, episodic_return=0.0\n", + "global_step=23160, episodic_return=-257.26409404900744\n", + "global_step=23160, episodic_return=0.0\n", + "global_step=23264, episodic_return=-138.40662275010527\n", + "global_step=23264, episodic_return=0.0\n", + "global_step=23264, episodic_return=0.0\n", + "global_step=23264, episodic_return=0.0\n", + "global_step=23380, episodic_return=0.0\n", + "global_step=23380, episodic_return=-85.26213749052313\n", + "global_step=23380, episodic_return=0.0\n", + "global_step=23380, episodic_return=0.0\n", + "global_step=23516, episodic_return=0.0\n", + "global_step=23516, episodic_return=0.0\n", + "global_step=23516, episodic_return=-129.43057898832495\n", + "global_step=23516, episodic_return=0.0\n", + "SPS: 1506\n", + "global_step=23572, episodic_return=0.0\n", + "global_step=23572, episodic_return=0.0\n", + "global_step=23572, episodic_return=0.0\n", + "global_step=23572, episodic_return=-199.2500877698294\n", + "global_step=23640, episodic_return=-143.0129640248081\n", + "global_step=23640, episodic_return=0.0\n", + "global_step=23640, episodic_return=0.0\n", + "global_step=23640, episodic_return=0.0\n", + "global_step=23744, episodic_return=0.0\n", + "global_step=23744, episodic_return=-237.53230191741204\n", + "global_step=23744, episodic_return=0.0\n", + "global_step=23744, episodic_return=0.0\n", + "global_step=23784, episodic_return=0.0\n", + "global_step=23784, episodic_return=0.0\n", + "global_step=23784, episodic_return=-108.92324305314077\n", + "global_step=23784, episodic_return=0.0\n", + "global_step=23980, episodic_return=0.0\n", + "global_step=23980, episodic_return=0.0\n", + "global_step=23980, episodic_return=0.0\n", + "global_step=23980, episodic_return=-311.89787299888053\n", + "global_step=24000, episodic_return=-123.48900879068347\n", + "global_step=24000, episodic_return=0.0\n", + "global_step=24000, episodic_return=0.0\n", + "global_step=24000, episodic_return=0.0\n", + "global_step=24036, episodic_return=0.0\n", + "global_step=24036, episodic_return=-58.431768665566864\n", + "global_step=24036, episodic_return=0.0\n", + "global_step=24036, episodic_return=0.0\n", + "SPS: 1494\n", + "global_step=24244, episodic_return=0.0\n", + "global_step=24244, episodic_return=0.0\n", + "global_step=24244, episodic_return=-144.45198406792866\n", + "global_step=24244, episodic_return=0.0\n", + "global_step=24300, episodic_return=0.0\n", + "global_step=24300, episodic_return=0.0\n", + "global_step=24300, episodic_return=0.0\n", + "global_step=24300, episodic_return=-88.00449746411478\n", + "global_step=24536, episodic_return=0.0\n", + "global_step=24536, episodic_return=-245.03075796751608\n", + "global_step=24536, episodic_return=0.0\n", + "global_step=24536, episodic_return=0.0\n", + "global_step=24544, episodic_return=-15.187527882012262\n", + "global_step=24544, episodic_return=0.0\n", + "global_step=24544, episodic_return=0.0\n", + "global_step=24544, episodic_return=0.0\n", + "SPS: 1489\n", + "global_step=24728, episodic_return=0.0\n", + "global_step=24728, episodic_return=0.0\n", + "global_step=24728, episodic_return=-275.9882862821758\n", + "global_step=24728, episodic_return=0.0\n", + "global_step=24820, episodic_return=0.0\n", + "global_step=24820, episodic_return=-68.48330770844464\n", + "global_step=24820, episodic_return=0.0\n", + "global_step=24820, episodic_return=0.0\n", + "global_step=24828, episodic_return=0.0\n", + "global_step=24828, episodic_return=0.0\n", + "global_step=24828, episodic_return=0.0\n", + "global_step=24828, episodic_return=-243.65354205254135\n", + "global_step=24892, episodic_return=-58.027840137957455\n", + "global_step=24892, episodic_return=0.0\n", + "global_step=24892, episodic_return=0.0\n", + "global_step=24892, episodic_return=0.0\n", + "global_step=25088, episodic_return=0.0\n", + "global_step=25088, episodic_return=0.0\n", + "global_step=25088, episodic_return=-139.2107802844261\n", + "global_step=25088, episodic_return=0.0\n", + "SPS: 1490\n", + "global_step=25104, episodic_return=0.0\n", + "global_step=25104, episodic_return=0.0\n", + "global_step=25104, episodic_return=0.0\n", + "global_step=25104, episodic_return=-91.82580590146509\n", + "global_step=25180, episodic_return=0.0\n", + "global_step=25180, episodic_return=-99.25246903482808\n", + "global_step=25180, episodic_return=0.0\n", + "global_step=25180, episodic_return=0.0\n", + "global_step=25248, episodic_return=-169.17655104277117\n", + "global_step=25248, episodic_return=0.0\n", + "global_step=25248, episodic_return=0.0\n", + "global_step=25248, episodic_return=0.0\n", + "global_step=25380, episodic_return=0.0\n", + "global_step=25380, episodic_return=0.0\n", + "global_step=25380, episodic_return=-72.86365277340317\n", + "global_step=25380, episodic_return=0.0\n", + "global_step=25492, episodic_return=0.0\n", + "global_step=25492, episodic_return=0.0\n", + "global_step=25492, episodic_return=0.0\n", + "global_step=25492, episodic_return=-112.07887072878836\n", + "SPS: 1493\n", + "global_step=25664, episodic_return=0.0\n", + "global_step=25664, episodic_return=-61.64456701554823\n", + "global_step=25664, episodic_return=0.0\n", + "global_step=25664, episodic_return=0.0\n", + "global_step=25676, episodic_return=-24.223959006635567\n", + "global_step=25676, episodic_return=0.0\n", + "global_step=25676, episodic_return=0.0\n", + "global_step=25676, episodic_return=0.0\n", + "global_step=25732, episodic_return=0.0\n", + "global_step=25732, episodic_return=0.0\n", + "global_step=25732, episodic_return=-147.10147330874057\n", + "global_step=25732, episodic_return=0.0\n", + "global_step=25988, episodic_return=0.0\n", + "global_step=25988, episodic_return=-150.02833508756385\n", + "global_step=25988, episodic_return=0.0\n", + "global_step=25988, episodic_return=0.0\n", + "global_step=26020, episodic_return=-193.56758730207446\n", + "global_step=26020, episodic_return=0.0\n", + "global_step=26020, episodic_return=0.0\n", + "global_step=26020, episodic_return=0.0\n", + "global_step=26108, episodic_return=0.0\n", + "global_step=26108, episodic_return=0.0\n", + "global_step=26108, episodic_return=-269.9491257418164\n", + "global_step=26108, episodic_return=0.0\n", + "SPS: 1495\n", + "global_step=26116, episodic_return=0.0\n", + "global_step=26116, episodic_return=0.0\n", + "global_step=26116, episodic_return=0.0\n", + "global_step=26116, episodic_return=-368.07037772432034\n", + "global_step=26296, episodic_return=-120.68105457014958\n", + "global_step=26296, episodic_return=0.0\n", + "global_step=26296, episodic_return=0.0\n", + "global_step=26296, episodic_return=0.0\n", + "global_step=26380, episodic_return=0.0\n", + "global_step=26380, episodic_return=-108.24855628553271\n", + "global_step=26380, episodic_return=0.0\n", + "global_step=26380, episodic_return=0.0\n", + "global_step=26388, episodic_return=0.0\n", + "global_step=26388, episodic_return=0.0\n", + "global_step=26388, episodic_return=-108.34152462761234\n", + "global_step=26388, episodic_return=0.0\n", + "global_step=26412, episodic_return=0.0\n", + "global_step=26412, episodic_return=0.0\n", + "global_step=26412, episodic_return=0.0\n", + "global_step=26412, episodic_return=-111.52558520049122\n", + "global_step=26596, episodic_return=-173.95208564248142\n", + "global_step=26596, episodic_return=0.0\n", + "global_step=26596, episodic_return=0.0\n", + "global_step=26596, episodic_return=0.0\n", + "SPS: 1497\n", + "global_step=26684, episodic_return=0.0\n", + "global_step=26684, episodic_return=0.0\n", + "global_step=26684, episodic_return=0.0\n", + "global_step=26684, episodic_return=-93.61357373864672\n", + "global_step=26688, episodic_return=0.0\n", + "global_step=26688, episodic_return=0.0\n", + "global_step=26688, episodic_return=-105.2537863132624\n", + "global_step=26688, episodic_return=0.0\n", + "global_step=26848, episodic_return=0.0\n", + "global_step=26848, episodic_return=-75.82087537582191\n", + "global_step=26848, episodic_return=0.0\n", + "global_step=26848, episodic_return=0.0\n", + "global_step=26980, episodic_return=0.0\n", + "global_step=26980, episodic_return=0.0\n", + "global_step=26980, episodic_return=-160.8949380123361\n", + "global_step=26980, episodic_return=0.0\n", + "global_step=26984, episodic_return=-283.54648934216937\n", + "global_step=26984, episodic_return=0.0\n", + "global_step=26984, episodic_return=0.0\n", + "global_step=26984, episodic_return=0.0\n", + "global_step=27080, episodic_return=0.0\n", + "global_step=27080, episodic_return=0.0\n", + "global_step=27080, episodic_return=0.0\n", + "global_step=27080, episodic_return=18.005768107875568\n", + "global_step=27096, episodic_return=0.0\n", + "global_step=27096, episodic_return=-113.18755855481527\n", + "global_step=27096, episodic_return=0.0\n", + "global_step=27096, episodic_return=0.0\n", + "SPS: 1498\n", + "global_step=27268, episodic_return=0.0\n", + "global_step=27268, episodic_return=0.0\n", + "global_step=27268, episodic_return=-84.97122220614892\n", + "global_step=27268, episodic_return=0.0\n", + "global_step=27336, episodic_return=-220.81561682450035\n", + "global_step=27336, episodic_return=0.0\n", + "global_step=27336, episodic_return=0.0\n", + "global_step=27336, episodic_return=0.0\n", + "global_step=27420, episodic_return=0.0\n", + "global_step=27420, episodic_return=-331.0230134031756\n", + "global_step=27420, episodic_return=0.0\n", + "global_step=27420, episodic_return=0.0\n", + "global_step=27544, episodic_return=0.0\n", + "global_step=27544, episodic_return=0.0\n", + "global_step=27544, episodic_return=0.0\n", + "global_step=27544, episodic_return=-189.62645127590758\n", + "SPS: 1500\n", + "global_step=27672, episodic_return=-318.84696912805424\n", + "global_step=27672, episodic_return=0.0\n", + "global_step=27672, episodic_return=0.0\n", + "global_step=27672, episodic_return=0.0\n", + "global_step=27676, episodic_return=0.0\n", + "global_step=27676, episodic_return=-137.8813772859641\n", + "global_step=27676, episodic_return=0.0\n", + "global_step=27676, episodic_return=0.0\n", + "global_step=27820, episodic_return=0.0\n", + "global_step=27820, episodic_return=0.0\n", + "global_step=27820, episodic_return=-94.47401998068418\n", + "global_step=27820, episodic_return=0.0\n", + "global_step=27824, episodic_return=0.0\n", + "global_step=27824, episodic_return=0.0\n", + "global_step=27824, episodic_return=0.0\n", + "global_step=27824, episodic_return=-88.78026451846794\n", + "global_step=27948, episodic_return=-88.63825887518074\n", + "global_step=27948, episodic_return=0.0\n", + "global_step=27948, episodic_return=0.0\n", + "global_step=27948, episodic_return=0.0\n", + "global_step=27960, episodic_return=0.0\n", + "global_step=27960, episodic_return=-77.61834227758251\n", + "global_step=27960, episodic_return=0.0\n", + "global_step=27960, episodic_return=0.0\n", + "SPS: 1502\n", + "global_step=28304, episodic_return=0.0\n", + "global_step=28304, episodic_return=0.0\n", + "global_step=28304, episodic_return=0.0\n", + "global_step=28304, episodic_return=-218.5092455834062\n", + "global_step=28308, episodic_return=-222.56537248787726\n", + "global_step=28308, episodic_return=0.0\n", + "global_step=28308, episodic_return=0.0\n", + "global_step=28308, episodic_return=0.0\n", + "global_step=28368, episodic_return=0.0\n", + "global_step=28368, episodic_return=0.0\n", + "global_step=28368, episodic_return=-228.60600067770983\n", + "global_step=28368, episodic_return=0.0\n", + "global_step=28392, episodic_return=0.0\n", + "global_step=28392, episodic_return=-150.79180157054495\n", + "global_step=28392, episodic_return=0.0\n", + "global_step=28392, episodic_return=0.0\n", + "global_step=28560, episodic_return=0.0\n", + "global_step=28560, episodic_return=0.0\n", + "global_step=28560, episodic_return=0.0\n", + "global_step=28560, episodic_return=-126.93862437968569\n", + "global_step=28624, episodic_return=-90.25179179235042\n", + "global_step=28624, episodic_return=0.0\n", + "global_step=28624, episodic_return=0.0\n", + "global_step=28624, episodic_return=0.0\n", + "SPS: 1503\n", + "global_step=28716, episodic_return=0.0\n", + "global_step=28716, episodic_return=-177.03344244133933\n", + "global_step=28716, episodic_return=0.0\n", + "global_step=28716, episodic_return=0.0\n", + "global_step=28776, episodic_return=0.0\n", + "global_step=28776, episodic_return=0.0\n", + "global_step=28776, episodic_return=-125.78422880843736\n", + "global_step=28776, episodic_return=0.0\n", + "global_step=28980, episodic_return=0.0\n", + "global_step=28980, episodic_return=-112.23208015063497\n", + "global_step=28980, episodic_return=0.0\n", + "global_step=28980, episodic_return=0.0\n", + "global_step=29040, episodic_return=-204.72568064052416\n", + "global_step=29040, episodic_return=0.0\n", + "global_step=29040, episodic_return=0.0\n", + "global_step=29040, episodic_return=0.0\n", + "global_step=29136, episodic_return=0.0\n", + "global_step=29136, episodic_return=0.0\n", + "global_step=29136, episodic_return=0.0\n", + "global_step=29136, episodic_return=-176.74921754031647\n", + "SPS: 1506\n", + "global_step=29204, episodic_return=0.0\n", + "global_step=29204, episodic_return=0.0\n", + "global_step=29204, episodic_return=-456.3578470509466\n", + "global_step=29204, episodic_return=0.0\n", + "global_step=29312, episodic_return=-133.71933663238252\n", + "global_step=29312, episodic_return=0.0\n", + "global_step=29312, episodic_return=0.0\n", + "global_step=29312, episodic_return=0.0\n", + "global_step=29404, episodic_return=0.0\n", + "global_step=29404, episodic_return=-166.95670615276543\n", + "global_step=29404, episodic_return=0.0\n", + "global_step=29404, episodic_return=0.0\n", + "global_step=29612, episodic_return=0.0\n", + "global_step=29612, episodic_return=0.0\n", + "global_step=29612, episodic_return=-139.93598223482667\n", + "global_step=29612, episodic_return=0.0\n", + "global_step=29652, episodic_return=0.0\n", + "global_step=29652, episodic_return=0.0\n", + "global_step=29652, episodic_return=0.0\n", + "global_step=29652, episodic_return=-90.5319935650166\n", + "SPS: 1507\n", + "global_step=29768, episodic_return=-115.56377322172202\n", + "global_step=29768, episodic_return=0.0\n", + "global_step=29768, episodic_return=0.0\n", + "global_step=29768, episodic_return=0.0\n", + "global_step=29816, episodic_return=0.0\n", + "global_step=29816, episodic_return=-139.74007206244752\n", + "global_step=29816, episodic_return=0.0\n", + "global_step=29816, episodic_return=0.0\n", + "global_step=29940, episodic_return=0.0\n", + "global_step=29940, episodic_return=0.0\n", + "global_step=29940, episodic_return=0.0\n", + "global_step=29940, episodic_return=-80.20639864962344\n", + "global_step=30076, episodic_return=0.0\n", + "global_step=30076, episodic_return=0.0\n", + "global_step=30076, episodic_return=-264.92643778778256\n", + "global_step=30076, episodic_return=0.0\n", + "global_step=30168, episodic_return=-287.6842438701519\n", + "global_step=30168, episodic_return=0.0\n", + "global_step=30168, episodic_return=0.0\n", + "global_step=30168, episodic_return=0.0\n", + "global_step=30196, episodic_return=0.0\n", + "global_step=30196, episodic_return=-372.9392072315559\n", + "global_step=30196, episodic_return=0.0\n", + "global_step=30196, episodic_return=0.0\n", + "SPS: 1508\n", + "global_step=30404, episodic_return=0.0\n", + "global_step=30404, episodic_return=0.0\n", + "global_step=30404, episodic_return=-104.37868351078741\n", + "global_step=30404, episodic_return=0.0\n", + "global_step=30444, episodic_return=0.0\n", + "global_step=30444, episodic_return=0.0\n", + "global_step=30444, episodic_return=0.0\n", + "global_step=30444, episodic_return=-230.46948994544988\n", + "global_step=30468, episodic_return=0.0\n", + "global_step=30468, episodic_return=-97.07808115134836\n", + "global_step=30468, episodic_return=0.0\n", + "global_step=30468, episodic_return=0.0\n", + "global_step=30520, episodic_return=-130.51799220045734\n", + "global_step=30520, episodic_return=0.0\n", + "global_step=30520, episodic_return=0.0\n", + "global_step=30520, episodic_return=0.0\n", + "global_step=30648, episodic_return=0.0\n", + "global_step=30648, episodic_return=0.0\n", + "global_step=30648, episodic_return=-78.09499724262108\n", + "global_step=30648, episodic_return=0.0\n", + "SPS: 1509\n", + "global_step=30824, episodic_return=-218.31250108860016\n", + "global_step=30824, episodic_return=0.0\n", + "global_step=30824, episodic_return=0.0\n", + "global_step=30824, episodic_return=0.0\n", + "global_step=30860, episodic_return=0.0\n", + "global_step=30860, episodic_return=0.0\n", + "global_step=30860, episodic_return=0.0\n", + "global_step=30860, episodic_return=-195.76778157896024\n", + "global_step=30924, episodic_return=0.0\n", + "global_step=30924, episodic_return=-513.6954632828515\n", + "global_step=30924, episodic_return=0.0\n", + "global_step=30924, episodic_return=0.0\n", + "global_step=30972, episodic_return=0.0\n", + "global_step=30972, episodic_return=0.0\n", + "global_step=30972, episodic_return=-80.66064506131318\n", + "global_step=30972, episodic_return=0.0\n", + "global_step=31112, episodic_return=0.0\n", + "global_step=31112, episodic_return=0.0\n", + "global_step=31112, episodic_return=0.0\n", + "global_step=31112, episodic_return=-59.61553813131465\n", + "SPS: 1511\n", + "global_step=31244, episodic_return=0.0\n", + "global_step=31244, episodic_return=0.0\n", + "global_step=31244, episodic_return=-87.13829378463281\n", + "global_step=31244, episodic_return=0.0\n", + "global_step=31336, episodic_return=-110.48588426284496\n", + "global_step=31336, episodic_return=0.0\n", + "global_step=31336, episodic_return=0.0\n", + "global_step=31336, episodic_return=0.0\n", + "global_step=31368, episodic_return=0.0\n", + "global_step=31368, episodic_return=-124.66710460460504\n", + "global_step=31368, episodic_return=0.0\n", + "global_step=31368, episodic_return=0.0\n", + "global_step=31388, episodic_return=0.0\n", + "global_step=31388, episodic_return=0.0\n", + "global_step=31388, episodic_return=0.0\n", + "global_step=31388, episodic_return=-81.68203897147619\n", + "global_step=31588, episodic_return=0.0\n", + "global_step=31588, episodic_return=0.0\n", + "global_step=31588, episodic_return=-154.58932744574219\n", + "global_step=31588, episodic_return=0.0\n", + "SPS: 1511\n", + "global_step=31856, episodic_return=0.0\n", + "global_step=31856, episodic_return=0.0\n", + "global_step=31856, episodic_return=0.0\n", + "global_step=31856, episodic_return=-258.4220980891322\n", + "global_step=31888, episodic_return=-132.90342380008042\n", + "global_step=31888, episodic_return=0.0\n", + "global_step=31888, episodic_return=0.0\n", + "global_step=31888, episodic_return=0.0\n", + "global_step=31900, episodic_return=0.0\n", + "global_step=31900, episodic_return=-83.38840636867394\n", + "global_step=31900, episodic_return=0.0\n", + "global_step=31900, episodic_return=0.0\n", + "global_step=32068, episodic_return=0.0\n", + "global_step=32068, episodic_return=0.0\n", + "global_step=32068, episodic_return=-467.5719453684326\n", + "global_step=32068, episodic_return=0.0\n", + "global_step=32176, episodic_return=-49.909879976052444\n", + "global_step=32176, episodic_return=0.0\n", + "global_step=32176, episodic_return=0.0\n", + "global_step=32176, episodic_return=0.0\n", + "global_step=32200, episodic_return=0.0\n", + "global_step=32200, episodic_return=-117.03404619213556\n", + "global_step=32200, episodic_return=0.0\n", + "global_step=32200, episodic_return=0.0\n", + "SPS: 1510\n", + "global_step=32384, episodic_return=0.0\n", + "global_step=32384, episodic_return=0.0\n", + "global_step=32384, episodic_return=0.0\n", + "global_step=32384, episodic_return=-106.92186293036224\n", + "global_step=32532, episodic_return=-167.81742666044164\n", + "global_step=32532, episodic_return=0.0\n", + "global_step=32532, episodic_return=0.0\n", + "global_step=32532, episodic_return=0.0\n", + "global_step=32628, episodic_return=0.0\n", + "global_step=32628, episodic_return=-121.93528829548383\n", + "global_step=32628, episodic_return=0.0\n", + "global_step=32628, episodic_return=0.0\n", + "global_step=32636, episodic_return=0.0\n", + "global_step=32636, episodic_return=0.0\n", + "global_step=32636, episodic_return=-106.02374892502246\n", + "global_step=32636, episodic_return=0.0\n", + "SPS: 1512\n", + "global_step=32840, episodic_return=0.0\n", + "global_step=32840, episodic_return=0.0\n", + "global_step=32840, episodic_return=0.0\n", + "global_step=32840, episodic_return=-276.45472855891035\n", + "global_step=32868, episodic_return=-161.66146771683134\n", + "global_step=32868, episodic_return=0.0\n", + "global_step=32868, episodic_return=0.0\n", + "global_step=32868, episodic_return=0.0\n", + "global_step=32968, episodic_return=0.0\n", + "global_step=32968, episodic_return=0.0\n", + "global_step=32968, episodic_return=-134.17310658866302\n", + "global_step=32968, episodic_return=0.0\n", + "global_step=33076, episodic_return=0.0\n", + "global_step=33076, episodic_return=-142.69632560851377\n", + "global_step=33076, episodic_return=0.0\n", + "global_step=33076, episodic_return=0.0\n", + "global_step=33248, episodic_return=0.0\n", + "global_step=33248, episodic_return=0.0\n", + "global_step=33248, episodic_return=-97.33289279513758\n", + "global_step=33248, episodic_return=0.0\n", + "SPS: 1513\n", + "global_step=33308, episodic_return=0.0\n", + "global_step=33308, episodic_return=0.0\n", + "global_step=33308, episodic_return=0.0\n", + "global_step=33308, episodic_return=-79.78013565557515\n", + "global_step=33328, episodic_return=-270.0731627435165\n", + "global_step=33328, episodic_return=0.0\n", + "global_step=33328, episodic_return=0.0\n", + "global_step=33328, episodic_return=0.0\n", + "global_step=33468, episodic_return=0.0\n", + "global_step=33468, episodic_return=-97.09013878612409\n", + "global_step=33468, episodic_return=0.0\n", + "global_step=33468, episodic_return=0.0\n", + "global_step=33564, episodic_return=0.0\n", + "global_step=33564, episodic_return=0.0\n", + "global_step=33564, episodic_return=0.0\n", + "global_step=33564, episodic_return=-99.03484647343022\n", + "global_step=33712, episodic_return=0.0\n", + "global_step=33712, episodic_return=0.0\n", + "global_step=33712, episodic_return=-302.4151343303304\n", + "global_step=33712, episodic_return=0.0\n", + "SPS: 1514\n", + "global_step=33820, episodic_return=0.0\n", + "global_step=33820, episodic_return=0.0\n", + "global_step=33820, episodic_return=0.0\n", + "global_step=33820, episodic_return=-34.912698561950194\n", + "global_step=33852, episodic_return=-144.4864242951091\n", + "global_step=33852, episodic_return=0.0\n", + "global_step=33852, episodic_return=0.0\n", + "global_step=33852, episodic_return=0.0\n", + "global_step=33872, episodic_return=0.0\n", + "global_step=33872, episodic_return=-214.48028703097276\n", + "global_step=33872, episodic_return=0.0\n", + "global_step=33872, episodic_return=0.0\n", + "global_step=34064, episodic_return=0.0\n", + "global_step=34064, episodic_return=0.0\n", + "global_step=34064, episodic_return=0.0\n", + "global_step=34064, episodic_return=-111.90236875406072\n", + "global_step=34176, episodic_return=-133.42294323161286\n", + "global_step=34176, episodic_return=0.0\n", + "global_step=34176, episodic_return=0.0\n", + "global_step=34176, episodic_return=0.0\n", + "global_step=34200, episodic_return=0.0\n", + "global_step=34200, episodic_return=0.0\n", + "global_step=34200, episodic_return=-121.01955085692236\n", + "global_step=34200, episodic_return=0.0\n", + "SPS: 1516\n", + "global_step=34312, episodic_return=0.0\n", + "global_step=34312, episodic_return=-152.52743101052417\n", + "global_step=34312, episodic_return=0.0\n", + "global_step=34312, episodic_return=0.0\n", + "global_step=34548, episodic_return=0.0\n", + "global_step=34548, episodic_return=0.0\n", + "global_step=34548, episodic_return=0.0\n", + "global_step=34548, episodic_return=-134.285926682617\n", + "global_step=34588, episodic_return=0.0\n", + "global_step=34588, episodic_return=0.0\n", + "global_step=34588, episodic_return=-82.7047452787356\n", + "global_step=34588, episodic_return=0.0\n", + "global_step=34720, episodic_return=-256.9035786991632\n", + "global_step=34720, episodic_return=0.0\n", + "global_step=34720, episodic_return=0.0\n", + "global_step=34720, episodic_return=0.0\n", + "global_step=34792, episodic_return=0.0\n", + "global_step=34792, episodic_return=-88.64626398833587\n", + "global_step=34792, episodic_return=0.0\n", + "global_step=34792, episodic_return=0.0\n", + "SPS: 1518\n", + "global_step=34904, episodic_return=0.0\n", + "global_step=34904, episodic_return=0.0\n", + "global_step=34904, episodic_return=-76.15499171779362\n", + "global_step=34904, episodic_return=0.0\n", + "global_step=34980, episodic_return=0.0\n", + "global_step=34980, episodic_return=0.0\n", + "global_step=34980, episodic_return=0.0\n", + "global_step=34980, episodic_return=-171.1809998922983\n", + "global_step=35144, episodic_return=-84.83896503067254\n", + "global_step=35144, episodic_return=0.0\n", + "global_step=35144, episodic_return=0.0\n", + "global_step=35144, episodic_return=0.0\n", + "global_step=35208, episodic_return=0.0\n", + "global_step=35208, episodic_return=-354.3947620913443\n", + "global_step=35208, episodic_return=0.0\n", + "global_step=35208, episodic_return=0.0\n", + "SPS: 1518\n", + "global_step=35368, episodic_return=0.0\n", + "global_step=35368, episodic_return=0.0\n", + "global_step=35368, episodic_return=0.0\n", + "global_step=35368, episodic_return=-168.95972257561277\n", + "global_step=35412, episodic_return=-138.6798099481939\n", + "global_step=35412, episodic_return=0.0\n", + "global_step=35412, episodic_return=0.0\n", + "global_step=35412, episodic_return=0.0\n", + "global_step=35456, episodic_return=0.0\n", + "global_step=35456, episodic_return=-89.2759329917311\n", + "global_step=35456, episodic_return=0.0\n", + "global_step=35456, episodic_return=0.0\n", + "global_step=35512, episodic_return=0.0\n", + "global_step=35512, episodic_return=0.0\n", + "global_step=35512, episodic_return=-102.3468677715049\n", + "global_step=35512, episodic_return=0.0\n", + "global_step=35688, episodic_return=-40.55342811831136\n", + "global_step=35688, episodic_return=0.0\n", + "global_step=35688, episodic_return=0.0\n", + "global_step=35688, episodic_return=0.0\n", + "global_step=35728, episodic_return=0.0\n", + "global_step=35728, episodic_return=-77.09126472816686\n", + "global_step=35728, episodic_return=0.0\n", + "global_step=35728, episodic_return=0.0\n", + "SPS: 1519\n", + "global_step=35856, episodic_return=0.0\n", + "global_step=35856, episodic_return=0.0\n", + "global_step=35856, episodic_return=0.0\n", + "global_step=35856, episodic_return=-202.94398146566093\n", + "global_step=35996, episodic_return=0.0\n", + "global_step=35996, episodic_return=-34.7360893268561\n", + "global_step=35996, episodic_return=0.0\n", + "global_step=35996, episodic_return=0.0\n", + "global_step=36004, episodic_return=-76.61693284142844\n", + "global_step=36004, episodic_return=0.0\n", + "global_step=36004, episodic_return=0.0\n", + "global_step=36004, episodic_return=0.0\n", + "global_step=36008, episodic_return=0.0\n", + "global_step=36008, episodic_return=0.0\n", + "global_step=36008, episodic_return=-201.6820361816305\n", + "global_step=36008, episodic_return=0.0\n", + "global_step=36132, episodic_return=0.0\n", + "global_step=36132, episodic_return=0.0\n", + "global_step=36132, episodic_return=0.0\n", + "global_step=36132, episodic_return=-30.750629866171522\n", + "global_step=36264, episodic_return=0.0\n", + "global_step=36264, episodic_return=-79.73642965644778\n", + "global_step=36264, episodic_return=0.0\n", + "global_step=36264, episodic_return=0.0\n", + "SPS: 1520\n", + "global_step=36436, episodic_return=0.0\n", + "global_step=36436, episodic_return=0.0\n", + "global_step=36436, episodic_return=-141.40208745441836\n", + "global_step=36436, episodic_return=0.0\n", + "global_step=36528, episodic_return=-295.5142533404924\n", + "global_step=36528, episodic_return=0.0\n", + "global_step=36528, episodic_return=0.0\n", + "global_step=36528, episodic_return=0.0\n", + "global_step=36644, episodic_return=0.0\n", + "global_step=36644, episodic_return=-142.24072924906577\n", + "global_step=36644, episodic_return=0.0\n", + "global_step=36644, episodic_return=0.0\n", + "global_step=36692, episodic_return=0.0\n", + "global_step=36692, episodic_return=0.0\n", + "global_step=36692, episodic_return=0.0\n", + "global_step=36692, episodic_return=-144.54227613608305\n", + "global_step=36824, episodic_return=-93.66343713904958\n", + "global_step=36824, episodic_return=0.0\n", + "global_step=36824, episodic_return=0.0\n", + "global_step=36824, episodic_return=0.0\n", + "SPS: 1520\n", + "global_step=36924, episodic_return=0.0\n", + "global_step=36924, episodic_return=0.0\n", + "global_step=36924, episodic_return=-195.7537581519847\n", + "global_step=36924, episodic_return=0.0\n", + "global_step=36960, episodic_return=0.0\n", + "global_step=36960, episodic_return=0.0\n", + "global_step=36960, episodic_return=0.0\n", + "global_step=36960, episodic_return=-59.03140256272225\n", + "global_step=37092, episodic_return=0.0\n", + "global_step=37092, episodic_return=-106.04525424661657\n", + "global_step=37092, episodic_return=0.0\n", + "global_step=37092, episodic_return=0.0\n", + "global_step=37176, episodic_return=0.0\n", + "global_step=37176, episodic_return=0.0\n", + "global_step=37176, episodic_return=-91.30151179645681\n", + "global_step=37176, episodic_return=0.0\n", + "global_step=37304, episodic_return=-183.68685054001554\n", + "global_step=37304, episodic_return=0.0\n", + "global_step=37304, episodic_return=0.0\n", + "global_step=37304, episodic_return=0.0\n", + "SPS: 1521\n", + "global_step=37492, episodic_return=0.0\n", + "global_step=37492, episodic_return=-118.27382527460192\n", + "global_step=37492, episodic_return=0.0\n", + "global_step=37492, episodic_return=0.0\n", + "global_step=37500, episodic_return=0.0\n", + "global_step=37500, episodic_return=0.0\n", + "global_step=37500, episodic_return=0.0\n", + "global_step=37500, episodic_return=-173.61561867093044\n", + "global_step=37664, episodic_return=0.0\n", + "global_step=37664, episodic_return=0.0\n", + "global_step=37664, episodic_return=-238.66732236644822\n", + "global_step=37664, episodic_return=0.0\n", + "global_step=37784, episodic_return=0.0\n", + "global_step=37784, episodic_return=-115.09306286948325\n", + "global_step=37784, episodic_return=0.0\n", + "global_step=37784, episodic_return=0.0\n", + "global_step=37792, episodic_return=-365.7762743906785\n", + "global_step=37792, episodic_return=0.0\n", + "global_step=37792, episodic_return=0.0\n", + "global_step=37792, episodic_return=0.0\n", + "SPS: 1523\n", + "global_step=37912, episodic_return=0.0\n", + "global_step=37912, episodic_return=0.0\n", + "global_step=37912, episodic_return=0.0\n", + "global_step=37912, episodic_return=-162.05530151450986\n", + "global_step=38060, episodic_return=0.0\n", + "global_step=38060, episodic_return=0.0\n", + "global_step=38060, episodic_return=-187.1244846611091\n", + "global_step=38060, episodic_return=0.0\n", + "global_step=38120, episodic_return=0.0\n", + "global_step=38120, episodic_return=-105.25682097592748\n", + "global_step=38120, episodic_return=0.0\n", + "global_step=38120, episodic_return=0.0\n", + "global_step=38184, episodic_return=-130.22091023521756\n", + "global_step=38184, episodic_return=0.0\n", + "global_step=38184, episodic_return=0.0\n", + "global_step=38184, episodic_return=0.0\n", + "global_step=38360, episodic_return=0.0\n", + "global_step=38360, episodic_return=0.0\n", + "global_step=38360, episodic_return=0.0\n", + "global_step=38360, episodic_return=-83.6813832294981\n", + "global_step=38392, episodic_return=0.0\n", + "global_step=38392, episodic_return=0.0\n", + "global_step=38392, episodic_return=-164.6510744949171\n", + "global_step=38392, episodic_return=0.0\n", + "SPS: 1523\n", + "global_step=38540, episodic_return=-114.50084877192238\n", + "global_step=38540, episodic_return=0.0\n", + "global_step=38540, episodic_return=0.0\n", + "global_step=38540, episodic_return=0.0\n", + "global_step=38716, episodic_return=0.0\n", + "global_step=38716, episodic_return=-93.34309671951691\n", + "global_step=38716, episodic_return=0.0\n", + "global_step=38716, episodic_return=0.0\n", + "global_step=38824, episodic_return=0.0\n", + "global_step=38824, episodic_return=0.0\n", + "global_step=38824, episodic_return=-103.47519202125204\n", + "global_step=38824, episodic_return=0.0\n", + "global_step=38856, episodic_return=0.0\n", + "global_step=38856, episodic_return=0.0\n", + "global_step=38856, episodic_return=0.0\n", + "global_step=38856, episodic_return=-153.3558031363482\n", + "global_step=38912, episodic_return=-155.89797512633425\n", + "global_step=38912, episodic_return=0.0\n", + "global_step=38912, episodic_return=0.0\n", + "global_step=38912, episodic_return=0.0\n", + "SPS: 1525\n", + "global_step=39124, episodic_return=0.0\n", + "global_step=39124, episodic_return=0.0\n", + "global_step=39124, episodic_return=0.0\n", + "global_step=39124, episodic_return=-71.07997264706191\n", + "global_step=39160, episodic_return=0.0\n", + "global_step=39160, episodic_return=-147.66884551167013\n", + "global_step=39160, episodic_return=0.0\n", + "global_step=39160, episodic_return=0.0\n", + "global_step=39268, episodic_return=0.0\n", + "global_step=39268, episodic_return=0.0\n", + "global_step=39268, episodic_return=-116.5922084763587\n", + "global_step=39268, episodic_return=0.0\n", + "global_step=39424, episodic_return=-148.15731305489103\n", + "global_step=39424, episodic_return=0.0\n", + "global_step=39424, episodic_return=0.0\n", + "global_step=39424, episodic_return=0.0\n", + "SPS: 1526\n", + "global_step=39500, episodic_return=0.0\n", + "global_step=39500, episodic_return=-122.56519928247248\n", + "global_step=39500, episodic_return=0.0\n", + "global_step=39500, episodic_return=0.0\n", + "global_step=39540, episodic_return=0.0\n", + "global_step=39540, episodic_return=0.0\n", + "global_step=39540, episodic_return=0.0\n", + "global_step=39540, episodic_return=-100.62625250425444\n", + "global_step=39600, episodic_return=0.0\n", + "global_step=39600, episodic_return=0.0\n", + "global_step=39600, episodic_return=-104.30969672493457\n", + "global_step=39600, episodic_return=0.0\n", + "global_step=39784, episodic_return=-115.9324618418651\n", + "global_step=39784, episodic_return=0.0\n", + "global_step=39784, episodic_return=0.0\n", + "global_step=39784, episodic_return=0.0\n", + "global_step=39884, episodic_return=0.0\n", + "global_step=39884, episodic_return=-113.19836585586869\n", + "global_step=39884, episodic_return=0.0\n", + "global_step=39884, episodic_return=-109.45124792474174\n", + "global_step=39916, episodic_return=0.0\n", + "global_step=39916, episodic_return=0.0\n", + "global_step=39916, episodic_return=-200.44995534848707\n", + "global_step=39916, episodic_return=0.0\n", + "SPS: 1526\n", + "global_step=40172, episodic_return=0.0\n", + "global_step=40172, episodic_return=0.0\n", + "global_step=40172, episodic_return=-92.39789623740064\n", + "global_step=40172, episodic_return=0.0\n", + "global_step=40216, episodic_return=0.0\n", + "global_step=40216, episodic_return=-198.90007027832823\n", + "global_step=40216, episodic_return=0.0\n", + "global_step=40216, episodic_return=0.0\n", + "global_step=40232, episodic_return=0.0\n", + "global_step=40232, episodic_return=0.0\n", + "global_step=40232, episodic_return=0.0\n", + "global_step=40232, episodic_return=-98.23431143357031\n", + "global_step=40288, episodic_return=-111.91793902888902\n", + "global_step=40288, episodic_return=0.0\n", + "global_step=40288, episodic_return=0.0\n", + "global_step=40288, episodic_return=0.0\n", + "SPS: 1525\n", + "global_step=40476, episodic_return=0.0\n", + "global_step=40476, episodic_return=0.0\n", + "global_step=40476, episodic_return=0.0\n", + "global_step=40476, episodic_return=-64.5505665200252\n", + "global_step=40496, episodic_return=0.0\n", + "global_step=40496, episodic_return=0.0\n", + "global_step=40496, episodic_return=-90.60733405536902\n", + "global_step=40496, episodic_return=0.0\n", + "global_step=40596, episodic_return=0.0\n", + "global_step=40596, episodic_return=-207.30513897855406\n", + "global_step=40596, episodic_return=0.0\n", + "global_step=40596, episodic_return=0.0\n", + "global_step=40672, episodic_return=-112.08002865173495\n", + "global_step=40672, episodic_return=0.0\n", + "global_step=40672, episodic_return=0.0\n", + "global_step=40672, episodic_return=0.0\n", + "global_step=40840, episodic_return=0.0\n", + "global_step=40840, episodic_return=0.0\n", + "global_step=40840, episodic_return=-260.3074911375628\n", + "global_step=40840, episodic_return=0.0\n", + "SPS: 1520\n", + "global_step=40992, episodic_return=0.0\n", + "global_step=40992, episodic_return=0.0\n", + "global_step=40992, episodic_return=0.0\n", + "global_step=40992, episodic_return=-168.77711911105595\n", + "global_step=41104, episodic_return=0.0\n", + "global_step=41104, episodic_return=-215.51036330242295\n", + "global_step=41104, episodic_return=0.0\n", + "global_step=41104, episodic_return=0.0\n", + "global_step=41136, episodic_return=-235.3882120179783\n", + "global_step=41136, episodic_return=0.0\n", + "global_step=41136, episodic_return=0.0\n", + "global_step=41136, episodic_return=0.0\n", + "global_step=41160, episodic_return=0.0\n", + "global_step=41160, episodic_return=0.0\n", + "global_step=41160, episodic_return=-100.50230666035506\n", + "global_step=41160, episodic_return=0.0\n", + "global_step=41316, episodic_return=0.0\n", + "global_step=41316, episodic_return=0.0\n", + "global_step=41316, episodic_return=0.0\n", + "global_step=41316, episodic_return=-67.94384646942923\n", + "global_step=41408, episodic_return=-86.70306122830908\n", + "global_step=41408, episodic_return=0.0\n", + "global_step=41408, episodic_return=0.0\n", + "global_step=41408, episodic_return=0.0\n", + "SPS: 1516\n", + "global_step=41500, episodic_return=0.0\n", + "global_step=41500, episodic_return=-124.30121575229896\n", + "global_step=41500, episodic_return=0.0\n", + "global_step=41500, episodic_return=0.0\n", + "global_step=41512, episodic_return=0.0\n", + "global_step=41512, episodic_return=0.0\n", + "global_step=41512, episodic_return=-132.43912388118224\n", + "global_step=41512, episodic_return=0.0\n", + "global_step=41656, episodic_return=-73.97513128515271\n", + "global_step=41656, episodic_return=0.0\n", + "global_step=41656, episodic_return=0.0\n", + "global_step=41656, episodic_return=0.0\n", + "global_step=41704, episodic_return=0.0\n", + "global_step=41704, episodic_return=0.0\n", + "global_step=41704, episodic_return=0.0\n", + "global_step=41704, episodic_return=-115.58497595554091\n", + "global_step=41912, episodic_return=0.0\n", + "global_step=41912, episodic_return=0.0\n", + "global_step=41912, episodic_return=-106.74039182702457\n", + "global_step=41912, episodic_return=0.0\n", + "global_step=41924, episodic_return=0.0\n", + "global_step=41924, episodic_return=-109.67677942639511\n", + "global_step=41924, episodic_return=0.0\n", + "global_step=41924, episodic_return=0.0\n", + "global_step=41940, episodic_return=-52.75747715627173\n", + "global_step=41940, episodic_return=0.0\n", + "global_step=41940, episodic_return=0.0\n", + "global_step=41940, episodic_return=0.0\n", + "SPS: 1513\n", + "global_step=42084, episodic_return=0.0\n", + "global_step=42084, episodic_return=0.0\n", + "global_step=42084, episodic_return=0.0\n", + "global_step=42084, episodic_return=-176.34476877173893\n", + "global_step=42376, episodic_return=-84.69550061650752\n", + "global_step=42376, episodic_return=0.0\n", + "global_step=42376, episodic_return=0.0\n", + "global_step=42376, episodic_return=0.0\n", + "global_step=42412, episodic_return=0.0\n", + "global_step=42412, episodic_return=0.0\n", + "global_step=42412, episodic_return=-318.0056332645644\n", + "global_step=42412, episodic_return=0.0\n", + "global_step=42420, episodic_return=0.0\n", + "global_step=42420, episodic_return=-134.07607874747202\n", + "global_step=42420, episodic_return=0.0\n", + "global_step=42420, episodic_return=0.0\n", + "global_step=42468, episodic_return=0.0\n", + "global_step=42468, episodic_return=0.0\n", + "global_step=42468, episodic_return=0.0\n", + "global_step=42468, episodic_return=-103.15592089257423\n", + "SPS: 1505\n", + "global_step=42672, episodic_return=0.0\n", + "global_step=42672, episodic_return=0.0\n", + "global_step=42672, episodic_return=-99.9923834788316\n", + "global_step=42672, episodic_return=0.0\n", + "global_step=42696, episodic_return=0.0\n", + "global_step=42696, episodic_return=-33.0372447711576\n", + "global_step=42696, episodic_return=0.0\n", + "global_step=42696, episodic_return=0.0\n", + "global_step=42860, episodic_return=-335.62353800485107\n", + "global_step=42860, episodic_return=0.0\n", + "global_step=42860, episodic_return=0.0\n", + "global_step=42860, episodic_return=0.0\n", + "global_step=42940, episodic_return=0.0\n", + "global_step=42940, episodic_return=0.0\n", + "global_step=42940, episodic_return=0.0\n", + "global_step=42940, episodic_return=-258.06406010684157\n", + "SPS: 1501\n", + "global_step=43052, episodic_return=0.0\n", + "global_step=43052, episodic_return=0.0\n", + "global_step=43052, episodic_return=-110.05862352768072\n", + "global_step=43052, episodic_return=0.0\n", + "global_step=43228, episodic_return=-123.17337317672845\n", + "global_step=43228, episodic_return=0.0\n", + "global_step=43228, episodic_return=0.0\n", + "global_step=43228, episodic_return=0.0\n", + "global_step=43268, episodic_return=0.0\n", + "global_step=43268, episodic_return=42.89226029648424\n", + "global_step=43268, episodic_return=0.0\n", + "global_step=43268, episodic_return=0.0\n", + "global_step=43360, episodic_return=0.0\n", + "global_step=43360, episodic_return=0.0\n", + "global_step=43360, episodic_return=0.0\n", + "global_step=43360, episodic_return=-300.1519622743209\n", + "global_step=43464, episodic_return=0.0\n", + "global_step=43464, episodic_return=0.0\n", + "global_step=43464, episodic_return=-193.3202337582009\n", + "global_step=43464, episodic_return=0.0\n", + "SPS: 1502\n", + "global_step=43568, episodic_return=-76.81559577045022\n", + "global_step=43568, episodic_return=0.0\n", + "global_step=43568, episodic_return=0.0\n", + "global_step=43568, episodic_return=0.0\n", + "global_step=43788, episodic_return=0.0\n", + "global_step=43788, episodic_return=-136.59903813154608\n", + "global_step=43788, episodic_return=0.0\n", + "global_step=43788, episodic_return=0.0\n", + "global_step=43836, episodic_return=0.0\n", + "global_step=43836, episodic_return=0.0\n", + "global_step=43836, episodic_return=0.0\n", + "global_step=43836, episodic_return=-82.24306083728668\n", + "global_step=43908, episodic_return=-144.37465032135398\n", + "global_step=43908, episodic_return=0.0\n", + "global_step=43908, episodic_return=0.0\n", + "global_step=43908, episodic_return=0.0\n", + "global_step=43960, episodic_return=0.0\n", + "global_step=43960, episodic_return=0.0\n", + "global_step=43960, episodic_return=-402.4363200745798\n", + "global_step=43960, episodic_return=0.0\n", + "SPS: 1502\n", + "global_step=44092, episodic_return=0.0\n", + "global_step=44092, episodic_return=-90.84795435825852\n", + "global_step=44092, episodic_return=0.0\n", + "global_step=44092, episodic_return=0.0\n", + "global_step=44368, episodic_return=-414.48147313592165\n", + "global_step=44368, episodic_return=0.0\n", + "global_step=44368, episodic_return=0.0\n", + "global_step=44368, episodic_return=0.0\n", + "global_step=44480, episodic_return=0.0\n", + "global_step=44480, episodic_return=0.0\n", + "global_step=44480, episodic_return=0.0\n", + "global_step=44480, episodic_return=-232.84403422841984\n", + "global_step=44488, episodic_return=0.0\n", + "global_step=44488, episodic_return=0.0\n", + "global_step=44488, episodic_return=-62.43996352577196\n", + "global_step=44488, episodic_return=0.0\n", + "SPS: 1504\n", + "global_step=44600, episodic_return=0.0\n", + "global_step=44600, episodic_return=-335.09876524566533\n", + "global_step=44600, episodic_return=0.0\n", + "global_step=44600, episodic_return=0.0\n", + "global_step=44740, episodic_return=0.0\n", + "global_step=44740, episodic_return=0.0\n", + "global_step=44740, episodic_return=0.0\n", + "global_step=44740, episodic_return=-130.81766396743538\n", + "global_step=44764, episodic_return=-215.31845174047834\n", + "global_step=44764, episodic_return=0.0\n", + "global_step=44764, episodic_return=0.0\n", + "global_step=44764, episodic_return=0.0\n", + "global_step=44912, episodic_return=0.0\n", + "global_step=44912, episodic_return=-104.05662914255566\n", + "global_step=44912, episodic_return=0.0\n", + "global_step=44912, episodic_return=0.0\n", + "global_step=44948, episodic_return=0.0\n", + "global_step=44948, episodic_return=0.0\n", + "global_step=44948, episodic_return=-111.29951010794045\n", + "global_step=44948, episodic_return=0.0\n", + "SPS: 1504\n", + "global_step=45072, episodic_return=0.0\n", + "global_step=45072, episodic_return=0.0\n", + "global_step=45072, episodic_return=0.0\n", + "global_step=45072, episodic_return=-202.0370293270174\n", + "global_step=45188, episodic_return=-127.59096132961787\n", + "global_step=45188, episodic_return=0.0\n", + "global_step=45188, episodic_return=0.0\n", + "global_step=45188, episodic_return=0.0\n", + "global_step=45228, episodic_return=0.0\n", + "global_step=45228, episodic_return=-59.557131517727825\n", + "global_step=45228, episodic_return=0.0\n", + "global_step=45228, episodic_return=0.0\n", + "global_step=45324, episodic_return=0.0\n", + "global_step=45324, episodic_return=0.0\n", + "global_step=45324, episodic_return=-90.04873590118079\n", + "global_step=45324, episodic_return=0.0\n", + "global_step=45408, episodic_return=0.0\n", + "global_step=45408, episodic_return=0.0\n", + "global_step=45408, episodic_return=0.0\n", + "global_step=45408, episodic_return=-192.46227216399626\n", + "global_step=45532, episodic_return=-119.63851301807338\n", + "global_step=45532, episodic_return=0.0\n", + "global_step=45532, episodic_return=0.0\n", + "global_step=45532, episodic_return=0.0\n", + "global_step=45560, episodic_return=0.0\n", + "global_step=45560, episodic_return=-208.37953316673145\n", + "global_step=45560, episodic_return=0.0\n", + "global_step=45560, episodic_return=0.0\n", + "SPS: 1505\n", + "global_step=45632, episodic_return=0.0\n", + "global_step=45632, episodic_return=0.0\n", + "global_step=45632, episodic_return=-130.00431643059557\n", + "global_step=45632, episodic_return=0.0\n", + "global_step=45776, episodic_return=-79.14405956077562\n", + "global_step=45776, episodic_return=0.0\n", + "global_step=45776, episodic_return=0.0\n", + "global_step=45776, episodic_return=0.0\n", + "global_step=45816, episodic_return=0.0\n", + "global_step=45816, episodic_return=0.0\n", + "global_step=45816, episodic_return=0.0\n", + "global_step=45816, episodic_return=-153.1638726967988\n", + "global_step=45940, episodic_return=0.0\n", + "global_step=45940, episodic_return=-61.77034645505779\n", + "global_step=45940, episodic_return=0.0\n", + "global_step=45940, episodic_return=0.0\n", + "SPS: 1506\n", + "global_step=46136, episodic_return=-96.7509804693675\n", + "global_step=46136, episodic_return=0.0\n", + "global_step=46136, episodic_return=0.0\n", + "global_step=46136, episodic_return=0.0\n", + "global_step=46160, episodic_return=0.0\n", + "global_step=46160, episodic_return=0.0\n", + "global_step=46160, episodic_return=-100.17253615306421\n", + "global_step=46160, episodic_return=0.0\n", + "global_step=46232, episodic_return=0.0\n", + "global_step=46232, episodic_return=-66.258694646864\n", + "global_step=46232, episodic_return=0.0\n", + "global_step=46232, episodic_return=0.0\n", + "global_step=46312, episodic_return=0.0\n", + "global_step=46312, episodic_return=0.0\n", + "global_step=46312, episodic_return=0.0\n", + "global_step=46312, episodic_return=-282.2957877980816\n", + "global_step=46444, episodic_return=-86.87676818064423\n", + "global_step=46444, episodic_return=0.0\n", + "global_step=46444, episodic_return=0.0\n", + "global_step=46444, episodic_return=0.0\n", + "SPS: 1508\n", + "global_step=46624, episodic_return=0.0\n", + "global_step=46624, episodic_return=0.0\n", + "global_step=46624, episodic_return=-130.98261171667951\n", + "global_step=46624, episodic_return=0.0\n", + "global_step=46704, episodic_return=0.0\n", + "global_step=46704, episodic_return=-41.62535082141453\n", + "global_step=46704, episodic_return=0.0\n", + "global_step=46704, episodic_return=0.0\n", + "global_step=46732, episodic_return=-64.58844904884363\n", + "global_step=46732, episodic_return=0.0\n", + "global_step=46732, episodic_return=0.0\n", + "global_step=46732, episodic_return=0.0\n", + "global_step=46840, episodic_return=0.0\n", + "global_step=46840, episodic_return=0.0\n", + "global_step=46840, episodic_return=0.0\n", + "global_step=46840, episodic_return=-79.86341710269015\n", + "global_step=47028, episodic_return=0.0\n", + "global_step=47028, episodic_return=0.0\n", + "global_step=47028, episodic_return=-119.39029905655015\n", + "global_step=47028, episodic_return=0.0\n", + "global_step=47040, episodic_return=-131.11397209308637\n", + "global_step=47040, episodic_return=0.0\n", + "global_step=47040, episodic_return=0.0\n", + "global_step=47040, episodic_return=0.0\n", + "global_step=47056, episodic_return=0.0\n", + "global_step=47056, episodic_return=-220.05257566353825\n", + "global_step=47056, episodic_return=0.0\n", + "global_step=47056, episodic_return=0.0\n", + "SPS: 1509\n", + "global_step=47304, episodic_return=0.0\n", + "global_step=47304, episodic_return=0.0\n", + "global_step=47304, episodic_return=0.0\n", + "global_step=47304, episodic_return=-115.67627899617287\n", + "global_step=47352, episodic_return=0.0\n", + "global_step=47352, episodic_return=0.0\n", + "global_step=47352, episodic_return=-99.98355102156644\n", + "global_step=47352, episodic_return=0.0\n", + "global_step=47392, episodic_return=-155.89139235339408\n", + "global_step=47392, episodic_return=0.0\n", + "global_step=47392, episodic_return=0.0\n", + "global_step=47392, episodic_return=0.0\n", + "global_step=47548, episodic_return=0.0\n", + "global_step=47548, episodic_return=0.0\n", + "global_step=47548, episodic_return=0.0\n", + "global_step=47548, episodic_return=-125.46148134798624\n", + "global_step=47556, episodic_return=0.0\n", + "global_step=47556, episodic_return=-253.71640608526536\n", + "global_step=47556, episodic_return=0.0\n", + "global_step=47556, episodic_return=0.0\n", + "SPS: 1510\n", + "global_step=47756, episodic_return=-308.84811159541226\n", + "global_step=47756, episodic_return=0.0\n", + "global_step=47756, episodic_return=0.0\n", + "global_step=47756, episodic_return=0.0\n", + "global_step=47768, episodic_return=0.0\n", + "global_step=47768, episodic_return=0.0\n", + "global_step=47768, episodic_return=-162.42308349192245\n", + "global_step=47768, episodic_return=0.0\n", + "global_step=47804, episodic_return=0.0\n", + "global_step=47804, episodic_return=0.0\n", + "global_step=47804, episodic_return=0.0\n", + "global_step=47804, episodic_return=-147.78946225192746\n", + "global_step=47996, episodic_return=0.0\n", + "global_step=47996, episodic_return=-72.80429086996239\n", + "global_step=47996, episodic_return=0.0\n", + "global_step=47996, episodic_return=0.0\n", + "SPS: 1511\n", + "global_step=48156, episodic_return=0.0\n", + "global_step=48156, episodic_return=0.0\n", + "global_step=48156, episodic_return=-243.8773446393264\n", + "global_step=48156, episodic_return=0.0\n", + "global_step=48176, episodic_return=0.0\n", + "global_step=48176, episodic_return=0.0\n", + "global_step=48176, episodic_return=0.0\n", + "global_step=48176, episodic_return=-102.31519912298796\n", + "global_step=48344, episodic_return=0.0\n", + "global_step=48344, episodic_return=-128.06037368088545\n", + "global_step=48344, episodic_return=0.0\n", + "global_step=48344, episodic_return=0.0\n", + "global_step=48352, episodic_return=-189.4805248194835\n", + "global_step=48352, episodic_return=0.0\n", + "global_step=48352, episodic_return=0.0\n", + "global_step=48352, episodic_return=0.0\n", + "global_step=48492, episodic_return=0.0\n", + "global_step=48492, episodic_return=0.0\n", + "global_step=48492, episodic_return=0.0\n", + "global_step=48492, episodic_return=-123.20025075596399\n", + "global_step=48560, episodic_return=0.0\n", + "global_step=48560, episodic_return=0.0\n", + "global_step=48560, episodic_return=-287.3283106546849\n", + "global_step=48560, episodic_return=0.0\n", + "global_step=48620, episodic_return=0.0\n", + "global_step=48620, episodic_return=-130.5931019536206\n", + "global_step=48620, episodic_return=0.0\n", + "global_step=48620, episodic_return=0.0\n", + "SPS: 1512\n", + "global_step=48808, episodic_return=9.089811988811348\n", + "global_step=48808, episodic_return=0.0\n", + "global_step=48808, episodic_return=0.0\n", + "global_step=48808, episodic_return=0.0\n", + "global_step=48896, episodic_return=0.0\n", + "global_step=48896, episodic_return=0.0\n", + "global_step=48896, episodic_return=0.0\n", + "global_step=48896, episodic_return=-101.96250220688296\n", + "global_step=48912, episodic_return=0.0\n", + "global_step=48912, episodic_return=0.0\n", + "global_step=48912, episodic_return=-89.50128012525491\n", + "global_step=48912, episodic_return=0.0\n", + "global_step=48916, episodic_return=0.0\n", + "global_step=48916, episodic_return=-128.79994965405038\n", + "global_step=48916, episodic_return=0.0\n", + "global_step=48916, episodic_return=0.0\n", + "global_step=49080, episodic_return=-47.85055085354773\n", + "global_step=49080, episodic_return=0.0\n", + "global_step=49080, episodic_return=0.0\n", + "global_step=49080, episodic_return=0.0\n", + "SPS: 1512\n", + "global_step=49320, episodic_return=0.0\n", + "global_step=49320, episodic_return=-110.01764772576249\n", + "global_step=49320, episodic_return=0.0\n", + "global_step=49320, episodic_return=0.0\n", + "global_step=49364, episodic_return=0.0\n", + "global_step=49364, episodic_return=0.0\n", + "global_step=49364, episodic_return=0.0\n", + "global_step=49364, episodic_return=-273.37295025643186\n", + "global_step=49380, episodic_return=0.0\n", + "global_step=49380, episodic_return=0.0\n", + "global_step=49380, episodic_return=-345.2010040269387\n", + "global_step=49380, episodic_return=0.0\n", + "global_step=49508, episodic_return=23.426672944491997\n", + "global_step=49508, episodic_return=0.0\n", + "global_step=49508, episodic_return=0.0\n", + "global_step=49508, episodic_return=0.0\n", + "SPS: 1514\n", + "\u001b[38;5;4mβ„Ή This function will save, evaluate, generate a video of your agent,\n", + "create a model card and push everything to the hub. It might take up to 1min.\n", + "This is a work in progress: if you encounter a bug, please open an issue.\u001b[0m\n" + ] + }, + { + "output_type": "stream", + "name": "stderr", + "text": [ + "WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).\n" + ] + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[38;5;4mβ„Ή Pushing repo Anish13/ppo-LunarLander to the Hugging Face Hub\u001b[0m\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "events.out.tfevents.1745998595.1c54943520c2.688.17: 0%| | 0.00/303k [00:00 args.clip_coef).float().mean().item()]\n", + "\n", + " mb_advantages = b_advantages[mb_inds]\n", + " if args.norm_adv:\n", + " mb_advantages = (mb_advantages - mb_advantages.mean()) / (mb_advantages.std() + 1e-8)\n", + "\n", + " # Policy loss\n", + " pg_loss1 = -mb_advantages * ratio\n", + " pg_loss2 = -mb_advantages * torch.clamp(ratio, 1 - args.clip_coef, 1 + args.clip_coef)\n", + " pg_loss = torch.max(pg_loss1, pg_loss2).mean()\n", + "\n", + " # Value loss\n", + " newvalue = newvalue.view(-1)\n", + " if args.clip_vloss:\n", + " v_loss_unclipped = (newvalue - b_returns[mb_inds]) ** 2\n", + " v_clipped = b_values[mb_inds] + torch.clamp(\n", + " newvalue - b_values[mb_inds],\n", + " -args.clip_coef,\n", + " args.clip_coef,\n", + " )\n", + " v_loss_clipped = (v_clipped - b_returns[mb_inds]) ** 2\n", + " v_loss_max = torch.max(v_loss_unclipped, v_loss_clipped)\n", + " v_loss = 0.5 * v_loss_max.mean()\n", + " else:\n", + " v_loss = 0.5 * ((newvalue - b_returns[mb_inds]) ** 2).mean()\n", + "\n", + " entropy_loss = entropy.mean()\n", + " loss = pg_loss - args.ent_coef * entropy_loss + v_loss * args.vf_coef\n", + "\n", + " optimizer.zero_grad()\n", + " loss.backward()\n", + " nn.utils.clip_grad_norm_(agent.parameters(), args.max_grad_norm)\n", + " optimizer.step()\n", + "\n", + " if args.target_kl is not None:\n", + " if approx_kl > args.target_kl:\n", + " break\n", + "\n", + " y_pred, y_true = b_values.cpu().numpy(), b_returns.cpu().numpy()\n", + " var_y = np.var(y_true)\n", + " explained_var = np.nan if var_y == 0 else 1 - np.var(y_true - y_pred) / var_y\n", + "\n", + " # TRY NOT TO MODIFY: record rewards for plotting purposes\n", + " writer.add_scalar(\"charts/learning_rate\", optimizer.param_groups[0][\"lr\"], global_step)\n", + " writer.add_scalar(\"losses/value_loss\", v_loss.item(), global_step)\n", + " writer.add_scalar(\"losses/policy_loss\", pg_loss.item(), global_step)\n", + " writer.add_scalar(\"losses/entropy\", entropy_loss.item(), global_step)\n", + " writer.add_scalar(\"losses/old_approx_kl\", old_approx_kl.item(), global_step)\n", + " writer.add_scalar(\"losses/approx_kl\", approx_kl.item(), global_step)\n", + " writer.add_scalar(\"losses/clipfrac\", np.mean(clipfracs), global_step)\n", + " writer.add_scalar(\"losses/explained_variance\", explained_var, global_step)\n", + " print(\"SPS:\", int(global_step / (time.time() - start_time)))\n", + " writer.add_scalar(\"charts/SPS\", int(global_step / (time.time() - start_time)), global_step)\n", + "\n", + " envs.close()\n", + " writer.close()\n", + "\n", + " # Create the evaluation environment\n", + " eval_env = gym.make(args.env_id, render_mode=\"rgb_array\")\n", + "\n", + " package_to_hub(\n", + " repo_id=args.repo_id,\n", + " model=agent,\n", + " hyperparameters=args,\n", + " eval_env=gym.make(args.env_id, render_mode=\"rgb_array\"), # <-- fix here too\n", + " logs=f\"runs/{run_name}\",\n", + " )\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "JquRrWytA6eo" + }, + "source": [ + "To be able to share your model with the community there are three more steps to follow:\n", + "\n", + "1️⃣ (If it's not already done) create an account to HF ➑ https://huggingface.co/join\n", + "\n", + "2️⃣ Sign in and then, you need to store your authentication token from the Hugging Face website.\n", + "- Create a new token (https://huggingface.co/settings/tokens) **with write role**\n", + "\n", + "\"Create\n", + "\n", + "- Copy the token\n", + "- Run the cell below and paste the token" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "id": "GZiFBBlzxzxY", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 17, + "referenced_widgets": [ + "8e1b70e8c04f483f8bfaa0aeee91fc7b", + "92bbd1688aad426787beb03c64ff376d", + "84365d8a62b1425c88c36502955d69e0", + "a270854b78344917b8fe93a4908ff4c5", + "e21d3b7feda346a38411c1210cbb45ec", + "86ee3825be1c4252bd38e4b977e746d9", + "851db24d8a9c48fcae9b9722ccde17f7", + "0682113a4aa7467a96de1c1bc67c5e12", + "ee72035aa2284875a53e31944c8fef55", + "1ecc478206884bb88b4513fedbdbe582", + "043221d1db184078a603cabfad55ad05", + "119c4e28ab6a402497dcdf7d722fd3c2", + "62337f40579d422e98cd19cbbd836408", + "9ebf3f78bcb04484a5da2c751445b3b5", + "70d43337effd4e0fa1f7d3da5e126fed", + "e7bec908fb064220892c5e931540511f", + "2b50423c03a64935b0d00cf2fb8ea30a", + "0c65e6b3fa894fdc864989e3e21ad491", + "dc574c859a8c493bb60f87278a877b3e", + "9bbf2ad5bb0242ba971eaddc00119001" + ] + }, + "outputId": "80085bf8-c545-4a79-aced-fb0ec05f16fd" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "VBox(children=(HTML(value='
" + ], + "metadata": { + "id": "Sq0My0LOjPYR" + } + }, + { + "cell_type": "markdown", + "source": [ + "\"PPO\"/" + ], + "metadata": { + "id": "A8C-Q5ZyjUe3" + } + }, + { + "cell_type": "markdown", + "metadata": { + "id": "VrS80GmMu_j5" + }, + "source": [ + "- Now we just need to run this python script using `python .py` with the additional parameters we defined with `argparse`\n", + "\n", + "- You should modify more hyperparameters otherwise the training will not be super stable." + ] + }, + { + "cell_type": "code", + "source": [ + "!python ppo.py --env-id=\"LunarLander-v2\" --repo-id=\"YOUR_REPO_ID\" --total-timesteps=50000" + ], + "metadata": { + "id": "KXLih6mKseBs", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "6d6fcc1f-7581-4d98-e9c4-c0a51ba530ce" + }, + "execution_count": 34, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "python3: can't open file '/content/ppo.py': [Errno 2] No such file or directory\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "eVsVJ5AdqLE7" + }, + "source": [ + "## Some additional challenges πŸ†\n", + "The best way to learn **is to try things by your own**! Why not trying another environment?\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "nYdl758GqLXT" + }, + "source": [ + "See you on Unit 8, part 2 where we going to train agents to play Doom πŸ”₯\n", + "## Keep learning, stay awesome πŸ€—" + ] + } + ], + "metadata": { + "colab": { + "provenance": [], + "gpuType": "T4" + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + }, + "accelerator": "GPU", + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "1bc55520f7224951be56724f2f8c56eb": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_d19388de6acb4619b09d4134c883bc12", + "IPY_MODEL_e97db22db95b4737939291ebdc027542", + "IPY_MODEL_e5d1de9ee5804a79af2de598ac189d23" + ], + "layout": "IPY_MODEL_1222af9728444699878d6eff3eadb422" + } + }, + "d19388de6acb4619b09d4134c883bc12": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_da290322509e40a38b773e29203beb08", + "placeholder": "​", + "style": "IPY_MODEL_76691d2225a140be84cb039bfa5ddc66", + "value": "events.out.tfevents.1745998595.1c54943520c2.688.17: 100%" + } + }, + "e97db22db95b4737939291ebdc027542": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e2494ca3eaa94f878910f0fb3294124e", + "max": 302507, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_ef94b647379e425c91b30a41d319bf81", + "value": 302507 + } + }, + "e5d1de9ee5804a79af2de598ac189d23": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_5f043d406d0b4d4e8d70f20142605b9e", + "placeholder": "​", + "style": "IPY_MODEL_868e6b0273bc4c0b970fea9d3ad8c6a3", + "value": " 303k/303k [00:00<00:00, 529kB/s]" + } + }, + "1222af9728444699878d6eff3eadb422": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "da290322509e40a38b773e29203beb08": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "76691d2225a140be84cb039bfa5ddc66": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "e2494ca3eaa94f878910f0fb3294124e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ef94b647379e425c91b30a41d319bf81": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "5f043d406d0b4d4e8d70f20142605b9e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "868e6b0273bc4c0b970fea9d3ad8c6a3": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "17f1ac1f06f145bc8b94eb7c3c110205": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_b73bf3572a0345c19f35575e522b7781", + "IPY_MODEL_a749f78e8bd7444fb45c4c2d0c3ccfd0", + "IPY_MODEL_b1a7db22d14a46f8a645f688cb34f8a5" + ], + "layout": "IPY_MODEL_dff438dd40884b3b8eb30355d41ce00d" + } + }, + "b73bf3572a0345c19f35575e522b7781": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_d7e5a99e71a94a68a932354faa110c12", + "placeholder": "​", + "style": "IPY_MODEL_8509fab4cc404b159b14e51b04c90b25", + "value": "model.pt: 100%" + } + }, + "a749f78e8bd7444fb45c4c2d0c3ccfd0": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_a3133154031f45df9a0f2acb50c5251e", + "max": 43026, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_03164def3ed94865b5ba32e4b050322a", + "value": 43026 + } + }, + "b1a7db22d14a46f8a645f688cb34f8a5": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_0e991537a7c0444b83fd37b9eeefbc0c", + "placeholder": "​", + "style": "IPY_MODEL_22aba18c4ffb42dfac6ad66f2cbd55aa", + "value": " 43.0k/43.0k [00:00<00:00, 175kB/s]" + } + }, + "dff438dd40884b3b8eb30355d41ce00d": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d7e5a99e71a94a68a932354faa110c12": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "8509fab4cc404b159b14e51b04c90b25": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "a3133154031f45df9a0f2acb50c5251e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "03164def3ed94865b5ba32e4b050322a": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "0e991537a7c0444b83fd37b9eeefbc0c": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "22aba18c4ffb42dfac6ad66f2cbd55aa": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "6c57350f8d0a41f5a46b0dacabb0f644": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_5014d74eaa114bbd8b691521125e981b", + "IPY_MODEL_500437911cc845b4b2a0036cf37ee7cf", + "IPY_MODEL_b0dc84fd87464ec88b3233c2ce93974b" + ], + "layout": "IPY_MODEL_946f8a4fcd4a490a900d1cefdd9f89dd" + } + }, + "5014d74eaa114bbd8b691521125e981b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ab5415ee8f8940e894e48ad5f0f0b564", + "placeholder": "​", + "style": "IPY_MODEL_1af9913925b0426c92769f0c031f20ed", + "value": "Upload 2 LFS files: 100%" + } + }, + "500437911cc845b4b2a0036cf37ee7cf": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_58e9c158205f43cb9d6323e3dbffce87", + "max": 2, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_66cb3f30d59140508a100b82389f1a2c", + "value": 2 + } + }, + "b0dc84fd87464ec88b3233c2ce93974b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_04e290bcbd7448a1b2df9b0624f01b67", + "placeholder": "​", + "style": "IPY_MODEL_d3453f32a0fe4cf6b9518892d6c7b633", + "value": " 2/2 [00:00<00:00,  1.33it/s]" + } + }, + "946f8a4fcd4a490a900d1cefdd9f89dd": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ab5415ee8f8940e894e48ad5f0f0b564": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "1af9913925b0426c92769f0c031f20ed": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "58e9c158205f43cb9d6323e3dbffce87": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "66cb3f30d59140508a100b82389f1a2c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "04e290bcbd7448a1b2df9b0624f01b67": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d3453f32a0fe4cf6b9518892d6c7b633": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "8e1b70e8c04f483f8bfaa0aeee91fc7b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "VBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "VBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "VBoxView", + "box_style": "", + "children": [], + "layout": "IPY_MODEL_851db24d8a9c48fcae9b9722ccde17f7" + } + }, + "92bbd1688aad426787beb03c64ff376d": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_0682113a4aa7467a96de1c1bc67c5e12", + "placeholder": "​", + "style": "IPY_MODEL_ee72035aa2284875a53e31944c8fef55", + "value": "

Copy a token from your Hugging Face\ntokens page and paste it below.
Immediately click login after copying\nyour token or it might be stored in plain text in this notebook file.
" + } + }, + "84365d8a62b1425c88c36502955d69e0": { + "model_module": "@jupyter-widgets/controls", + "model_name": "PasswordModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "PasswordModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "PasswordView", + "continuous_update": true, + "description": "Token:", + "description_tooltip": null, + "disabled": false, + "layout": "IPY_MODEL_1ecc478206884bb88b4513fedbdbe582", + "placeholder": "​", + "style": "IPY_MODEL_043221d1db184078a603cabfad55ad05", + "value": "" + } + }, + "a270854b78344917b8fe93a4908ff4c5": { + "model_module": "@jupyter-widgets/controls", + "model_name": "CheckboxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "CheckboxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "CheckboxView", + "description": "Add token as git credential?", + "description_tooltip": null, + "disabled": false, + "indent": true, + "layout": "IPY_MODEL_119c4e28ab6a402497dcdf7d722fd3c2", + "style": "IPY_MODEL_62337f40579d422e98cd19cbbd836408", + "value": true + } + }, + "e21d3b7feda346a38411c1210cbb45ec": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ButtonModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ButtonModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ButtonView", + "button_style": "", + "description": "Login", + "disabled": false, + "icon": "", + "layout": "IPY_MODEL_9ebf3f78bcb04484a5da2c751445b3b5", + "style": "IPY_MODEL_70d43337effd4e0fa1f7d3da5e126fed", + "tooltip": "" + } + }, + "86ee3825be1c4252bd38e4b977e746d9": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e7bec908fb064220892c5e931540511f", + "placeholder": "​", + "style": "IPY_MODEL_2b50423c03a64935b0d00cf2fb8ea30a", + "value": "\nPro Tip: If you don't already have one, you can create a dedicated\n'notebooks' token with 'write' access, that you can then easily reuse for all\nnotebooks.
" + } + }, + "851db24d8a9c48fcae9b9722ccde17f7": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": "center", + "align_self": null, + "border": null, + "bottom": null, + "display": "flex", + "flex": null, + "flex_flow": "column", + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": "50%" + } + }, + "0682113a4aa7467a96de1c1bc67c5e12": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ee72035aa2284875a53e31944c8fef55": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "1ecc478206884bb88b4513fedbdbe582": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "043221d1db184078a603cabfad55ad05": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "119c4e28ab6a402497dcdf7d722fd3c2": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "62337f40579d422e98cd19cbbd836408": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "9ebf3f78bcb04484a5da2c751445b3b5": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "70d43337effd4e0fa1f7d3da5e126fed": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ButtonStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ButtonStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "button_color": null, + "font_weight": "" + } + }, + "e7bec908fb064220892c5e931540511f": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "2b50423c03a64935b0d00cf2fb8ea30a": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "0c65e6b3fa894fdc864989e3e21ad491": { + "model_module": "@jupyter-widgets/controls", + "model_name": "LabelModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "LabelModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "LabelView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_dc574c859a8c493bb60f87278a877b3e", + "placeholder": "​", + "style": "IPY_MODEL_9bbf2ad5bb0242ba971eaddc00119001", + "value": "Connecting..." + } + }, + "dc574c859a8c493bb60f87278a877b3e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9bbf2ad5bb0242ba971eaddc00119001": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file