Delete pretrained_model

pretrained_model/README.md

@@ -1,204 +0,0 @@
----
-library_name: peft
-tags:
-- base_model:adapter:/fsx/caroline_pascal/models/so101
-- lora
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.21.0

pretrained_model/adapter_config.json

@@ -1,51 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": {
-    "base_model_class": "Flux3Policy",
-    "parent_library": "lerobot.policies.flux3.modeling_flux3"
-  },
-  "base_model_name_or_path": "/fsx/caroline_pascal/models/so101",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": false,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "kasa_config": null,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.0,
-  "lora_ga_config": null,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "dit.emb_in.action",
-    "dit.emb_in.action_cond",
-    "dit.final_layer.action",
-    "dit.final_layer.action_cond"
-  ],
-  "monteclora_config": null,
-  "peft_type": "LORA",
-  "peft_version": "0.21.0",
-  "qalora_group_size": 16,
-  "r": 32,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": ".*\\.(q_proj|k_proj|v_proj|attn_out|mlp_in|mlp_out)",
-  "target_parameters": null,
-  "task_type": null,
-  "trainable_token_indices": null,
-  "use_bdlora": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false,
-  "velora_config": null
-}

pretrained_model/adapter_model.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f3b2864bbe5c195833e219c054ac237c8ea241780df4e181403f285cd4eb786b
-size 471701520

pretrained_model/config.json

@@ -1,142 +0,0 @@
-{
-    "type": "flux3",
-    "n_obs_steps": 1,
-    "input_features": {
-        "observation.images.scene": {
-            "type": "VISUAL",
-            "shape": [
-                3,
-                256,
-                256
-            ]
-        },
-        "observation.images.wrist": {
-            "type": "VISUAL",
-            "shape": [
-                3,
-                256,
-                256
-            ]
-        },
-        "observation.state": {
-            "type": "STATE",
-            "shape": [
-                6
-            ]
-        }
-    },
-    "output_features": {
-        "action": {
-            "type": "ACTION",
-            "shape": [
-                6
-            ]
-        }
-    },
-    "device": "cuda",
-    "use_amp": false,
-    "use_peft": true,
-    "push_to_hub": false,
-    "repo_id": null,
-    "private": null,
-    "tags": null,
-    "license": null,
-    "pretrained_path": "/fsx/caroline_pascal/models/so101",
-    "pretrained_revision": null,
-    "conditioning": "history",
-    "packer": null,
-    "condition_on_past_actions": false,
-    "action_representation": "delta",
-    "delta_absolute_dims": [
-        -1
-    ],
-    "text_fixed_length": 320,
-    "video_position_fps": 24.0,
-    "history_snapshots": 1,
-    "normalization_stats": null,
-    "normalization_clip": 6.0,
-    "separate_timesteps": true,
-    "video_logit_mean": 1.08,
-    "video_logit_std": 1.0,
-    "conditioning_noise_max": 0.2,
-    "loss_reduction": "modalities",
-    "action_channel_weights": [
-        1.0,
-        1.0,
-        1.0,
-        1.0,
-        1.0,
-        2.0
-    ],
-    "gradient_checkpointing": true,
-    "chunk_size": 32,
-    "n_action_steps": 30,
-    "fps": 30.0,
-    "camera_layout": "side_by_side",
-    "camera_keys": [
-        "observation.images.scene",
-        "observation.images.wrist"
-    ],
-    "canvas_hw": [
-        256,
-        512
-    ],
-    "action_modality": "action",
-    "action_scale": 2.0,
-    "gripper_flip_dims": [],
-    "use_relative_actions": false,
-    "relative_exclude_joints": [
-        "gripper"
-    ],
-    "action_feature_names": [
-        "shoulder_pan.pos",
-        "shoulder_lift.pos",
-        "elbow_flex.pos",
-        "wrist_flex.pos",
-        "wrist_roll.pos",
-        "gripper.pos"
-    ],
-    "normalization_mapping": {
-        "VISUAL": "IDENTITY",
-        "STATE": "IDENTITY",
-        "ACTION": "IDENTITY"
-    },
-    "trunk_weights": null,
-    "video_vae_id": "/fsx/caroline_pascal/models/so101/components/video_vae.safetensors",
-    "text_encoder_id": "/fsx/caroline_pascal/models/so101/components/text_encoder",
-    "dit_config": null,
-    "content_streams": [
-        "video",
-        "video_cond"
-    ],
-    "attn_mode": "torch",
-    "dtype": "bfloat16",
-    "compile_model": false,
-    "sampler": "cosmos_unipc",
-    "num_inference_steps": 4,
-    "guidance_scale": 3.0,
-    "guidance_scale_action": null,
-    "sampler_shift": 5.0,
-    "inference_seed": 0,
-    "head_init_seed": 0,
-    "train_timestep_width": 0.75,
-    "train_timestep_shift": 42.0,
-    "action_loss_weight": 0.5,
-    "video_loss_weight": 1.0,
-    "caption_dropout": 0.1,
-    "augment": false,
-    "optimizer_lr": 0.0001,
-    "optimizer_lr_heads_multiplier": 5.0,
-    "optimizer_betas": [
-        0.9,
-        0.999
-    ],
-    "optimizer_eps": 1e-08,
-    "optimizer_weight_decay": 0.0,
-    "optimizer_grad_clip_norm": 1.0,
-    "scheduler_freeze_backbone_steps": 0,
-    "scheduler_warmup_steps": 0,
-    "scheduler_warmup_steps_heads": 0,
-    "scheduler_decay_steps": 0,
-    "scheduler_cooldown_steps": 0
-}

pretrained_model/policy_postprocessor.json

@@ -1,24 +0,0 @@
-{
-  "name": "policy_postprocessor",
-  "steps": [
-    {
-      "registry_name": "flux3_action_history_unnormalizer",
-      "config": {
-        "action_dim": 6,
-        "action_representation": "delta",
-        "absolute_dims": [
-          -1
-        ],
-        "normalization_clip": 6.0
-      },
-      "state_file": "policy_postprocessor_step_0_flux3_action_history_unnormalizer.safetensors"
-    },
-    {
-      "registry_name": "device_processor",
-      "config": {
-        "device": "cpu",
-        "float_dtype": null
-      }
-    }
-  ]
-}

pretrained_model/policy_postprocessor_step_0_flux3_action_history_unnormalizer.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:96a5f8de8c33db904ba2cfadeac7a4477f64bca1e0a5bba8da6f4ed576ff4331
-size 184

pretrained_model/policy_preprocessor.json

@@ -1,44 +0,0 @@
-{
-  "name": "policy_preprocessor",
-  "steps": [
-    {
-      "registry_name": "rename_observations_processor",
-      "config": {
-        "rename_map": {
-          "observation.images.on_top": "observation.images.scene",
-          "observation.images.camera_on_follower": "observation.images.wrist"
-        }
-      }
-    },
-    {
-      "registry_name": "to_batch_processor",
-      "config": {}
-    },
-    {
-      "registry_name": "device_processor",
-      "config": {
-        "device": "cuda",
-        "float_dtype": null
-      }
-    },
-    {
-      "registry_name": "flux3_action_history_normalizer",
-      "config": {
-        "action_dim": 6,
-        "action_representation": "delta",
-        "absolute_dims": [
-          -1
-        ],
-        "normalization_clip": 6.0,
-        "n_obs_steps": 1,
-        "chunk_size": 32,
-        "camera_keys": [
-          "observation.images.scene",
-          "observation.images.wrist"
-        ],
-        "condition_on_past_actions": false
-      },
-      "state_file": "policy_preprocessor_step_3_flux3_action_history_normalizer.safetensors"
-    }
-  ]
-}

pretrained_model/policy_preprocessor_step_3_flux3_action_history_normalizer.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:fb0b16f0b7909cb2604c089e1fdda2caf34d7ea168574594a5f5cf673d326d97
-size 360

pretrained_model/train_config.json

@@ -1,356 +0,0 @@
-{
-    "dataset": {
-        "repo_id": "cetiennec/yellow_on_purple_super",
-        "repo_type": "dataset",
-        "root": null,
-        "episodes": null,
-        "exclude_episodes": null,
-        "image_transforms": {
-            "enable": false,
-            "max_num_transforms": 3,
-            "random_order": false,
-            "tfs": {
-                "brightness": {
-                    "weight": 1.0,
-                    "type": "ColorJitter",
-                    "kwargs": {
-                        "brightness": [
-                            0.8,
-                            1.2
-                        ]
-                    }
-                },
-                "contrast": {
-                    "weight": 1.0,
-                    "type": "ColorJitter",
-                    "kwargs": {
-                        "contrast": [
-                            0.8,
-                            1.2
-                        ]
-                    }
-                },
-                "saturation": {
-                    "weight": 1.0,
-                    "type": "ColorJitter",
-                    "kwargs": {
-                        "saturation": [
-                            0.5,
-                            1.5
-                        ]
-                    }
-                },
-                "hue": {
-                    "weight": 1.0,
-                    "type": "ColorJitter",
-                    "kwargs": {
-                        "hue": [
-                            -0.05,
-                            0.05
-                        ]
-                    }
-                },
-                "sharpness": {
-                    "weight": 1.0,
-                    "type": "SharpnessJitter",
-                    "kwargs": {
-                        "sharpness": [
-                            0.5,
-                            1.5
-                        ]
-                    }
-                },
-                "affine": {
-                    "weight": 1.0,
-                    "type": "RandomAffine",
-                    "kwargs": {
-                        "degrees": [
-                            -5.0,
-                            5.0
-                        ],
-                        "translate": [
-                            0.05,
-                            0.05
-                        ]
-                    }
-                }
-            }
-        },
-        "revision": null,
-        "use_imagenet_stats": true,
-        "video_backend": "torchcodec",
-        "return_uint8": false,
-        "depth_output_unit": "mm",
-        "streaming": false,
-        "eval_split": 0.2
-    },
-    "env": null,
-    "policy": {
-        "type": "flux3",
-        "n_obs_steps": 1,
-        "input_features": {
-            "observation.images.scene": {
-                "type": "VISUAL",
-                "shape": [
-                    3,
-                    256,
-                    256
-                ]
-            },
-            "observation.images.wrist": {
-                "type": "VISUAL",
-                "shape": [
-                    3,
-                    256,
-                    256
-                ]
-            },
-            "observation.state": {
-                "type": "STATE",
-                "shape": [
-                    6
-                ]
-            }
-        },
-        "output_features": {
-            "action": {
-                "type": "ACTION",
-                "shape": [
-                    6
-                ]
-            }
-        },
-        "device": "cuda",
-        "use_amp": false,
-        "use_peft": true,
-        "push_to_hub": false,
-        "repo_id": null,
-        "private": null,
-        "tags": null,
-        "license": null,
-        "pretrained_path": "/fsx/caroline_pascal/models/so101",
-        "pretrained_revision": null,
-        "conditioning": "history",
-        "packer": null,
-        "condition_on_past_actions": false,
-        "action_representation": "delta",
-        "delta_absolute_dims": [
-            -1
-        ],
-        "text_fixed_length": 320,
-        "video_position_fps": 24.0,
-        "history_snapshots": 1,
-        "normalization_stats": null,
-        "normalization_clip": 6.0,
-        "separate_timesteps": true,
-        "video_logit_mean": 1.08,
-        "video_logit_std": 1.0,
-        "conditioning_noise_max": 0.2,
-        "loss_reduction": "modalities",
-        "action_channel_weights": [
-            1.0,
-            1.0,
-            1.0,
-            1.0,
-            1.0,
-            2.0
-        ],
-        "gradient_checkpointing": true,
-        "chunk_size": 32,
-        "n_action_steps": 30,
-        "fps": 30.0,
-        "camera_layout": "side_by_side",
-        "camera_keys": [
-            "observation.images.scene",
-            "observation.images.wrist"
-        ],
-        "canvas_hw": [
-            256,
-            512
-        ],
-        "action_modality": "action",
-        "action_scale": 2.0,
-        "gripper_flip_dims": [],
-        "use_relative_actions": false,
-        "relative_exclude_joints": [
-            "gripper"
-        ],
-        "action_feature_names": [
-            "shoulder_pan.pos",
-            "shoulder_lift.pos",
-            "elbow_flex.pos",
-            "wrist_flex.pos",
-            "wrist_roll.pos",
-            "gripper.pos"
-        ],
-        "normalization_mapping": {
-            "VISUAL": "IDENTITY",
-            "STATE": "IDENTITY",
-            "ACTION": "IDENTITY"
-        },
-        "trunk_weights": null,
-        "video_vae_id": "/fsx/caroline_pascal/models/so101/components/video_vae.safetensors",
-        "text_encoder_id": "/fsx/caroline_pascal/models/so101/components/text_encoder",
-        "dit_config": null,
-        "content_streams": [
-            "video",
-            "video_cond"
-        ],
-        "attn_mode": "torch",
-        "dtype": "bfloat16",
-        "compile_model": false,
-        "sampler": "cosmos_unipc",
-        "num_inference_steps": 4,
-        "guidance_scale": 3.0,
-        "guidance_scale_action": null,
-        "sampler_shift": 5.0,
-        "inference_seed": 0,
-        "head_init_seed": 0,
-        "train_timestep_width": 0.75,
-        "train_timestep_shift": 42.0,
-        "action_loss_weight": 0.5,
-        "video_loss_weight": 1.0,
-        "caption_dropout": 0.1,
-        "augment": false,
-        "optimizer_lr": 0.0001,
-        "optimizer_lr_heads_multiplier": 5.0,
-        "optimizer_betas": [
-            0.9,
-            0.999
-        ],
-        "optimizer_eps": 1e-08,
-        "optimizer_weight_decay": 0.0,
-        "optimizer_grad_clip_norm": 1.0,
-        "scheduler_freeze_backbone_steps": 0,
-        "scheduler_warmup_steps": 0,
-        "scheduler_warmup_steps_heads": 0,
-        "scheduler_decay_steps": 0,
-        "scheduler_cooldown_steps": 0
-    },
-    "reward_model": null,
-    "output_dir": "/fsx/caroline_pascal/outputs/yellow_on_purple_lora_4gpu",
-    "job_name": "flux3",
-    "resume": false,
-    "seed": 42,
-    "cudnn_deterministic": false,
-    "num_workers": 8,
-    "batch_size": 2,
-    "prefetch_factor": 4,
-    "persistent_workers": true,
-    "dataloader_multiprocessing_context": "spawn",
-    "steps": 10000,
-    "env_eval_freq": 0,
-    "log_freq": 20,
-    "eval_steps": 500,
-    "max_eval_samples": 0,
-    "tolerance_s": 0.0001,
-    "save_checkpoint": true,
-    "save_freq": 500,
-    "checkpoint_format": "safetensors",
-    "use_policy_training_preset": true,
-    "optimizer": {
-        "type": "adamw",
-        "lr": 0.0001,
-        "weight_decay": 0.0,
-        "grad_clip_norm": 1.0,
-        "betas": [
-            0.9,
-            0.999
-        ],
-        "eps": 1e-08
-    },
-    "scheduler": {
-        "type": "frozen_warmup_constant",
-        "num_warmup_steps": 0,
-        "freeze_steps": 0,
-        "warmup_steps_heads": 0,
-        "decay_steps": 0,
-        "cooldown_steps": 0
-    },
-    "parallelism": {
-        "dp_replicate": 4,
-        "dp_shard": 1,
-        "context_parallel": {
-            "ring_degree": 1,
-            "ulysses_degree": 1
-        },
-        "cfg_parallel": 1
-    },
-    "accelerator": {
-        "mixed_precision": "bf16",
-        "gradient_accumulation": {
-            "steps": 1
-        },
-        "fsdp": {
-            "reshard_after_forward": true,
-            "wrap_modules": null,
-            "min_num_params": null,
-            "cpu_offload": false,
-            "ignored_modules": null
-        },
-        "ddp": {
-            "find_unused_parameters": true,
-            "gradient_as_bucket_view": false,
-            "static_graph": false
-        },
-        "compile": {
-            "enabled": false,
-            "backend": "inductor",
-            "mode": null,
-            "regional": true
-        },
-        "activation_checkpointing": {
-            "mode": "none"
-        }
-    },
-    "eval": {
-        "n_episodes": 50,
-        "batch_size": 50,
-        "use_async_envs": true,
-        "recording": false,
-        "recording_repo_id": null,
-        "recording_private": false
-    },
-    "ema": {
-        "enable": true,
-        "decay": 0.999,
-        "update_after_step": 0,
-        "inv_gamma": 1.0,
-        "power": 0.75,
-        "min_decay": 0.0,
-        "max_decay": 0.9999,
-        "use_for_eval": true
-    },
-    "wandb": {
-        "enable": true,
-        "disable_artifact": false,
-        "project": "flux3-so101",
-        "entity": null,
-        "notes": null,
-        "run_id": "yrahmxpq",
-        "mode": null,
-        "add_tags": true
-    },
-    "peft": {
-        "target_modules": null,
-        "full_training_modules": null,
-        "method_type": "LORA",
-        "init_type": null,
-        "r": 32,
-        "lora_alpha": 32
-    },
-    "job": {
-        "target": null,
-        "image": "huggingface/lerobot-gpu:latest",
-        "timeout": "2d",
-        "detach": false,
-        "tags": []
-    },
-    "save_checkpoint_to_hub": false,
-    "sample_weighting": null,
-    "rename_map": {
-        "observation.images.on_top": "observation.images.scene",
-        "observation.images.camera_on_follower": "observation.images.wrist"
-    },
-    "checkpoint_path": null
-}