LogicDeposit/adamu_act_pickplace

LogicMimic ACT Policy - AdamU Left Arm Pick & Place (HD720)

This is a trained Action Chunking with Transformers (ACT) policy for the AdamU dual-arm humanoid robot, specifically controlling the left arm for pick and place tasks. The model is trained using the LogicMimic framework, which is built on top of Hugging Face's LeRobot.

Model Details

• Architecture: Action Chunking with Transformers (ACT)
• Vision Backbone: ResNet-18 (ImageNet1K V1 pre-trained)
• Framework: LeRobot / PyTorch
• Device Training: CUDA
• Input Types: Visual (Images) + State (Joint Positions)
• Output Types: Action (Joint Positions)
• Training Checkpoint: 100,000 steps

Hyperparameters

• Chunk Size: 50
• Action Steps (n_action_steps): 50
• Optimizer LR: 1e-05
• Backbone LR: 1e-05
• Weight Decay: 0.0001
• KL Weight: 10.0
• Dropout: 0.1

Model Configuration

• Transformer Encoder Layers: 4
• Transformer Decoder Layers: 1
• Latent Dimension (VAE): 32
• Model Dimension: 512
• Attention Heads: 8
• Normalization: MEAN_STD for Visual, State, and Action

Intended Use

This model is intended to be deployed on the Jetson Orin NX (or a workstation) as part of the LogicMimic inference pipeline. It sends control commands over the ZMQ-DDS bridge to the AdamU robot hardware.

• Robot: AdamU Dual-Arm Humanoid
• Task: Left Arm Pick and Place
• Hardware Requirement: Tested on Jetson Orin NX with ~50ms inference time

Training Data

The model was trained on a local dataset derived from MCAP recordings.

• Dataset Path: data/processed/adam_u_left_pp_20260130/adam_u_left_pp_20260130_v1_HD720
• Total Episodes: 71
• Total Frames: 9149
• Frame Rate (FPS): 8 Hz
• Camera Resolution: 640x360 (ZED Mini)

Observation Space

• observation.images.zedm: [3, 360, 640] (RGB Video)
• observation.state: [8] (Float32 Joint Positions)
  • Left Shoulder Pitch, Roll, Yaw
  • Left Elbow
  • Left Wrist Yaw, Pitch, Roll
  • Left Hand Index Finger

Action Space

• action: [8] (Float32 Joint Commands)
  • Dimensions match the observation state exactly.

Deployment

To deploy this policy, use the LogicMimic inference pipeline.

Synchronous Inference (Single Machine)

# Inference on Macbook (MPS)
python scripts/launch_sync.py \
    --model_path <MODEL_PATH> \
    --dataset_path <DATASET_PATH> \
    --camera_source zmq \
    --zmq_camera_host <NX_IP> \
    --device mps\
    --robot_ip <NUC_IP> \
    --fps 10 \
    --task 'pick the bottle and place into the box' \
    --temporal_ensemble_coeff 0.01

Asynchronous Inference (Recommended)

# Start Inference Server (on GPU Workstation or Jetson)
python scripts/launch_async_server.py \
  --host 0.0.0.0 \
  --port 8000 \
  --fps 10

# Start Robot Client (on Jetson connected to Robot NUC)
python scripts/launch_async_client.py \
  --model_path <MODEL_PATH> \
  --robot_ip <NUC_IP> \
  --server_address <SERVER_IP>:8000 \
  --model_path <MODEL_PATH> \
  --dataset_path <DATASET_PATH> \
  --device mps  \
  --policy_type act \
  --fps 10