README.md

---
license: mit
language:
  - en
tags:
  - model-predictive-control
  - mpc
  - pytorch
  - aerospace
  - flight-control
  - boeing-747
  - learned-dynamics
  - neural-network
  - continuous-control
  - gymnasium
library_name: tensoraerospace
pipeline_tag: reinforcement-learning
model-index:
  - name: MPC-OneStepMLP-B747-PitchControl
    results:
      - task:
          type: model-predictive-control
          name: Pitch Angle Tracking Control
        dataset:
          type: custom
          name: Boeing 747 Longitudinal Dynamics Simulation
        metrics:
          - type: overshoot
            value: 0.27
            name: Overshoot (%)
          - type: settling_time
            value: 1.40
            name: Settling Time (s)
          - type: rise_time
            value: 0.80
            name: Rise Time (s)
          - type: peak_time
            value: 1.70
            name: Peak Time (s)
          - type: static_error
            value: 0.038
            name: Static Error
          - type: oscillation_count
            value: 5
            name: Oscillation Count
          - type: performance_index
            value: 72.62
            name: Performance Index
          - type: iae
            value: 41.25
            name: IAE
          - type: ise
            value: 147.43
            name: ISE
          - type: itae
            value: 33.99
            name: ITAE
          - type: dynamics_loss
            value: 8.69e-6
            name: Dynamics Model MSE Loss
---

# TorchMPC with Learned Dynamics (OneStepMLP) for Boeing 747 Pitch Angle Control

<div align="center">

![TensorAeroSpace](https://raw.githubusercontent.com/TensorAeroSpace/TensorAeroSpace/main/img/logo-no-background.png)

**Model Predictive Control with Neural Network Dynamics for Longitudinal Aircraft Control**

[![TensorAeroSpace](https://img.shields.io/badge/%F0%9F%9A%80-TensorAeroSpace-blue)](https://github.com/TensorAeroSpace/TensorAeroSpace)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![PyTorch](https://img.shields.io/badge/PyTorch-2.0+-red.svg)](https://pytorch.org/)

</div>

## Model Description

This model combines **Model Predictive Control (MPC)** with a **learned neural network dynamics model (OneStepMLP)** to control the pitch angle (θ) of a **Boeing 747** aircraft in a longitudinal flight dynamics simulation. The approach first learns the aircraft dynamics from exploration data, then uses gradient-based MPC optimization to compute optimal control actions for reference tracking.

![image](https://cdn-uploads.huggingface.co/production/uploads/602bf7c9c4f8038e9a1e0a65/yZYzPcK_PU7uFh_j6pZii.png)

### Key Features

- **Data-driven dynamics**: Learns one-step transition model f(x, u) → Δx from exploration data
- **Gradient-based MPC**: Differentiable optimization through learned dynamics
- **Step response optimization**: Custom cost function for overshoot/settling time minimization
- **Warm-starting**: Efficient action sequence initialization across timesteps

### Intended Uses

- **Primary Use**: Automatic pitch angle tracking and stabilization for Boeing 747 aircraft simulation
- **Research Applications**: Benchmarking learning-based MPC algorithms for aerospace control systems
- **Educational**: Learning MPC concepts with neural network dynamics in aerospace applications
- **Hybrid Control**: Can be combined with analytical models for robust flight control

## Model Architecture

### Dynamics Model (OneStepMLP)

The dynamics model predicts state transitions using a multi-layer perceptron:

| Layer | Configuration |
|-------|---------------|
| Input | 5 (state_dim=4 + action_dim=1) |
| Hidden 1 | Linear(5, 256) + ReLU |
| Hidden 2 | Linear(256, 256) + ReLU |
| Output | Linear(256, 4) |
| Mode | Predict Δx (delta dynamics) |

**Total Parameters**: ~70K

### MPC Controller

| Parameter | Value |
|-----------|-------|
| Horizon | 20 steps |
| Iterations per step | 60 |
| Optimizer | Adam |
| MPC Learning Rate | 0.02 |
| Warm Start | Enabled |
| Track Best | Enabled |

### State Space

The observation vector consists of 4 states representing the longitudinal dynamics:

| Index | State | Description | Units |
|-------|-------|-------------|-------|
| 0 | u | Forward velocity perturbation | m/s (rad internally) |
| 1 | w | Vertical velocity perturbation | m/s (rad internally) |
| 2 | q | Pitch rate | rad/s |
| 3 | θ | Pitch angle (tracking target) | rad |

### Action Space

| Dimension | Description | Range | Rate Limit |
|-----------|-------------|-------|------------|
| 1 | Elevator deflection | [-25°, 25°] | ±10°/step |

## Training Details

### Data Collection

| Parameter | Value |
|-----------|-------|
| Collection Episodes | 1500 |
| Transitions Collected | 297,000 |
| Exploration Strategy | Multi-signal exploration |
| Signal Types | random_steps, unit_step, multi_step, ramp, sinusoid, multisine, chirp, square_wave, triangular_wave, sawtooth, doublet, pulse, gaussian_pulse, exponential, damped_sinusoid |
| Action Amplitude | 100% of action space |

### Dynamics Training

| Parameter | Value |
|-----------|-------|
| Epochs | 120 |
| Batch Size | 2048 |
| Learning Rate | 1e-4 |
| Loss Function | MSE |
| Final Loss | 8.69e-6 |
| Normalization | Enabled |

### MPC Cost Weights

| Weight | Value | Description |
|--------|-------|-------------|
| W_θ | 2000.0 | Pitch tracking weight |
| W_q | 0.2 | Pitch rate weight |
| W_action | 0.01 | Control effort weight |
| W_Δu | 5.0 | Control rate weight |
| Terminal | 10.0 | Terminal cost multiplier |

### Step Response Cost Configuration

| Parameter | Value |
|-----------|-------|
| W_overshoot | 8,000 |
| W_settle | 8,000 |
| W_sse_steady | 40,000 |
| W_time | 800 |
| W_osc | 500 |
| W_jerk | 50 |
| Overshoot limit | 0.05° |
| Settle band | 0.10° |
| Settle time target | 1.0 s |

### Environment Configuration

| Parameter | Value |
|-----------|-------|
| Environment | `LinearLongitudinalB747-v0` |
| Time Step (dt) | 0.1 s |
| Episode Duration | 20 s |
| Initial State | [0, 0, 0, 0] |
| Reference Signal | Step function |
| Step Amplitude | 1.0° |
| Step Time | 5.0 s |

### Training Infrastructure

- **Hardware**: CUDA GPU (recommended) / CPU
- **Framework**: PyTorch 2.0+
- **Compile Mode**: reduce-overhead (CUDA only)

## Evaluation Results

### Performance Metrics

| Metric | Value |
|--------|-------|
| **Overshoot** | 0.27% |
| **Settling Time (±5%)** | 1.40 s |
| **Rise Time** | 0.80 s |
| **Peak Time** | 1.70 s |
| **Static Error** | 0.038 |
| **Oscillation Count** | 5 |
| **Performance Index** | 72.62 |
| **Damping Degree** | -0.002 |

### Integral Criteria

| Criterion | Value |
|-----------|-------|
| IAE (Integral Absolute Error) | 41.25 |
| ISE (Integral Squared Error) | 147.43 |
| ITAE (Integral Time-weighted Absolute Error) | 33.99 |

### Step Response Characteristics

The MPC controller demonstrates good step tracking performance with:
- ✅ Very low overshoot (~0.27%)
- ✅ Fast settling time (1.4s)
- ✅ Quick rise time (0.8s)
- ⚠️ Some oscillations (5 cycles)
- ⚠️ Small static error (0.038)

## Usage

### Installation

```bash
pip install tensoraerospace
```

### Quick Start

```python
import numpy as np
import gymnasium as gym
import torch
from tensoraerospace.signals.standart import unit_step
from tensoraerospace.agent.mpc import MPCAgent

def pick_device() -> str:
    if torch.cuda.is_available():
        return "cuda"
    if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
        return "mps"
    return "cpu"

# Setup environment
DT = 0.1
TN = 20.0
N_STEPS = int(TN / DT) + 1
T = np.arange(N_STEPS, dtype=np.float32) * DT

# Create step reference signal (1 degree step at t=5s)
reference_signal = unit_step(
    tp=T,
    degree=1.0,
    time_step=5.0,
    output_rad=True,
).reshape(1, -1)

env = gym.make(
    "LinearLongitudinalB747-v0",
    number_time_steps=N_STEPS,
    initial_state=np.array([[0.0], [0.0], [0.0], [0.0]], dtype=np.float32),
    reference_signal=reference_signal,
    dt=DT,
)

# Load pretrained agent
agent = MPCAgent.from_pretrained("TensorAeroSpace/torchmpc-mlp-b747-step-response")
agent.env = env
agent.to_device(pick_device())

# Run evaluation
_ = env.reset()
agent.reset()

ref_theta_rad = reference_signal[0]
x_ref = np.zeros((21, 4), dtype=np.float32)  # horizon + 1

for step in range(N_STEPS - 2):
    k = int(env.unwrapped.current_step)
    x0 = np.asarray(env.unwrapped.model.xt, dtype=np.float32).reshape(-1)
    
    # Set reference for horizon
    ref_k = float(ref_theta_rad[min(k, len(ref_theta_rad) - 1)])
    x_ref[:, 3] = ref_k
    
    action = agent.select_action(x0, x_ref=x_ref)
    obs, reward, terminated, truncated, info = env.step(action)
    
    if terminated or truncated:
        break
```

### Custom Dynamics Training

```python
# Collect exploration data
agent.collect_data(
    num_episodes=1500,
    max_steps=199,
    exploration="signals",
    signal_kinds=["random_steps", "sinusoid", "chirp", ...],
    dt=0.1,
    action_amplitude_frac=1.0,
)

# Train dynamics model
metrics = agent.train_dynamics(
    epochs=120,
    batch_size=2048,
    loss="mse",
)
print(f"Final dynamics loss: {metrics['loss']:.2e}")
```

## Comparison with Other Methods

| Method | Overshoot | Settling Time | Rise Time | Static Error |
|--------|-----------|---------------|-----------|--------------|
| **MPC-MLP** | 0.27% | 1.40 s | 0.80 s | 0.038 |
| DSAC | 0.99% | 0.40 s | 0.40 s | 0.0002 |
| PID (tuned) | ~5% | ~2.0 s | ~1.0 s | ~0 |

## Limitations

- **Fixed Aircraft Model**: Trained specifically on Boeing 747 longitudinal dynamics; may not generalize to other aircraft
- **Step Reference Focus**: Optimized for step reference tracking; performance on other signal types may vary
- **Simulation Gap**: Trained in simulation; real-world deployment would require additional validation
- **Computational Cost**: MPC optimization at each step requires more computation than pure RL policies
- **Linear Dynamics**: Based on linearized aircraft model around trim conditions
- **Some Oscillations**: The controller exhibits 5 oscillation cycles during settling

## Ethical Considerations

- **Not for Real Flight Control**: This model is for research and educational purposes only. It should NOT be used for actual aircraft control systems without extensive testing, certification, and regulatory approval.
- **Simulation Only**: All training and evaluation performed in simulation environments.

## Citation

If you use this model in your research, please cite:

```bibtex
@software{tensoraerospace2024,
  title = {TensorAeroSpace: Advanced Aerospace Control Systems \& Reinforcement Learning Framework},
  author = {TensorAeroSpace Team},
  year = {2024},
  url = {https://github.com/TensorAeroSpace/TensorAeroSpace},
  license = {MIT}
}
```

## Model Card Authors

TensorAeroSpace Team

## Model Card Contact

- **GitHub**: [TensorAeroSpace/TensorAeroSpace](https://github.com/TensorAeroSpace/TensorAeroSpace)
- **Documentation**: [tensoraerospace.readthedocs.io](https://tensoraerospace.readthedocs.io/)
- **Hugging Face**: [TensorAeroSpace](https://huggingface.co/TensorAeroSpace)
```