anonymous commited on
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,36 +1,114 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
|
| 5 |
-
# Open-Source Demonstration Reproduction
|
| 6 |
|
| 7 |
-
|
| 8 |
-
**MIRA: Medical Time Series Foundation Model for Real-World Health Data**
|
| 9 |
-
(arXiv: https://arxiv.org/abs/2506.07584).
|
| 10 |
|
| 11 |
-
|
| 12 |
-
|
|
|
|
| 13 |
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
This demonstration
|
| 19 |
|
| 20 |
-
-
|
| 21 |
-
Enables geometric time encoding for irregularly sampled signals. The **CT-RoPE temporal unit is set to 0.1**.
|
| 22 |
|
| 23 |
-
|
| 24 |
-
Allows experts to specialize on low- vs. high-frequency clinical patterns.
|
| 25 |
|
| 26 |
-
- **
|
| 27 |
-
|
| 28 |
|
| 29 |
---
|
| 30 |
|
| 31 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
-
-
|
| 34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
|
| 36 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
license: mit
|
| 5 |
+
tags:
|
| 6 |
+
- time-series
|
| 7 |
+
- healthcare
|
| 8 |
+
- medical
|
| 9 |
+
- foundation-model
|
| 10 |
+
- mira
|
| 11 |
+
datasets:
|
| 12 |
+
- public-datasets
|
| 13 |
+
model_name: MIRA-Large (Demo)
|
| 14 |
+
base_model: microsoft/MIRA
|
| 15 |
---
|
| 16 |
|
| 17 |
+
# MIRA-Large: Open-Source Demonstration Reproduction
|
| 18 |
|
| 19 |
+
<div align="center">
|
|
|
|
|
|
|
| 20 |
|
| 21 |
+
[](https://arxiv.org/abs/2506.07584)
|
| 22 |
+
[](https://github.com/microsoft/MIRA)
|
| 23 |
+
[]()
|
| 24 |
|
| 25 |
+
</div>
|
| 26 |
+
|
| 27 |
+
## Model Summary
|
| 28 |
+
|
| 29 |
+
This repository hosts the **Large-scale configuration** of the open-source demonstration for **MIRA: Medical Time Series Foundation Model for Real-World Health Data**.
|
| 30 |
|
| 31 |
+
The goal of this release is to offer a **fully accessible and reproducible version** of the MIRA framework. Unlike the internal version described in the paper which utilizes private clinical data, this model was trained **exclusively on publicly available time-series datasets**. It is designed to showcase the core architectural innovations of MIRA—such as CT-RoPE and Frequency-Specialized MoE—without compromising patient privacy.
|
| 32 |
|
| 33 |
+
**Disclaimer:** This is a demonstration reproduction. While it shares the same "Large" architecture and mechanisms as the original MIRA, its performance on downstream clinical tasks may differ due to the restriction to public training data.
|
| 34 |
|
| 35 |
+
---
|
|
|
|
| 36 |
|
| 37 |
+
## Key Resources
|
|
|
|
| 38 |
|
| 39 |
+
- **📄 Paper:** [MIRA: Medical Time Series Foundation Model for Real-World Health Data (arXiv)](https://arxiv.org/abs/2506.07584)
|
| 40 |
+
- **💻 Official Code:** [github.com/microsoft/MIRA](https://github.com/microsoft/MIRA)
|
| 41 |
|
| 42 |
---
|
| 43 |
|
| 44 |
+
## Technical Specifications
|
| 45 |
+
|
| 46 |
+
### Architecture Highlights
|
| 47 |
+
This demonstration reproduces the key components of the MIRA architecture:
|
| 48 |
+
|
| 49 |
+
- **Continuous-Time Rotary Positional Encoding (CT-RoPE):** Enables geometric time encoding specifically designed for irregularly sampled medical signals.
|
| 50 |
+
*Configuration:* The temporal unit is set to **0.1**.
|
| 51 |
|
| 52 |
+
- **Frequency-Specialized Mixture-of-Experts (MoE):** A routing mechanism that allows experts to specialize in distinguishing between low-frequency trends and high-frequency clinical anomalies.
|
| 53 |
+
|
| 54 |
+
- **Neural ODE–Based Continuous Dynamics Modeling:** Captures long-range physiological trajectories using continuous-time formulations, making it robust to missing data.
|
| 55 |
+
|
| 56 |
+
### Model Configuration
|
| 57 |
+
- **Size:** Large
|
| 58 |
+
- **Training Data:** Publicly available medical time-series datasets only.
|
| 59 |
|
| 60 |
---
|
| 61 |
+
|
| 62 |
+
## Usage
|
| 63 |
+
|
| 64 |
+
You can load this model using the official codebase. Please ensure you have cloned the repository first.
|
| 65 |
+
|
| 66 |
+
```python
|
| 67 |
+
import torch
|
| 68 |
+
from MIRA.mira.models.modeling_mira import MIRAForPrediction
|
| 69 |
+
from MIRA.mira.models.utils_time_normalization import normalize_time_for_ctrope
|
| 70 |
+
|
| 71 |
+
# Load the pre-trained model
|
| 72 |
+
model = MIRAForPrediction.from_pretrained(ckpt_path).cuda()
|
| 73 |
+
model.eval()
|
| 74 |
+
|
| 75 |
+
# Example inference (pseudo-code)
|
| 76 |
+
device = next(model.parameters()).device
|
| 77 |
+
hist_vals = hist_vals.to(device)
|
| 78 |
+
hist_times = hist_times.to(device)
|
| 79 |
+
future_times = future_times.to(device)
|
| 80 |
+
|
| 81 |
+
cur_vals = hist_vals.clone()
|
| 82 |
+
cur_times = hist_times.clone()
|
| 83 |
+
|
| 84 |
+
preds_norm = []
|
| 85 |
+
|
| 86 |
+
for i in range(P):
|
| 87 |
+
|
| 88 |
+
# model input
|
| 89 |
+
inp_vals = cur_vals.unsqueeze(-1) # [1, L, 1]
|
| 90 |
+
inp_times = cur_times # [1, L]
|
| 91 |
+
|
| 92 |
+
with torch.no_grad():
|
| 93 |
+
out = model(
|
| 94 |
+
input_ids=inp_vals,
|
| 95 |
+
time_values=inp_times,
|
| 96 |
+
next_target_time_values=None, # no ODE for 1-step
|
| 97 |
+
return_dict=True,
|
| 98 |
+
)
|
| 99 |
+
|
| 100 |
+
next_norm = out.logits[:, -1, :] # [1, 1]
|
| 101 |
+
preds_norm.append(next_norm.squeeze(0))
|
| 102 |
+
|
| 103 |
+
next_t = future_times[:, i:i+1]
|
| 104 |
+
|
| 105 |
+
cur_vals = torch.cat([cur_vals, next_norm], dim=1)
|
| 106 |
+
cur_times = torch.cat([cur_times, next_t], dim=1)
|
| 107 |
+
|
| 108 |
+
|
| 109 |
+
preds_norm = torch.stack(preds_norm, dim=1) # [1, P]
|
| 110 |
+
|
| 111 |
+
preds = preds_norm * std[:, :, :] + mean[:, :, :]
|
| 112 |
+
preds = preds.squeeze(0)
|
| 113 |
+
print(preds)
|
| 114 |
+
```
|