anonymous commited on
Commit
2d6ab56
·
verified ·
1 Parent(s): 07d2e23

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +96 -18
README.md CHANGED
@@ -1,36 +1,114 @@
1
  ---
 
 
2
  license: mit
 
 
 
 
 
 
 
 
 
 
3
  ---
4
 
5
- # Open-Source Demonstration Reproduction of MIRA
6
 
7
- This repository provides an **open-source demonstration reproduction** of
8
- **MIRA: Medical Time Series Foundation Model for Real-World Health Data**
9
- (arXiv: https://arxiv.org/abs/2506.07584).
10
 
11
- The goal of this demo is to offer a **fully accessible and reproducible version** of the MIRA framework, trained **exclusively on publicly available time-series datasets**, without using any proprietary or sensitive clinical data.
12
- It is designed to showcase the core ideas of MIRA rather than replicate the full-scale training described in the original paper.
 
13
 
14
- ---
 
 
 
 
15
 
16
- ## Overview
17
 
18
- This demonstration model reproduces the key architectural components of MIRA:
19
 
20
- - **Continuous-Time Rotary Positional Encoding (CT-RoPE)**
21
- Enables geometric time encoding for irregularly sampled signals. The **CT-RoPE temporal unit is set to 0.1**.
22
 
23
- - **Frequency-Specialized Mixture-of-Experts (MoE)**
24
- Allows experts to specialize on low- vs. high-frequency clinical patterns.
25
 
26
- - **Neural ODE–Based Continuous Dynamics Modeling**
27
- Captures long-range physiological trajectories with continuous-time formulations.
28
 
29
  ---
30
 
31
- ## Notes
 
 
 
 
 
 
32
 
33
- - This model is a **demonstration reproduction**, not the full-scale MIRA model.
34
- - Performance will differ from the original paper due to dataset differences.
 
 
 
 
 
35
 
36
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
  license: mit
5
+ tags:
6
+ - time-series
7
+ - healthcare
8
+ - medical
9
+ - foundation-model
10
+ - mira
11
+ datasets:
12
+ - public-datasets
13
+ model_name: MIRA-Large (Demo)
14
+ base_model: microsoft/MIRA
15
  ---
16
 
17
+ # MIRA-Large: Open-Source Demonstration Reproduction
18
 
19
+ <div align="center">
 
 
20
 
21
+ [![arXiv](https://img.shields.io/badge/arXiv-2506.07584-b31b1b.svg)](https://arxiv.org/abs/2506.07584)
22
+ [![GitHub](https://img.shields.io/badge/GitHub-microsoft%2FMIRA-blue.svg)](https://github.com/microsoft/MIRA)
23
+ [![Model Size](https://img.shields.io/badge/Model%20Size-Large-orange)]()
24
 
25
+ </div>
26
+
27
+ ## Model Summary
28
+
29
+ This repository hosts the **Large-scale configuration** of the open-source demonstration for **MIRA: Medical Time Series Foundation Model for Real-World Health Data**.
30
 
31
+ The goal of this release is to offer a **fully accessible and reproducible version** of the MIRA framework. Unlike the internal version described in the paper which utilizes private clinical data, this model was trained **exclusively on publicly available time-series datasets**. It is designed to showcase the core architectural innovations of MIRA—such as CT-RoPE and Frequency-Specialized MoE—without compromising patient privacy.
32
 
33
+ **Disclaimer:** This is a demonstration reproduction. While it shares the same "Large" architecture and mechanisms as the original MIRA, its performance on downstream clinical tasks may differ due to the restriction to public training data.
34
 
35
+ ---
 
36
 
37
+ ## Key Resources
 
38
 
39
+ - **📄 Paper:** [MIRA: Medical Time Series Foundation Model for Real-World Health Data (arXiv)](https://arxiv.org/abs/2506.07584)
40
+ - **💻 Official Code:** [github.com/microsoft/MIRA](https://github.com/microsoft/MIRA)
41
 
42
  ---
43
 
44
+ ## Technical Specifications
45
+
46
+ ### Architecture Highlights
47
+ This demonstration reproduces the key components of the MIRA architecture:
48
+
49
+ - **Continuous-Time Rotary Positional Encoding (CT-RoPE):** Enables geometric time encoding specifically designed for irregularly sampled medical signals.
50
+ *Configuration:* The temporal unit is set to **0.1**.
51
 
52
+ - **Frequency-Specialized Mixture-of-Experts (MoE):** A routing mechanism that allows experts to specialize in distinguishing between low-frequency trends and high-frequency clinical anomalies.
53
+
54
+ - **Neural ODE–Based Continuous Dynamics Modeling:** Captures long-range physiological trajectories using continuous-time formulations, making it robust to missing data.
55
+
56
+ ### Model Configuration
57
+ - **Size:** Large
58
+ - **Training Data:** Publicly available medical time-series datasets only.
59
 
60
  ---
61
+
62
+ ## Usage
63
+
64
+ You can load this model using the official codebase. Please ensure you have cloned the repository first.
65
+
66
+ ```python
67
+ import torch
68
+ from MIRA.mira.models.modeling_mira import MIRAForPrediction
69
+ from MIRA.mira.models.utils_time_normalization import normalize_time_for_ctrope
70
+
71
+ # Load the pre-trained model
72
+ model = MIRAForPrediction.from_pretrained(ckpt_path).cuda()
73
+ model.eval()
74
+
75
+ # Example inference (pseudo-code)
76
+ device = next(model.parameters()).device
77
+ hist_vals = hist_vals.to(device)
78
+ hist_times = hist_times.to(device)
79
+ future_times = future_times.to(device)
80
+
81
+ cur_vals = hist_vals.clone()
82
+ cur_times = hist_times.clone()
83
+
84
+ preds_norm = []
85
+
86
+ for i in range(P):
87
+
88
+ # model input
89
+ inp_vals = cur_vals.unsqueeze(-1) # [1, L, 1]
90
+ inp_times = cur_times # [1, L]
91
+
92
+ with torch.no_grad():
93
+ out = model(
94
+ input_ids=inp_vals,
95
+ time_values=inp_times,
96
+ next_target_time_values=None, # no ODE for 1-step
97
+ return_dict=True,
98
+ )
99
+
100
+ next_norm = out.logits[:, -1, :] # [1, 1]
101
+ preds_norm.append(next_norm.squeeze(0))
102
+
103
+ next_t = future_times[:, i:i+1]
104
+
105
+ cur_vals = torch.cat([cur_vals, next_norm], dim=1)
106
+ cur_times = torch.cat([cur_times, next_t], dim=1)
107
+
108
+
109
+ preds_norm = torch.stack(preds_norm, dim=1) # [1, P]
110
+
111
+ preds = preds_norm * std[:, :, :] + mean[:, :, :]
112
+ preds = preds.squeeze(0)
113
+ print(preds)
114
+ ```