Experiment Report
Summary
- opt_gapは最適なpartition+mappingとの差分ノイズで、0が最適一致を示す。
- 監督学習とRLを同一スケールで比較するため、val_opt_gap/opt_gapを採用。
- opt_gap正規化係数: min=0.0000, std=157.6755 (報酬もstdで正規化)。
- RLはpolicy loss/entropy/approx_klを学習状況の指標として記録する。
- PPOはvalue loss/clip frac/explained varも診断指標として記録する。
Tables
Supervised summary
| run |
model |
hidden |
epochs |
final_val_opt_gap |
best_val_opt_gap |
| cnn_h128-64_lr0.001 |
cnn |
128,64 |
5 |
0.6025 |
0.6025 |
| mlp_h128-64_lr0.001 |
mlp |
128,64 |
5 |
0.1741 |
0.1701 |
| vector_h128-64_lr0.001 |
vector |
128,64 |
5 |
0.0681 |
0.0681 |
RL summary
| run |
algo |
hidden |
episodes |
final_reward_ma |
opt_gap_ep200 |
opt_gap_ep500 |
final_opt_gap |
best_opt_gap |
avg_opt_gap |
| ppo_h64-32_ep500 |
ppo |
64,32 |
500 |
-0.0956 |
0.1053 |
0.0949 |
0.0949 |
0.0902 |
0.2247 |
| rl_h64-32_ep500 |
rl |
64,32 |
500 |
-0.8185 |
1.2689 |
0.6225 |
0.6225 |
0.5287 |
0.9880 |
Random baseline
- ランダム分割(9960回)の平均opt_gap: 1.0038
Sample inputs/outputs
Sample 960
{
"index": 960,
"input": {
"circuit_layers": [
{
"type": "1q",
"qubits": [
0,
1,
3,
4,
5
]
},
{
"type": "2q",
"qubits": [
0,
3
]
},
{
"type": "1q",
"qubits": [
2,
4,
5
]
},
{
"type": "2q",
"qubits": [
2,
3
]
}
],
"machine": {
"one_q_order": [
3,
4,
5,
0,
1,
2
],
"two_q_order": [
[
0,
5
],
[
0,
2
],
[
4,
5
],
[
1,
3
],
[
0,
1
],
[
0,
4
],
[
1,
2
],
[
1,
5
],
[
3,
4
],
[
2,
5
],
[
2,
3
],
[
0,
3
],
[
3,
5
],
[
1,
4
],
[
2,
4
]
]
}
},
"optimal_partition": [
false,
true,
false,
false,
false,
false
],
"supervised": {
"cnn": {
"partition": [
true,
false,
false,
false,
false,
false
],
"mapping": [
0,
1,
2,
4,
5,
3
],
"opt_gap": 0.7800830861646056
},
"mlp": {
"partition": [
false,
true,
false,
false,
false,
false
],
"mapping": [
0,
1,
2,
3,
4,
5
],
"opt_gap": 0.13952705606196197
},
"vector": {
"partition": [
false,
true,
false,
false,
false,
false
],
"mapping": [
0,
3,
5,
4,
1,
2
],
"opt_gap": 0.08878994476670309
}
},
"rl": {
"start_partition": [
false,
false,
false,
false,
true,
true
],
"start_noise": 39.0,
"final_partition": [
false,
false,
false,
false,
false,
false
],
"final_noise": 36.0,
"start_mapping": [
0,
1,
2,
3,
4,
5
],
"final_mapping": [
2,
1,
0,
4,
5,
3
],
"start_opt_gap": 0.13952705606196197,
"opt_gap": 0.1205006393262399
}
}
Sample 143
{
"index": 143,
"input": {
"circuit_layers": [
{
"type": "1q",
"qubits": [
1,
3,
5
]
},
{
"type": "2q",
"qubits": [
1,
4
]
},
{
"type": "1q",
"qubits": [
0,
2,
5
]
},
{
"type": "2q",
"qubits": [
0,
3
]
}
],
"machine": {
"one_q_order": [
1,
4,
5,
0,
3,
2
],
"two_q_order": [
[
0,
1
],
[
1,
3
],
[
1,
4
],
[
2,
4
],
[
2,
3
],
[
0,
4
],
[
1,
2
],
[
0,
5
],
[
3,
4
],
[
1,
5
],
[
3,
5
],
[
0,
2
],
[
0,
3
],
[
2,
5
],
[
4,
5
]
]
}
},
"optimal_partition": [
false,
false,
true,
false,
false,
false
],
"supervised": {
"cnn": {
"partition": [
true,
false,
false,
false,
false,
false
],
"mapping": [
0,
1,
2,
4,
5,
3
],
"opt_gap": 0.57713464098357
},
"mlp": {
"partition": [
false,
false,
true,
false,
false,
false
],
"mapping": [
0,
3,
2,
5,
4,
1
],
"opt_gap": 0.11415850041433254
},
"vector": {
"partition": [
false,
false,
true,
false,
false,
false
],
"mapping": [
1,
5,
2,
4,
0,
3
],
"opt_gap": 0.06342138911907363
}
},
"rl": {
"start_partition": [
true,
true,
true,
true,
false,
false
],
"start_noise": 132.0,
"final_partition": [
false,
false,
false,
false,
false,
false
],
"final_noise": 25.0,
"start_mapping": [
0,
1,
2,
3,
4,
5
],
"final_mapping": [
2,
1,
0,
4,
5,
3
],
"start_opt_gap": 0.7547145305169762,
"opt_gap": 0.07610566694288835
}
}
Sample 1887
{
"index": 1887,
"input": {
"circuit_layers": [
{
"type": "1q",
"qubits": []
},
{
"type": "2q",
"qubits": [
2,
3
]
},
{
"type": "1q",
"qubits": [
0,
2,
3,
5
]
},
{
"type": "2q",
"qubits": [
1,
5
]
}
],
"machine": {
"one_q_order": [
0,
1,
4,
2,
3,
5
],
"two_q_order": [
[
2,
5
],
[
1,
4
],
[
1,
2
],
[
2,
4
],
[
0,
1
],
[
4,
5
],
[
0,
4
],
[
0,
2
],
[
2,
3
],
[
1,
3
],
[
1,
5
],
[
0,
5
],
[
3,
4
],
[
0,
3
],
[
3,
5
]
]
}
},
"optimal_partition": [
true,
false,
false,
false,
false,
false
],
"supervised": {
"cnn": {
"partition": [
true,
false,
false,
false,
false,
false
],
"mapping": [
0,
1,
2,
4,
5,
3
],
"opt_gap": 0.07610566694288835
},
"mlp": {
"partition": [
true,
false,
false,
false,
false,
false
],
"mapping": [
0,
1,
5,
3,
4,
2
],
"opt_gap": 0.11415850041433254
},
"vector": {
"partition": [
true,
false,
false,
false,
false,
false
],
"mapping": [
3,
1,
0,
5,
4,
2
],
"opt_gap": 0.09513208367861045
}
},
"rl": {
"start_partition": [
false,
false,
true,
true,
false,
true
],
"start_noise": 100.0,
"final_partition": [
false,
false,
false,
false,
false,
false
],
"final_noise": 24.0,
"start_mapping": [
0,
1,
2,
3,
4,
5
],
"final_mapping": [
2,
1,
0,
4,
5,
3
],
"start_opt_gap": 0.57713464098357,
"opt_gap": 0.09513208367861045
}
}
RL validation (100 samples)
- after avg_opt_gap: 0.0949, best: 0.0000, worst: 0.1839
- before avg_opt_gap: 1.0043, best: 0.0127, worst: 3.7102
Images
- learning curves: reports_6q/learning_curves.png
- zoom (mlp/vector): reports_6q/learning_curves_zoom.png
- rl val opt_gap (after): reports_6q/rl_val_opt_gap_after.png
- rl val opt_gap (before): reports_6q/rl_val_opt_gap_before.png
- rl reward moving avg: reports_6q/rl_reward_mavg.png
- rl policy metrics: reports_6q/rl_policy_metrics.png
- ppo diagnostics: reports_6q/ppo_diagnostics.png
Image notes
- learning_curves.png: 2x2でmlp/vector/cnnのval opt_gapとrl opt_gapを表示。凡例の
h=64,32は隠れ層構成、rl h=64,32/ppo h=64,32はアルゴリズムと隠れ層を示す。
- learning_curves_zoom.png: mlp/vectorのみのval opt_gapをズーム表示。凡例はモデル名と隠れ層構成。
- rl_val_opt_gap_after.png: RLで得た最終partition+mappingのopt_gapヒストグラム。凡例なし。
- rl_val_opt_gap_before.png: 初期partition+mappingのopt_gapヒストグラム。凡例なし。
- rl_reward_mavg.png: window=20の報酬移動平均。凡例はアルゴリズムと隠れ層。
- rl_policy_metrics.png: policy loss/entropy/approx_klの推移。凡例はアルゴリズムと隠れ層。
- ppo_diagnostics.png: value loss/clip frac/explained var/approx_kl/policy entropyの推移。凡例はPPOの隠れ層。
