| # Experiment Report |
|
|
| ## Summary |
| - opt_gapは最適なpartition+mappingとの差分ノイズで、0が最適一致を示す。 |
| - 監督学習とRLを同一スケールで比較するため、val_opt_gap/opt_gapを採用。 |
| - opt_gap正規化係数: min=0.0000, std=157.6755 (報酬もstdで正規化)。 |
| - RLはpolicy loss/entropy/approx_klを学習状況の指標として記録する。 |
| - PPOはvalue loss/clip frac/explained varも診断指標として記録する。 |
|
|
| ## Tables |
|
|
| ### Supervised summary |
|
|
| | run | model | hidden | epochs | final_val_opt_gap | best_val_opt_gap | |
| | --- | --- | --- | --- | --- | --- | |
| | cnn_h128-64_lr0.001 | cnn | 128,64 | 5 | 0.6025 | 0.6025 | |
| | mlp_h128-64_lr0.001 | mlp | 128,64 | 5 | 0.1741 | 0.1701 | |
| | vector_h128-64_lr0.001 | vector | 128,64 | 5 | 0.0681 | 0.0681 | |
|
|
| ### RL summary |
|
|
| | run | algo | hidden | episodes | final_reward_ma | opt_gap_ep200 | opt_gap_ep500 | final_opt_gap | best_opt_gap | avg_opt_gap | |
| | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | |
| | ppo_h64-32_ep500 | ppo | 64,32 | 500 | -0.0956 | 0.1053 | 0.0949 | 0.0949 | 0.0902 | 0.2247 | |
| | rl_h64-32_ep500 | rl | 64,32 | 500 | -0.8185 | 1.2689 | 0.6225 | 0.6225 | 0.5287 | 0.9880 | |
|
|
| ## Random baseline |
|
|
| - ランダム分割(9960回)の平均opt_gap: 1.0038 |
| |
| ## Sample inputs/outputs |
| |
| ### Sample 960 |
| |
| ```json |
| { |
| "index": 960, |
| "input": { |
| "circuit_layers": [ |
| { |
| "type": "1q", |
| "qubits": [ |
| 0, |
| 1, |
| 3, |
| 4, |
| 5 |
| ] |
| }, |
| { |
| "type": "2q", |
| "qubits": [ |
| 0, |
| 3 |
| ] |
| }, |
| { |
| "type": "1q", |
| "qubits": [ |
| 2, |
| 4, |
| 5 |
| ] |
| }, |
| { |
| "type": "2q", |
| "qubits": [ |
| 2, |
| 3 |
| ] |
| } |
| ], |
| "machine": { |
| "one_q_order": [ |
| 3, |
| 4, |
| 5, |
| 0, |
| 1, |
| 2 |
| ], |
| "two_q_order": [ |
| [ |
| 0, |
| 5 |
| ], |
| [ |
| 0, |
| 2 |
| ], |
| [ |
| 4, |
| 5 |
| ], |
| [ |
| 1, |
| 3 |
| ], |
| [ |
| 0, |
| 1 |
| ], |
| [ |
| 0, |
| 4 |
| ], |
| [ |
| 1, |
| 2 |
| ], |
| [ |
| 1, |
| 5 |
| ], |
| [ |
| 3, |
| 4 |
| ], |
| [ |
| 2, |
| 5 |
| ], |
| [ |
| 2, |
| 3 |
| ], |
| [ |
| 0, |
| 3 |
| ], |
| [ |
| 3, |
| 5 |
| ], |
| [ |
| 1, |
| 4 |
| ], |
| [ |
| 2, |
| 4 |
| ] |
| ] |
| } |
| }, |
| "optimal_partition": [ |
| false, |
| true, |
| false, |
| false, |
| false, |
| false |
| ], |
| "supervised": { |
| "cnn": { |
| "partition": [ |
| true, |
| false, |
| false, |
| false, |
| false, |
| false |
| ], |
| "mapping": [ |
| 0, |
| 1, |
| 2, |
| 4, |
| 5, |
| 3 |
| ], |
| "opt_gap": 0.7800830861646056 |
| }, |
| "mlp": { |
| "partition": [ |
| false, |
| true, |
| false, |
| false, |
| false, |
| false |
| ], |
| "mapping": [ |
| 0, |
| 1, |
| 2, |
| 3, |
| 4, |
| 5 |
| ], |
| "opt_gap": 0.13952705606196197 |
| }, |
| "vector": { |
| "partition": [ |
| false, |
| true, |
| false, |
| false, |
| false, |
| false |
| ], |
| "mapping": [ |
| 0, |
| 3, |
| 5, |
| 4, |
| 1, |
| 2 |
| ], |
| "opt_gap": 0.08878994476670309 |
| } |
| }, |
| "rl": { |
| "start_partition": [ |
| false, |
| false, |
| false, |
| false, |
| true, |
| true |
| ], |
| "start_noise": 39.0, |
| "final_partition": [ |
| false, |
| false, |
| false, |
| false, |
| false, |
| false |
| ], |
| "final_noise": 36.0, |
| "start_mapping": [ |
| 0, |
| 1, |
| 2, |
| 3, |
| 4, |
| 5 |
| ], |
| "final_mapping": [ |
| 2, |
| 1, |
| 0, |
| 4, |
| 5, |
| 3 |
| ], |
| "start_opt_gap": 0.13952705606196197, |
| "opt_gap": 0.1205006393262399 |
| } |
| } |
| ``` |
| |
| ### Sample 143 |
|
|
| ```json |
| { |
| "index": 143, |
| "input": { |
| "circuit_layers": [ |
| { |
| "type": "1q", |
| "qubits": [ |
| 1, |
| 3, |
| 5 |
| ] |
| }, |
| { |
| "type": "2q", |
| "qubits": [ |
| 1, |
| 4 |
| ] |
| }, |
| { |
| "type": "1q", |
| "qubits": [ |
| 0, |
| 2, |
| 5 |
| ] |
| }, |
| { |
| "type": "2q", |
| "qubits": [ |
| 0, |
| 3 |
| ] |
| } |
| ], |
| "machine": { |
| "one_q_order": [ |
| 1, |
| 4, |
| 5, |
| 0, |
| 3, |
| 2 |
| ], |
| "two_q_order": [ |
| [ |
| 0, |
| 1 |
| ], |
| [ |
| 1, |
| 3 |
| ], |
| [ |
| 1, |
| 4 |
| ], |
| [ |
| 2, |
| 4 |
| ], |
| [ |
| 2, |
| 3 |
| ], |
| [ |
| 0, |
| 4 |
| ], |
| [ |
| 1, |
| 2 |
| ], |
| [ |
| 0, |
| 5 |
| ], |
| [ |
| 3, |
| 4 |
| ], |
| [ |
| 1, |
| 5 |
| ], |
| [ |
| 3, |
| 5 |
| ], |
| [ |
| 0, |
| 2 |
| ], |
| [ |
| 0, |
| 3 |
| ], |
| [ |
| 2, |
| 5 |
| ], |
| [ |
| 4, |
| 5 |
| ] |
| ] |
| } |
| }, |
| "optimal_partition": [ |
| false, |
| false, |
| true, |
| false, |
| false, |
| false |
| ], |
| "supervised": { |
| "cnn": { |
| "partition": [ |
| true, |
| false, |
| false, |
| false, |
| false, |
| false |
| ], |
| "mapping": [ |
| 0, |
| 1, |
| 2, |
| 4, |
| 5, |
| 3 |
| ], |
| "opt_gap": 0.57713464098357 |
| }, |
| "mlp": { |
| "partition": [ |
| false, |
| false, |
| true, |
| false, |
| false, |
| false |
| ], |
| "mapping": [ |
| 0, |
| 3, |
| 2, |
| 5, |
| 4, |
| 1 |
| ], |
| "opt_gap": 0.11415850041433254 |
| }, |
| "vector": { |
| "partition": [ |
| false, |
| false, |
| true, |
| false, |
| false, |
| false |
| ], |
| "mapping": [ |
| 1, |
| 5, |
| 2, |
| 4, |
| 0, |
| 3 |
| ], |
| "opt_gap": 0.06342138911907363 |
| } |
| }, |
| "rl": { |
| "start_partition": [ |
| true, |
| true, |
| true, |
| true, |
| false, |
| false |
| ], |
| "start_noise": 132.0, |
| "final_partition": [ |
| false, |
| false, |
| false, |
| false, |
| false, |
| false |
| ], |
| "final_noise": 25.0, |
| "start_mapping": [ |
| 0, |
| 1, |
| 2, |
| 3, |
| 4, |
| 5 |
| ], |
| "final_mapping": [ |
| 2, |
| 1, |
| 0, |
| 4, |
| 5, |
| 3 |
| ], |
| "start_opt_gap": 0.7547145305169762, |
| "opt_gap": 0.07610566694288835 |
| } |
| } |
| ``` |
|
|
| ### Sample 1887 |
|
|
| ```json |
| { |
| "index": 1887, |
| "input": { |
| "circuit_layers": [ |
| { |
| "type": "1q", |
| "qubits": [] |
| }, |
| { |
| "type": "2q", |
| "qubits": [ |
| 2, |
| 3 |
| ] |
| }, |
| { |
| "type": "1q", |
| "qubits": [ |
| 0, |
| 2, |
| 3, |
| 5 |
| ] |
| }, |
| { |
| "type": "2q", |
| "qubits": [ |
| 1, |
| 5 |
| ] |
| } |
| ], |
| "machine": { |
| "one_q_order": [ |
| 0, |
| 1, |
| 4, |
| 2, |
| 3, |
| 5 |
| ], |
| "two_q_order": [ |
| [ |
| 2, |
| 5 |
| ], |
| [ |
| 1, |
| 4 |
| ], |
| [ |
| 1, |
| 2 |
| ], |
| [ |
| 2, |
| 4 |
| ], |
| [ |
| 0, |
| 1 |
| ], |
| [ |
| 4, |
| 5 |
| ], |
| [ |
| 0, |
| 4 |
| ], |
| [ |
| 0, |
| 2 |
| ], |
| [ |
| 2, |
| 3 |
| ], |
| [ |
| 1, |
| 3 |
| ], |
| [ |
| 1, |
| 5 |
| ], |
| [ |
| 0, |
| 5 |
| ], |
| [ |
| 3, |
| 4 |
| ], |
| [ |
| 0, |
| 3 |
| ], |
| [ |
| 3, |
| 5 |
| ] |
| ] |
| } |
| }, |
| "optimal_partition": [ |
| true, |
| false, |
| false, |
| false, |
| false, |
| false |
| ], |
| "supervised": { |
| "cnn": { |
| "partition": [ |
| true, |
| false, |
| false, |
| false, |
| false, |
| false |
| ], |
| "mapping": [ |
| 0, |
| 1, |
| 2, |
| 4, |
| 5, |
| 3 |
| ], |
| "opt_gap": 0.07610566694288835 |
| }, |
| "mlp": { |
| "partition": [ |
| true, |
| false, |
| false, |
| false, |
| false, |
| false |
| ], |
| "mapping": [ |
| 0, |
| 1, |
| 5, |
| 3, |
| 4, |
| 2 |
| ], |
| "opt_gap": 0.11415850041433254 |
| }, |
| "vector": { |
| "partition": [ |
| true, |
| false, |
| false, |
| false, |
| false, |
| false |
| ], |
| "mapping": [ |
| 3, |
| 1, |
| 0, |
| 5, |
| 4, |
| 2 |
| ], |
| "opt_gap": 0.09513208367861045 |
| } |
| }, |
| "rl": { |
| "start_partition": [ |
| false, |
| false, |
| true, |
| true, |
| false, |
| true |
| ], |
| "start_noise": 100.0, |
| "final_partition": [ |
| false, |
| false, |
| false, |
| false, |
| false, |
| false |
| ], |
| "final_noise": 24.0, |
| "start_mapping": [ |
| 0, |
| 1, |
| 2, |
| 3, |
| 4, |
| 5 |
| ], |
| "final_mapping": [ |
| 2, |
| 1, |
| 0, |
| 4, |
| 5, |
| 3 |
| ], |
| "start_opt_gap": 0.57713464098357, |
| "opt_gap": 0.09513208367861045 |
| } |
| } |
| ``` |
|
|
| ## RL validation (100 samples) |
|
|
| - after avg_opt_gap: 0.0949, best: 0.0000, worst: 0.1839 |
| - before avg_opt_gap: 1.0043, best: 0.0127, worst: 3.7102 |
|
|
| ## Images |
|
|
| - learning curves: reports_6q/learning_curves.png |
| - zoom (mlp/vector): reports_6q/learning_curves_zoom.png |
| - rl val opt_gap (after): reports_6q/rl_val_opt_gap_after.png |
| - rl val opt_gap (before): reports_6q/rl_val_opt_gap_before.png |
| - rl reward moving avg: reports_6q/rl_reward_mavg.png |
| - rl policy metrics: reports_6q/rl_policy_metrics.png |
| - ppo diagnostics: reports_6q/ppo_diagnostics.png |
| |
| ## Image notes |
| |
| - learning_curves.png: 2x2でmlp/vector/cnnのval opt_gapとrl opt_gapを表示。凡例の`h=64,32`は隠れ層構成、`rl h=64,32`/`ppo h=64,32`はアルゴリズムと隠れ層を示す。 |
| - learning_curves_zoom.png: mlp/vectorのみのval opt_gapをズーム表示。凡例はモデル名と隠れ層構成。 |
| - rl_val_opt_gap_after.png: RLで得た最終partition+mappingのopt_gapヒストグラム。凡例なし。 |
| - rl_val_opt_gap_before.png: 初期partition+mappingのopt_gapヒストグラム。凡例なし。 |
| - rl_reward_mavg.png: window=20の報酬移動平均。凡例はアルゴリズムと隠れ層。 |
| - rl_policy_metrics.png: policy loss/entropy/approx_klの推移。凡例はアルゴリズムと隠れ層。 |
| - ppo_diagnostics.png: value loss/clip frac/explained var/approx_kl/policy entropyの推移。凡例はPPOの隠れ層。 |
|
|
