Spaces:
Running
Running
File size: 66,572 Bytes
3d9db9c d4c815e 3d9db9c 3fe7722 3d9db9c 154a7f8 3d9db9c 3fe7722 3d9db9c 36e76dd 3fe7722 36e76dd 3fe7722 d4c815e 3fe7722 38c848b 3fe7722 38c848b 3fe7722 3d9db9c 38c848b 3d9db9c 3fe7722 38c848b 3fe7722 d4c815e 3fe7722 38c848b 3fe7722 38c848b d4c815e 38c848b 3fe7722 cf72f92 3fe7722 cf72f92 3fe7722 38c848b 3fe7722 38c848b afed715 d4c815e afed715 38c848b 3fe7722 38c848b 3fe7722 38c848b 490cead 38c848b 490cead 38c848b 490cead 38c848b d4c815e 38c848b 96c3790 3fe7722 2b0b804 38c848b 2b0b804 38c848b 2b0b804 38c848b 2b0b804 cf72f92 2b0b804 38c848b 2b0b804 38c848b 3fe7722 2b0b804 38c848b 96c3790 0ac7051 d4c815e 3fe7722 96c3790 38c848b 96c3790 3fe7722 d4c815e 3fe7722 96c3790 3fe7722 96c3790 3fe7722 d4c815e 38c848b d4c815e 3fe7722 36e76dd d4c815e cf72f92 38c848b cf72f92 d4c815e cf72f92 38c848b d4c815e cf72f92 d4c815e cf72f92 38c848b cf72f92 38c848b cf72f92 38c848b cf72f92 38c848b cf72f92 38c848b cf72f92 38c848b cf72f92 38c848b cf72f92 38c848b cf72f92 38c848b cf72f92 38c848b cf72f92 3fe7722 38c848b 3fe7722 38c848b 3d9db9c 38c848b 3d9db9c 3fe7722 38c848b 3fe7722 38c848b 3fe7722 38c848b 3fe7722 38c848b cf72f92 3d9db9c 38c848b 3d9db9c 96c3790 3fe7722 38c848b 3fe7722 38c848b 3fe7722 38c848b 3fe7722 38c848b 3fe7722 38c848b 3fe7722 38c848b 3fe7722 38c848b 3fe7722 96c3790 38c848b 96c3790 3fe7722 38c848b 3fe7722 38c848b 3fe7722 43f19af 38c848b 3fe7722 38c848b 43f19af cf72f92 38c848b cf72f92 38c848b cf72f92 3fe7722 2b0b804 38c848b 3fe7722 43f19af 2b0b804 38c848b 3fe7722 43f19af 2b0b804 38c848b d4c815e cf72f92 3fe7722 38c848b 2b0b804 38c848b 3fe7722 0ac7051 96c3790 3fe7722 38c848b 3fe7722 38c848b 3fe7722 2b0b804 38c848b cf72f92 2b0b804 96c3790 cf72f92 3fe7722 3d9db9c 38c848b 3fe7722 38c848b 3fe7722 3d9db9c 3fe7722 154a7f8 d4c815e 3fe7722 38c848b 3fe7722 38c848b 3fe7722 38c848b 3fe7722 38c848b d4c815e 3fe7722 38c848b 3fe7722 38c848b 3fe7722 38c848b d4c815e 38c848b d4c815e 38c848b 3d9db9c 38c848b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 | # ================================================================
# 教育大模型MIA攻防研究 - Gradio演示系统 终极防重叠版
# 1. 严格基于你发来的完整代码底座,一字不漏
# 2. 修复:散点图(Trade-off)图例移至左下角,增加点位透明度防遮挡
# 3. 修复:折线图(Trend)图例移至下方空白处,不再遮挡数据线和阴影
# ================================================================
import os
import json
import re
import numpy as np
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
from sklearn.metrics import roc_curve, roc_auc_score
import gradio as gr
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
# ================================================================
# 数据加载
# ================================================================
def load_json(path):
full = os.path.join(BASE_DIR, path)
with open(full, 'r', encoding='utf-8') as f:
return json.load(f)
def clean_text(text):
if not isinstance(text, str):
return str(text)
text = re.sub(r'[\U00010000-\U0010ffff]', '', text)
text = re.sub(r'[\ufff0-\uffff]', '', text)
text = re.sub(r'[\u200b-\u200f\u2028-\u202f\u2060-\u206f\ufeff]', '', text)
return text.strip()
# 尝试加载数据,如果不存在则使用虚拟数据以确保运行
try:
member_data = load_json("data/member.json")
non_member_data = load_json("data/non_member.json")
config = load_json("config.json")
all_data = load_json("results/all_results.json")
mia_results = all_data["mia_results"]
perturb_results = all_data["perturbation_results"]
utility_results = all_data["utility_results"]
full_losses = all_data["full_losses"]
model_name = config.get('model_name', 'Qwen/Qwen2.5-Math-1.5B-Instruct')
except FileNotFoundError:
print("⚠️ 警告: 未找到数据文件。将使用虚拟数据进行演示。")
member_data = [{"question": "Q"+str(i), "answer": "A"+str(i), "task_type": "calculation", "metadata": {"name": "Student"+str(i), "student_id": str(1000+i), "class": "Class A", "score": 90}} for i in range(1000)]
non_member_data = [{"question": "Q"+str(i+1000), "answer": "A"+str(i+1000), "task_type": "word_problem", "metadata": {"name": "Student"+str(i+1000), "student_id": str(2000+i), "class": "Class B", "score": 85}} for i in range(1000)]
model_name = "Demo Model"
mia_results = {"baseline": {"auc": 0.6230, "attack_accuracy": 0.6055, "precision": 0.6779, "recall": 0.4020, "f1": 0.5047, "tpr_at_5fpr": 0.1850, "tpr_at_1fpr": 0.0930, "loss_gap": 0.0107, "member_loss_mean": 0.2494, "non_member_loss_mean": 0.2601, "member_loss_std": 0.03, "non_member_loss_std": 0.03}}
perturb_results = {}
utility_results = {"baseline": {"accuracy": 0.660}}
full_losses = {"baseline": {"member_losses": np.random.normal(0.2494, 0.03, 1000).tolist(), "non_member_losses": np.random.normal(0.2601, 0.03, 1000).tolist()}}
for e in [0.02, 0.05, 0.1, 0.2]:
k = f"smooth_eps_{e}"
mia_results[k] = {m: v*0.9 for m, v in mia_results["baseline"].items()}
utility_results[k] = {"accuracy": 0.660 + e*0.1}
for s in [0.005, 0.01, 0.015, 0.02, 0.025, 0.03]:
k = f"perturbation_{s}"
perturb_results[k] = {m: v*0.85 for m, v in mia_results["baseline"].items()}
perturb_results[k]["member_loss_std"] = np.sqrt(0.03**2 + s**2)
perturb_results[k]["non_member_loss_std"] = np.sqrt(0.03**2 + s**2)
# ================================================================
# 全局图表配置
# ================================================================
COLORS = {
'bg': '#FFFFFF',
'panel': '#F5F7FA',
'grid': '#E2E8F0',
'text': '#1E293B',
'text_dim': '#64748B',
'accent': '#007AFF',
'accent2': '#5856D6',
'danger': '#FF3B30',
'success': '#34C759',
'warning': '#FF9500',
'baseline': '#8E8E93',
'ls_colors': ['#A0C4FF', '#70A1FF', '#478EFF', '#007AFF'],
'op_colors': ['#98F5E1', '#6EE7B7', '#34D399', '#10B981', '#059669', '#047857'],
}
CHART_W = 14
def apply_light_style(fig, ax_or_axes):
fig.patch.set_facecolor(COLORS['bg'])
axes = ax_or_axes if hasattr(ax_or_axes, '__iter__') else [ax_or_axes]
for ax in axes:
ax.set_facecolor(COLORS['panel'])
for spine in ax.spines.values():
spine.set_color(COLORS['grid'])
spine.set_linewidth(1)
ax.tick_params(colors=COLORS['text_dim'], labelsize=10, width=1)
ax.xaxis.label.set_color(COLORS['text'])
ax.yaxis.label.set_color(COLORS['text'])
ax.title.set_color(COLORS['text'])
ax.title.set_fontweight('semibold')
ax.grid(True, color=COLORS['grid'], alpha=0.6, linestyle='-', linewidth=0.8)
ax.set_axisbelow(True)
# ================================================================
# 使用标准 Unicode ε 和 σ
# ================================================================
LS_KEYS = ["baseline", "smooth_eps_0.02", "smooth_eps_0.05", "smooth_eps_0.1", "smooth_eps_0.2"]
LS_LABELS_PLOT = ["Baseline", "LS(ε=0.02)", "LS(ε=0.05)", "LS(ε=0.1)", "LS(ε=0.2)"]
LS_LABELS_MD = ["基线(Baseline)", "LS(ε=0.02)", "LS(ε=0.05)", "LS(ε=0.1)", "LS(ε=0.2)"]
OP_SIGMAS = [0.005, 0.01, 0.015, 0.02, 0.025, 0.03]
OP_KEYS = [f"perturbation_{s}" for s in OP_SIGMAS]
OP_LABELS_PLOT = [f"OP(σ={s})" for s in OP_SIGMAS]
OP_LABELS_MD = [f"OP(σ={s})" for s in OP_SIGMAS]
ALL_KEYS = LS_KEYS + OP_KEYS
def gm(key, metric, default=0):
if key in mia_results: return mia_results[key].get(metric, default)
if key in perturb_results: return perturb_results[key].get(metric, default)
return default
def gu(key):
if key in utility_results: return utility_results[key].get("accuracy", 0) * 100
if key.startswith("perturbation_"): return utility_results.get("baseline", {}).get("accuracy", 0) * 100
return 0
bl_auc = gm("baseline", "auc")
bl_acc = gu("baseline")
bl_m_mean = gm("baseline", "member_loss_mean")
bl_nm_mean = gm("baseline", "non_member_loss_mean")
TYPE_CN = {'calculation': '基础计算', 'word_problem': '应用题', 'concept': '概念问答', 'error_correction': '错题订正'}
np.random.seed(777)
EVAL_POOL = []
_types = ['calculation']*120 + ['word_problem']*90 + ['concept']*60 + ['error_correction']*30
for _i in range(300):
_t = _types[_i]
if _t == 'calculation':
_a, _b = int(np.random.randint(10,500)), int(np.random.randint(10,500))
_op = ['+','-','x'][_i%3]
if _op=='+': _q,_ans=f"{_a} + {_b} = ?",str(_a+_b)
elif _op=='-': _q,_ans=f"{_a} - {_b} = ?",str(_a-_b)
else: _q,_ans=f"{_a} x {_b} = ?",str(_a*_b)
elif _t == 'word_problem':
_a,_b = int(np.random.randint(5,200)), int(np.random.randint(3,50))
_tpls = [(f"{_a} apples, ate {_b}, left?",str(_a-_b)), (f"{_a} per group, {_b} groups, total?",str(_a*_b))]
_q,_ans = _tpls[_i%len(_tpls)]
elif _t == 'concept':
_cs = [("area","Area = space occupied by a shape"),("perimeter","Perimeter = total boundary length")]
_cn,_df = _cs[_i%len(_cs)]; _q,_ans = f"What is {_cn}?",_df
else:
_a,_b = int(np.random.randint(10,99)), int(np.random.randint(10,99))
_w = _a+_b+int(np.random.choice([-1,1,-10,10]))
_q,_ans = f"Student got {_a}+{_b}={_w}, correct?",str(_a+_b)
item = {'question':_q,'answer':_ans,'type_cn':TYPE_CN[_t]}
for key in LS_KEYS:
acc = gu(key)/100; item[key] = bool(np.random.random()<acc)
EVAL_POOL.append(item)
# ================================================================
# 图表绘制函数 (保留你的原貌,只做防重叠修复)
# ================================================================
def fig_gauge(loss_val, m_mean, nm_mean, thr, m_std, nm_std):
fig, ax = plt.subplots(figsize=(10, 2.6)); fig.patch.set_facecolor(COLORS['bg']); ax.set_facecolor(COLORS['panel'])
xlo = min(m_mean - 3.0 * m_std, loss_val - 0.005); xhi = max(nm_mean + 3.0 * nm_std, loss_val + 0.005)
ax.axvspan(xlo, thr, alpha=0.2, color=COLORS['accent']); ax.axvspan(thr, xhi, alpha=0.2, color=COLORS['danger'])
ax.axvline(m_mean, color=COLORS['accent'], lw=2, ls=':', alpha=0.8, zorder=2)
ax.text(m_mean - 0.002, 1.02, f'Member Mean\n{m_mean:.4f}', ha='right', va='bottom', fontsize=9, color=COLORS['accent'], transform=ax.get_xaxis_transform())
ax.axvline(nm_mean, color=COLORS['danger'], lw=2, ls=':', alpha=0.8, zorder=2)
ax.text(nm_mean + 0.002, 1.02, f'Non-Member Mean\n{nm_mean:.4f}', ha='left', va='bottom', fontsize=9, color=COLORS['danger'], transform=ax.get_xaxis_transform())
ax.axvline(thr, color=COLORS['text_dim'], lw=2.5, ls='--', zorder=3)
ax.text(thr, 1.25, f'Threshold\n{thr:.4f}', ha='center', va='bottom', fontsize=10, fontweight='bold', color=COLORS['text_dim'], transform=ax.get_xaxis_transform())
mc = COLORS['accent'] if loss_val < thr else COLORS['danger']
ax.plot(loss_val, 0.5, marker='o', ms=16, color='white', mec=mc, mew=3, zorder=5, transform=ax.get_xaxis_transform())
ax.text(loss_val, 0.75, f'Current Loss\n{loss_val:.4f}', ha='center', fontsize=11, fontweight='bold', color=mc, transform=ax.get_xaxis_transform())
ax.text((xlo+thr)/2, 0.25, 'MEMBER', ha='center', fontsize=12, color=COLORS['accent'], alpha=0.6, fontweight='bold', transform=ax.get_xaxis_transform())
ax.text((thr+xhi)/2, 0.25, 'NON-MEMBER', ha='center', fontsize=12, color=COLORS['danger'], alpha=0.6, fontweight='bold', transform=ax.get_xaxis_transform())
ax.set_xlim(xlo, xhi); ax.set_yticks([])
for s in ax.spines.values(): s.set_visible(False)
ax.spines['bottom'].set_visible(True); ax.spines['bottom'].set_color(COLORS['grid']); ax.tick_params(colors=COLORS['text_dim'], width=1)
ax.set_xlabel('Loss Value', fontsize=11, color=COLORS['text'], fontweight='medium'); plt.tight_layout(pad=0.5)
return fig
def fig_auc_bar():
names, vals, clrs = [], [], []
ls_c = [COLORS['baseline']] + COLORS['ls_colors']
for i,(k,l) in enumerate(zip(LS_KEYS, LS_LABELS_PLOT)):
if k in mia_results: names.append(l); vals.append(mia_results[k]['auc']); clrs.append(ls_c[i])
for i,(k,l) in enumerate(zip(OP_KEYS, OP_LABELS_PLOT)):
if k in perturb_results: names.append(l); vals.append(perturb_results[k]['auc']); clrs.append(COLORS['op_colors'][i])
fig, ax = plt.subplots(figsize=(14, 6)); apply_light_style(fig, ax)
bars = ax.bar(range(len(names)), vals, color=clrs, width=0.65, edgecolor='none', zorder=3)
for b,v in zip(bars, vals): ax.text(b.get_x()+b.get_width()/2, v+0.01, f'{v:.4f}', ha='center', fontsize=10, fontweight='semibold', color=COLORS['text'])
ax.axhline(0.5, color=COLORS['text_dim'], ls='--', lw=1.5, alpha=0.6, label='Random Guess (0.5)', zorder=2)
ax.axhline(bl_auc, color=COLORS['danger'], ls=':', lw=1.5, alpha=0.8, label=f'Baseline ({bl_auc:.4f})', zorder=2)
ax.set_ylabel('MIA Attack AUC', fontsize=12, fontweight='medium'); ax.set_title('Defense Effectiveness: MIA AUC Comparison', fontsize=14, fontweight='bold', pad=20)
ax.set_ylim(0.45, max(vals)+0.05); ax.set_xticks(range(len(names))); ax.set_xticklabels(names, rotation=30, ha='right', fontsize=11)
ax.legend(facecolor=COLORS['bg'], edgecolor='none', labelcolor=COLORS['text'], fontsize=10, loc='upper right'); plt.tight_layout()
return fig
def fig_radar():
ms = ['AUC', 'Atk Acc', 'Prec', 'Recall', 'F1', 'TPR@5%', 'TPR@1%', 'Gap']
mk = ['auc', 'attack_accuracy', 'precision', 'recall', 'f1', 'tpr_at_5fpr', 'tpr_at_1fpr', 'loss_gap']
N = len(ms); ag = np.linspace(0, 2 * np.pi, N, endpoint=False).tolist() + [0]
fig, axes = plt.subplots(1, 2, figsize=(CHART_W + 2, 7), subplot_kw=dict(polar=True)); fig.patch.set_facecolor('white')
ls_cfgs = [("Baseline", "baseline", '#F04438'), ("LS(ε=0.02)", "smooth_eps_0.02", '#B2DDFF'), ("LS(ε=0.05)", "smooth_eps_0.05", '#84CAFF'), ("LS(ε=0.1)", "smooth_eps_0.1", '#2E90FA'), ("LS(ε=0.2)", "smooth_eps_0.2", '#7A5AF8')]
op_cfgs = [("Baseline", "baseline", '#F04438'), ("OP(σ=0.005)", "perturbation_0.005", '#A6F4C5'), ("OP(σ=0.01)", "perturbation_0.01", '#6CE9A6'), ("OP(σ=0.015)", "perturbation_0.015", '#32D583'), ("OP(σ=0.02)", "perturbation_0.02", '#12B76A'), ("OP(σ=0.025)", "perturbation_0.025", '#039855'), ("OP(σ=0.03)", "perturbation_0.03", '#027A48')]
# 🌟 核心修改:全局归一化,锁定统一的天花板
all_configs_keys = ["baseline"] + LS_KEYS[1:] + OP_KEYS
mx_global = [max(gm(k, m_key) for k in all_configs_keys) for m_key in mk]
mx_global = [m if m > 0 else 1 for m in mx_global]
for ax_idx, (ax, cfgs, title) in enumerate([(axes[0], ls_cfgs, 'Radar Analysis: Label Smoothing'), (axes[1], op_cfgs, 'Radar Analysis: Output Perturbation')]):
ax.set_facecolor('white')
for nm, ky, cl in cfgs:
# 🌟 这里统一除以 mx_global,确保红线形状完全一致
v = [gm(ky, m_key) / mx_global[i] for i, m_key in enumerate(mk)]; v += [v[0]]
ax.plot(ag, v, 'o-', lw=2.8 if ky == 'baseline' else 1.8, label=nm, color=cl, ms=5, alpha=0.95 if ky == 'baseline' else 0.85)
ax.fill(ag, v, alpha=0.10 if ky == 'baseline' else 0.04, color=cl)
ax.set_xticks(ag[:-1]); ax.set_xticklabels(ms, fontsize=10, color=COLORS['text']); ax.set_yticklabels([])
ax.set_title(title, fontsize=14, fontweight='bold', color=COLORS['text'], pad=25)
ax.legend(loc='upper right', bbox_to_anchor=(1.35 if ax_idx == 1 else 1.30, 1.12), fontsize=9, framealpha=0.9, edgecolor=COLORS['grid'])
ax.spines['polar'].set_color(COLORS['grid']); ax.grid(color=COLORS['grid'], alpha=0.5)
plt.tight_layout()
return fig
# 🌟 修复:这是我专门加的 3联 Loss 直方图横向对比
def fig_d3_dist_compare():
configs = [
("Baseline (No Defense)", "baseline", COLORS['danger'], None),
("Label Smoothing (ε=0.2)", "smooth_eps_0.2", COLORS['accent2'], None),
("Output Perturbation (σ=0.03)", "baseline", COLORS['success'], 0.03),
]
fig, axes = plt.subplots(1, 3, figsize=(18, 5.5))
apply_light_style(fig, axes)
for idx, (title, key, color, sigma) in enumerate(configs):
ax = axes[idx]
if key in full_losses:
m_losses = np.array(full_losses[key]['member_losses'])
nm_losses = np.array(full_losses[key]['non_member_losses'])
if sigma:
rm=np.random.RandomState(42); rn=np.random.RandomState(137)
m_losses = m_losses + rm.normal(0, sigma, len(m_losses))
nm_losses = nm_losses + rn.normal(0, sigma, len(nm_losses))
all_v = np.concatenate([m_losses, nm_losses])
bins = np.linspace(all_v.min(), all_v.max(), 35)
ax.hist(m_losses, bins=bins, alpha=0.6, color=COLORS['accent'], label='Member', density=True, edgecolor='white')
ax.hist(nm_losses, bins=bins, alpha=0.6, color=COLORS['danger'], label='Non-Member', density=True, edgecolor='white')
m_mean = np.mean(m_losses); nm_mean = np.mean(nm_losses)
gap = nm_mean - m_mean
ax.axvline(m_mean, color=COLORS['accent'], ls='--', lw=2, alpha=0.8)
ax.axvline(nm_mean, color=COLORS['danger'], ls='--', lw=2, alpha=0.8)
ax.annotate(f'Gap={gap:.4f}', xy=((m_mean+nm_mean)/2, ax.get_ylim()[1]*0.85 if ax.get_ylim()[1]>0 else 5),
fontsize=11, fontweight='bold', color=color, ha='center',
bbox=dict(boxstyle='round,pad=0.4', fc='white', ec=color, alpha=0.9))
ax.set_title(title, fontsize=13, fontweight='bold', color=color, pad=15)
ax.set_xlabel('Loss', fontsize=12)
if idx == 0: ax.set_ylabel('Density', fontsize=12)
ax.legend(fontsize=10, facecolor=COLORS['bg'], edgecolor='none')
fig.suptitle('Loss Distribution: Baseline vs LS vs OP', fontsize=16, fontweight='bold', color=COLORS['text'], y=1.05)
plt.tight_layout(); return fig
def fig_loss_dist():
items = [(k, l, gm(k, 'auc')) for k, l in zip(LS_KEYS, LS_LABELS_PLOT) if k in full_losses]; n = len(items)
if n == 0: return plt.figure()
fig, axes = plt.subplots(1, n, figsize=(4.5*n, 4.5)); axes = [axes] if n == 1 else axes; apply_light_style(fig, axes)
for ax, (k, l, a) in zip(axes, items):
m = full_losses[k]['member_losses']; nm = full_losses[k]['non_member_losses']; bins = np.linspace(min(min(m),min(nm)), max(max(m),max(nm)), 30)
ax.hist(m, bins=bins, alpha=0.6, color=COLORS['accent'], label='Member', density=True, edgecolor='white')
ax.hist(nm, bins=bins, alpha=0.6, color=COLORS['danger'], label='Non-Member', density=True, edgecolor='white')
ax.set_title(f'{l}\nAUC={a:.4f}', fontsize=11, fontweight='semibold'); ax.set_xlabel('Loss', fontsize=10); ax.set_ylabel('Density', fontsize=10)
ax.legend(fontsize=9, facecolor=COLORS['bg'], edgecolor='none', labelcolor=COLORS['text'])
plt.tight_layout(); return fig
def fig_perturb_dist():
if 'baseline' not in full_losses: return plt.figure()
ml = np.array(full_losses['baseline']['member_losses']); nl = np.array(full_losses['baseline']['non_member_losses'])
fig, axes = plt.subplots(2, 3, figsize=(16, 9)); axes_flat = axes.flatten(); apply_light_style(fig, axes_flat)
for i, (ax, s) in enumerate(zip(axes_flat, OP_SIGMAS)):
rng_m = np.random.RandomState(42); rng_nm = np.random.RandomState(137)
mp = ml + rng_m.normal(0, s, len(ml)); np_ = nl + rng_nm.normal(0, s, len(nl)); v = np.concatenate([mp, np_])
bins = np.linspace(v.min(), v.max(), 28)
ax.hist(mp, bins=bins, alpha=0.6, color=COLORS['accent'], label='Mem+noise', density=True, edgecolor='white')
ax.hist(np_, bins=bins, alpha=0.6, color=COLORS['danger'], label='Non+noise', density=True, edgecolor='white')
pa = gm(f'perturbation_{s}', 'auc')
ax.set_title(f'OP(σ={s})\nAUC={pa:.4f}', fontsize=11, fontweight='semibold'); ax.set_xlabel('Loss', fontsize=10)
ax.legend(fontsize=9, facecolor=COLORS['bg'], edgecolor='none', labelcolor=COLORS['text'])
plt.tight_layout(); return fig
def fig_roc_curves():
fig, axes = plt.subplots(1, 2, figsize=(16, 7)); apply_light_style(fig, axes)
ax = axes[0]; ls_colors = [COLORS['danger'], COLORS['ls_colors'][0], COLORS['ls_colors'][1], COLORS['ls_colors'][2], COLORS['ls_colors'][3]]
for i, (k, l) in enumerate(zip(LS_KEYS, LS_LABELS_PLOT)):
if k not in full_losses: continue
m = np.array(full_losses[k]['member_losses']); nm = np.array(full_losses[k]['non_member_losses'])
y_true = np.concatenate([np.ones(len(m)), np.zeros(len(nm))]); y_scores = np.concatenate([-m, -nm])
fpr, tpr, _ = roc_curve(y_true, y_scores); auc_val = roc_auc_score(y_true, y_scores)
ax.plot(fpr, tpr, color=ls_colors[i], lw=2.5, label=f'{l} (AUC={auc_val:.4f})')
ax.plot([0,1], [0,1], '--', color=COLORS['text_dim'], lw=1.5, label='Random')
ax.set_xlabel('False Positive Rate', fontsize=12, fontweight='medium'); ax.set_ylabel('True Positive Rate', fontsize=12, fontweight='medium'); ax.set_title('ROC Curves: Label Smoothing', fontsize=14, fontweight='bold', pad=15); ax.legend(fontsize=10, facecolor=COLORS['bg'], edgecolor='none', labelcolor=COLORS['text'])
ax = axes[1]
if 'baseline' in full_losses:
ml_base = np.array(full_losses['baseline']['member_losses']); nl_base = np.array(full_losses['baseline']['non_member_losses']); y_true = np.concatenate([np.ones(len(ml_base)), np.zeros(len(nl_base))]); y_scores = np.concatenate([-ml_base, -nl_base])
fpr, tpr, _ = roc_curve(y_true, y_scores); ax.plot(fpr, tpr, color=COLORS['danger'], lw=2.5, label=f'Baseline (AUC={bl_auc:.4f})')
for i, s in enumerate(OP_SIGMAS):
rng_m = np.random.RandomState(42); rng_nm = np.random.RandomState(137); mp = ml_base + rng_m.normal(0, s, len(ml_base)); np_ = nl_base + rng_nm.normal(0, s, len(nl_base)); y_scores_p = np.concatenate([-mp, -np_]); fpr_p, tpr_p, _ = roc_curve(y_true, y_scores_p); auc_p = roc_auc_score(y_true, y_scores_p)
ax.plot(fpr_p, tpr_p, color=COLORS['op_colors'][i], lw=2, label=f'OP(σ={s}) (AUC={auc_p:.4f})')
ax.plot([0,1], [0,1], '--', color=COLORS['text_dim'], lw=1.5, label='Random')
ax.set_xlabel('False Positive Rate', fontsize=12, fontweight='medium'); ax.set_ylabel('True Positive Rate', fontsize=12, fontweight='medium'); ax.set_title('ROC Curves: Output Perturbation', fontsize=14, fontweight='bold', pad=15); ax.legend(fontsize=10, facecolor=COLORS['bg'], edgecolor='none', labelcolor=COLORS['text'], loc='lower right'); plt.tight_layout()
return fig
def fig_tpr_at_low_fpr():
fig, axes = plt.subplots(1, 2, figsize=(16, 6.5)); apply_light_style(fig, axes); labels_all, tpr5_all, tpr1_all, colors_all = [], [], [], []; ls_c = [COLORS['baseline']] + COLORS['ls_colors']
for i, (k, l) in enumerate(zip(LS_KEYS, LS_LABELS_PLOT)): labels_all.append(l); tpr5_all.append(gm(k, 'tpr_at_5fpr')); tpr1_all.append(gm(k, 'tpr_at_1fpr')); colors_all.append(ls_c[i])
for i, (k, l) in enumerate(zip(OP_KEYS, OP_LABELS_PLOT)): labels_all.append(l); tpr5_all.append(gm(k, 'tpr_at_5fpr')); tpr1_all.append(gm(k, 'tpr_at_1fpr')); colors_all.append(COLORS['op_colors'][i])
x = range(len(labels_all)); ax = axes[0]; bars = ax.bar(x, tpr5_all, color=colors_all, width=0.65, edgecolor='none', zorder=3)
for b, v in zip(bars, tpr5_all): ax.text(b.get_x()+b.get_width()/2, v+0.005, f'{v:.3f}', ha='center', fontsize=9, fontweight='semibold', color=COLORS['text'])
ax.set_ylabel('TPR @ 5% FPR', fontsize=12, fontweight='medium'); ax.set_title('Attack Power at 5% FPR', fontsize=14, fontweight='bold', pad=15); ax.set_xticks(x); ax.set_xticklabels(labels_all, rotation=35, ha='right', fontsize=11); ax.axhline(0.05, color=COLORS['warning'], ls='--', lw=1.5, alpha=0.7, label='Random (0.05)'); ax.legend(facecolor=COLORS['bg'], edgecolor='none', labelcolor=COLORS['text'], fontsize=10)
ax = axes[1]; bars = ax.bar(x, tpr1_all, color=colors_all, width=0.65, edgecolor='none', zorder=3)
for b, v in zip(bars, tpr1_all): ax.text(b.get_x()+b.get_width()/2, v+0.003, f'{v:.3f}', ha='center', fontsize=9, fontweight='semibold', color=COLORS['text'])
ax.set_ylabel('TPR @ 1% FPR', fontsize=12, fontweight='medium'); ax.set_title('Attack Power at 1% FPR (Strict)', fontsize=14, fontweight='bold', pad=15); ax.set_xticks(x); ax.set_xticklabels(labels_all, rotation=35, ha='right', fontsize=11); ax.axhline(0.01, color=COLORS['warning'], ls='--', lw=1.5, alpha=0.7, label='Random (0.01)'); ax.legend(facecolor=COLORS['bg'], edgecolor='none', labelcolor=COLORS['text'], fontsize=10); plt.tight_layout()
return fig
def fig_loss_gap_waterfall():
fig, ax = plt.subplots(figsize=(14, 6)); apply_light_style(fig, ax); names, gaps, clrs = [], [], []; ls_c = [COLORS['baseline']] + COLORS['ls_colors']
for i, (k, l) in enumerate(zip(LS_KEYS, LS_LABELS_PLOT)): names.append(l); gaps.append(gm(k, 'loss_gap')); clrs.append(ls_c[i])
for i, (k, l) in enumerate(zip(OP_KEYS, OP_LABELS_PLOT)): names.append(l); gaps.append(gm(k, 'loss_gap')); clrs.append(COLORS['op_colors'][i])
bars = ax.bar(range(len(names)), gaps, color=clrs, width=0.65, edgecolor='none', zorder=3)
for b, v in zip(bars, gaps): ax.text(b.get_x()+b.get_width()/2, v+0.0005, f'{v:.4f}', ha='center', fontsize=10, fontweight='semibold', color=COLORS['text'])
ax.set_ylabel('Loss Gap', fontsize=12, fontweight='medium'); ax.set_title('Member vs Non-Member Loss Gap', fontsize=14, fontweight='bold', pad=20); ax.set_xticks(range(len(names))); ax.set_xticklabels(names, rotation=30, ha='right', fontsize=11); ax.annotate('Smaller gap = Better Privacy', xy=(8, gaps[0]*0.4), fontsize=11, color=COLORS['success'], fontstyle='italic', ha='center', backgroundcolor=COLORS['bg'], bbox=dict(boxstyle='round,pad=0.4', facecolor=COLORS['panel'], edgecolor=COLORS['success'], alpha=0.8)); plt.tight_layout()
return fig
# 🌟 效用页面柱状图
def fig_acc_bar():
names, vals, clrs = [], [], []; ls_c = [COLORS['baseline']] + COLORS['ls_colors']
for i, (k, l) in enumerate(zip(LS_KEYS, LS_LABELS_PLOT)):
if k in utility_results: names.append(l); vals.append(utility_results[k]['accuracy']*100); clrs.append(ls_c[i])
for i, (k, l) in enumerate(zip(OP_KEYS, OP_LABELS_PLOT)):
if k in perturb_results: names.append(l); vals.append(bl_acc); clrs.append(COLORS['op_colors'][i])
fig, ax = plt.subplots(figsize=(12, 7)); apply_light_style(fig, ax); bars = ax.bar(range(len(names)), vals, color=clrs, width=0.65, edgecolor='none', zorder=3)
for b, v in zip(bars, vals): ax.text(b.get_x()+b.get_width()/2, v+1, f'{v:.1f}%', ha='center', fontsize=11, fontweight='bold', color=COLORS['text'])
ax.set_ylabel('Test Accuracy (%)', fontsize=12, fontweight='medium'); ax.set_title('Model Utility: Test Accuracy', fontsize=15, fontweight='bold', pad=20)
ax.set_ylim(0, 105); ax.set_xticks(range(len(names))); ax.set_xticklabels(names, rotation=35, ha='right', fontsize=12); plt.tight_layout()
return fig
# 🌟 修复:散点图增加透明度防遮挡,并且图例移至左下角空白处
def fig_tradeoff():
fig, ax = plt.subplots(figsize=(12, 7)); apply_light_style(fig, ax); markers_ls = ['o', 's', 's', 's', 's']; ls_c = [COLORS['baseline']] + COLORS['ls_colors']
for i, (k, l) in enumerate(zip(LS_KEYS, LS_LABELS_PLOT)):
if k in mia_results and k in utility_results: ax.scatter(utility_results[k]['accuracy']*100, mia_results[k]['auc'], label=l, marker=markers_ls[i], color=ls_c[i], s=250, edgecolors='white', lw=2, zorder=5, alpha=0.9)
op_markers = ['^', 'D', 'v', 'P', 'X', 'h']
for i, (k, l) in enumerate(zip(OP_KEYS, OP_LABELS_PLOT)):
# 增加透明度 alpha=0.75 防止叠放时遮盖
if k in perturb_results: ax.scatter(bl_acc, perturb_results[k]['auc'], label=l, marker=op_markers[i], color=COLORS['op_colors'][i], s=200, edgecolors='white', lw=1.5, zorder=6, alpha=0.75)
ax.axhline(0.5, color=COLORS['text_dim'], ls='--', alpha=0.6, label='Random (AUC=0.5)')
ax.annotate('IDEAL ZONE\nHigh Utility, Low Risk', xy=(85, 0.51), fontsize=11, fontweight='bold', color=COLORS['success'], alpha=0.7, ha='center', backgroundcolor=COLORS['bg'])
ax.annotate('HIGH RISK ZONE\nLow Utility, High Risk', xy=(62, 0.61), fontsize=11, fontweight='bold', color=COLORS['danger'], alpha=0.7, ha='center', backgroundcolor=COLORS['bg'])
ax.set_xlabel('Model Utility (Accuracy %)', fontsize=12, fontweight='medium'); ax.set_ylabel('Privacy Risk (MIA AUC)', fontsize=12, fontweight='medium')
ax.set_title('Privacy-Utility Trade-off Analysis', fontsize=15, fontweight='bold', pad=20)
# 图例放到绝对安全的左下角空白区域
ax.legend(fontsize=11, loc='lower left', ncol=2, facecolor=COLORS['bg'], edgecolor='none'); plt.tight_layout()
return fig
# 🌟 修复:趋势图图例移位,避免遮挡数据线和阴影
def fig_auc_trend():
fig, axes = plt.subplots(1, 2, figsize=(16, 6.5)); apply_light_style(fig, axes); ax = axes[0]; eps_vals = [0.0, 0.02, 0.05, 0.1, 0.2]; auc_vals = [gm(k, 'auc') for k in LS_KEYS]; acc_vals = [gu(k) for k in LS_KEYS]
ax2 = ax.twinx(); line1 = ax.plot(eps_vals, auc_vals, 'o-', color=COLORS['danger'], lw=3, ms=9, label='MIA AUC (Risk)', zorder=5); line2 = ax2.plot(eps_vals, acc_vals, 's--', color=COLORS['accent'], lw=3, ms=9, label='Utility % (right)', zorder=5); ax.axhline(0.5, color=COLORS['text_dim'], ls=':', alpha=0.5)
ax.fill_between(eps_vals, auc_vals, 0.5, alpha=0.08, color=COLORS['danger'])
ax.set_xlabel('Label Smoothing ε', fontsize=12, fontweight='medium'); ax.set_ylabel('MIA AUC', fontsize=12, fontweight='medium', color=COLORS['danger']); ax2.set_ylabel('Utility (%)', fontsize=12, fontweight='medium', color=COLORS['accent']); ax.set_title('Label Smoothing Trends', fontsize=14, fontweight='bold', pad=15); ax.tick_params(axis='y', labelcolor=COLORS['danger']); ax2.tick_params(axis='y', labelcolor=COLORS['accent']); ax2.spines['right'].set_color(COLORS['accent']); ax2.spines['left'].set_color(COLORS['danger']); lines = line1 + line2; labels = [l.get_label() for l in lines]
# 图例移至右下角安全区
ax.legend(lines, labels, fontsize=10, facecolor=COLORS['bg'], edgecolor='none', loc='lower right')
ax = axes[1]; sig_vals = OP_SIGMAS; auc_op = [gm(k, 'auc') for k in OP_KEYS]; ax.plot(sig_vals, auc_op, 'o-', color=COLORS['success'], lw=3, ms=9, zorder=5, label='MIA AUC'); ax.axhline(bl_auc, color=COLORS['danger'], ls='--', lw=2, alpha=0.6, label=f'Baseline ({bl_auc:.4f})'); ax.axhline(0.5, color=COLORS['text_dim'], ls=':', alpha=0.5, label='Random (0.5)'); ax.fill_between(sig_vals, auc_op, bl_auc, alpha=0.2, color=COLORS['success'], label='AUC Reduction')
ax2r = ax.twinx(); ax2r.axhline(bl_acc, color=COLORS['success'], ls='-', lw=2.5, alpha=0.8); ax2r.set_ylabel(f'Utility = {bl_acc:.1f}% (unchanged)', fontsize=12, fontweight='medium', color=COLORS['success']); ax2r.set_ylim(0,100); ax2r.tick_params(axis='y', labelcolor=COLORS['success']); ax2r.spines['right'].set_color(COLORS['success'])
ax.set_xlabel('Perturbation σ', fontsize=12, fontweight='medium'); ax.set_ylabel('MIA AUC', fontsize=12, fontweight='medium'); ax.set_title('Output Perturbation Trends', fontsize=14, fontweight='bold', pad=15)
# 图例移至左下角安全区
ax.legend(fontsize=10, facecolor=COLORS['bg'], edgecolor='none', loc='lower left'); plt.tight_layout()
return fig
# ================================================================
# 回调函数 (维持你原有的所有核心逻辑不变)
# ================================================================
def cb_sample(src):
pool = member_data if "训练集" in src else non_member_data
s = pool[np.random.randint(len(pool))]
m = s['metadata']
md = f"""
<table style="width:100%; border-collapse: collapse; border: 1px solid #E2E8F0; border-radius: 8px; overflow: hidden;">
<tr style="background-color: #F9F9F9;">
<th style="padding: 10px; text-align: left; color: #86868B; font-weight: 600; border-bottom: 1px solid #E2E8F0;">字段</th>
<th style="padding: 10px; text-align: left; color: #86868B; font-weight: 600; border-bottom: 1px solid #E2E8F0;">值</th>
</tr>
<tr><td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">姓名</td><td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">{clean_text(str(m.get('name','')))}</td></tr>
<tr><td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">学号</td><td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">{clean_text(str(m.get('student_id','')))}</td></tr>
<tr><td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">班级</td><td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">{clean_text(str(m.get('class','')))}</td></tr>
<tr><td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">成绩</td><td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">{clean_text(str(m.get('score','')))} 分</td></tr>
<tr><td style="padding: 10px; color: #1D1D1F;">类型</td><td style="padding: 10px; color: #1D1D1F;">{TYPE_CN.get(s.get('task_type',''), '')}</td></tr>
</table>
"""
return md, clean_text(s.get('question', '')), clean_text(s.get('answer', ''))
ATK_CHOICES = (
["基线模型 (Baseline)"] +
[f"标签平滑 (ε={e})" for e in [0.02, 0.05, 0.1, 0.2]] +
[f"输出扰动 (σ={s})" for s in OP_SIGMAS]
)
ATK_MAP = {"基线模型 (Baseline)": "baseline"}
for e in [0.02, 0.05, 0.1, 0.2]: ATK_MAP[f"标签平滑 (ε={e})"] = f"smooth_eps_{e}"
for s in OP_SIGMAS: ATK_MAP[f"输出扰动 (σ={s})"] = f"perturbation_{s}"
def cb_attack(idx, src, target):
is_mem = "训练集" in src
pool = member_data if is_mem else non_member_data
idx = min(int(idx), len(pool)-1)
sample = pool[idx]
key = ATK_MAP.get(target, "baseline")
is_op = key.startswith("perturbation_")
if is_op:
sigma = float(key.split("_")[1])
fr = full_losses.get('baseline', {})
lk = 'member_losses' if is_mem else 'non_member_losses'
ll = fr.get(lk, [])
base_loss = ll[idx] if idx < len(ll) else float(np.random.normal(bl_m_mean if is_mem else bl_nm_mean, 0.02))
np.random.seed(idx*1000 + int(sigma*10000))
loss = base_loss + np.random.normal(0, sigma)
mm = gm(key, "member_loss_mean", 0.19)
nm_m = gm(key, "non_member_loss_mean", 0.20)
ms = gm(key, "member_loss_std", np.sqrt(0.03**2 + sigma**2))
ns = gm(key, "non_member_loss_std", np.sqrt(0.03**2 + sigma**2))
auc_v = gm(key, "auc")
lbl = f"OP(σ={sigma})"
else:
info = mia_results.get(key, mia_results.get('baseline', {}))
fr = full_losses.get(key, full_losses.get('baseline', {}))
lk = 'member_losses' if is_mem else 'non_member_losses'
ll = fr.get(lk, [])
loss = ll[idx] if idx < len(ll) else float(np.random.normal(info.get('member_loss_mean',0.19), 0.02))
mm = info.get('member_loss_mean', 0.19); nm_m = info.get('non_member_loss_mean', 0.20)
ms = info.get('member_loss_std', 0.03); ns = info.get('non_member_loss_std', 0.03)
auc_v = info.get('auc', 0)
lbl = "Baseline" if key == "baseline" else f"LS(ε={key.replace('smooth_eps_','')})"
thr = (mm + nm_m) / 2
pred = loss < thr
correct = pred == is_mem
gauge = fig_gauge(loss, mm, nm_m, thr, ms, ns)
pl = "🔴 训练成员" if pred else "🟢 非训练成员"
al = "🔴 训练成员" if is_mem else "🟢 非训练成员"
if correct and pred and is_mem:
v = f"<div style='background-color: #FFEBEE; border-left: 4px solid {COLORS['danger']}; padding: 12px; border-radius: 8px; color: {COLORS['danger']}; box-shadow: 0 2px 5px rgba(0,0,0,0.05); margin-top: 0px;'>⚠️ <b>攻击成功:隐私泄露</b><br><span style='font-size: 0.9em; color: #B71C1C;'>模型对该样本过于熟悉(Loss < 阈值),攻击者成功判定为训练数据。</span></div>"
elif correct:
v = f"<div style='background-color: #E8F5E9; border-left: 4px solid {COLORS['success']}; padding: 12px; border-radius: 8px; color: {COLORS['success']}; box-shadow: 0 2px 5px rgba(0,0,0,0.05); margin-top: 0px;'>✅ <b>判定正确</b><br><span style='font-size: 0.9em; color: #1B5E20;'>攻击者判定与真实身份一致。</span></div>"
else:
v = f"<div style='background-color: #E3F2FD; border-left: 4px solid {COLORS['accent']}; padding: 12px; border-radius: 8px; color: {COLORS['accent']}; box-shadow: 0 2px 5px rgba(0,0,0,0.05); margin-top: 0px;'>🛡️ <b>防御成功</b><br><span style='font-size: 0.9em; color: #0D47A1;'>攻击者判定错误,防御起到了保护作用。</span></div>"
table_html = f"""
<table style="width:100%; border-collapse: collapse; margin-top: 10px; border: 1px solid #E2E8F0; border-radius: 8px; overflow: hidden;">
<thead style="background-color: #F9F9F9;">
<tr>
<th style="padding: 10px; text-align: left; color: #86868B; font-weight: 600; border-bottom: 1px solid #E2E8F0;">项目</th>
<th style="padding: 10px; text-align: left; color: #86868B; font-weight: 600; border-bottom: 1px solid #E2E8F0;">攻击者判定</th>
<th style="padding: 10px; text-align: left; color: #86868B; font-weight: 600; border-bottom: 1px solid #E2E8F0;">真实身份</th>
</tr>
</thead>
<tbody>
<tr>
<td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">身份</td>
<td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">{pl}</td>
<td style="padding: 10px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">{al}</td>
</tr>
<tr>
<td style="padding: 10px; color: #1D1D1F;">Loss / 阈值</td>
<td style="padding: 10px; color: #1D1D1F;">Loss: {loss:.4f}</td>
<td style="padding: 10px; color: #1D1D1F;">阈值: {thr:.4f}</td>
</tr>
</tbody>
</table>
"""
res = v + f"<div style='font-weight: 600; margin: 12px 0 8px 0;'>🎯 攻击目标: {lbl} <span style='margin-left: 20px; color: #86868B;'>📊 AUC: {auc_v:.4f}</span></div>" + table_html
qtxt = f"**样本 #{idx}**\n\n" + clean_text(sample.get('question',''))[:500]
return qtxt, gauge, res
EVAL_CHOICES = (
["基线模型"] +
[f"标签平滑 (ε={e})" for e in [0.02, 0.05, 0.1, 0.2]] +
[f"输出扰动 (σ={s})" for s in OP_SIGMAS]
)
EVAL_KEY_MAP = {"基线模型": "baseline"}
for e in [0.02, 0.05, 0.1, 0.2]: EVAL_KEY_MAP[f"标签平滑 (ε={e})"] = f"smooth_eps_{e}"
for s in OP_SIGMAS: EVAL_KEY_MAP[f"输出扰动 (σ={s})"] = "baseline"
def cb_eval(model_choice):
k = EVAL_KEY_MAP.get(model_choice, "baseline")
acc = gu(k) if "输出扰动" not in model_choice else bl_acc
q = EVAL_POOL[np.random.randint(len(EVAL_POOL))]
ok = q.get(k, q.get('baseline', False))
ic = "✅ 正确" if ok else "❌ 错误"
note = "\n\n> 输出扰动不改变模型参数,准确率与基线一致。" if "输出扰动" in model_choice else ""
table_html = f"""
<table style="width:100%; border-collapse: collapse; margin-top: 15px; border: 1px solid #E2E8F0; border-radius: 8px; overflow: hidden;">
<tbody>
<tr><td style="padding: 12px; color: #86868B; font-weight: 600; border-bottom: 1px solid #E2E8F0; width: 100px;">类型</td><td style="padding: 12px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">{q['type_cn']}</td></tr>
<tr><td style="padding: 12px; color: #86868B; font-weight: 600; border-bottom: 1px solid #E2E8F0;">题目</td><td style="padding: 12px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">{q['question']}</td></tr>
<tr><td style="padding: 12px; color: #86868B; font-weight: 600; border-bottom: 1px solid #E2E8F0;">正确答案</td><td style="padding: 12px; color: #1D1D1F; border-bottom: 1px solid #E2E8F0;">{q['answer']}</td></tr>
<tr><td style="padding: 12px; color: #86868B; font-weight: 600;">判定</td><td style="padding: 12px; color: #1D1D1F;">{ic}</td></tr>
</tbody>
</table>
"""
return (f"<div style='font-weight: 600; margin-bottom: 10px;'>🤖 模型: {model_choice} <span style='margin-left: 20px; color: #86868B;'>🎯 准确率: {acc:.1f}%</span></div>" + table_html + note)
def build_full_table():
rows = []
for k, l in zip(LS_KEYS, LS_LABELS_MD):
if k in mia_results:
m = mia_results[k]; u = gu(k)
t = "—" if k == "baseline" else "训练期"; d = "" if k == "baseline" else f"{m['auc']-bl_auc:+.4f}"
rows.append(f"| {l} | {t} | {m['auc']:.4f} | {m['attack_accuracy']:.4f} | {m['precision']:.4f} | {m['recall']:.4f} | {m['f1']:.4f} | {m['tpr_at_5fpr']:.4f} | {m['tpr_at_1fpr']:.4f} | {m['loss_gap']:.4f} | {u:.1f}% | {d} |")
for k, l in zip(OP_KEYS, OP_LABELS_MD):
if k in perturb_results:
m = perturb_results[k]; d = f"{m['auc']-bl_auc:+.4f}"
rows.append(f"| {l} | 推理期 | {m['auc']:.4f} | {m['attack_accuracy']:.4f} | {m['precision']:.4f} | {m['recall']:.4f} | {m['f1']:.4f} | {m['tpr_at_5fpr']:.4f} | {m['tpr_at_1fpr']:.4f} | {m['loss_gap']:.4f} | {bl_acc:.1f}% | {d} |")
header = ("| 策略 | 类型 | AUC | Acc | Prec | Rec | F1 | TPR@5% | TPR@1% | LossGap | 效用 | ΔAUC |\n"
"|---|---|---|---|---|---|---|---|---|---|---|---|")
return header + "\n" + "\n".join(rows)
# ================================================================
# CSS - 简约苹果风
# ================================================================
CSS = """
:root {
--primary-blue: #007AFF;
--bg-light: #F5F5F7;
--card-bg: #FFFFFF;
--text-dark: #1D1D1F;
--text-gray: #86868B;
--border-color: #D2D2D7;
}
body { background-color: var(--bg-light) !important; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif !important; color: var(--text-dark) !important; }
.gradio-container { max-width: 1350px !important; margin: 40px auto !important; }
.title-area { background-color: var(--card-bg); padding: 32px 40px; border-radius: 18px; box-shadow: 0 4px 12px rgba(0, 0, 0, 0.05); margin-bottom: 30px; text-align: center; }
.title-area h1 { color: var(--text-dark) !important; font-size: 2.2rem !important; font-weight: 700 !important; margin-bottom: 10px !important; letter-spacing: -0.5px; }
.title-area p { color: var(--text-gray) !important; font-size: 1.1rem !important; margin-bottom: 15px !important; }
.title-area .badge { display: inline-block; background-color: #E5F1FF; color: var(--primary-blue); padding: 6px 16px; border-radius: 20px; font-size: 0.9rem; font-weight: 600; }
.tabitem { background-color: var(--card-bg) !important; border-radius: 18px !important; border: none !important; box-shadow: 0 8px 24px rgba(0, 0, 0, 0.08) !important; padding: 40px !important; margin-top: 20px !important; }
.tab-nav { border-bottom: none !important; gap: 10px !important; background: transparent !important; padding-bottom: 5px !important; }
.tab-nav button { font-size: 15px !important; padding: 10px 20px !important; font-weight: 500 !important; color: var(--text-gray) !important; background: rgba(0,0,0,0.03) !important; border: none !important; border-radius: 12px !important; transition: all 0.2s ease !important; }
.tab-nav button:hover { background: rgba(0,0,0,0.06) !important; color: var(--text-dark) !important; }
.tab-nav button.selected { color: var(--primary-blue) !important; background: #E5F1FF !important; font-weight: 600 !important; }
.prose { color: var(--text-dark) !important; }
.prose h2 { color: var(--text-dark) !important; font-weight: 700 !important; border-bottom: 1px solid var(--border-color) !important; padding-bottom: 12px !important; margin-top: 30px !important; }
.prose h3 { color: var(--text-dark) !important; font-weight: 600 !important; margin-top: 24px !important; }
.prose h4 { color: var(--text-gray) !important; font-weight: 600 !important; margin-bottom: 12px !important; }
.prose table { border-collapse: separate !important; border-spacing: 0 !important; width: 100% !important; border: 1px solid var(--border-color) !important; border-radius: 12px !important; overflow: hidden !important; box-shadow: 0 2px 8px rgba(0,0,0,0.04) !important; }
.prose th { background: #F9F9F9 !important; color: var(--text-gray) !important; font-weight: 600 !important; padding: 14px 18px !important; text-align: left !important; border-bottom: 1px solid var(--border-color) !important; white-space: nowrap !important; }
.prose td { padding: 14px 18px !important; color: var(--text-dark) !important; border-bottom: 1px solid var(--border-color) !important; background: var(--card-bg) !important; white-space: nowrap !important; }
.prose tr:last-child td { border-bottom: none !important; }
.prose tr:hover td { background: #F5F7FA !important; }
button.primary { background-color: var(--primary-blue) !important; color: white !important; border: none !important; border-radius: 10px !important; font-weight: 600 !important; padding: 12px 24px !important; box-shadow: 0 2px 6px rgba(0, 122, 255, 0.25) !important; transition: all 0.2s !important; }
button.primary:hover { background-color: #0062CC !important; box-shadow: 0 4px 10px rgba(0, 122, 255, 0.35) !important; transform: translateY(-1px) !important; }
.card-wrap { background: var(--card-bg) !important; border: 1px solid var(--border-color) !important; border-radius: 14px !important; padding: 24px !important; box-shadow: 0 2px 8px rgba(0,0,0,0.04) !important; }
.block.svelte-12cmxck { border-radius: 12px !important; border-color: var(--border-color) !important; }
.input-label { color: var(--text-gray) !important; font-weight: 500 !important; }
footer { display: none !important; }
"""
# ================================================================
# UI 布局构建
# ================================================================
with gr.Blocks(title="MIA攻防研究", theme=gr.themes.Soft(), css=CSS) as demo:
gr.HTML("""<div class="title-area">
<h1>🎓 教育大模型中的成员推理攻击及其防御研究</h1>
<p>Membership Inference Attack & Defense on Educational LLM</p>
<div class="badge">✨ 11组实验 × 8维指标 × 2种策略</div>
</div>""")
with gr.Tab("📊 实验总览"):
gr.Markdown(f"""
## 📌 研究背景:为什么教育大模型需要防范 MIA?
在教育领域,大模型(如虚拟辅导老师)的训练往往离不开学生真实的互动数据,而这些数据中包含了大量**极度敏感的个人隐私**。本研究基于 **{model_name}** 微调的数学辅导模型,系统揭示并解决这一安全隐患。
### 1️⃣ 什么是成员推理攻击 (MIA)?
**成员推理攻击 (Membership Inference Attack)** 的核心目的,是判断“某一条特定的数据,到底有没有被用来训练过这个AI?”
* **测谎仪原理**:大模型有一种“偷懒”的天性,对于它在训练时见过的“旧题”(成员数据),它回答得会极其顺畅,**损失值(Loss)非常低**;而面对没见过的“新题”(非成员数据),Loss 会偏高。攻击者正是利用这个 Loss 差距来做判定。
### 2️⃣ 教育大模型中的 MIA 危害有多大?(结合实验数据)
想象一下,我们系统后台有这样一条真实的训练数据:
> *“老师您好,我是**李明(学号20231001)**。我上次数学只考了**55分**,计算题老是错,请问 25+37 等于多少?”*
如果学校直接用这些记录训练了AI,恶意攻击者就可以拿着这句话去“套话”。如果 AI 表现出“极度熟悉”(Loss极低),攻击者就能推断出:**“李明确实在这个学校,且上次数学不及格。”** 学生的姓名、学号、成绩短板等核心隐私将彻底暴露!
### 3️⃣ 我们如何进行防御?
为了打破攻击者的“测谎仪”,本研究引入了两大防御流派,并探讨了它们在保护隐私与维持 AI 教学智商(效用)之间的平衡:
* 🛡️ **标签平滑 (Label Smoothing, 训练期)**:从小教育 AI“不要死记硬背”。在训练时强行引入不确定性,逼迫 AI 去学习加减乘除的通用规律,而不是死记李明的名字和分数。
* 🛡️ **输出扰动 (Output Perturbation, 推理期)**:给 AI 的输出加上“变声器”。在攻击者探查 Loss 值时,强行混入高斯噪声(加沙子),让攻击者看到的 Loss 忽高忽低,彻底瞎掉,但普通用户看到的文字回答依然绝对正确。
""")
if os.path.exists(os.path.join(BASE_DIR, "figures", "algo4_overview_cn_final.png")):
gr.Image(os.path.join(BASE_DIR, "figures", "algo4_overview_cn_final.png"), label="实验体系总览", show_label=True)
gr.HTML(f"""<div style="display:grid;grid-template-columns:repeat(4,1fr);gap:20px;margin:30px 0;">
<div class="card-wrap" style="text-align:center;">
<div style="font-size:32px;font-weight:700;color:{COLORS['accent']};margin-bottom:8px;">5</div>
<div style="font-size:14px;color:{COLORS['text_dim']};font-weight:600;">训练模型</div>
<div style="font-size:30px;margin-top:10px;">🤖</div>
</div>
<div class="card-wrap" style="text-align:center;">
<div style="font-size:32px;font-weight:700;color:{COLORS['accent2']};margin-bottom:8px;">6</div>
<div style="font-size:14px;color:{COLORS['text_dim']};font-weight:600;">扰动配置</div>
<div style="font-size:30px;margin-top:10px;">🎛️</div>
</div>
<div class="card-wrap" style="text-align:center;">
<div style="font-size:32px;font-weight:700;color:{COLORS['success']};margin-bottom:8px;">8</div>
<div style="font-size:14px;color:{COLORS['text_dim']};font-weight:600;">评估指标</div>
<div style="font-size:30px;margin-top:10px;">📈</div>
</div>
<div class="card-wrap" style="text-align:center;">
<div style="font-size:32px;font-weight:700;color:{COLORS['warning']};margin-bottom:8px;">2000</div>
<div style="font-size:14px;color:{COLORS['text_dim']};font-weight:600;">测试样本</div>
<div style="font-size:30px;margin-top:10px;">📄</div>
</div>
</div>""")
with gr.Accordion("📋 完整实验结果表(11组 × 8维度)", open=True):
gr.Markdown(build_full_table())
with gr.Tab("📁 数据与模型"):
gr.HTML("""<div style="display:flex; flex-direction:row; gap:25px; margin-bottom:30px; align-items:stretch;">
<div class="card-wrap" style="flex:1; display:flex; flex-direction:column;">
<h3 style="margin:0 0 15px;font-size:18px;color:#1D1D1F;">📦 数据组成</h3>
<table style="width:100%;border-collapse:collapse;font-size:14px; margin-bottom:auto;">
<tr style="background:#F9F9F9;"><th style="padding:12px;text-align:left;color:#86868B;">数据组</th><th style="padding:12px;text-align:left;color:#86868B;">数量</th><th style="padding:12px;text-align:left;color:#86868B;">用途</th><th style="padding:12px;text-align:left;color:#86868B;">说明</th></tr>
<tr><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">🔴 成员数据</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">1000条</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">模型训练</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">Loss偏低</td></tr>
<tr><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">🟢 非成员数据</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">1000条</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">攻击对照</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">Loss偏高</td></tr>
</table>
</div>
<div class="card-wrap" style="flex:1; display:flex; flex-direction:column;">
<h3 style="margin:0 0 15px;font-size:18px;color:#1D1D1F;">📚 任务分布</h3>
<table style="width:100%;border-collapse:collapse;font-size:14px; margin-bottom:auto;">
<tr style="background:#F9F9F9;"><th style="padding:12px;text-align:left;color:#86868B;">类别</th><th style="padding:12px;text-align:left;color:#86868B;">数量</th><th style="padding:12px;text-align:left;color:#86868B;">占比</th></tr>
<tr><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">🔢 基础计算</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">800</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">40%</td></tr>
<tr><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">📝 应用题</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">600</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">30%</td></tr>
<tr><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">💬 概念问答</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">400</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">20%</td></tr>
<tr><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">✏️ 错题订正</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">200</td><td style="padding:12px;border-bottom:1px solid #E2E8F0;color:#1D1D1F;">10%</td></tr>
</table>
</div></div>""")
gr.HTML(f'<div style="background:#FFF4E5; border-left:4px solid {COLORS["warning"]}; padding:16px; border-radius:12px; margin-bottom:30px; font-size:14px; color:#663C00; box-shadow: 0 2px 6px rgba(0,0,0,0.05);">⚠️ <b>注意:</b>两组数据格式完全相同(均含隐私字段),这是MIA实验的标准设置——攻击者无法从格式区分。</div>')
gr.Markdown("### 🔍 数据样例提取")
with gr.Row():
with gr.Column(scale=1):
gr.Markdown("#### ⚙️ 提取控制台")
d_src = gr.Radio(["成员数据(训练集)", "非成员数据(测试集)"], value="成员数据(训练集)", label="目标数据源")
d_btn = gr.Button("🎲 随机提取样本", variant="primary")
d_meta = gr.HTML()
with gr.Column(scale=2):
gr.Markdown("#### 📄 样本详情")
d_q = gr.Textbox(label="🧑🎓 学生提问 (Prompt)", lines=6, interactive=False)
d_a = gr.Textbox(label="💡 标准回答 (Ground Truth)", lines=6, interactive=False)
d_btn.click(cb_sample, [d_src], [d_meta, d_q, d_a])
with gr.Tab("🧠 算法原理"):
gr.Markdown("## 算法流程图与伪代码")
gr.Markdown("### Algorithm 1: 基于Loss的成员推理攻击 (MIA)")
if os.path.exists(os.path.join(BASE_DIR, "figures", "algo1_mia_attack.png")):
gr.Image(os.path.join(BASE_DIR, "figures", "algo1_mia_attack.png"), show_label=False)
gr.Markdown(f"""\
> **原理讲解:** MIA利用了“模型对训练数据记忆更深”这一现象。当模型“见过”某条数据时,它的预测不确定性更低,表现为**Loss偏低**。攻击者正是利用这个差异来判断数据是否属于训练集。
>
> 本实验中,基线模型的成员平均Loss={bl_m_mean:.4f},非成员平均Loss={bl_nm_mean:.4f},差距{bl_nm_mean-bl_m_mean:.4f},足以被攻击者利用。
""")
gr.Markdown("---\n### Algorithm 2: 标签平滑防御(训练期)")
if os.path.exists(os.path.join(BASE_DIR, "figures", "algo2_label_smoothing.png")):
gr.Image(os.path.join(BASE_DIR, "figures", "algo2_label_smoothing.png"), show_label=False)
gr.Markdown("""\
> **原理讲解:** 标签平滑将one-hot硬标签软化为概率分布。例如,原始标签[0,0,1,0]变为[0.033,0.033,0.9,0.033]。这迫使模型不再“100%确定”某个答案,从而降低对训练数据的过度记忆。
>
> 副作用:正则化效应还能防止过拟合,提升泛化能力。这就是为什么效用会反升的原因。
""")
gr.Markdown("---\n### Algorithm 3: 输出扰动防御(推理期)")
if os.path.exists(os.path.join(BASE_DIR, "figures", "algo3_output_perturbation.png")):
gr.Image(os.path.join(BASE_DIR, "figures", "algo3_output_perturbation.png"), show_label=False)
gr.Markdown("""\
> **原理讲解:** 输出扰动不修改模型本身,而是在返回给攻击者的Loss值上加入随机噪声。攻击者看到的是被噪声污染的Loss,无法精确判断是否低于阈值。
>
> 优势:①不需重新训练 ②即插即用 ③不影响模型回答质量(因为只扰动Loss,不扰动生成结果)
""")
with gr.Tab("🎯 攻击验证"):
gr.Markdown("## 🕵️ 成员推理攻击交互演示\n\n配置攻击目标与数据源,系统将执行 Loss 计算并映射判定边界。")
with gr.Row():
with gr.Column(scale=1):
gr.Markdown("#### ⚙️ 攻击配置台")
a_t = gr.Dropdown(choices=ATK_CHOICES, value=ATK_CHOICES[0], label="🎯 选择被攻击模型", interactive=True)
a_s = gr.Radio(["成员数据(训练集)", "非成员数据(测试集)"], value="成员数据(训练集)", label="📂 输入数据源")
a_i = gr.Slider(0, 999, step=1, value=12, label="📌 定位样本 ID")
a_b = gr.Button("⚡ 执行成员推理攻击", variant="primary")
a_qt = gr.HTML()
with gr.Column(scale=2):
gr.Markdown("#### 📉 攻击结果与 Loss 边界")
a_g = gr.Plot(label="Loss位置判定 (Decision Boundary)")
a_r = gr.HTML()
a_b.click(cb_attack, [a_i, a_s, a_t], [a_qt, a_g, a_r])
with gr.Tab("🛡️ 五维度攻防分析"):
gr.Markdown("## 多维度攻防效果完整论证")
gr.HTML('<div style="margin:20px 0 8px;"><span class="dim-label dim1">维度一</span><strong style="font-size:18px;color:#1D2939;">宏观评价维度 — 证明“总体攻防能力”</strong></div>')
gr.Markdown(f"""\
> **攻击有效性:** 基线(Baseline)状态下,ROC 曲线明显凸起,AUC 达到了 **{bl_auc:.4f}**。证明模型确实记住了学生的隐私。
> **防御有效性:** 施加防御后(无论是 LS 还是 OP),随着参数强度的增加,AUC 柱子显著变矮,且 ROC 曲线几乎被完全压平(贴近对角线)。防御从根本上瓦解了攻击。
""")
gr.Plot(value=fig_auc_bar())
gr.Plot(value=fig_roc_curves())
gr.HTML('<div style="margin:40px 0 8px;"><span class="dim-label dim2">维度二</span><strong style="font-size:18px;color:#1D2939;">极限实战维度 — 证明“极低误报下的安全底线”</strong></div>')
gr.Markdown(f"""\
> **实战意义:** 现实中黑客只允许极低的误报(如 1%)。在 Baseline 中,1% 误报率下黑客依然能精准窃取 **{gm('baseline','tpr_at_1fpr')*100:.1f}%** 的真实隐私(红柱子极高)。
> 开启 OP(σ=0.03) 防御后,该成功率被死死压制到了 **{gm('perturbation_0.03','tpr_at_1fpr')*100:.1f}%**。这证明在最极端的实战条件下,防线依然坚固。
""")
gr.Plot(value=fig_tpr_at_low_fpr())
gr.HTML('<div style="margin:40px 0 8px;"><span class="dim-label dim3">维度三</span><strong style="font-size:18px;color:#1D2939;">机制溯源维度 — 证明“底层物理逻辑”</strong></div>')
gr.Markdown(f"""\
> **攻击根源:** 模型对“背过”的数据给的 Loss 更低。基线状态下,蓝红两座山峰明显错位,均值差距达到了 **{gm('baseline','loss_gap'):.4f}**。
> **LS 的防御本质:** 随着 ε 增大,两座山峰趋于完美重合,均值差距缩小到了 {gm('smooth_eps_0.2','loss_gap'):.4f}。这是从物理上抹除了模型记忆。
> **OP 的防御本质:** 均值差距未变,但高斯噪声导致分布变得极其扁平宽阔,红蓝区域被完全搅混,蒙蔽了攻击者的双眼。
""")
# 🌟 调用专门添加的三联横向对比直方图
gr.Plot(value=fig_d3_dist_compare())
gr.Plot(value=fig_loss_gap_waterfall())
with gr.Accordion("📉 查看所有模型详细 Loss 分布直方图", open=False):
gr.Plot(value=fig_loss_dist())
gr.Plot(value=fig_perturb_dist())
gr.HTML('<div style="margin:40px 0 8px;"><span class="dim-label dim4">维度四</span><strong style="font-size:18px;color:#1D2939;">无死角压制维度 — 证明“防御没有偏科”</strong></div>')
gr.Markdown("""\
> **为什么要看雷达图?** 为了证明防御不是拆东墙补西墙。红色的基线圈面积最大,代表攻击方在精确率、召回率、F1 等各个维度都非常嚣张。
> 无论是左图的 LS 还是右图的 OP,随着参数增加,**整个多边形在极其均匀地向内收缩**。证明我们的防线是 360 度无死角的。
""")
gr.Plot(value=fig_radar())
with gr.Accordion("📖 查看详细防御参数表格", open=False):
detail_md = ""
for eps in [0.02, 0.05, 0.1, 0.2]:
k = f"smooth_eps_{eps}"
detail_md += f"""\
### 标签平滑 LS(ε={eps})
| 指标 | 值 | vs基线 | 变化 |
|---|---|---|---|
| AUC | {gm(k,'auc'):.4f} | {bl_auc:.4f} | {gm(k,'auc')-bl_auc:+.4f} ({(gm(k,'auc')-bl_auc)/bl_auc*100:+.1f}%) |
| 攻击准确率 | {gm(k,'attack_accuracy'):.4f} | {gm('baseline','attack_accuracy'):.4f} | {gm(k,'attack_accuracy')-gm('baseline','attack_accuracy'):+.4f} |
| F1 | {gm(k,'f1'):.4f} | {gm('baseline','f1'):.4f} | {gm(k,'f1')-gm('baseline','f1'):+.4f} |
| TPR@5%FPR | {gm(k,'tpr_at_5fpr'):.4f} | {gm('baseline','tpr_at_5fpr'):.4f} | {gm(k,'tpr_at_5fpr')-gm('baseline','tpr_at_5fpr'):+.4f} |
| TPR@1%FPR | {gm(k,'tpr_at_1fpr'):.4f} | {gm('baseline','tpr_at_1fpr'):.4f} | {gm(k,'tpr_at_1fpr')-gm('baseline','tpr_at_1fpr'):+.4f} |
| Loss差距 | {gm(k,'loss_gap'):.4f} | {gm('baseline','loss_gap'):.4f} | {gm(k,'loss_gap')-gm('baseline','loss_gap'):+.4f} |
| 效用 | {gu(k):.1f}% | {bl_acc:.1f}% | {gu(k)-bl_acc:+.1f}% |
---
"""
for sigma in OP_SIGMAS:
k = f"perturbation_{sigma}"
detail_md += f"""\
### 输出扰动 OP(σ={sigma})
| 指标 | 值 | vs基线 | 变化 |
|---|---|---|---|
| AUC | {gm(k,'auc'):.4f} | {bl_auc:.4f} | {gm(k,'auc')-bl_auc:+.4f} ({(gm(k,'auc')-bl_auc)/bl_auc*100:+.1f}%) |
| 攻击准确率 | {gm(k,'attack_accuracy'):.4f} | {gm('baseline','attack_accuracy'):.4f} | {gm(k,'attack_accuracy')-gm('baseline','attack_accuracy'):+.4f} |
| F1 | {gm(k,'f1'):.4f} | {gm('baseline','f1'):.4f} | {gm(k,'f1')-gm('baseline','f1'):+.4f} |
| TPR@5%FPR | {gm(k,'tpr_at_5fpr'):.4f} | {gm('baseline','tpr_at_5fpr'):.4f} | {gm(k,'tpr_at_5fpr')-gm('baseline','tpr_at_5fpr'):+.4f} |
| TPR@1%FPR | {gm(k,'tpr_at_1fpr'):.4f} | {gm('baseline','tpr_at_1fpr'):.4f} | {gm(k,'tpr_at_1fpr')-gm('baseline','tpr_at_1fpr'):+.4f} |
| Loss差距 | {gm(k,'loss_gap'):.4f} | {gm('baseline','loss_gap'):.4f} | {gm(k,'loss_gap')-gm('baseline','loss_gap'):+.4f} |
| 效用 | {bl_acc:.1f}% | {bl_acc:.1f}% | 0.0% (零损失) |
---
"""
gr.Markdown(detail_md)
with gr.Tab("⚖️ 效用评估"):
gr.HTML('<div style="margin:20px 0 8px;"><span class="dim-label dim5">维度五</span><strong style="font-size:18px;color:#1D2939;">落地代价维度 — 证明“隐私与效用的完美平衡”</strong></div>')
gr.Markdown(f"""\
> 抛开模型能力谈安全是纸上谈兵。
> **输出扰动 (OP):** 实现了完美的 **零效用损耗(维持 {bl_acc:.1f}%)**。
> **标签平滑 (LS):** 打出了惊艳的 **双赢 (Win-Win)**,效用曲线逆势上扬到了 {gu('smooth_eps_0.2'):.1f}%(不仅保护了隐私,还治好了过拟合)。
""")
gr.Plot(value=fig_auc_trend())
gr.Markdown("## 📊 模型测试集准确率全景分析")
with gr.Row():
with gr.Column(): gr.Plot(value=fig_acc_bar())
with gr.Column(): gr.Plot(value=fig_tradeoff())
gr.Markdown("### 🧪 在线抽题演示")
with gr.Row():
with gr.Column(scale=1):
gr.Markdown("#### ⚙️ 测试配置")
e_m = gr.Dropdown(choices=EVAL_CHOICES, value="基线模型", label="🤖 选择测试模型", interactive=True)
e_b = gr.Button("🎲 随机抽题测试", variant="primary")
with gr.Column(scale=2):
gr.Markdown("#### 📝 模型作答结果")
e_r = gr.HTML()
e_b.click(cb_eval, [e_m], [e_r])
with gr.Tab("📝 研究结论"):
gr.Markdown(f"""\
## 核心研究发现
---
### 🎯 结论一:教育大模型存在可量化的MIA风险
基线模型的MIA攻击 AUC = **{bl_auc:.4f}**,显著高于随机猜测的0.5。攻击准确率达 **{gm('baseline','attack_accuracy')*100:.1f}%**,远超50%。在TPR@5%FPR={gm('baseline','tpr_at_5fpr'):.4f}的严格条件下,攻击者仍能识别近五分之一的训练成员。这证明教育大模型确实存在学生隐私泄露风险。
### 🛡️ 结论二:标签平滑是有效的训练期防御 (实现双赢)
| ε 参数 | AUC | AUC降幅 | 效用 | 效用变化 |
|---|---|---|---|---|
| ε=0.02 | {gm('smooth_eps_0.02','auc'):.4f} | {bl_auc-gm('smooth_eps_0.02','auc'):.4f} | {gu('smooth_eps_0.02'):.1f}% | {gu('smooth_eps_0.02')-bl_acc:+.1f}% |
| ε=0.05 | {gm('smooth_eps_0.05','auc'):.4f} | {bl_auc-gm('smooth_eps_0.05','auc'):.4f} | {gu('smooth_eps_0.05'):.1f}% | {gu('smooth_eps_0.05')-bl_acc:+.1f}% |
| ε=0.1 | {gm('smooth_eps_0.1','auc'):.4f} | {bl_auc-gm('smooth_eps_0.1','auc'):.4f} | {gu('smooth_eps_0.1'):.1f}% | {gu('smooth_eps_0.1')-bl_acc:+.1f}% |
| ε=0.2 | {gm('smooth_eps_0.2','auc'):.4f} | {bl_auc-gm('smooth_eps_0.2','auc'):.4f} | {gu('smooth_eps_0.2'):.1f}% | {gu('smooth_eps_0.2')-bl_acc:+.1f}% |
**重要发现:ε≥0.05时,隐私保护和模型效用同时提升。标签平滑的正则化效应防止了过拟合。**
### 🎭 结论三:输出扰动是有效的推理期防御 (零效用损耗)
| σ 参数 | AUC | AUC降幅 | 效用 |
|---|---|---|---|
| σ=0.005 | {gm('perturbation_0.005','auc'):.4f} | {bl_auc-gm('perturbation_0.005','auc'):.4f} | {bl_acc:.1f}% |
| σ=0.01 | {gm('perturbation_0.01','auc'):.4f} | {bl_auc-gm('perturbation_0.01','auc'):.4f} | {bl_acc:.1f}% |
| σ=0.015 | {gm('perturbation_0.015','auc'):.4f} | {bl_auc-gm('perturbation_0.015','auc'):.4f} | {bl_acc:.1f}% |
| σ=0.02 | {gm('perturbation_0.02','auc'):.4f} | {bl_auc-gm('perturbation_0.02','auc'):.4f} | {bl_acc:.1f}% |
| σ=0.025 | {gm('perturbation_0.025','auc'):.4f} | {bl_auc-gm('perturbation_0.025','auc'):.4f} | {bl_acc:.1f}% |
| σ=0.03 | {gm('perturbation_0.03','auc'):.4f} | {bl_auc-gm('perturbation_0.03','auc'):.4f} | {bl_acc:.1f}% |
**核心发现:零效用损失,不需重新训练,即插即用。**
### 💡 结论四:最佳实践建议
> **推荐组合方案: LS(ε=0.1) + OP(σ=0.02)**
>
> - **训练期 (治本):** 标签平滑从源头降低记忆,缩小Loss差距
> - **推理期 (治标):** 输出扰动遮蔽残余信号,进一步降低AUC
> - **两者机制互补,可叠加使用。**
""")
demo.launch(theme=gr.themes.Soft(), css=CSS) |