Hugging Face's logo

NLP-beginner
/
cs3319-project2

Model card Files
xet
 Community
cs3319-project2 / figures_paper /README_figures.md
NLP-beginner's picture
NLP-beginner
CS3319 Project 2 final deliverable (public F1 = 0.96626)
f28d994
preview code
|
Raw
History Blame Contribute Delete
12.3 kB

Figures for the CS3319 Project 2 paper

Auto-generated by figures_paper/scripts/make_all_figures.py. Every figure is emitted as .pdf (vector, pdf.fonttype=42 TrueType-embedded — venue-safe), .png (300 dpi preview) and .svg. Widths follow ACM sigconf: single-column 3.25 in, double-column 6.75 in.

Generation 

python figures_paper/scripts/make_all_figures.py --package-root .

Status

figure status files suggested placement
fig1_graph_overview ok fig1_graph_overview.pdf, fig1_graph_overview.png, fig1_graph_overview.svg Sec. Introduction / Task definition
fig2_sparsity ok fig2_sparsity.pdf, fig2_sparsity.png, fig2_sparsity.svg Sec. Dataset
fig3_roadmap ok fig3_roadmap.pdf, fig3_roadmap.png, fig3_roadmap.svg Sec. Results (overview)
fig4_architecture ok fig4_architecture.pdf, fig4_architecture.png, fig4_architecture.svg Sec. Method
fig5_ablation_highorder ok fig5_ablation_highorder.pdf, fig5_ablation_highorder.png, fig5_ablation_highorder.svg Sec. Ablation
fig6_rank_vs_threshold ok fig6_rank_vs_threshold.pdf, fig6_rank_vs_threshold.png, fig6_rank_vs_threshold.svg Sec. Method / Decision rule
fig7_error_heatmap ok fig7_error_heatmap.pdf, fig7_error_heatmap.png, fig7_error_heatmap.svg Sec. Error analysis
figA1_hyperparam ok figA1_hyperparam.pdf, figA1_hyperparam.png, figA1_hyperparam.svg Appendix
figA2_rw_ablation ok figA2_rw_ablation.pdf, figA2_rw_ablation.png, figA2_rw_ablation.svg Appendix
figA3_feature_importance ok figA3_feature_importance.pdf, figA3_feature_importance.png, figA3_feature_importance.svg Appendix
figA4_oof_pr ok figA4_oof_pr.pdf, figA4_oof_pr.png, figA4_oof_pr.svg Appendix

Captions, sources and notes

Figure 1. Task and heterogeneous graph overview

  • status: ok
  • data sources: schematic (no real graph)
  • files: figures_paper/{pdf,png,svg}/fig1_graph_overview.pdf, figures_paper/{pdf,png,svg}/fig1_graph_overview.png, figures_paper/{pdf,png,svg}/fig1_graph_overview.svg
  • caption draft: Heterogeneous author–paper graph and link-prediction task. Nodes are authors (circles, 6,611) and papers (squares, 79,937); relations are author–paper historical read/cite, author–author coauthorship, and directed paper→paper citation. For each test pair (author, paper) the model predicts a binary label; performance is measured by F1 over 2,047,262 test pairs.

Figure 2. Dataset sparsity and long-tail degree structure

  • status: ok
  • data sources: D:\reps\26H1_cs3319_final_deliverable\data_and_docs\author_file_ann.txt; D:\reps\26H1_cs3319_final_deliverable\data_and_docs\paper_file_ann.txt; D:\reps\26H1_cs3319_final_deliverable\data_and_docs\bipartite_train_ann.txt
  • files: figures_paper/{pdf,png,svg}/fig2_sparsity.pdf, figures_paper/{pdf,png,svg}/fig2_sparsity.png, figures_paper/{pdf,png,svg}/fig2_sparsity.svg
  • caption draft: Dataset sparsity and long-tail degree structure. (a–c) Complementary CDFs (log–log) of co-author degree, paper citation in-degree, and paper author-interaction degree; straight-line fits confirm approximate power-law tails. (d) The mass of nodes sits at degree 1 — 56% of authors have a single connection — so cold-start nodes dominate and motivate structural / high-order features.

Figure 3. Performance evolution roadmap

  • status: ok
  • data sources: validation_runs/dynamic_summary.csv; post95/extra/content/node2vec/high_order ablations; README public-leaderboard stage table
  • files: figures_paper/{pdf,png,svg}/fig3_roadmap.pdf, figures_paper/{pdf,png,svg}/fig3_roadmap.png, figures_paper/{pdf,png,svg}/fig3_roadmap.svg
  • caption draft: Performance evolution roadmap. Validation F1 (blue) and public-leaderboard F1 (green, dashed) across the method stages. Three conceptual leaps are shaded: (I) LightGCN captures the collaborative-filtering main signal; (II) explicit graph / meta-path features with a LightGBM stacker fill the local-structure blind spot (+0.0174 val); (III) random-walk embeddings and directed high-order citation propagation inject high-order structural semantics. The final model — rich content + 7 random-walk blocks + directed high-order propagation + LightGBM — reaches public F1 = 0.9663.

Figure 4. Final pipeline architecture

  • status: ok
  • data sources: schematic
  • files: figures_paper/{pdf,png,svg}/fig4_architecture.pdf, figures_paper/{pdf,png,svg}/fig4_architecture.png, figures_paper/{pdf,png,svg}/fig4_architecture.svg
  • caption draft: Final pipeline. Five feature streams — LightGCN score/rank, explicit graph and meta-path features, content similarity and rich author–content profiles, DeepWalk/Node2Vec pair features, and high-order propagation features — are concatenated into a 259-d vector and fed to a LightGBM out-of-fold stacker. A rank-cutoff decision (top 50% positive, known positives forced to 1) produces the submission. High-order features are defined by $H_k = R,C^{k}$ (author reaches papers $k$ citation hops away) and $G_k = S,R,C^{k}$ (same, mediated by coauthorship), where $R$ is the author–paper matrix, $C$ the citation matrix and $S$ the coauthorship matrix.

Figure 5. Ablation of high-order propagation

  • status: ok
  • data sources: D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\high_order_graph_stack\validation_summary.csv
  • files: figures_paper/{pdf,png,svg}/fig5_ablation_highorder.pdf, figures_paper/{pdf,png,svg}/fig5_ablation_highorder.png, figures_paper/{pdf,png,svg}/fig5_ablation_highorder.svg
  • caption draft: Ablation of high-order citation propagation (validation, seed=202). Bars are incremental F1 gains over a broken-axis floor; diamonds show AUC. Adding rich content and the 7-block random-walk ensemble lifts F1 by +0.00068; re-introducing undirected high-order propagation gives the largest single jump (+0.00161); the directed citation-aware variant contributes the decisive final +0.00032, reaching F1 = 0.96687 / AUC = 0.99492 and the public-best 0.9663. (Numbers from validation_summary.csv; fallback values used if the file is absent.)

Figure 6. Rank-cutoff vs probability-threshold robustness

  • status: ok
  • data sources: D:\reps\26H1_cs3319_final_deliverable\validation_runs\stack_ratio_analysis.csv; D:\reps\26H1_cs3319_final_deliverable\validation_runs\stack_threshold_summary.csv; D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\high_order_graph_stack\submission_summary.csv; final_report: test positive ratio 0.524195 at val-opt threshold
  • files: figures_paper/{pdf,png,svg}/fig6_rank_vs_threshold.pdf, figures_paper/{pdf,png,svg}/fig6_rank_vs_threshold.png, figures_paper/{pdf,png,svg}/fig6_rank_vs_threshold.svg
  • caption draft: Rank-cutoff vs probability-threshold robustness. The validation split is artificially 1:1 positive/negative, so the LightGBM probability does not transfer to the imbalanced test set: the validation-optimal probability threshold (0.462, and varying 0.44–0.49 across seeds) drifts to a 0.524 predicted-positive ratio on test, off the F1 peak (a). The rank-cutoff rule (predict the top-50% plus known positives) is class-balance invariant and stays exactly at 0.50 on both splits (b), making it the more robust decision rule.

Figure 7. Error bucket heatmap

  • status: ok
  • data sources: D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\error_group_calibration\error_analysis_buckets.csv
  • files: figures_paper/{pdf,png,svg}/fig7_error_heatmap.pdf, figures_paper/{pdf,png,svg}/fig7_error_heatmap.png, figures_paper/{pdf,png,svg}/fig7_error_heatmap.svg
  • caption draft: Error-bucket heatmap (validation, seed=202). Rows are curated node/pair buckets (author and paper degree, citation in-degree, author internal rank, local-evidence flag); columns are precision, recall and F1. Red-outlined rows have F1 below 0.90. The hard regions are cold-start nodes (author/paper degree ≤ 2), low in-citation papers, mid-range LightGCN-score pairs, and pairs without any local structural evidence — exactly the cases the high-order propagation and random-walk features are designed to rescue.

Appendix Figure A1. LightGCN hyperparameter sweep

  • status: ok
  • data sources: D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_summary.csv
  • files: figures_paper/{pdf,png,svg}/figA1_hyperparam.pdf, figures_paper/{pdf,png,svg}/figA1_hyperparam.png, figures_paper/{pdf,png,svg}/figA1_hyperparam.svg
  • caption draft: LightGCN hyperparameter sweep (best validation F1 per dim×layers cell, seed=202). Two propagation layers with a 512-d embedding is optimal (0.93858); a single layer under-fits and very large embeddings over-fit. This is the configuration used throughout.

Appendix Figure A2. Random-walk ensemble size vs F1

  • status: ok
  • data sources: D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\randomwalk_systematic\small_ablation_table.csv; D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\randomwalk_systematic\ensemble_5_ablation.csv; D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\randomwalk_systematic\ensemble_7_ablation.csv
  • files: figures_paper/{pdf,png,svg}/figA2_rw_ablation.pdf, figures_paper/{pdf,png,svg}/figA2_rw_ablation.png, figures_paper/{pdf,png,svg}/figA2_rw_ablation.svg
  • caption draft: Random-walk ensemble size vs validation F1 (same base features). Moving from the best single DeepWalk/Node2Vec config to a 5-block and then 7-block ensemble of diverse configs yields a steady +0.00182; 7 blocks are used in the final model.

Appendix Figure A3. Feature-group contribution

  • status: ok
  • data sources: post95/extra/content_rich/node2vec/randomwalk/high_order incremental ablations
  • files: figures_paper/{pdf,png,svg}/figA3_feature_importance.pdf, figures_paper/{pdf,png,svg}/figA3_feature_importance.png, figures_paper/{pdf,png,svg}/figA3_feature_importance.svg
  • caption draft: Cumulative F1 contribution by feature group (incremental ablation on seed=202). The graph/meta-path stacking with the LightGBM stacker dominates (+0.0174); the random-walk blocks, DeepWalk/Node2Vec and higher-order propagation form the second tier. Gains sum to 0.0283, exactly the LightGCN 0.9386 -> 0.9669 improvement, confirming internal consistency.

Appendix Figure A4. OOF discrimination across stages

  • status: ok
  • data sources: D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\val_labels_seed202.npy; D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\dyn202_l2d512_bpr_bigbatch_more\scores\val_vanilla_ensemble_mean.npy; D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\post95_ablation\ensemble_lgcn_oof.npy; D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\node2vec_deepwalk\node2vec_stack_oof.npy; D:\reps\26H1_cs3319_final_deliverable\validation_runs\dynamic_seed202\high_order_graph_stack\rich_rw7_highorder_directed_oof.npy
  • files: figures_paper/{pdf,png,svg}/figA4_oof_pr.pdf, figures_paper/{pdf,png,svg}/figA4_oof_pr.png, figures_paper/{pdf,png,svg}/figA4_oof_pr.svg
  • caption draft: Out-of-fold discrimination across stacking stages (validation, seed=202; labels alignment-verified). (a) Precision–recall curves move outward at each stage; the final high-order model reaches AP≈0.995. (b) Score distributions separate positives from negatives with a clean margin.

Inserting into LaTeX (ACM sigconf) 

\usepackage{graphicx}

% single-column (3.25 in)
\begin{figure}
  \centering
  \includegraphics[width=\columnwidth]{figures_paper/pdf/fig2_sparsity.pdf}
  \caption{...}
\end{figure}

% double-column (6.75 in)
\begin{figure*}
  \centering
  \includegraphics[width=\textwidth]{figures_paper/pdf/fig3_roadmap.pdf}
  \caption{...}
\end{figure*}

Use the .pdf for the camera-ready (vector, embedded fonts); the .png is only for previews / draft review.