Initial commit

.gitignore

@@ -0,0 +1,62 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# Virtual Environments
+venv/
+env/
+.env
+.venv/
+
+# IDEs
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# macOS
+.DS_Store
+.AppleDouble
+.LSOverride
+
+# Project Specific Outputs
+*.txt
+*.md
+!README.md
+*_only_used_entry.bib
+
+# LaTeX and Bibliography (User Data)
+# Ignoring these to prevent committing personal paper content
+*.tex
+*.bib
+*.pdf
+*.log
+*.aux
+*.out
+*.bbl
+*.blg
+*.synctex.gz
+*.fls
+*.fdb_latexmk
+
+# cache
+.cache

.gradio/certificate.pem

@@ -0,0 +1,31 @@
+-----BEGIN CERTIFICATE-----
+MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
+TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
+cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4
+WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu
+ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY
+MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc
+h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+
+0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U
+A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW
+T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH
+B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC
+B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv
+KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn
+OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn
+jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw
+qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI
+rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV
+HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq
+hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL
+ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ
+3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK
+NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5
+ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur
+TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC
+jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc
+oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq
+4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA
+mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d
+emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc=
+-----END CERTIFICATE-----

README.md

@@ -0,0 +1,84 @@
+# CiteScan: Check References, Confirm Truth.
+
+<!-- - **🎉 CiteScan is now available online. [https://huggingface.co/spaces/thinkwee/CiteScan](https://huggingface.co/spaces/thinkwee/CiteScan)!  🎉** -->
+
+
+**CiteScan**  is an open-source and free tool designed to detect hallucinated references in academic writing. As AI coding assistants and writing tools become more prevalent, they sometimes generate plausible-sounding citations that do not actually exist. **CiteScan**  addresses this issue by validating every bibliography entry against multiple authoritative academic databases—including arXiv, CrossRef, DBLP, Semantic Scholar, OpenAlex, and Google Scholar—to confirm their authenticity.
+
+Going beyond simple verification, **CiteScan** used rule-based algorithms to analyze whether the cited papers genuinely support the claims made in your text. Thanks to the free accessibility for academic databases across CS and AI areas, our system will **cost $0 for maintenance after development**. 
+
+
+## 🛡 Why CiteScan?
+
+-   **🚫 NO Hallucinations**: Annotate citations that don't exist or have mismatched metadata across year, authors, and title. 
+
+-   **📋 Ground Truth Reference**: Provide the link if the citations are flagged to *issued entry*. You can click the **Open paper** or **DOI** button to access the real-world metadata, and Then cite the BibTex from the press website.
+ 
+![Functions](assets/screenshot_performance_zh.png)
+
+-   **🏠 Top-tier Research Organizations**: Cooperate with National University of Singapore (NUS) and Shanghai Jiao Tong University (SJTU). 
+
+## References Validation
+-   **Multi-Source Verification**: Validates metadata against arXiv, CrossRef, DBLP, Semantic Scholar, OpenAlex, and Google Scholar
+
+
+
+![Citation](assets/screenshot_semantic_scholar.png)
+
+
+### Understanding Mismatches
+
+*False positive cases* occur for CiteScan:
+
+1.  **Authors Mismatch**:
+    - *Reason*: Different databases deal with a longer list of  authors with different strategies, like truncation.
+    - *Action*: Verify if main authors match
+
+2.  **Venues Mismatch**:
+    - *Reason*: Abbreviations vs. full names, such as "ICLR" v.s. "International Conference on Learning Representations"
+    - *Action*: Both are correct.
+
+3.  **Year GAP (±1 Year)**:
+    - *Reason*: Delay between preprint (arXiv) and final version publication 
+    - *Action*: Verify which version you intend to cite, We recommend you to cite the version from the official press website. Lower pre-print version bib will make your submission more confidence. 
+
+4.  **Non-academic Sources**:
+    - *Reason*: Blogs, and APIs are not indexed in academic databases.
+    - *Action*: Verify URL, year, and title manually.
+
+
+## 🙏 Acknowledgments
+
+CiteScan uses multiple data sources:
+- arXiv API
+- CrossRef API
+- Semantic Scholar API
+- DBLP API
+- OpenAlex API
+
+
+### Push
+```shell
+
+# 如果已经添加过 modelscope 远程，先删掉再加：
+git remote remove modelscope
+git remote add modelscope "http://oauth2:ms-28735aa7-04b6-4b21-b4e0-cfb464f3587f@www.modelscope.cn/studios/aivolcano/CiteScan.git"
+
+# 添加 ModelScope 为远程仓库
+git remote add modelscope "http://oauth2:ms-28735aa7-04b6-4b21-b4e0-cfb464f3587f@www.modelscope.cn/studios/aivolcano/CiteScan.git"
+
+# 推送到创空间(主分支是 main 时)
+git push modelscope main
+
+# 推送到创空间(主分支是 master 时)
+git push modelscope master
+
+
+# 若推送时报错
+git push modelscope main --force
+
+# 推送成功后
+https://www.modelscope.cn/studios/aivolcano/CiteScan
+# 创空间里点 「上线空间展示」 或 「立即发布」，等部署完成即可访问 Gradio 应用。
+```
+

Untitled

@@ -0,0 +1 @@
+citescan.yaml

app.py

@@ -0,0 +1,626 @@
+#!/usr/bin/env python3
+from tkinter.constants import TRUE
+import base64
+import gradio as gr
+import tempfile
+from pathlib import Path
+from concurrent.futures import ThreadPoolExecutor, as_completed
+import threading
+
+from src.parsers import BibParser
+from src.fetchers import ArxivFetcher, ScholarFetcher, CrossRefFetcher, SemanticScholarFetcher, OpenAlexFetcher, DBLPFetcher
+from src.analyzers import MetadataComparator, DuplicateDetector
+from src.report.generator import EntryReport
+from src.config.workflow import get_default_workflow
+
+
+def format_entry_card(entry_report, duplicate_groups=None):
+    """格式化单个文献条目为 HTML 卡片"""
+    entry = entry_report.entry
+    comparison = entry_report.comparison
+
+    # 判断状态
+    if comparison and comparison.is_match:
+        card_class = "verified"
+        status_icon = "✓"
+        status_text = "Verified"
+    elif comparison and comparison.has_issues:
+        card_class = "warning"
+        status_icon = "⚠️"
+        status_text = "Issues Found"
+    else:
+        card_class = "error"
+        status_icon = "✗"
+        status_text = "Not Found"
+
+    # 格式化 BibTeX（BibEntry 用 raw_entry 存原始字段）
+    bibtex_str = f"@{entry.entry_type}{{{entry.key},\n"
+    for field, value in (entry.raw_entry or {}).items():
+        if field in ("ID", "ENTRYTYPE"):
+            continue
+        if value is not None and str(value).strip():
+            bibtex_str += f"  {field}={{{value}}},\n"
+    bibtex_str = bibtex_str.rstrip(",\n") + "\n}"
+
+    # Link for header (Open paper / DOI) when we have reference
+    link_url = ""
+    link_label = "Open paper"
+    if comparison and getattr(comparison, "source", "") != "unable":
+        if getattr(comparison, "fetched_doi", None) and str(comparison.fetched_doi).strip():
+            link_url = "https://doi.org/" + str(comparison.fetched_doi).strip()
+            link_label = "DOI"
+        elif getattr(comparison, "fetched_url", None) and str(comparison.fetched_url).strip():
+            link_url = str(comparison.fetched_url).strip()
+
+    # 收集标签
+    tags = []
+    if comparison:
+        if comparison.is_match:
+            tags.append(('<span class="tag success">✓ Verified</span>', 0))
+        if comparison.source:
+            tags.append((f'<span class="tag info">Source: {comparison.source}</span>', 0))
+
+        # 问题标签（ComparisonResult 使用 *_match，用 not *_match 表示 mismatch）
+        if not comparison.title_match:
+            tags.append(('<span class="tag warning">⚠️ Title Mismatch</span>', 1))
+        if not comparison.author_match:
+            tags.append(('<span class="tag warning">⚠️ Author Mismatch</span>', 1))
+        if not comparison.year_match:
+            tags.append(('<span class="tag warning">⚠️ Year Mismatch</span>', 1))
+        if hasattr(comparison, 'venue_match') and not comparison.venue_match:
+            tags.append(('<span class="tag warning">⚠️ Venue Mismatch</span>', 1))
+        if not comparison.is_match and not comparison.has_issues:
+            tags.append(('<span class="tag error">✗ Not Found</span>', 2))
+
+    # 检查是否是重复条目
+    if duplicate_groups:
+        for group in duplicate_groups:
+            if entry.key in group.entry_keys:
+                tags.append(('<span class="tag warning">⚠️ Duplicate Entry</span>', 1))
+                break
+
+    # 按优先级排序标签
+    tags.sort(key=lambda x: x[1])
+    tags_html = '\n'.join([tag[0] for tag in tags])
+
+    # 详细信息
+    metadata_info = ""
+    if comparison:
+        if comparison.is_match:
+            confidence = getattr(comparison, 'confidence', 0)
+            metadata_info = f"<strong>Verification Info:</strong> All fields matched successfully | Confidence: {confidence * 100:.2f}%"
+        elif comparison.has_issues:
+            issues = []
+            if not comparison.title_match:
+                issues.append("• Title mismatch detected")
+            if not comparison.author_match:
+                issues.append("• Author list differs from database")
+            if not comparison.year_match:
+                issues.append("• Publication year mismatch")
+            if hasattr(comparison, 'venue_match') and not comparison.venue_match:
+                issues.append("• Venue/journal name differs")
+            metadata_info = f"<strong>Issue Details:</strong><br>" + "<br>".join(issues)
+        else:
+            metadata_info = f"""<strong>Issue Details:</strong><br>
+• Entry not found in any database<br>
+• Possible causes: incorrect title, author errors, or non-existent reference<br>
+• Suggestion: verify the original source or use a search engine"""
+
+    # Ground truth (reference): compact title, author, year, doi only (no Copy, no full BibTeX)
+    fetched_bibtex_html = ""
+    if comparison and getattr(comparison, "source", "") != "unable" and (
+        getattr(comparison, "fetched_title", None) or getattr(comparison, "fetched_authors", None)
+    ):
+        src = getattr(comparison, "source", "reference")
+        fa = getattr(comparison, "fetched_authors", None)
+        authors_str = " and ".join(fa) if isinstance(fa, list) else (fa or "")
+        ft = (getattr(comparison, "fetched_title", None) or "").strip()
+        fy = (getattr(comparison, "fetched_year", None) or "").strip()
+        fdoi = (getattr(comparison, "fetched_doi", None) or "").strip()
+
+        def _line(label, value):
+            if not value:
+                return ""
+            esc = (value or "").replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
+            return f'<div style="margin: 0.15em 0; font-size: 0.9em;"><strong>{label}:</strong> {esc}</div>'
+
+        rows = []
+        if ft:
+            rows.append(_line("Title", ft))
+        if authors_str:
+            rows.append(_line("Authors", authors_str))
+        if fy:
+            rows.append(_line("Year", fy))
+        if fdoi:
+            rows.append(_line("DOI", fdoi))
+        fetched_bibtex_html = f"""
+        <div class="metadata-info" style="margin-top: 0.5em; font-size: 0.95em;">
+            <strong>Reference (from {src}):</strong>
+            {"".join(rows)}
+        </div>"""
+
+    header_link_html = ""
+    if link_url:
+        header_link_html = (
+            f'<a href="{link_url}" target="_blank" rel="noopener" '
+            'style="margin-left: auto; padding: 0.35em 0.75em; background: #2563eb; color: white; border-radius: 6px; font-size: 0.9em; font-weight: 500; text-decoration: none;">'
+            f'{link_label}</a>'
+        )
+
+    html = f"""
+    <div class="entry-card {card_class}">
+        <div class="entry-header" style="display: flex; align-items: center; gap: 0.5em; flex-wrap: wrap;">
+            <span class="status-icon">{status_icon}</span>
+            <span class="entry-key">{entry.key}</span>
+            {header_link_html}
+        </div>
+        <div class="bibtex-content">{bibtex_str}</div>
+        <div class="tags-container">
+            {tags_html}
+        </div>
+        <div class="metadata-info">
+            {metadata_info}
+        </div>
+        {fetched_bibtex_html}
+    </div>
+    """
+    return html, card_class
+
+
+def get_card_class(entry_report):
+    """Return 'verified' | 'warning' | 'error' for filtering."""
+    comparison = entry_report.comparison
+    if comparison and comparison.is_match:
+        return "verified"
+    if comparison and comparison.has_issues:
+        return "warning"
+    return "error"
+
+
+FILTER_TO_CLASS = {"Total": None, "Verified": "verified", "Issues Found": "warning", "Not Found": "error"}
+
+# CSS 样式（供 render_results 使用）
+REPORT_CSS = """
+<style>
+    .ci te scan-container { max-width: 1200px; margin: 0 auto; }
+    .entry-card { border: 1px solid #e0e0e0; border-radius: 8px; padding: 20px; margin-bottom: 20px; background: white; box-shadow: 0 2px 4px rgba(0,0,0,0.1); }
+    .entry-card.verified { border-left: 4px solid #4caf50; }
+    .entry-card.warning { border-left: 4px solid #ff9800; }
+    .entry-card.error { border-left: 4px solid #f44336; }
+    .entry-header { display: flex; align-items: center; margin-bottom: 15px; }
+    .status-icon { font-size: 24px; margin-right: 10px; }
+    .entry-key { font-size: 18px; font-weight: bold; color: #333; }
+    .bibtex-content { background: #f5f5f5; padding: 15px; border-radius: 4px; font-family: 'Courier New', monospace; font-size: 13px; margin: 15px 0; overflow-x: auto; white-space: pre-wrap; color: #1a1a1a; }
+    .tags-container { display: flex; flex-wrap: wrap; gap: 8px; margin-top: 15px; }
+    .tag { padding: 6px 12px; border-radius: 16px; font-size: 12px; font-weight: 500; }
+    .tag.success { background: #e8f5e9; color: #2e7d32; }
+    .tag.warning { background: #fff3e0; color: #e65100; }
+    .tag.error { background: #ffebee; color: #c62828; }
+    .tag.info { background: #e3f2fd; color: #1565c0; }
+    .metadata-info { margin-top: 10px; padding: 10px; background: #fafafa; border-radius: 4px; font-size: 13px; color: #666; }
+    .summary-stats { background: #f5f5f5; padding: 20px; border-radius: 8px; margin-bottom: 30px; display: flex; justify-content: space-around; text-align: center; }
+    .stat-item { flex: 1; }
+    .stat-number { font-size: 32px; font-weight: bold; color: #333; }
+    .stat-label { font-size: 14px; color: #666; margin-top: 5px; }
+    @media (prefers-color-scheme: dark) {
+        .citescan-container { color: #e4e4e7; }
+        .entry-card { background: #27272a; border-color: #3f3f46; box-shadow: 0 2px 4px rgba(0,0,0,0.3); }
+        .entry-key { color: #fafafa; }
+        .bibtex-content { background: #18181b; color: #d4d4d8; }
+        .tag.success { background: #166534; color: #bbf7d0; }
+        .tag.warning { background: #9a3412; color: #fed7aa; }
+        .tag.error { background: #991b1b; color: #fecaca; }
+        .tag.info { background: #1e3a8a; color: #bfdbfe; }
+        .metadata-info { background: #3f3f46; color: #a1a1aa; }
+        .metadata-info strong { color: #e4e4e7; }
+        .summary-stats { background: #3f3f46; color: #e4e4e7; }
+        .stat-number { color: #fafafa; }
+        .stat-label { color: #a1a1aa; }
+    }
+</style>
+"""
+
+
+def render_results(entry_reports, duplicate_groups, filter_choice, include_summary=True):
+    """Build HTML for (optionally) summary + filtered cards. include_summary=False when bar is a separate Gradio component."""
+    verified_count = warning_count = error_count = 0
+    for er in entry_reports:
+        c = get_card_class(er)
+        if c == "verified":
+            verified_count += 1
+        elif c == "warning":
+            warning_count += 1
+        else:
+            error_count += 1
+
+    summary_html = ""
+    if include_summary:
+        summary_html = f"""
+    <div class="summary-stats">
+        <div class="stat-item"><div class="stat-number" style="color: #4caf50;">{verified_count}</div><div class="stat-label">✓ Verified</div></div>
+        <div class="stat-item"><div class="stat-number" style="color: #ff9800;">{warning_count}</div><div class="stat-label">⚠️ Issues Found</div></div>
+        <div class="stat-item"><div class="stat-number" style="color: #f44336;">{error_count}</div><div class="stat-label">✗ Not Found</div></div>
+        <div class="stat-item"><div class="stat-number">{len(entry_reports)}</div><div class="stat-label">Total</div></div>
+    </div>
+    """
+
+    want_class = FILTER_TO_CLASS.get(filter_choice)
+    if want_class is None:
+        filtered = entry_reports
+    else:
+        filtered = [er for er in entry_reports if get_card_class(er) == want_class]
+
+    cards_html = ""
+    for entry_report in filtered:
+        card_html, _ = format_entry_card(entry_report, duplicate_groups)
+        cards_html += card_html
+    if not cards_html:
+        cards_html = "<p style='color: #666; margin: 1em 0;'>No entries in this category.</p>"
+
+    return f"{REPORT_CSS}<div class='citescan-container'>{summary_html}{cards_html}</div>"
+
+
+def filter_display(state, filter_choice):
+    """Re-render results with filter. state = (entry_reports, duplicate_groups) or None."""
+    if state is None:
+        return "<p style='color: #666;'>Please run Verify first.</p>"
+    entry_reports, duplicate_groups = state
+    return render_results(entry_reports, duplicate_groups, filter_choice, include_summary=False)
+
+
+# Bar 单段 HTML（大数字 + 小标签），配色与图片一致
+def _bar_segment_html(num, label, num_color):
+    return f'<div class="bar-seg"><span class="bar-num" style="color:{num_color}">{num}</span><span class="bar-label">{label}</span></div>'
+
+
+def bar_segments_html(verified_count, warning_count, error_count, total):
+    """返回 4 段 (Verified, Issues Found, Not Found, Total) 的 HTML，用于图片式 bar。"""
+    return (
+        _bar_segment_html(verified_count, "✓ Verified", "#32CD32"),
+        _bar_segment_html(warning_count, "⚠️ Issues Found", "#FFA500"),
+        _bar_segment_html(error_count, "✗ Not Found", "#FF0000"),
+        _bar_segment_html(total, "Total", "#ffffff"),
+    )
+
+
+def process_bibtex(bibtex_input, progress=gr.Progress()):
+    """处理用户输入的 BibTeX 并进行检测。返回 (html, state, seg1, seg2, seg3, seg4) 供 bar 展示与筛选。"""
+    zero_segs = bar_segments_html(0, 0, 0, 0)
+    if not bibtex_input.strip():
+        return "<p style='color: red;'>Please enter BibTeX content</p>", None, *zero_segs
+
+    try:
+        # 解析 BibTeX
+        progress(0, desc="Parsing BibTeX...")
+        parser = BibParser()
+
+        # 写入临时文件
+        with tempfile.NamedTemporaryFile(mode='w', suffix='.bib', delete=False) as f:
+            f.write(bibtex_input)
+            temp_bib_path = f.name
+
+        entries = parser.parse_file(temp_bib_path)
+        Path(temp_bib_path).unlink()  # 删除临时文件
+
+        if not entries:
+            return "<p style='color: red;'>No valid BibTeX entries found</p>", None, *bar_segments_html(0, 0, 0, 0)
+
+        # 初始化检测器
+        progress(0.1, desc="Initializing fetchers...")
+        arxiv_fetcher = ArxivFetcher()
+        crossref_fetcher = CrossRefFetcher()
+        scholar_fetcher = ScholarFetcher()
+        semantic_scholar_fetcher = SemanticScholarFetcher()
+        openalex_fetcher = OpenAlexFetcher()
+        dblp_fetcher = DBLPFetcher()
+        comparator = MetadataComparator()
+        duplicate_detector = DuplicateDetector()
+
+        # 检测重复
+        duplicate_groups = duplicate_detector.find_duplicates(entries)
+
+        # 获取工作流
+        workflow_config = get_default_workflow()
+
+        # 处理每个条目
+        entry_reports = []
+        progress_lock = threading.Lock()
+        verified_count = 0
+        warning_count = 0
+        error_count = 0
+
+        def process_single_entry(entry, idx, total):
+            from src.utils.normalizer import TextNormalizer
+
+            comparison_result = None
+            all_results = []
+
+            for step in workflow_config.get_enabled_steps():
+                result = None
+                if step.name == "arxiv_id" and entry.has_arxiv and arxiv_fetcher:
+                    arxiv_meta = arxiv_fetcher.fetch_by_id(entry.arxiv_id)
+                    if arxiv_meta:
+                        result = comparator.compare_with_arxiv(entry, arxiv_meta)
+                elif step.name == "crossref_doi" and entry.doi and crossref_fetcher:
+                    crossref_result = crossref_fetcher.search_by_doi(entry.doi)
+                    if crossref_result:
+                        result = comparator.compare_with_crossref(entry, crossref_result)
+                elif step.name == "semantic_scholar" and entry.title and semantic_scholar_fetcher:
+                    ss_result = semantic_scholar_fetcher.fetch_by_doi(entry.doi) if entry.doi else None
+                    if not ss_result:
+                        ss_result = semantic_scholar_fetcher.search_by_title(entry.title)
+                    if ss_result:
+                        result = comparator.compare_with_semantic_scholar(entry, ss_result)
+                elif step.name == "dblp" and entry.title and dblp_fetcher:
+                    dblp_result = dblp_fetcher.search_by_title(entry.title)
+                    if dblp_result:
+                        result = comparator.compare_with_dblp(entry, dblp_result)
+                elif step.name == "openalex" and entry.title and openalex_fetcher:
+                    oa_result = openalex_fetcher.fetch_by_doi(entry.doi) if entry.doi else None
+                    if not oa_result:
+                        oa_result = openalex_fetcher.search_by_title(entry.title)
+                    if oa_result:
+                        result = comparator.compare_with_openalex(entry, oa_result)
+                elif step.name == "arxiv_title" and entry.title and arxiv_fetcher:
+                    results = arxiv_fetcher.search_by_title(entry.title, max_results=3)
+                    if results:
+                        best_result = None
+                        best_sim = 0.0
+                        norm1 = TextNormalizer.normalize_for_comparison(entry.title)
+                        for r in results:
+                            sim = TextNormalizer.similarity_ratio(norm1, TextNormalizer.normalize_for_comparison(r.title))
+                            if sim > best_sim:
+                                best_sim, best_result = sim, r
+                        if best_result and best_sim > 0.5:
+                            result = comparator.compare_with_arxiv(entry, best_result)
+                elif step.name == "crossref_title" and entry.title and crossref_fetcher:
+                    crossref_result = crossref_fetcher.search_by_title(entry.title)
+                    if crossref_result:
+                        result = comparator.compare_with_crossref(entry, crossref_result)
+                elif step.name == "google_scholar" and entry.title and scholar_fetcher:
+                    scholar_result = scholar_fetcher.search_by_title(entry.title)
+                    if scholar_result:
+                        result = comparator.compare_with_scholar(entry, scholar_result)
+
+                if result:
+                    all_results.append(result)
+                    if result.is_match:
+                        comparison_result = result
+                        break
+
+            if not comparison_result and all_results:
+                all_results.sort(key=lambda r: r.confidence, reverse=True)
+                comparison_result = all_results[0]
+            elif not comparison_result:
+                comparison_result = comparator.create_unable_result(entry, "Unable to find this paper in any data source")
+
+            return EntryReport(entry=entry, comparison=comparison_result)
+
+        max_workers = min(10, len(entries))
+        with ThreadPoolExecutor(max_workers=max_workers) as executor:
+            future_to_entry = {executor.submit(process_single_entry, e, i, len(entries)): (e, i) for i, e in enumerate(entries)}
+
+            for future in as_completed(future_to_entry):
+                entry, idx = future_to_entry[future]
+                try:
+                    entry_report = future.result()
+                    with progress_lock:
+                        entry_reports.append(entry_report)
+
+                        if entry_report.comparison and entry_report.comparison.is_match:
+                            verified_count += 1
+                        elif entry_report.comparison and entry_report.comparison.has_issues:
+                            warning_count += 1
+                        else:
+                            error_count += 1
+
+                        progress((idx + 1) / len(entries), desc=f"Verifying entries {idx + 1}/{len(entries)}...")
+                except Exception as e:
+                    with progress_lock:
+                        error_count += 1
+                        print(f"Error processing {entry.key}: {e}")
+
+        # 生成 HTML（默认 Total 视图，不含 bar），并保存结果供分类筛选
+        progress(1.0, desc="Generating report...")
+        final_html = render_results(entry_reports, duplicate_groups, "Total", include_summary=False)
+        segs = bar_segments_html(verified_count, warning_count, error_count, len(entry_reports))
+        return final_html, (entry_reports, duplicate_groups), *segs
+
+    except Exception as e:
+        import traceback
+        error_msg = f"<p style='color: red;'>Error: {str(e)}</p><pre>{traceback.format_exc()}</pre>"
+        return error_msg, None, *bar_segments_html(0, 0, 0, 0)
+
+
+# 官网示例：点击即可填入输入框进行测试
+BIBTEX_EXAMPLES = [
+    (
+        """@article{gpt2,
+  title={Language models are unsupervised multitask},
+  author={Radford, Alec and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others},
+  journal={OpenAI blog},
+  volume={1},
+  number={8},
+  pages={9},
+  year={2021}
+}""",
+        "GPT-2 (OpenAI blog)",
+    ),
+    (
+        """@article{devlin2018bert,
+  year={2018},
+  journal={arXiv preprint arXiv:1810.04805},
+  author={Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
+  title={BERT: Pre-training of deep bidirectional transformers for language understanding}
+}""",
+        "BERT (arXiv)",
+    ),
+    (
+        """@article{vaswani2017attention,
+  title={Attention is all you need},
+  author={Vaswani, Ashish and Shazeer, Noam and others},
+  journal={Advances in neural information processing systems},
+  year={2017}
+}
+
+@article{brown2020language,
+  title={Language models are few-shot learners},
+  author={Brown, Tom B and Mann, Benjamin and others},
+  year={2020}
+}""",
+        "Attention + GPT-3 (multiple entries)",
+    ),
+]
+
+# Bar 图片式 UI：深灰背景 #2E3035，大数字 + 小标签，绿/橙/红/白
+BAR_CSS = """
+.status-bar-row { background: #2E3035 !important; border-radius: 8px !important; padding: 20px !important; margin-bottom: 20px !important; display: flex !important; justify-content: space-around !important; align-items: stretch !important; gap: 12px !important; }
+.bar-segment-col { flex: 1 !important; text-align: center !important; position: relative !important; min-width: 0 !important; }
+.bar-segment-col .bar-seg { display: flex !important; flex-direction: column !important; align-items: center !important; justify-content: center !important; padding: 8px 4px !important; }
+.bar-segment-col .bar-num { font-size: 32px !important; font-weight: bold !important; line-height: 1.2 !important; display: block !important; }
+.bar-segment-col .bar-label { font-size: 13px !important; color: #ffffff !important; margin-top: 4px !important; display: block !important; }
+.bar-segment-btn { position: absolute !important; top: 0 !important; left: 0 !important; right: 0 !important; bottom: 0 !important; opacity: 0 !important; cursor: pointer !important; }
+"""
+
+# 创建 Gradio 界面
+with gr.Blocks(title="CiteScan - Check References, Confirm Truth.", theme=gr.themes.Soft(), css=BAR_CSS) as demo:
+    gr.Markdown("""
+    # CiteScan - Check References, Confirm Truth.
+
+    1️⃣ Paste your BibTeX below, or **click an example** to load it.
+
+    2️⃣ Click "Verify" button to have the system verify the authenticity and accuracy of each reference.
+    
+    **Important: We check very carefully. Sometimes the result might look different from Google Scholar or ArXiv. We think it's best to use the official version from places like ACM, ACL, or CVF to cite your sources.
+
+    We will also add a feature soon to help change citations from pre-print versions (like arXiv or bioRxiv) into the final official ones (like from NeurIPS, ACL, or CVF).**
+    """)
+
+    with gr.Row():
+        with gr.Column():
+            bibtex_input = gr.Textbox(
+                label="📝 Paste your BibTeX",
+                placeholder="""Paste your BibTeX or click an example below. for example: 
+@article{gpt2,
+  title={Language models are unsupervised multitask},
+  author={Radford, Alec and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others},
+  journal={OpenAI blog},
+  volume={1},
+  number={8},
+  pages={9},
+  year={2021}
+}
+
+@article{devlin2018bert,
+  year={2018},
+  journal={arXiv preprint arXiv:1810.04805},
+  author={Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
+  title={BERT: Pre-training of deep bidirectional transformers for language understanding}
+}
+""",
+                lines=15,
+                max_lines=20
+            )
+
+            submit_btn = gr.Button("🚀 Verify", variant="primary", size="lg")
+
+            gr.Examples(
+                examples=[[ex[0]] for ex in BIBTEX_EXAMPLES],
+                inputs=[bibtex_input],
+                label="📋 Examples (click to load)",
+                examples_per_page=6,
+            )
+
+    result_state = gr.State(value=None)
+
+    # Bar：图片式 UI（深灰 #2E3035，大数字 + 小标签），每段可点击筛选
+    zero_segs = bar_segments_html(0, 0, 0, 0)
+    with gr.Row(elem_classes=["status-bar-row"]):
+        with gr.Column(elem_classes=["bar-segment-col"], scale=1):
+            bar_seg_verified = gr.HTML(zero_segs[0])
+            btn_verified = gr.Button("Verified", elem_classes=["bar-segment-btn"], visible=True)
+        with gr.Column(elem_classes=["bar-segment-col"], scale=1):
+            bar_seg_issues = gr.HTML(zero_segs[1])
+            btn_issues = gr.Button("Issues", elem_classes=["bar-segment-btn"], visible=True)
+        with gr.Column(elem_classes=["bar-segment-col"], scale=1):
+            bar_seg_notfound = gr.HTML(zero_segs[2])
+            btn_notfound = gr.Button("Not Found", elem_classes=["bar-segment-btn"], visible=True)
+        with gr.Column(elem_classes=["bar-segment-col"], scale=1):
+            bar_seg_total = gr.HTML(zero_segs[3])
+            btn_total = gr.Button("Total", elem_classes=["bar-segment-btn"], visible=True)
+
+    with gr.Row():
+        output_html = gr.HTML(label="Detection Results")
+
+    submit_btn.click(
+        fn=process_bibtex,
+        inputs=[bibtex_input],
+        outputs=[output_html, result_state, bar_seg_verified, bar_seg_issues, bar_seg_notfound, bar_seg_total],
+    )
+
+    def filter_to_verified(state):
+        return filter_display(state, "Verified")
+
+    def filter_to_issues(state):
+        return filter_display(state, "Issues Found")
+
+    def filter_to_notfound(state):
+        return filter_display(state, "Not Found")
+
+    def filter_to_total(state):
+        return filter_display(state, "Total")
+
+    btn_verified.click(fn=filter_to_verified, inputs=[result_state], outputs=[output_html])
+    btn_issues.click(fn=filter_to_issues, inputs=[result_state], outputs=[output_html])
+    btn_notfound.click(fn=filter_to_notfound, inputs=[result_state], outputs=[output_html])
+    btn_total.click(fn=filter_to_total, inputs=[result_state], outputs=[output_html])
+
+    gr.Markdown("""
+*False positive cases* occur for CiteScan:
+
+1.  **Authors Mismatch**:
+    - *Reason*: Different databases deal with a longer list of  authors with different strategies, like truncation.
+    - *Action*: Verify if main authors match
+
+2.  **Venues Mismatch**:
+    - *Reason*: Abbreviations vs. full names, such as "ICLR" v.s. "International Conference on Learning Representations"
+    - *Action*: Both are correct.
+
+3.  **Year GAP (±1 Year)**:
+    - *Reason*: Delay between preprint (arXiv) and final version publication 
+    - *Action*: Verify which version you intend to cite, We recommend you to cite the version from the official press website. Lower pre-print version bib will make your submission more confidence. 
+
+4.  **Non-academic Sources**:
+    - *Reason*: Blogs, and APIs are not indexed in academic databases.
+    - *Action*: Verify URL, year, and title manually.
+---
+**Supported Data Sources:** arXiv, CrossRef, DBLP, Semantic Scholar, ACL Anthology, ACM, theCVF,
+    """)
+
+    # Partner logos and contact (embed images as base64 so they work when served)
+    _root = Path(__file__).resolve().parent
+    def _logo_b64(path: Path) -> str | None:
+        if path.exists():
+            return base64.b64encode(path.read_bytes()).decode("utf-8")
+        return None
+    _nus = _logo_b64(_root / "assets" / "logo_nus.png")
+    _sjtu = _logo_b64(_root / "assets" / "logo_sjtu.png")
+    _logos_html = []
+    if _nus:
+        _logos_html.append(f'<img src="data:image/png;base64,{_nus}" alt="NUS" style="height:72px; margin-right:24px; vertical-align:middle; display:inline-block;" />')
+    if _sjtu:
+        _logos_html.append(f'<img src="data:image/png;base64,{_sjtu}" alt="Shanghai Jiao Tong University" style="height:72px; vertical-align:middle; display:inline-block;" />')
+    gr.HTML(f"""
+    <div style="margin-top:12px;">
+      <p><strong>Cooperations</strong></p>
+      <p style="display:flex; align-items:center; gap:24px; flex-wrap:nowrap;">{" ".join(_logos_html)}</p>
+      <p><strong> Feel free to reach out me by Email</strong> <a href="mailto:e1143641@u.nus.edu">e1143641@u.nus.edu</a></p>
+    </div>
+    """)
+
+if __name__ == "__main__":
+    demo.launch(
+        share=False, # To create a public link, set `share=True`
+        server_name="0.0.0.0", 
+        server_port=7860
+    )

citescan.yaml

@@ -0,0 +1,39 @@
+files:
+  bib: "test.bib"
+  output_dir: "test"
+
+bibliography:
+  check_metadata: true
+  check_duplicates: true
+  check_preprint_ratio: true
+  preprint_warning_threshold: 0.50
+
+workflow:
+  - name: arxiv_id
+    enabled: true
+    description: "Lookup by arXiv ID"
+  - name: crossref_doi
+    enabled: true
+    description: "Lookup by DOI"
+  - name: semantic_scholar
+    enabled: true
+    description: "Semantic Scholar API"
+  - name: dblp
+    enabled: true
+    description: "DBLP"
+  - name: openalex
+    enabled: true
+    description: "OpenAlex API"
+  - name: arxiv_title
+    enabled: true
+    description: "Search arXiv by title"
+  - name: crossref_title
+    enabled: true
+    description: "Search CrossRef by title"
+  - name: google_scholar
+    enabled: false
+    description: "Google Scholar (may be rate-limited)"
+
+output:
+  quiet: false
+  minimal_verified: false

config.yaml

@@ -0,0 +1,33 @@
+files:
+  bib: "paper.bib"
+  output_dir: "citescan_output"
+
+bibliography:
+  check_metadata: true
+  check_usage: true
+  check_duplicates: true
+  check_preprint_ratio: true
+  preprint_warning_threshold: 0.50
+  check_relevance: false
+
+submission:
+  caption: true
+  reference: true
+  formatting: true
+  equation: true
+  ai_artifacts: true
+  sentence: true
+  consistency: true
+  acronym: true
+  number: true
+  citation_quality: true
+  anonymization: true
+
+llm:
+  backend: "deepseeek"
+  model: "deepseek-chat"
+  api_key: "sk-d7c87a7386d94879a80282cee7bd3f45"
+
+output:
+  quiet: false
+  minimal_verified: false

requirements.txt

@@ -0,0 +1,8 @@
+gradio>=4.5.0
+bibtexparser>=1.4.0
+requests>=2.31.0
+beautifulsoup4>=4.12.0
+rich>=13.7.0
+Unidecode>=1.3.0
+lxml>=5.0.0
+PyYAML>=6.0

src/__init__.py

@@ -0,0 +1 @@
+"""Bibliography Checker Package"""

src/analyzers/__init__.py

@@ -0,0 +1,5 @@
+"""Analyzers package (bib-only)."""
+from .metadata_comparator import MetadataComparator
+from .duplicate_detector import DuplicateDetector
+
+__all__ = ["MetadataComparator", "DuplicateDetector"]

src/analyzers/duplicate_detector.py

@@ -0,0 +1,204 @@
+"""
+Duplicate entry detector for bibliography files.
+Uses fuzzy matching to find potential duplicates.
+"""
+from dataclasses import dataclass
+from typing import List, Tuple
+
+from ..parsers.bib_parser import BibEntry
+from ..utils.normalizer import TextNormalizer
+
+
+@dataclass
+class DuplicateGroup:
+    """A group of potentially duplicate entries."""
+    entries: List[BibEntry]
+    similarity_score: float
+    reason: str
+    
+    @property
+    def entry_keys(self) -> List[str]:
+        return [e.key for e in self.entries]
+
+
+class DuplicateDetector:
+    """Detects duplicate bibliography entries using fuzzy matching."""
+    
+    # Thresholds for duplicate detection
+    TITLE_SIMILARITY_THRESHOLD = 0.85
+    COMBINED_SIMILARITY_THRESHOLD = 0.80
+    
+    def __init__(self):
+        self.normalizer = TextNormalizer
+    
+    def find_duplicates(self, entries: List[BibEntry]) -> List[DuplicateGroup]:
+        """
+        Find all duplicate groups in the bibliography.
+        
+        Returns:
+            List of DuplicateGroup objects, each containing 2+ similar entries.
+        """
+        duplicates = []
+        processed = set()
+        
+        for i, entry1 in enumerate(entries):
+            if entry1.key in processed:
+                continue
+            
+            # Find all entries similar to this one
+            similar_entries = [entry1]
+            
+            for j, entry2 in enumerate(entries[i+1:], start=i+1):
+                if entry2.key in processed:
+                    continue
+                
+                similarity, reason = self._calculate_similarity(entry1, entry2)
+                
+                if similarity >= self.COMBINED_SIMILARITY_THRESHOLD:
+                    similar_entries.append(entry2)
+                    processed.add(entry2.key)
+            
+            # If we found duplicates, create a group
+            if len(similar_entries) > 1:
+                processed.add(entry1.key)
+                
+                # Calculate average similarity for the group
+                avg_similarity = self._calculate_group_similarity(similar_entries)
+                reason = self._generate_reason(similar_entries)
+                
+                duplicates.append(DuplicateGroup(
+                    entries=similar_entries,
+                    similarity_score=avg_similarity,
+                    reason=reason
+                ))
+        
+        # Sort by similarity score (highest first)
+        duplicates.sort(key=lambda g: g.similarity_score, reverse=True)
+        
+        return duplicates
+    
+    def _calculate_similarity(self, entry1: BibEntry, entry2: BibEntry) -> Tuple[float, str]:
+        """
+        Calculate similarity between two entries.
+        
+        Returns:
+            (similarity_score, reason_string)
+        """
+        # Normalize titles
+        title1 = self.normalizer.normalize_for_comparison(entry1.title)
+        title2 = self.normalizer.normalize_for_comparison(entry2.title)
+        
+        # Calculate title similarity
+        title_sim = self.normalizer.similarity_ratio(title1, title2)
+        
+        # If titles are very similar, likely a duplicate
+        if title_sim >= self.TITLE_SIMILARITY_THRESHOLD:
+            return title_sim, "Very similar titles"
+        
+        # Check author similarity
+        author_sim = self._calculate_author_similarity(entry1, entry2)
+        
+        # Combined score: weighted average
+        # Title is more important (70%) than authors (30%)
+        combined_sim = 0.7 * title_sim + 0.3 * author_sim
+        
+        if combined_sim >= self.COMBINED_SIMILARITY_THRESHOLD:
+            return combined_sim, f"Similar title ({title_sim:.0%}) and authors ({author_sim:.0%})"
+        
+        return combined_sim, ""
+    
+    def _calculate_author_similarity(self, entry1: BibEntry, entry2: BibEntry) -> float:
+        """Calculate similarity between author lists."""
+        # Parse author strings
+        authors1 = self._parse_authors(entry1.author)
+        authors2 = self._parse_authors(entry2.author)
+        
+        if not authors1 or not authors2:
+            return 0.0
+        
+        # Normalize author names
+        norm_authors1 = [self.normalizer.normalize_for_comparison(a) for a in authors1]
+        norm_authors2 = [self.normalizer.normalize_for_comparison(a) for a in authors2]
+        
+        # Count matching authors
+        matches = 0
+        for a1 in norm_authors1:
+            for a2 in norm_authors2:
+                if self._authors_match(a1, a2):
+                    matches += 1
+                    break
+        
+        # Calculate Jaccard similarity
+        total_unique = len(set(norm_authors1) | set(norm_authors2))
+        if total_unique == 0:
+            return 0.0
+        
+        return matches / total_unique
+    
+    def _parse_authors(self, author_string: str) -> List[str]:
+        """Parse author string into list of names."""
+        if not author_string:
+            return []
+        
+        # Split by 'and'
+        authors = author_string.split(' and ')
+        
+        # Clean up each author
+        cleaned = []
+        for author in authors:
+            # Remove extra whitespace
+            author = ' '.join(author.split())
+            if author:
+                cleaned.append(author)
+        
+        return cleaned
+    
+    def _authors_match(self, name1: str, name2: str) -> bool:
+        """Check if two author names match (handles initials)."""
+        # Simple exact match after normalization
+        if name1 == name2:
+            return True
+        
+        # Check if one is a substring of the other (handles initials)
+        if name1 in name2 or name2 in name1:
+            return True
+        
+        # Calculate string similarity
+        sim = self.normalizer.similarity_ratio(name1, name2)
+        return sim >= 0.8
+    
+    def _calculate_group_similarity(self, entries: List[BibEntry]) -> float:
+        """Calculate average similarity within a group."""
+        if len(entries) < 2:
+            return 1.0
+        
+        total_sim = 0.0
+        count = 0
+        
+        for i, entry1 in enumerate(entries):
+            for entry2 in entries[i+1:]:
+                sim, _ = self._calculate_similarity(entry1, entry2)
+                total_sim += sim
+                count += 1
+        
+        return total_sim / count if count > 0 else 0.0
+    
+    def _generate_reason(self, entries: List[BibEntry]) -> str:
+        """Generate a human-readable reason for the duplicate group."""
+        # Check if all titles are very similar
+        titles = [self.normalizer.normalize_for_comparison(e.title) for e in entries]
+        
+        # Calculate pairwise title similarities
+        title_sims = []
+        for i, t1 in enumerate(titles):
+            for t2 in titles[i+1:]:
+                title_sims.append(self.normalizer.similarity_ratio(t1, t2))
+        
+        avg_title_sim = sum(title_sims) / len(title_sims) if title_sims else 0.0
+        
+        if avg_title_sim >= 0.95:
+            return "Nearly identical titles"
+        elif avg_title_sim >= 0.85:
+            return "Very similar titles"
+        else:
+            return "Similar titles and authors"

src/analyzers/metadata_comparator.py

@@ -0,0 +1,502 @@
+"""
+Metadata comparison between bib entries and fetched metadata.
+"""
+from dataclasses import dataclass
+from typing import Optional
+
+from ..parsers.bib_parser import BibEntry
+from ..fetchers.arxiv_fetcher import ArxivMetadata
+from ..fetchers.scholar_fetcher import ScholarResult
+from ..fetchers.crossref_fetcher import CrossRefResult
+from ..fetchers.semantic_scholar_fetcher import SemanticScholarResult
+from ..fetchers.openalex_fetcher import OpenAlexResult
+from ..fetchers.dblp_fetcher import DBLPResult
+from ..utils.normalizer import TextNormalizer
+from ..utils.fetched_bibtex import build_fetched_bibtex
+
+
+@dataclass
+class ComparisonResult:
+    """Result of comparing bib entry with fetched metadata."""
+    entry_key: str
+    
+    # Title comparison
+    title_match: bool
+    title_similarity: float
+    bib_title: str
+    fetched_title: str
+    
+    # Author comparison
+    author_match: bool
+    author_similarity: float
+    bib_authors: list[str]
+    fetched_authors: list[str]
+    
+    # Year comparison
+    year_match: bool
+    bib_year: str
+    fetched_year: str
+    
+    # Overall assessment
+    is_match: bool
+    confidence: float
+    issues: list[str]
+    source: str  # 'arxiv', 'crossref', 'scholar', 'semantic_scholar', 'openalex', 'dblp', or 'unable'
+    fetched_bibtex: Optional[str] = None  # BibTeX string of the ground-truth entry when available
+    fetched_url: Optional[str] = None
+    fetched_doi: Optional[str] = None
+
+    @property
+    def has_issues(self) -> bool:
+        return len(self.issues) > 0
+
+
+class MetadataComparator:
+    """Compares bibliography entries with fetched metadata."""
+    
+    # Thresholds for matching
+    TITLE_THRESHOLD = 0.99
+    AUTHOR_THRESHOLD = 0.9
+    
+    def __init__(self):
+        self.normalizer = TextNormalizer
+    
+    def compare_with_arxiv(self, bib_entry: BibEntry, arxiv_meta: ArxivMetadata) -> ComparisonResult:
+        """Compare bib entry with arXiv metadata."""
+        issues = []
+        
+        # Compare titles
+        bib_title_norm = self.normalizer.normalize_for_comparison(bib_entry.title)
+        arxiv_title_norm = self.normalizer.normalize_for_comparison(arxiv_meta.title)
+        
+        title_similarity = self.normalizer.similarity_ratio(bib_title_norm, arxiv_title_norm)
+        if len(bib_title_norm) < 100:
+            lev_sim = self.normalizer.levenshtein_similarity(bib_title_norm, arxiv_title_norm)
+            title_similarity = min(title_similarity, lev_sim)
+        
+        title_match = title_similarity >= self.TITLE_THRESHOLD
+        
+        if not title_match:
+            issues.append(f"Title mismatch (similarity: {title_similarity:.2%})")
+        
+        # Compare authors
+        bib_authors = self.normalizer.normalize_author_list(bib_entry.author)
+        arxiv_authors = [self.normalizer.normalize_author_name(a) for a in arxiv_meta.authors]
+        
+        author_similarity = self._compare_author_lists(bib_authors, arxiv_authors)
+        author_match = author_similarity >= self.AUTHOR_THRESHOLD
+        
+        if not author_match:
+            issues.append(f"Author mismatch (similarity: {author_similarity:.2%})")
+        
+        # Compare years
+        bib_year = bib_entry.year.strip()
+        arxiv_year = arxiv_meta.year
+        year_match = bib_year == arxiv_year
+        
+        if not year_match and bib_year and arxiv_year:
+            issues.append(f"Year mismatch: bib={bib_year}, arxiv={arxiv_year}")
+        
+        # Overall assessment
+        is_match = title_match and author_match
+        confidence = (title_similarity * 0.5 + author_similarity * 0.3 + (1.0 if year_match else 0.5) * 0.2)
+        
+        return ComparisonResult(
+            entry_key=bib_entry.key,
+            title_match=title_match,
+            title_similarity=title_similarity,
+            bib_title=bib_entry.title,
+            fetched_title=arxiv_meta.title,
+            author_match=author_match,
+            author_similarity=author_similarity,
+            bib_authors=bib_authors,
+            fetched_authors=arxiv_authors,
+            year_match=year_match,
+            bib_year=bib_year,
+            fetched_year=arxiv_year,
+            is_match=is_match,
+            confidence=confidence,
+            issues=issues,
+            source="arxiv",
+            fetched_bibtex=build_fetched_bibtex("arxiv", arxiv_meta),
+            fetched_url=getattr(arxiv_meta, "abs_url", None) or "",
+            fetched_doi=getattr(arxiv_meta, "doi", None) or "",
+        )
+    
+    def compare_with_scholar(self, bib_entry: BibEntry, scholar_result: ScholarResult) -> ComparisonResult:
+        """Compare bib entry with Scholar search result."""
+        issues = []
+        
+        # Compare titles
+        bib_title_norm = self.normalizer.normalize_for_comparison(bib_entry.title)
+        scholar_title_norm = self.normalizer.normalize_for_comparison(scholar_result.title)
+        
+        title_similarity = self.normalizer.similarity_ratio(bib_title_norm, scholar_title_norm)
+        if len(bib_title_norm) < 100:
+            lev_sim = self.normalizer.levenshtein_similarity(bib_title_norm, scholar_title_norm)
+            title_similarity = min(title_similarity, lev_sim)
+        
+        title_match = title_similarity >= self.TITLE_THRESHOLD
+        
+        if not title_match:
+            issues.append(f"Title mismatch (similarity: {title_similarity:.2%})")
+        
+        # Compare authors (Scholar format is less structured)
+        bib_authors = self.normalizer.normalize_author_list(bib_entry.author)
+        # Scholar authors are comma-separated
+        scholar_authors_raw = scholar_result.authors.split(',')
+        scholar_authors = [self.normalizer.normalize_author_name(a.strip()) for a in scholar_authors_raw]
+        
+        author_similarity = self._compare_author_lists(bib_authors, scholar_authors)
+        author_match = author_similarity >= self.AUTHOR_THRESHOLD
+        
+        if not author_match:
+            issues.append(f"Author mismatch (similarity: {author_similarity:.2%})")
+        
+        # Compare years
+        bib_year = bib_entry.year.strip()
+        scholar_year = scholar_result.year
+        year_match = bib_year == scholar_year
+        
+        if not year_match and bib_year and scholar_year:
+            issues.append(f"Year mismatch: bib={bib_year}, scholar={scholar_year}")
+        
+        # Overall assessment
+        is_match = title_match and author_match
+        confidence = (title_similarity * 0.5 + author_similarity * 0.3 + (1.0 if year_match else 0.5) * 0.2)
+        
+        return ComparisonResult(
+            entry_key=bib_entry.key,
+            title_match=title_match,
+            title_similarity=title_similarity,
+            bib_title=bib_entry.title,
+            fetched_title=scholar_result.title,
+            author_match=author_match,
+            author_similarity=author_similarity,
+            bib_authors=bib_authors,
+            fetched_authors=scholar_authors,
+            year_match=year_match,
+            bib_year=bib_year,
+            fetched_year=scholar_year,
+            is_match=is_match,
+            confidence=confidence,
+            issues=issues,
+            source="scholar",
+            fetched_bibtex=build_fetched_bibtex("scholar", scholar_result),
+            fetched_url=getattr(scholar_result, "url", None) or "",
+            fetched_doi=None,
+        )
+    
+    def compare_with_crossref(self, bib_entry: BibEntry, crossref_result: CrossRefResult) -> ComparisonResult:
+        """Compare bib entry with CrossRef search result."""
+        issues = []
+        
+        # Compare titles
+        bib_title_norm = self.normalizer.normalize_for_comparison(bib_entry.title)
+        crossref_title_norm = self.normalizer.normalize_for_comparison(crossref_result.title)
+        
+        title_similarity = self.normalizer.similarity_ratio(bib_title_norm, crossref_title_norm)
+        if len(bib_title_norm) < 100:
+            lev_sim = self.normalizer.levenshtein_similarity(bib_title_norm, crossref_title_norm)
+            title_similarity = min(title_similarity, lev_sim)
+        
+        title_match = title_similarity >= self.TITLE_THRESHOLD
+        
+        if not title_match:
+            issues.append(f"Title mismatch (similarity: {title_similarity:.2%})")
+        
+        # Compare authors
+        bib_authors = self.normalizer.normalize_author_list(bib_entry.author)
+        crossref_authors = [self.normalizer.normalize_author_name(a) for a in crossref_result.authors]
+        
+        author_similarity = self._compare_author_lists(bib_authors, crossref_authors)
+        author_match = author_similarity >= self.AUTHOR_THRESHOLD
+        
+        if not author_match:
+            issues.append(f"Author mismatch (similarity: {author_similarity:.2%})")
+        
+        # Compare years
+        bib_year = bib_entry.year.strip()
+        crossref_year = crossref_result.year
+        year_match = bib_year == crossref_year
+        
+        if not year_match and bib_year and crossref_year:
+            issues.append(f"Year mismatch: bib={bib_year}, crossref={crossref_year}")
+        
+        # Overall assessment
+        is_match = title_match and author_match
+        confidence = (title_similarity * 0.5 + author_similarity * 0.3 + (1.0 if year_match else 0.5) * 0.2)
+        
+        return ComparisonResult(
+            entry_key=bib_entry.key,
+            title_match=title_match,
+            title_similarity=title_similarity,
+            bib_title=bib_entry.title,
+            fetched_title=crossref_result.title,
+            author_match=author_match,
+            author_similarity=author_similarity,
+            bib_authors=bib_authors,
+            fetched_authors=crossref_authors,
+            year_match=year_match,
+            bib_year=bib_year,
+            fetched_year=crossref_year,
+            is_match=is_match,
+            confidence=confidence,
+            issues=issues,
+            source="crossref",
+            fetched_bibtex=build_fetched_bibtex("crossref", crossref_result),
+            fetched_url=getattr(crossref_result, "url", None) or "",
+            fetched_doi=getattr(crossref_result, "doi", None) or "",
+        )
+    
+    def create_unable_result(self, bib_entry: BibEntry, reason: str = "Unable to fetch metadata") -> ComparisonResult:
+        """Create result when metadata couldn't be fetched."""
+        return ComparisonResult(
+            entry_key=bib_entry.key,
+            title_match=False,
+            title_similarity=0.0,
+            bib_title=bib_entry.title,
+            fetched_title="",
+            author_match=False,
+            author_similarity=0.0,
+            bib_authors=self.normalizer.normalize_author_list(bib_entry.author),
+            fetched_authors=[],
+            year_match=False,
+            bib_year=bib_entry.year,
+            fetched_year="",
+            is_match=False,
+            confidence=0.0,
+            issues=[reason],
+            source="unable"
+        )
+    
+    def _compare_author_lists(self, list1: list[str], list2: list[str]) -> float:
+        """Compare two author lists using Jaccard similarity (order-invariant, symmetric)."""
+        if not list1 and not list2:
+            return 1.0
+        if not list1 or not list2:
+            return 0.0
+
+        # Max matching: each author in list1 matches at most one in list2, and vice versa
+        used = set()
+        match_count = 0
+        for author1 in list1:
+            for j, author2 in enumerate(list2):
+                if j not in used and self._names_match(author1, author2):
+                    used.add(j)
+                    match_count += 1
+                    break
+
+        # Jaccard: |A ∩ B| / |A ∪ B| = match_count / (n + m - match_count)
+        union_size = len(list1) + len(list2) - match_count
+        if union_size <= 0:
+            return 1.0
+        return match_count / union_size
+    
+    def _names_match(self, name1: str, name2: str) -> bool:
+        """Check if two names match (handles abbreviated names)."""
+        words1 = name1.split()
+        words2 = name2.split()
+        
+        if not words1 or not words2:
+            return False
+        
+        # Check if last names match
+        if words1[-1] != words2[-1]:
+            # Try first word as last name too
+            if words1[0] != words2[-1] and words1[-1] != words2[0]:
+                return False
+        
+        return True
+    
+    def compare_with_semantic_scholar(self, bib_entry: BibEntry, ss_result: SemanticScholarResult) -> ComparisonResult:
+        """Compare bib entry with Semantic Scholar result."""
+        issues = []
+        
+        # Compare titles
+        bib_title_norm = self.normalizer.normalize_for_comparison(bib_entry.title)
+        ss_title_norm = self.normalizer.normalize_for_comparison(ss_result.title)
+        
+        title_similarity = self.normalizer.similarity_ratio(bib_title_norm, ss_title_norm)
+        if len(bib_title_norm) < 100:
+            lev_sim = self.normalizer.levenshtein_similarity(bib_title_norm, ss_title_norm)
+            title_similarity = min(title_similarity, lev_sim)
+        
+        title_match = title_similarity >= self.TITLE_THRESHOLD
+        
+        if not title_match:
+            issues.append(f"Title mismatch (similarity: {title_similarity:.2%})")
+        
+        # Compare authors
+        bib_authors = self.normalizer.normalize_author_list(bib_entry.author)
+        ss_authors = [self.normalizer.normalize_author_name(a) for a in ss_result.authors]
+        
+        author_similarity = self._compare_author_lists(bib_authors, ss_authors)
+        author_match = author_similarity >= self.AUTHOR_THRESHOLD
+        
+        if not author_match:
+            issues.append(f"Author mismatch (similarity: {author_similarity:.2%})")
+        
+        # Compare years
+        bib_year = bib_entry.year.strip()
+        ss_year = ss_result.year
+        year_match = bib_year == ss_year
+        
+        if not year_match and bib_year and ss_year:
+            issues.append(f"Year mismatch: bib={bib_year}, semantic_scholar={ss_year}")
+        
+        # Overall assessment
+        is_match = title_match and author_match
+        confidence = (title_similarity * 0.5 + author_similarity * 0.3 + (1.0 if year_match else 0.5) * 0.2)
+        
+        return ComparisonResult(
+            entry_key=bib_entry.key,
+            title_match=title_match,
+            title_similarity=title_similarity,
+            bib_title=bib_entry.title,
+            fetched_title=ss_result.title,
+            author_match=author_match,
+            author_similarity=author_similarity,
+            bib_authors=bib_authors,
+            fetched_authors=ss_authors,
+            year_match=year_match,
+            bib_year=bib_year,
+            fetched_year=ss_year,
+            is_match=is_match,
+            confidence=confidence,
+            issues=issues,
+            source="semantic_scholar",
+            fetched_bibtex=build_fetched_bibtex("semantic_scholar", ss_result),
+            fetched_url=getattr(ss_result, "url", None) or "",
+            fetched_doi=None,
+        )
+    
+    def compare_with_openalex(self, bib_entry: BibEntry, oa_result: OpenAlexResult) -> ComparisonResult:
+        """Compare bib entry with OpenAlex result."""
+        issues = []
+        
+        # Compare titles
+        bib_title_norm = self.normalizer.normalize_for_comparison(bib_entry.title)
+        oa_title_norm = self.normalizer.normalize_for_comparison(oa_result.title)
+        
+        title_similarity = self.normalizer.similarity_ratio(bib_title_norm, oa_title_norm)
+        if len(bib_title_norm) < 100:
+            lev_sim = self.normalizer.levenshtein_similarity(bib_title_norm, oa_title_norm)
+            title_similarity = min(title_similarity, lev_sim)
+        
+        title_match = title_similarity >= self.TITLE_THRESHOLD
+        
+        if not title_match:
+            issues.append(f"Title mismatch (similarity: {title_similarity:.2%})")
+        
+        # Compare authors
+        bib_authors = self.normalizer.normalize_author_list(bib_entry.author)
+        oa_authors = [self.normalizer.normalize_author_name(a) for a in oa_result.authors]
+        
+        author_similarity = self._compare_author_lists(bib_authors, oa_authors)
+        author_match = author_similarity >= self.AUTHOR_THRESHOLD
+        
+        if not author_match:
+            issues.append(f"Author mismatch (similarity: {author_similarity:.2%})")
+        
+        # Compare years
+        bib_year = bib_entry.year.strip()
+        oa_year = oa_result.year
+        year_match = bib_year == oa_year
+        
+        if not year_match and bib_year and oa_year:
+            issues.append(f"Year mismatch: bib={bib_year}, openalex={oa_year}")
+        
+        # Overall assessment
+        is_match = title_match and author_match
+        confidence = (title_similarity * 0.5 + author_similarity * 0.3 + (1.0 if year_match else 0.5) * 0.2)
+        
+        return ComparisonResult(
+            entry_key=bib_entry.key,
+            title_match=title_match,
+            title_similarity=title_similarity,
+            bib_title=bib_entry.title,
+            fetched_title=oa_result.title,
+            author_match=author_match,
+            author_similarity=author_similarity,
+            bib_authors=bib_authors,
+            fetched_authors=oa_authors,
+            year_match=year_match,
+            bib_year=bib_year,
+            fetched_year=oa_year,
+            is_match=is_match,
+            confidence=confidence,
+            issues=issues,
+            source="openalex",
+            fetched_bibtex=build_fetched_bibtex("openalex", oa_result),
+            fetched_url=getattr(oa_result, "url", None) or "",
+            fetched_doi=getattr(oa_result, "doi", None) or "",
+        )
+    
+    def compare_with_dblp(self, bib_entry: BibEntry, dblp_result: DBLPResult) -> ComparisonResult:
+        """Compare bib entry with DBLP result."""
+        issues = []
+        
+        # Compare titles
+        bib_title_norm = self.normalizer.normalize_for_comparison(bib_entry.title)
+        dblp_title_norm = self.normalizer.normalize_for_comparison(dblp_result.title)
+        
+        title_similarity = self.normalizer.similarity_ratio(bib_title_norm, dblp_title_norm)
+        if len(bib_title_norm) < 100:
+            lev_sim = self.normalizer.levenshtein_similarity(bib_title_norm, dblp_title_norm)
+            title_similarity = min(title_similarity, lev_sim)
+        
+        title_match = title_similarity >= self.TITLE_THRESHOLD
+        
+        if not title_match:
+            issues.append(f"Title mismatch (similarity: {title_similarity:.2%})")
+        
+        # Compare authors
+        bib_authors = self.normalizer.normalize_author_list(bib_entry.author)
+        dblp_authors = [self.normalizer.normalize_author_name(a) for a in dblp_result.authors]
+        
+        author_similarity = self._compare_author_lists(bib_authors, dblp_authors)
+        author_match = author_similarity >= self.AUTHOR_THRESHOLD
+        
+        if not author_match:
+            issues.append(f"Author mismatch (similarity: {author_similarity:.2%})")
+        
+        # Compare years
+        bib_year = bib_entry.year.strip()
+        dblp_year = dblp_result.year
+        year_match = bib_year == dblp_year
+        
+        if not year_match and bib_year and dblp_year:
+            issues.append(f"Year mismatch: bib={bib_year}, dblp={dblp_year}")
+        
+        # Overall assessment
+        is_match = title_match and author_match
+        confidence = (title_similarity * 0.5 + author_similarity * 0.3 + (1.0 if year_match else 0.5) * 0.2)
+        
+        return ComparisonResult(
+            entry_key=bib_entry.key,
+            title_match=title_match,
+            title_similarity=title_similarity,
+            bib_title=bib_entry.title,
+            fetched_title=dblp_result.title,
+            author_match=author_match,
+            author_similarity=author_similarity,
+            bib_authors=bib_authors,
+            fetched_authors=dblp_authors,
+            year_match=year_match,
+            bib_year=bib_year,
+            fetched_year=dblp_year,
+            is_match=is_match,
+            confidence=confidence,
+            issues=issues,
+            source="dblp",
+            fetched_bibtex=build_fetched_bibtex("dblp", dblp_result),
+            fetched_url=getattr(dblp_result, "url", None) or "",
+            fetched_doi=getattr(dblp_result, "doi", None) or "",
+        )
+
+# TODO: Use API to compare metadata
+if __name__ == "__main__":
+    comparator = MetadataComparator()
+    result = comparator.compare_with_arxiv(BibEntry(key="test", title="Zero Bubble Pipeline Parallelism", author="Penghui Qi, Xinyi Wan, Guangxing Huang, Min Lin"), ArxivMetadata(title="Zero Bubble Pipeline Parallelism", authors=["Penghui Qi", "Xinyi Wan", "Guangxing Huang", "Min Lin"], year="2023"))
+    print('result: \n', result)

src/config/__init__.py

@@ -0,0 +1,4 @@
+"""Configuration module for citescan."""
+from .workflow import WorkflowConfig, WorkflowStep, DEFAULT_WORKFLOW, get_default_workflow
+
+__all__ = ['WorkflowConfig', 'WorkflowStep', 'DEFAULT_WORKFLOW', 'get_default_workflow']

src/config/workflow.py

@@ -0,0 +1,174 @@
+"""
+Workflow configuration for reference checking.
+
+Allows users to customize the order and enable/disable individual fetchers
+in the reference verification workflow.
+"""
+import json
+from dataclasses import dataclass, field, asdict
+from pathlib import Path
+from typing import List, Optional
+
+
+@dataclass
+class WorkflowStep:
+    """A single step in the reference checking workflow."""
+    name: str
+    display_name: str
+    description: str
+    enabled: bool = True
+    priority: int = 0
+    
+    # Step type: 'by_id', 'by_doi', 'by_title'
+    search_type: str = 'by_title'
+    
+    def to_dict(self) -> dict:
+        return asdict(self)
+    
+    @classmethod
+    def from_dict(cls, data: dict) -> 'WorkflowStep':
+        return cls(**data)
+
+
+@dataclass
+class WorkflowConfig:
+    """Configuration for the reference checking workflow."""
+    steps: List[WorkflowStep] = field(default_factory=list)
+    name: str = "default"
+    description: str = "Default workflow configuration"
+    
+    def get_enabled_steps(self) -> List[WorkflowStep]:
+        """Get only enabled steps, sorted by priority."""
+        return sorted(
+            [s for s in self.steps if s.enabled],
+            key=lambda x: x.priority
+        )
+    
+    def move_step_up(self, index: int) -> bool:
+        """Move a step up in priority (swap with previous)."""
+        if index <= 0 or index >= len(self.steps):
+            return False
+        self.steps[index], self.steps[index - 1] = self.steps[index - 1], self.steps[index]
+        self._update_priorities()
+        return True
+    
+    def move_step_down(self, index: int) -> bool:
+        """Move a step down in priority (swap with next)."""
+        if index < 0 or index >= len(self.steps) - 1:
+            return False
+        self.steps[index], self.steps[index + 1] = self.steps[index + 1], self.steps[index]
+        self._update_priorities()
+        return True
+    
+    def toggle_step(self, index: int) -> bool:
+        """Toggle enabled status of a step."""
+        if 0 <= index < len(self.steps):
+            self.steps[index].enabled = not self.steps[index].enabled
+            return True
+        return False
+    
+    def _update_priorities(self):
+        """Update priority values based on current order."""
+        for i, step in enumerate(self.steps):
+            step.priority = i
+    
+    def to_dict(self) -> dict:
+        return {
+            'name': self.name,
+            'description': self.description,
+            'steps': [s.to_dict() for s in self.steps]
+        }
+    
+    @classmethod
+    def from_dict(cls, data: dict) -> 'WorkflowConfig':
+        steps = [WorkflowStep.from_dict(s) for s in data.get('steps', [])]
+        return cls(
+            steps=steps,
+            name=data.get('name', 'custom'),
+            description=data.get('description', '')
+        )
+    
+    def save(self, filepath: str):
+        """Save workflow configuration to JSON file."""
+        path = Path(filepath)
+        path.parent.mkdir(parents=True, exist_ok=True)
+        with open(path, 'w', encoding='utf-8') as f:
+            json.dump(self.to_dict(), f, indent=2)
+    
+    @classmethod
+    def load(cls, filepath: str) -> 'WorkflowConfig':
+        """Load workflow configuration from JSON file."""
+        with open(filepath, 'r', encoding='utf-8') as f:
+            data = json.load(f)
+        return cls.from_dict(data)
+
+
+# Default workflow matching current implementation order
+DEFAULT_WORKFLOW = WorkflowConfig(
+    name="default",
+    description="Default reference checking workflow prioritizing reliable APIs",
+    steps=[
+        WorkflowStep(
+            name="arxiv_id",
+            display_name="arXiv by ID",
+            description="Look up paper by arXiv ID (highest priority for arXiv papers)",
+            priority=0,
+            search_type="by_id"
+        ),
+        WorkflowStep(
+            name="crossref_doi",
+            display_name="CrossRef by DOI",
+            description="Look up paper by DOI (authoritative for DOIs)",
+            priority=1,
+            search_type="by_doi"
+        ),
+        WorkflowStep(
+            name="semantic_scholar",
+            display_name="Semantic Scholar",
+            description="Official API with high quality metadata",
+            priority=2,
+            search_type="by_title"
+        ),
+        WorkflowStep(
+            name="dblp",
+            display_name="DBLP",
+            description="Official API, especially good for CS publications",
+            priority=3,
+            search_type="by_title"
+        ),
+        WorkflowStep(
+            name="openalex",
+            display_name="OpenAlex",
+            description="Official API with broad coverage",
+            priority=4,
+            search_type="by_title"
+        ),
+        WorkflowStep(
+            name="arxiv_title",
+            display_name="arXiv by Title",
+            description="Search arXiv by title (fallback for non-ID lookups)",
+            priority=5,
+            search_type="by_title"
+        ),
+        WorkflowStep(
+            name="crossref_title",
+            display_name="CrossRef by Title",
+            description="Search CrossRef by title",
+            priority=6,
+            search_type="by_title"
+        ),
+        WorkflowStep(
+            name="google_scholar",
+            display_name="Google Scholar",
+            description="Web scraping fallback (may be rate-limited or blocked)",
+            priority=7,
+            search_type="by_title",
+            enabled=True  # Still enabled but lowest priority
+        ),
+    ]
+)
+
+
+def get_default_workflow() -> WorkflowConfig:
+    """Get a fresh copy of the default workflow."""
+    return WorkflowConfig.from_dict(DEFAULT_WORKFLOW.to_dict())

src/config/yaml_config.py

@@ -0,0 +1,170 @@
+"""
+YAML configuration loader for citescan (bib-only).
+"""
+import yaml
+from pathlib import Path
+from dataclasses import dataclass, field
+from typing import Optional, List
+
+
+@dataclass
+class FilesConfig:
+    """File path configuration (bib-only)."""
+    bib: str = ""
+    input_dir: str = ""
+    output_dir: str = "citescan_output"
+
+
+@dataclass
+class BibliographyConfig:
+    """Bibliography check configuration."""
+    check_metadata: bool = True
+    check_duplicates: bool = True
+    check_preprint_ratio: bool = True
+    preprint_warning_threshold: float = 0.50
+
+
+@dataclass
+class WorkflowStep:
+    """Single step in the reference check workflow."""
+    name: str
+    enabled: bool = True
+    description: str = ""
+
+
+@dataclass
+class OutputConfig:
+    """Output configuration."""
+    quiet: bool = False
+    minimal_verified: bool = False
+
+
+@dataclass
+class CiteScanConfig:
+    """citescan configuration"""
+    files: FilesConfig = field(default_factory=FilesConfig)
+    bibliography: BibliographyConfig = field(default_factory=BibliographyConfig)
+    workflow: List[WorkflowStep] = field(default_factory=list)
+    output: OutputConfig = field(default_factory=OutputConfig)
+
+    _bib_files: List[Path] = field(default_factory=list)
+    _config_dir: Path = field(default_factory=lambda: Path.cwd())
+
+    def resolve_path(self, path: str) -> Path:
+        p = Path(path)
+        if p.is_absolute():
+            return p
+        return self._config_dir / p
+
+    @property
+    def bib_path(self) -> Path:
+        return self.resolve_path(self.files.bib)
+
+    @property
+    def input_dir_path(self) -> Path:
+        return self.resolve_path(self.files.input_dir)
+
+    @property
+    def output_dir_path(self) -> Path:
+        return self.resolve_path(self.files.output_dir)
+
+
+def load_config(config_path: str) -> CiteScanConfig:
+    """Load configuration from YAML file."""
+    path = Path(config_path)
+    if not path.exists():
+        raise FileNotFoundError(f"Config file not found: {config_path}")
+    with open(path, "r", encoding="utf-8") as f:
+        data = yaml.safe_load(f) or {}
+
+    config = CiteScanConfig()
+    config._config_dir = path.parent.absolute()
+
+    if "files" in data:
+        files = data["files"]
+        config.files = FilesConfig(
+            bib=files.get("bib", ""),
+            input_dir=files.get("input_dir", ""),
+            output_dir=files.get("output_dir", "citescan_output"),
+        )
+    if "bibliography" in data:
+        bib = data["bibliography"]
+        config.bibliography = BibliographyConfig(
+            check_metadata=bib.get("check_metadata", True),
+            check_duplicates=bib.get("check_duplicates", True),
+            check_preprint_ratio=bib.get("check_preprint_ratio", True),
+            preprint_warning_threshold=bib.get("preprint_warning_threshold", 0.50),
+        )
+    if "workflow" in data:
+        config.workflow = [
+            WorkflowStep(
+                name=step.get("name", ""),
+                enabled=step.get("enabled", True),
+                description=step.get("description", ""),
+            )
+            for step in data["workflow"]
+        ]
+    if "output" in data:
+        out = data["output"]
+        config.output = OutputConfig(
+            quiet=out.get("quiet", False),
+            minimal_verified=out.get("minimal_verified", False),
+        )
+    return config
+
+
+def find_config_file() -> Optional[Path]:
+    """Find config file in current directory or parent directories."""
+    names = ["config.yaml", "citescan.yaml", "citescan.yml", ".citescan.yaml", ".citescan.yml"]
+    current = Path.cwd()
+    for _ in range(5):
+        for name in names:
+            p = current / name
+            if p.exists():
+                return p
+        parent = current.parent
+        if parent == current:
+            break
+        current = parent
+    return None
+
+
+def create_default_config(output_path: str = "config.yaml") -> str:
+    """Create a default config file (bib-only)."""
+    default = """# citescan Configuration (bib-only)
+
+files:
+  bib: "paper.bib"
+  output_dir: "citescan_output"
+
+bibliography:
+  check_metadata: true
+  check_duplicates: true
+  check_preprint_ratio: true
+  preprint_warning_threshold: 0.50
+
+workflow:
+  - name: arxiv_id
+    enabled: true
+  - name: crossref_doi
+    enabled: true
+  - name: semantic_scholar
+    enabled: true
+  - name: dblp
+    enabled: true
+  - name: openalex
+    enabled: true
+  - name: arxiv_title
+    enabled: true
+  - name: crossref_title
+    enabled: true
+  - name: google_scholar
+    enabled: false
+
+output:
+  quiet: false
+  minimal_verified: false
+"""
+    with open(output_path, "w", encoding="utf-8") as f:
+        f.write(default)
+    return output_path

src/fetchers/__init__.py

@@ -0,0 +1,16 @@
+"""Fetchers package"""
+from .arxiv_fetcher import ArxivFetcher
+from .scholar_fetcher import ScholarFetcher
+from .crossref_fetcher import CrossRefFetcher
+from .semantic_scholar_fetcher import SemanticScholarFetcher
+from .openalex_fetcher import OpenAlexFetcher
+from .dblp_fetcher import DBLPFetcher
+
+__all__ = [
+    'ArxivFetcher', 
+    'ScholarFetcher', 
+    'CrossRefFetcher',
+    'SemanticScholarFetcher',
+    'OpenAlexFetcher',
+    'DBLPFetcher'
+]

src/fetchers/arxiv_fetcher.py

@@ -0,0 +1,228 @@
+"""
+arXiv metadata fetcher using the public API.
+"""
+import re
+import time
+import xml.etree.ElementTree as ET
+from dataclasses import dataclass
+from typing import Optional
+from urllib.parse import quote
+
+import requests
+
+
+@dataclass
+class ArxivMetadata:
+    """Metadata fetched from arXiv."""
+    arxiv_id: str
+    title: str
+    authors: list[str]
+    abstract: str
+    published: str
+    updated: str
+    categories: list[str]
+    primary_category: str
+    doi: str
+    journal_ref: str
+    comment: str
+    pdf_url: str
+    abs_url: str
+    
+    @property
+    def year(self) -> str:
+        """Extract year from published date."""
+        if self.published:
+            match = re.match(r'(\d{4})', self.published)
+            if match:
+                return match.group(1)
+        return ""
+
+
+class ArxivFetcher:
+    """Fetches metadata from arXiv API."""
+    
+    API_BASE = "http://export.arxiv.org/api/query"
+    RATE_LIMIT_DELAY = 3.0  # seconds between requests
+    
+    def __init__(self):
+        self._last_request_time = 0.0
+    
+    def _rate_limit(self):
+        """Ensure rate limiting between requests."""
+        elapsed = time.time() - self._last_request_time
+        if elapsed < self.RATE_LIMIT_DELAY:
+            time.sleep(self.RATE_LIMIT_DELAY - elapsed)
+        self._last_request_time = time.time()
+    
+    def fetch_by_id(self, arxiv_id: str) -> Optional[ArxivMetadata]:
+        """Fetch metadata by arXiv ID."""
+        # Clean up ID
+        arxiv_id = arxiv_id.strip()
+        arxiv_id = re.sub(r'^arXiv:', '', arxiv_id, flags=re.IGNORECASE)
+        
+        self._rate_limit()
+        
+        params = {
+            'id_list': arxiv_id,
+            'max_results': 1
+        }
+        
+        try:
+            response = requests.get(
+                self.API_BASE,
+                params=params,
+                timeout=30,
+                headers={'User-Agent': 'BibChecker/1.0 (mailto:user@example.com)'}
+            )
+            response.raise_for_status()
+        except requests.RequestException as e:
+            return None
+        
+        return self._parse_response(response.text)
+    
+    def search_by_title(self, title: str, max_results: int = 5) -> list[ArxivMetadata]:
+        """Search arXiv by title."""
+        self._rate_limit()
+        
+        # Clean up title for search
+        clean_title = re.sub(r'[^\w\s]', ' ', title)
+        clean_title = re.sub(r'\s+', ' ', clean_title).strip()
+        
+        # Build search query
+        search_query = f'ti:"{clean_title}"'
+        
+        params = {
+            'search_query': search_query,
+            'max_results': max_results,
+            'sortBy': 'relevance',
+            'sortOrder': 'descending'
+        }
+        
+        try:
+            response = requests.get(
+                self.API_BASE,
+                params=params,
+                timeout=30,
+                headers={'User-Agent': 'BibChecker/1.0 (mailto:user@example.com)'}
+            )
+            response.raise_for_status()
+        except requests.RequestException as e:
+            return []
+        
+        return self._parse_response_multiple(response.text)
+    
+    def _parse_response(self, xml_content: str) -> Optional[ArxivMetadata]:
+        """Parse single entry response."""
+        results = self._parse_response_multiple(xml_content)
+        return results[0] if results else None
+    
+    def _parse_response_multiple(self, xml_content: str) -> list[ArxivMetadata]:
+        """Parse multiple entries from response."""
+        results = []
+        
+        try:
+            root = ET.fromstring(xml_content)
+        except ET.ParseError:
+            return results
+        
+        # Define namespaces
+        ns = {
+            'atom': 'http://www.w3.org/2005/Atom',
+            'arxiv': 'http://arxiv.org/schemas/atom'
+        }
+        
+        entries = root.findall('atom:entry', ns)
+        
+        for entry in entries:
+            try:
+                metadata = self._parse_entry(entry, ns)
+                if metadata:
+                    results.append(metadata)
+            except Exception:
+                continue
+        
+        return results
+    
+    def _parse_entry(self, entry: ET.Element, ns: dict) -> Optional[ArxivMetadata]:
+        """Parse a single entry element."""
+        # Get ID
+        id_elem = entry.find('atom:id', ns)
+        if id_elem is None or id_elem.text is None:
+            return None
+        
+        abs_url = id_elem.text.strip()
+        
+        # Extract arXiv ID from URL
+        match = re.search(r'arxiv\.org/abs/(.+)$', abs_url)
+        arxiv_id = match.group(1) if match else ""
+        
+        # Get title
+        title_elem = entry.find('atom:title', ns)
+        title = self._clean_text(title_elem.text) if title_elem is not None and title_elem.text else ""
+        
+        # Get abstract
+        summary_elem = entry.find('atom:summary', ns)
+        abstract = self._clean_text(summary_elem.text) if summary_elem is not None and summary_elem.text else ""
+        
+        # Get authors
+        authors = []
+        for author_elem in entry.findall('atom:author', ns):
+            name_elem = author_elem.find('atom:name', ns)
+            if name_elem is not None and name_elem.text:
+                authors.append(name_elem.text.strip())
+        
+        # Get dates
+        published_elem = entry.find('atom:published', ns)
+        published = published_elem.text.strip() if published_elem is not None and published_elem.text else ""
+        
+        updated_elem = entry.find('atom:updated', ns)
+        updated = updated_elem.text.strip() if updated_elem is not None and updated_elem.text else ""
+        
+        # Get categories
+        categories = []
+        for cat_elem in entry.findall('atom:category', ns):
+            term = cat_elem.get('term')
+            if term:
+                categories.append(term)
+        
+        primary_cat_elem = entry.find('arxiv:primary_category', ns)
+        primary_category = primary_cat_elem.get('term', '') if primary_cat_elem is not None else ""
+        
+        # Get DOI
+        doi_elem = entry.find('arxiv:doi', ns)
+        doi = doi_elem.text.strip() if doi_elem is not None and doi_elem.text else ""
+        
+        # Get journal reference
+        journal_elem = entry.find('arxiv:journal_ref', ns)
+        journal_ref = journal_elem.text.strip() if journal_elem is not None and journal_elem.text else ""
+        
+        # Get comment
+        comment_elem = entry.find('arxiv:comment', ns)
+        comment = comment_elem.text.strip() if comment_elem is not None and comment_elem.text else ""
+        
+        # Build PDF URL
+        pdf_url = abs_url.replace('/abs/', '/pdf/') + '.pdf'
+        
+        return ArxivMetadata(
+            arxiv_id=arxiv_id,
+            title=title,
+            authors=authors,
+            abstract=abstract,
+            published=published,
+            updated=updated,
+            categories=categories,
+            primary_category=primary_category,
+            doi=doi,
+            journal_ref=journal_ref,
+            comment=comment,
+            pdf_url=pdf_url,
+            abs_url=abs_url
+        )
+    
+    def _clean_text(self, text: str) -> str:
+        """Clean up text from XML."""
+        if not text:
+            return ""
+        # Normalize whitespace
+        text = re.sub(r'\s+', ' ', text)
+        return text.strip()

src/fetchers/crossref_fetcher.py

@@ -0,0 +1,203 @@
+"""
+CrossRef API fetcher for bibliography metadata.
+
+CrossRef provides free, reliable access to metadata for academic publications.
+No API key required, no rate limiting for reasonable use.
+"""
+import requests
+from dataclasses import dataclass
+from typing import Optional, List
+import time
+
+
+@dataclass
+class CrossRefResult:
+    """Metadata result from CrossRef API."""
+    title: str
+    authors: List[str]
+    year: str
+    doi: str
+    publisher: str
+    container_title: str  # Journal/conference name
+    abstract: str = ""
+    url: str = ""
+    
+    
+class CrossRefFetcher:
+    """
+    Fetcher for CrossRef API.
+    
+    CrossRef is a reliable, free API for academic metadata.
+    Much more reliable than Google Scholar scraping.
+    """
+    
+    BASE_URL = "https://api.crossref.org/works"
+    RATE_LIMIT_DELAY = 1.0  # Be polite
+    
+    def __init__(self, mailto: str = "e1143641@u.nus.edu"):
+        """
+        Initialize CrossRef fetcher.
+        
+        Args:
+            mailto: Email for polite pool (gets better rate limits)
+        """
+        self.mailto = mailto
+        self._last_request_time = 0.0
+        self._session = requests.Session()
+    
+    def _rate_limit(self):
+        """Ensure rate limiting between requests."""
+        elapsed = time.time() - self._last_request_time
+        if elapsed < self.RATE_LIMIT_DELAY:
+            time.sleep(self.RATE_LIMIT_DELAY - elapsed)
+        self._last_request_time = time.time()
+    
+    def _get_headers(self) -> dict:
+        """Get request headers with mailto for polite pool."""
+        return {
+            'User-Agent': f'CiteScan/1.0 (mailto:{self.mailto})',
+            'Accept': 'application/json',
+        }
+    
+    def search_by_title(self, title: str, max_results: int = 5) -> Optional[CrossRefResult]:
+        """
+        Search for a paper by title.
+        
+        Args:
+            title: Paper title to search for
+            max_results: Maximum number of results to retrieve
+            
+        Returns:
+            Best matching CrossRefResult or None if not found
+        """
+        self._rate_limit()
+        
+        params = {
+            'query.title': title,
+            'rows': max_results,
+            'select': 'title,author,published-print,published-online,DOI,publisher,container-title,abstract'
+        }
+        
+        try:
+            response = self._session.get(
+                self.BASE_URL,
+                params=params,
+                headers=self._get_headers(),
+                timeout=30
+            )
+            response.raise_for_status()
+            
+            data = response.json()
+            
+            if data.get('status') != 'ok':
+                return None
+            
+            items = data.get('message', {}).get('items', [])
+            
+            if not items:
+                return None
+            
+            # Return best match (first result, as CrossRef ranks by relevance)
+            return self._parse_item(items[0])
+            
+        except requests.RequestException:
+            return None
+    
+    def search_by_doi(self, doi: str) -> Optional[CrossRefResult]:
+        """
+        Fetch metadata by DOI.
+        
+        Args:
+            doi: DOI of the paper
+            
+        Returns:
+            CrossRefResult or None if not found
+        """
+        self._rate_limit()
+        
+        # Clean DOI (remove https://doi.org/ prefix if present)
+        doi = doi.replace('https://doi.org/', '').replace('http://doi.org/', '')
+        
+        try:
+            response = self._session.get(
+                f"{self.BASE_URL}/{doi}",
+                headers=self._get_headers(),
+                timeout=30
+            )
+            response.raise_for_status()
+            
+            data = response.json()
+            
+            if data.get('status') != 'ok':
+                return None
+            
+            item = data.get('message', {})
+            return self._parse_item(item)
+            
+        except requests.RequestException:
+            return None
+    
+    def _parse_item(self, item: dict) -> Optional[CrossRefResult]:
+        """Parse a CrossRef API item into CrossRefResult."""
+        try:
+            # Get title
+            titles = item.get('title', [])
+            title = titles[0] if titles else ""
+            
+            if not title:
+                return None
+            
+            # Get authors
+            authors = []
+            for author in item.get('author', []):
+                given_name = author.get('given', '')
+                family_name = author.get('family', '')
+                if family_name:
+                    if given_name:
+                        authors.append(f"{given_name} {family_name}")
+                    else:
+                        authors.append(family_name)
+            
+            # Get year (try published-print first, then published-online)
+            year = ""
+            for date_field in ['published-print', 'published-online', 'created']:
+                date_parts = item.get(date_field, {}).get('date-parts', [[]])
+                if date_parts and date_parts[0]:
+                    year = str(date_parts[0][0])
+                    break
+            
+            # Get DOI
+            doi = item.get('DOI', '')
+            
+            # Get publisher
+            publisher = item.get('publisher', '')
+            
+            # Get container title (journal/conference name)
+            container_titles = item.get('container-title', [])
+            container_title = container_titles[0] if container_titles else ""
+            
+            # Get abstract (if available)
+            abstract = item.get('abstract', '')
+            
+            # Build URL
+            url = f"https://doi.org/{doi}" if doi else ""
+            
+            return CrossRefResult(
+                title=title,
+                authors=authors,
+                year=year,
+                doi=doi,
+                publisher=publisher,
+                container_title=container_title,
+                abstract=abstract,
+                url=url
+            )
+            
+        except (KeyError, IndexError, TypeError):
+            return None
+
+# TODO: Use API to fetch crossref results
+if __name__ == "__main__":
+    crossref_fetcher = CrossRefFetcher(mailto="e1143641@u.nus.edu")
+    results = crossref_fetcher.search_by_title("Zero Bubble Pipeline Parallelism")
+    print('results: \n', results)

src/fetchers/dblp_fetcher.py

@@ -0,0 +1,121 @@
+import requests
+import time
+import logging
+from typing import Optional, List, Dict, Any
+from dataclasses import dataclass
+
+@dataclass
+class DBLPResult:
+    title: str
+    authors: List[str]
+    year: str
+    venue: str
+    url: str
+    doi: Optional[str] = None
+
+class DBLPFetcher:
+    """Fetcher for DBLP API."""
+    
+    BASE_URL = "https://dblp.org/search/publ/api"
+     
+    def __init__(self):
+        self.last_request_time = 0
+        # DBLP asks for 1-2 seconds between requests. We'll use 1.5s to be safe.
+        self.rate_limit_delay = 1.5
+        self.logger = logging.getLogger(__name__)
+
+    def _wait_for_rate_limit(self):
+        """Ensure we don't exceed rate limits."""
+        elapsed = time.time() - self.last_request_time
+        if elapsed < self.rate_limit_delay:
+            time.sleep(self.rate_limit_delay - elapsed)
+        self.last_request_time = time.time()
+
+    def search_by_title(self, title: str) -> Optional[DBLPResult]:
+        """Search DBLP by title.
+        reference: https://dblp.org/faq/How+to+use+the+dblp+search+API.html
+        """
+        self._wait_for_rate_limit()
+
+        params = {
+            "q": title,
+            "format": "json",
+            "h": 5
+        }
+        
+        try:
+            response = requests.get(self.BASE_URL, params=params, timeout=10)
+            
+            if response.status_code == 429:
+                self.logger.warning("DBLP rate limit exceeded. Waiting longer...")
+                time.sleep(5)
+                return None
+                
+            if response.status_code != 200:
+                self.logger.warning(f"DBLP API error: {response.status_code}")
+                return None
+                
+            data = response.json()
+            return self._parse_response(data, title)
+            
+        except Exception as e:
+            self.logger.error(f"Error fetching from DBLP: {e}")
+            return None
+
+    def _parse_response(self, data: Dict[str, Any], query_title: str) -> Optional[DBLPResult]:
+        """Parse DBLP JSON response."""
+        try:
+            result = data.get("result", {})
+            hits = result.get("hits", {}).get("hit", [])
+            
+            if not hits:
+                return None
+            
+            # Find best match
+            best_hit = None
+            
+            # Simple check: first hit is usually the best in DBLP for exact title match
+            # But we can do a quick normalization check if needed.
+            # For now, let's take the first hit that is a publication (not a person/venue)
+            # The search/publ/api endpoint should only return publications.
+            
+            best_hit = hits[0]
+            info = best_hit.get("info", {})
+            
+            # Extract authors
+            authors_data = info.get("authors", {}).get("author", [])
+            authors = []
+            if isinstance(authors_data, list):
+                authors = [a.get("text", "") for a in authors_data]
+            elif isinstance(authors_data, dict):
+                authors = [authors_data.get("text", "")]
+                
+            # Extract other fields
+            title = info.get("title", "")
+            year = info.get("year", "")
+            venue = info.get("venue", "")
+            url = info.get("url", "")
+            doi = info.get("doi", "")
+            
+            # Clean title (DBLP titles often end with a dot)
+            if title.endswith("."):
+                title = title[:-1]
+            
+            return DBLPResult(
+                title=title,
+                authors=authors,
+                year=year,
+                venue=venue,
+                url=url,
+                doi=doi if doi else None
+            )
+            
+        except Exception as e:
+            self.logger.error(f"Error parsing DBLP response: {e}")
+            return None
+
+# TODO: Use API to fetch dblp results
+if __name__ == "__main__":
+    dblp_fetcher = DBLPFetcher()
+    results = dblp_fetcher.search_by_title("Zero Bubble Pipeline Parallelism")
+    print('results: \n', results)

src/fetchers/openalex_fetcher.py

@@ -0,0 +1,201 @@
+"""
+OpenAlex API fetcher.
+Free and open API for scholarly metadata.
+"""
+import time
+from dataclasses import dataclass
+from typing import Optional
+
+import requests
+
+
+@dataclass
+class OpenAlexResult:
+    """Search result from OpenAlex API."""
+    title: str
+    authors: list[str]
+    year: str
+    abstract: str
+    doi: str
+    citation_count: int
+    url: str
+
+
+class OpenAlexFetcher:
+    """
+    Fetcher using OpenAlex's free API.
+    
+    API Docs: https://docs.openalex.org/
+    Rate Limits:
+    - 100,000 requests per day
+    - 10 requests per second (very generous)
+    - No API key required (but polite pool recommended)
+    """
+    
+    BASE_URL = "https://api.openalex.org"
+    RATE_LIMIT_DELAY = 0.1  # 10 req/sec max
+    
+    def __init__(self, email: Optional[str] = None):
+        """
+        Initialize OpenAlex fetcher.
+        
+        Args:
+            email: Optional email for polite pool (faster rate limits)
+        """
+        self.email = email
+        self._last_request_time = 0.0
+        self._session = requests.Session()
+        
+        # Set user agent (required by OpenAlex)
+        self._session.headers.update({
+            'User-Agent': 'CiteScan/1.0 (https://github.com/aivolcano/CiteScan; mailto:e1143641@u.nus.edu)'
+        })
+        
+        # Add email to polite pool if provided
+        if email:
+            self._session.headers.update({'From': email})
+    
+    def _rate_limit(self):
+        """Ensure rate limiting between requests."""
+        elapsed = time.time() - self._last_request_time
+        if elapsed < self.RATE_LIMIT_DELAY:
+            time.sleep(self.RATE_LIMIT_DELAY - elapsed)
+        self._last_request_time = time.time()
+    
+    def search_by_title(self, title: str, max_results: int = 5) -> Optional[OpenAlexResult]:
+        """
+        Search for a paper by title.
+        
+        Args:
+            title: Paper title to search for
+            max_results: Maximum number of results to fetch (default: 5)
+            
+        Returns:
+            OpenAlexResult if found, None otherwise
+        """
+        self._rate_limit()
+        
+        url = f"{self.BASE_URL}/works"
+        params = {
+            'search': title,
+            'per-page': max_results
+        }
+        
+        try:
+            response = self._session.get(url, params=params, timeout=10)
+            response.raise_for_status()
+            data = response.json()
+            
+            results = data.get('results', [])
+            if not results:
+                return None
+            
+            # Return the first (most relevant) result
+            return self._parse_work(results[0])
+            
+        except requests.RequestException:
+            return None
+    
+    def fetch_by_doi(self, doi: str) -> Optional[OpenAlexResult]:
+        """
+        Fetch paper metadata by DOI.
+        
+        Args:
+            doi: DOI of the paper
+            
+        Returns:
+            OpenAlexResult if found, None otherwise
+        """
+        self._rate_limit()
+        
+        # OpenAlex uses DOI URLs
+        doi_url = f"https://doi.org/{doi}"
+        url = f"{self.BASE_URL}/works/{doi_url}"
+        
+        try:
+            response = self._session.get(url, timeout=10)
+            response.raise_for_status()
+            data = response.json()
+            return self._parse_work(data)
+            
+        except requests.RequestException:
+            return None
+    
+    def _parse_work(self, work_data: dict) -> Optional[OpenAlexResult]:
+        """Parse work data from API response."""
+        try:
+            # Extract title
+            title = work_data.get('title', '')
+            
+            # Extract authors
+            authors = []
+            authorships = work_data.get('authorships', [])
+            for authorship in authorships:
+                author = authorship.get('author', {})
+                name = author.get('display_name', '')
+                if name:
+                    authors.append(name)
+            
+            # Get publication year
+            year = work_data.get('publication_year')
+            year_str = str(year) if year else ""
+            
+            # Get abstract (inverted index format)
+            abstract = ""
+            abstract_inverted = work_data.get('abstract_inverted_index')
+            if abstract_inverted:
+                # Reconstruct abstract from inverted index
+                abstract = self._reconstruct_abstract(abstract_inverted)
+            
+            # Get DOI
+            doi = work_data.get('doi', '')
+            if doi and doi.startswith('https://doi.org/'):
+                doi = doi.replace('https://doi.org/', '')
+            
+            # Get citation count
+            citation_count = work_data.get('cited_by_count', 0)
+            
+            # Get URL
+            url = work_data.get('id', '')  # OpenAlex ID URL
+            
+            return OpenAlexResult(
+                title=title,
+                authors=authors,
+                year=year_str,
+                abstract=abstract,
+                doi=doi,
+                citation_count=citation_count,
+                url=url
+            )
+        except (KeyError, TypeError):
+            return None
+    
+    def _reconstruct_abstract(self, inverted_index: dict) -> str:
+        """
+        Reconstruct abstract text from inverted index.
+        
+        OpenAlex stores abstracts in inverted index format:
+        {"word": [position1, position2, ...], ...}
+        """
+        if not inverted_index:
+            return ""
+        
+        try:
+            # Create a list to hold words at their positions
+            max_pos = max(max(positions) for positions in inverted_index.values())
+            words = [''] * (max_pos + 1)
+            
+            # Place each word at its positions
+            for word, positions in inverted_index.items():
+                for pos in positions:
+                    words[pos] = word
+            
+            # Join words with spaces
+            return ' '.join(word for word in words if word)
+        except (ValueError, TypeError):
+            return ""
+
+if __name__ == "__main__":
+    openalex_fetcher = OpenAlexFetcher(email="e1143641@u.nus.edu")
+    results = openalex_fetcher.search_by_title("Zero Bubble Pipeline Parallelism")
+    print('results: \n', results)

src/fetchers/scholar_fetcher.py

@@ -0,0 +1,225 @@
+"""
+Google Scholar search (scraping-based fallback).
+"""
+import re
+import time
+import random
+from dataclasses import dataclass
+from typing import Optional
+
+import requests
+from bs4 import BeautifulSoup
+
+
+@dataclass
+class ScholarResult:
+    """Search result from Google Scholar."""
+    title: str
+    authors: str
+    year: str
+    snippet: str
+    url: str
+    cited_by: int
+    
+
+class ScholarFetcher:
+    """
+    Fallback fetcher using Google Scholar search.
+    
+    Note: This uses scraping and may be blocked. 
+    Use rate limiting and respect robots.txt.
+    """
+    
+    SEARCH_URL = "https://scholar.google.com/scholar"
+    RATE_LIMIT_DELAY = 10.0  # Conservative delay to avoid blocking (was 5.0)
+    MAX_RETRIES = 2  # Retry on failures
+    
+    USER_AGENTS = [
+        'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
+        'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
+        'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
+        'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Safari/605.1.15',
+        'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0',
+    ]
+    
+    def __init__(self):
+        self._last_request_time = 0.0
+        self._session = requests.Session()
+        self._request_count = 0
+        self._blocked = False  # Track if we've been blocked
+    
+    def _rate_limit(self):
+        """Ensure rate limiting between requests."""
+        elapsed = time.time() - self._last_request_time
+        # Add more randomness to avoid detection (3-5 seconds extra)
+        delay = self.RATE_LIMIT_DELAY + random.uniform(3, 5)
+        if elapsed < delay:
+            time.sleep(delay - elapsed)
+        self._last_request_time = time.time()
+    
+    def _get_headers(self) -> dict:
+        """Get request headers with random user agent."""
+        return {
+            'User-Agent': random.choice(self.USER_AGENTS),
+            'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
+            'Accept-Language': 'en-US,en;q=0.5',
+            'Accept-Encoding': 'gzip, deflate',
+            'Connection': 'keep-alive',
+            'Upgrade-Insecure-Requests': '1',
+        }
+    
+    def search(self, query: str, max_results: int = 5) -> list[ScholarResult]:
+        """
+        Search Google Scholar.
+        
+        Returns list of search results.
+        Note: This may fail if blocked by Google.
+        """
+        # If we've been blocked, don't waste time
+        if self._blocked:
+            return []
+        
+        self._rate_limit()
+        self._request_count += 1
+        
+        params = {
+            'q': query,
+            'hl': 'en',
+            'num': min(max_results, 10)  # Scholar max is 10 per page
+        }
+        
+        try:
+            response = self._session.get(
+                self.SEARCH_URL,
+                params=params,
+                headers=self._get_headers(),
+                timeout=30
+            )
+            response.raise_for_status()
+        except requests.RequestException as e:
+            return []
+        
+        # Check if we're blocked
+        if 'unusual traffic' in response.text.lower() or response.status_code == 429:
+            self._blocked = True
+            print(f"⚠️  Google Scholar blocked after {self._request_count} requests. Skipping further Scholar queries.")
+            return []
+        
+        return self._parse_results(response.text, max_results)
+    
+    def search_by_title(self, title: str) -> Optional[ScholarResult]:
+        """Search for a specific paper by title."""
+        # Use quotes for exact title match
+        query = f'"{title}"'
+        results = self.search(query, max_results=3)
+        
+        if not results:
+            # Try without quotes
+            results = self.search(title, max_results=5)
+        
+        return results[0] if results else None
+    
+    def _parse_results(self, html: str, max_results: int) -> list[ScholarResult]:
+        """Parse search results from HTML."""
+        results = []
+        soup = BeautifulSoup(html, 'lxml')
+        
+        # Find all result entries
+        entries = soup.find_all('div', class_='gs_ri')
+        
+        for entry in entries[:max_results]:
+            try:
+                result = self._parse_entry(entry)
+                if result:
+                    results.append(result)
+            except Exception:
+                continue
+        
+        return results
+    
+    def _parse_entry(self, entry) -> Optional[ScholarResult]:
+        """Parse a single search result entry."""
+        # Get title
+        title_elem = entry.find('h3', class_='gs_rt')
+        if not title_elem:
+            return None
+        
+        # Get title text and URL
+        title_link = title_elem.find('a')
+        if title_link:
+            title = title_link.get_text(strip=True)
+            url = title_link.get('href', '')
+        else:
+            title = title_elem.get_text(strip=True)
+            url = ''
+        
+        # Clean title (remove [PDF], [HTML] markers)
+        title = re.sub(r'^\[(PDF|HTML|BOOK|CITATION)\]\s*', '', title)
+        
+        # Get authors and year from the green line
+        meta_elem = entry.find('div', class_='gs_a')
+        authors = ""
+        year = ""
+        
+        if meta_elem:
+            meta_text = meta_elem.get_text(strip=True)
+            
+            # Extract year first
+            year_match = re.search(r'\b(19|20)\d{2}\b', meta_text)
+            if year_match:
+                year = year_match.group(0)
+            
+            # Parse authors more carefully
+            # Format is usually: "Author1, Author2 - Journal, Year - Publisher"
+            # or sometimes: "Author1, Author2 - Journal/Conference - Year"
+            parts = meta_text.split(' - ')
+            if parts:
+                author_part = parts[0].strip()
+                
+                # Clean up author field - remove year if it leaked in
+                if year:
+                    # Remove year and anything after it from author field
+                    author_part = re.sub(r',?\s*' + re.escape(year) + r'.*$', '', author_part)
+                
+                # Remove common journal/venue keywords that might have leaked
+                # Handle patterns like "the journal of", "the proceedings", etc.
+                author_part = re.sub(r'\s+the\s+(journal|proceedings|conference|symposium|workshop|transactions|magazine|review|annals)\s+.*$', '', author_part, flags=re.IGNORECASE)
+                
+                # Also handle without "the" prefix
+                author_part = re.sub(r'\s+(journal|proceedings|conference|symposium|workshop|transactions|magazine|review|annals)\s+.*$', '', author_part, flags=re.IGNORECASE)
+                
+                # Remove standalone "the" at the end (in case it's left over)
+                author_part = re.sub(r'\s+the\s*$', '', author_part, flags=re.IGNORECASE)
+                
+                # Remove trailing commas and whitespace
+                author_part = author_part.rstrip(', ').strip()
+                
+                authors = author_part
+        
+        # Get snippet
+        snippet_elem = entry.find('div', class_='gs_rs')
+        snippet = snippet_elem.get_text(strip=True) if snippet_elem else ""
+        
+        # Get cited by count
+        cited_by = 0
+        cited_elem = entry.find('a', string=re.compile(r'Cited by \d+'))
+        if cited_elem:
+            match = re.search(r'Cited by (\d+)', cited_elem.get_text())
+            if match:
+                cited_by = int(match.group(1))
+        
+        return ScholarResult(
+            title=title,
+            authors=authors,
+            year=year,
+            snippet=snippet,
+            url=url,
+            cited_by=cited_by
+        )
+
+# TODO: Use API to fetch scholar results
+# Fix the code, and provide the serper api key to fetch scholar results
+if __name__ == "__main__":
+    scholar_fetcher = ScholarFetcher()
+    results = scholar_fetcher.search("Zero Bubble Pipeline Parallelism")
+    print('results: \n', results)

src/fetchers/semantic_scholar_fetcher.py

@@ -0,0 +1,177 @@
+"""
+Semantic Scholar API fetcher.
+Official API with high quality metadata and generous rate limits.
+"""
+import time
+from dataclasses import dataclass
+from typing import Optional
+
+import requests, random
+
+
+@dataclass
+class SemanticScholarResult:
+    """Search result from Semantic Scholar API."""
+    title: str
+    authors: list[str]
+    year: str
+    abstract: str
+    paper_id: str
+    citation_count: int
+    url: str
+
+
+class SemanticScholarFetcher:
+    """
+    Fetcher using Semantic Scholar's official API.
+    
+    API Docs: https://api.semanticscholar.org/
+    Rate Limits:
+    - Without API key: 100 requests per 5 minutes
+    - With API key: 5,000 requests per 5 minutes (free)
+    """
+    
+    BASE_URL = "https://api.semanticscholar.org/graph/v1"
+    RATE_LIMIT_DELAY = 0.5  # Conservative delay (120 req/min max)
+    
+    def __init__(self, api_key: Optional[str] = None):
+        """
+        Initialize Semantic Scholar fetcher.
+        
+        Args:
+            api_key: Optional API key for higher rate limits (free from semanticscholar.org)
+        """
+        api_keys = ["oK11tdfAXm5Sdn2wZi2zA24xstu8iE3G6kqR4Jqj","LlAtlGzau01reNF1M8F754BXoVLpkhys78xVA5Mn","ZtJ3CQtjiWQL8iN5NUaU2QMdd8IIglP8RtikS6n7","HZjLYgpG863xuekLReTec7WFXSC78EFg44aPsmpq","I87z0IQQyJ11i3nVMV3fU39VIaM1DCOx8DUuMc9B","DNX92WcQcz8qV2NNABb3T6RrbFBWhy3W5lgbiwsH"]
+        self._last_request_time = 0.0
+        self._session = requests.Session()
+        
+        self._session.headers.update({'x-api-key': random.choice(api_keys)})
+    
+    def _rate_limit(self):
+        """Ensure rate limiting between requests."""
+        elapsed = time.time() - self._last_request_time
+        if elapsed < self.RATE_LIMIT_DELAY:
+            time.sleep(self.RATE_LIMIT_DELAY - elapsed)
+        self._last_request_time = time.time()
+    
+    def search_by_title(self, title: str, max_results: int = 5) -> Optional[SemanticScholarResult]:
+        """
+        Search for a paper by title.
+        
+        Args:
+            title: Paper title to search for
+            max_results: Maximum number of results to fetch (default: 5)
+            
+        Returns:
+            SemanticScholarResult if found, None otherwise
+        """
+        self._rate_limit()
+        
+        url = f"{self.BASE_URL}/paper/search"
+        params = {
+            'query': title,
+            'limit': max_results,
+            'fields': 'title,authors,year,abstract,paperId,citationCount,url'
+        }
+        
+        try:
+            response = self._session.get(url, params=params, timeout=10)
+            response.raise_for_status()
+            data = response.json()
+            
+            papers = data.get('data', [])
+            if not papers:
+                return None
+            
+            # Return the first (most relevant) result
+            return self._parse_paper(papers[0])
+            
+        except requests.RequestException:
+            return None
+    
+    def fetch_by_doi(self, doi: str) -> Optional[SemanticScholarResult]:
+        """
+        Fetch paper metadata by DOI.
+        
+        Args:
+            doi: DOI of the paper
+            
+        Returns:
+            SemanticScholarResult if found, None otherwise
+        """
+        self._rate_limit()
+        
+        url = f"{self.BASE_URL}/paper/DOI:{doi}"
+        params = {
+            'fields': 'title,authors,year,abstract,paperId,citationCount,url'
+        }
+        
+        try:
+            response = self._session.get(url, params=params, timeout=10)
+            response.raise_for_status()
+            data = response.json()
+            return self._parse_paper(data)
+            
+        except requests.RequestException:
+            return None
+    
+    def fetch_by_arxiv_id(self, arxiv_id: str) -> Optional[SemanticScholarResult]:
+        """
+        Fetch paper metadata by arXiv ID.
+        
+        Args:
+            arxiv_id: arXiv ID (e.g., "2301.12345" or "arXiv:2301.12345")
+            
+        Returns:
+            SemanticScholarResult if found, None otherwise
+        """
+        self._rate_limit()
+        
+        # Clean arXiv ID (remove "arXiv:" prefix if present)
+        clean_id = arxiv_id.replace('arXiv:', '')
+        
+        url = f"{self.BASE_URL}/paper/ARXIV:{clean_id}"
+        params = {
+            'fields': 'title,authors,year,abstract,paperId,citationCount,url'
+        }
+        
+        try:
+            response = self._session.get(url, params=params, timeout=10)
+            response.raise_for_status()
+            data = response.json()
+            return self._parse_paper(data)
+            
+        except requests.RequestException:
+            return None
+    
+    def _parse_paper(self, paper_data: dict) -> Optional[SemanticScholarResult]:
+        """Parse paper data from API response."""
+        try:
+            # Extract author names
+            authors = []
+            for author in paper_data.get('authors', []):
+                name = author.get('name', '')
+                if name:
+                    authors.append(name)
+            
+            # Get year (may be None)
+            year = paper_data.get('year')
+            year_str = str(year) if year else ""
+            
+            return SemanticScholarResult(
+                title=paper_data.get('title', ''),
+                authors=authors,
+                year=year_str,
+                abstract=paper_data.get('abstract', ''),
+                paper_id=paper_data.get('paperId', ''),
+                citation_count=paper_data.get('citationCount', 0),
+                url=paper_data.get('url', '')
+            )
+        except (KeyError, TypeError):
+            return None
+
+# TODO: Use API to fetch semantic scholar results
+if __name__ == "__main__":
+    semantic_scholar_fetcher = SemanticScholarFetcher(api_key="oK11tdfAXm5Sdn2wZi2zA24xstu8iE3G6kqR4Jqj")
+    results = semantic_scholar_fetcher.search_by_title("Zero Bubble Pipeline Parallelism")
+    print('results: \n', results)

src/parsers/__init__.py

@@ -0,0 +1,4 @@
+"""Parsers package (bib-only)."""
+from .bib_parser import BibParser
+
+__all__ = ["BibParser"]

src/parsers/bib_parser.py

@@ -0,0 +1,273 @@
+"""
+BibTeX file parser.
+"""
+import re
+from dataclasses import dataclass, field
+from typing import Optional
+from pathlib import Path
+
+import bibtexparser
+from bibtexparser.bparser import BibTexParser
+from bibtexparser.customization import convert_to_unicode
+
+
+@dataclass
+class BibEntry:
+    """Represents a parsed bibliography entry."""
+    key: str
+    entry_type: str
+    title: str = ""
+    author: str = ""
+    year: str = ""
+    abstract: str = ""
+    url: str = ""
+    doi: str = ""
+    arxiv_id: str = ""
+    journal: str = ""
+    booktitle: str = ""
+    publisher: str = ""
+    pages: str = ""
+    volume: str = ""
+    number: str = ""
+    raw_entry: dict = field(default_factory=dict)
+    
+    @property
+    def has_arxiv(self) -> bool:
+        """Check if entry has arXiv information."""
+        return bool(self.arxiv_id)
+    
+    @property
+    def search_query(self) -> str:
+        """Get search query for this entry."""
+        return self.title or self.key
+
+
+class BibParser:
+    """Parser for .bib files."""
+    
+    # Patterns for extracting arXiv IDs
+    ARXIV_PATTERNS = [
+        # New format: 2301.00001 or 2301.00001v1
+        r'(\d{4}\.\d{4,5}(?:v\d+)?)',
+        # Old format: hep-th/9901001 or math.GT/0309136
+        r'([a-z-]+(?:\.[A-Z]{2})?/\d{7}(?:v\d+)?)',
+        # arXiv: prefix
+        r'arXiv:(\d{4}\.\d{4,5}(?:v\d+)?)',
+        r'arXiv:([a-z-]+(?:\.[A-Z]{2})?/\d{7}(?:v\d+)?)',
+    ]
+    
+    # URL patterns for arXiv
+    ARXIV_URL_PATTERNS = [
+        r'arxiv\.org/abs/(\d{4}\.\d{4,5}(?:v\d+)?)',
+        r'arxiv\.org/abs/([a-z-]+(?:\.[A-Z]{2})?/\d{7}(?:v\d+)?)',
+        r'arxiv\.org/pdf/(\d{4}\.\d{4,5}(?:v\d+)?)(?:\.pdf)?',
+        r'arxiv\.org/pdf/([a-z-]+(?:\.[A-Z]{2})?/\d{7}(?:v\d+)?)(?:\.pdf)?',
+    ]
+    
+    def __init__(self):
+        self.entries: list[BibEntry] = []
+    
+    def parse_file(self, filepath: str) -> list[BibEntry]:
+        """Parse a .bib file into a list of entries."""
+        path = Path(filepath)
+        if not path.exists():
+            raise FileNotFoundError(f"Bib file not found: {filepath}")
+        
+        with open(path, 'r', encoding='utf-8', errors='replace') as f:
+            content = f.read()
+        
+        return self.parse_content(content)
+    
+    def parse_content(self, content: str) -> list[BibEntry]:
+        """Parse bib content string into a list of entries."""
+        parser = BibTexParser(common_strings=True)
+        parser.customization = convert_to_unicode
+        
+        try:
+            bib_database = bibtexparser.loads(content, parser=parser)
+        except Exception as e:
+            raise ValueError(f"Failed to parse bib content: {e}")
+        
+        self.entries = []
+        for entry in bib_database.entries:
+            bib_entry = self._convert_entry(entry)
+            self.entries.append(bib_entry)
+        
+        return self.entries
+    
+    def _convert_entry(self, entry: dict) -> BibEntry:
+        """Convert a bibtexparser entry to BibEntry."""
+        # Extract basic fields
+        bib_entry = BibEntry(
+            key=entry.get('ID', ''),
+            entry_type=entry.get('ENTRYTYPE', ''),
+            title=entry.get('title', ''),
+            author=entry.get('author', ''),
+            year=entry.get('year', ''),
+            abstract=entry.get('abstract', ''),
+            url=entry.get('url', ''),
+            doi=entry.get('doi', ''),
+            journal=entry.get('journal', ''),
+            booktitle=entry.get('booktitle', ''),
+            publisher=entry.get('publisher', ''),
+            pages=entry.get('pages', ''),
+            volume=entry.get('volume', ''),
+            number=entry.get('number', ''),
+            raw_entry=entry.copy()
+        )
+        
+        # Extract arXiv ID
+        bib_entry.arxiv_id = self._extract_arxiv_id(entry)
+        
+        return bib_entry
+    
+    def _extract_arxiv_id(self, entry: dict) -> str:
+        """Extract arXiv ID from entry."""
+        # Check eprint field first
+        eprint = entry.get('eprint', '')
+        if eprint:
+            arxiv_id = self._parse_arxiv_id(eprint)
+            if arxiv_id:
+                return arxiv_id
+        
+        # Check arxiv field
+        arxiv = entry.get('arxiv', '')
+        if arxiv:
+            arxiv_id = self._parse_arxiv_id(arxiv)
+            if arxiv_id:
+                return arxiv_id
+        
+        # Check URL field
+        url = entry.get('url', '')
+        if url:
+            for pattern in self.ARXIV_URL_PATTERNS:
+                match = re.search(pattern, url, re.IGNORECASE)
+                if match:
+                    return match.group(1)
+        
+        # Check journal field for "arXiv preprint arXiv:XXXX.XXXXX" format
+        journal = entry.get('journal', '')
+        if journal and 'arxiv' in journal.lower():
+            arxiv_id = self._parse_arxiv_id(journal)
+            if arxiv_id:
+                return arxiv_id
+        
+        # Check note field
+        note = entry.get('note', '')
+        if note:
+            arxiv_id = self._parse_arxiv_id(note)
+            if arxiv_id:
+                return arxiv_id
+        
+        return ""
+    
+    def _parse_arxiv_id(self, text: str) -> str:
+        """Parse arXiv ID from text."""
+        for pattern in self.ARXIV_PATTERNS:
+            match = re.search(pattern, text)
+            if match:
+                return match.group(1)
+        return ""
+    
+    def get_entry_by_key(self, key: str) -> Optional[BibEntry]:
+        """Get entry by citation key."""
+        for entry in self.entries:
+            if entry.key == key:
+                return entry
+        return None
+
+    def filter_file(self, input_path: str, output_path: str, keys_to_keep: set[str]):
+        """
+        Create a new bib file containing only specified keys.
+        Preserves original formatting, comments, and strings.
+        """
+        with open(input_path, 'r', encoding='utf-8') as f:
+            content = f.read()
+            
+        filtered_content = self._filter_content(content, keys_to_keep)
+        
+        with open(output_path, 'w', encoding='utf-8') as f:
+            f.write(filtered_content)
+            
+    def _filter_content(self, content: str, keys_to_keep: set[str]) -> str:
+        """Filter content string keeping only specified keys."""
+        ranges_to_remove = []
+        i = 0
+        length = len(content)
+        
+        while i < length:
+            if content[i] == '@':
+                start = i
+                # Find opening brace
+                brace_open = content.find('{', i)
+                if brace_open == -1:
+                    i += 1
+                    continue
+                
+                # Get entry type
+                entry_type = content[i+1:brace_open].strip().lower()
+                
+                # Skip comments
+                if entry_type == 'comment':
+                    i = brace_open + 1
+                    continue
+                
+                # Find matching closing brace to determine entry end
+                balance = 1
+                j = brace_open + 1
+                in_quote = False
+                
+                while j < length and balance > 0:
+                    char = content[j]
+                    
+                    # Handle escaped characters
+                    if char == '\\':
+                        j += 2
+                        continue
+                        
+                    if char == '"':
+                        in_quote = not in_quote
+                    elif not in_quote:
+                        if char == '{':
+                            balance += 1
+                        elif char == '}':
+                            balance -= 1
+                    j += 1
+                
+                end = j
+                
+                # Extract key (between { and ,)
+                # Only for standard entries, not @string or @preamble
+                if entry_type not in ('string', 'preamble'):
+                    # Find comma or end of entry
+                    # Key is usually the first token after {
+                    key_part = content[brace_open+1:end]
+                    comma_pos = key_part.find(',')
+                    
+                    if comma_pos != -1:
+                        key = key_part[:comma_pos].strip()
+                        
+                        # If key is NOT in keep list, mark for removal
+                        if key not in keys_to_keep:
+                            ranges_to_remove.append((start, end))
+                
+                i = end
+            else:
+                i += 1
+        
+        # Reconstruct content
+        new_content = []
+        last_pos = 0
+        for start, end in ranges_to_remove:
+            new_content.append(content[last_pos:start])
+            
+            # Clean up whitespace after removed entry
+            last_pos = end
+            while last_pos < length and content[last_pos] in ' \t\r':
+                last_pos += 1
+            if last_pos < length and content[last_pos] == '\n':
+                last_pos += 1
+        
+        new_content.append(content[last_pos:])
+        return "".join(new_content)
+

src/report/__init__.py

@@ -0,0 +1,4 @@
+"""Report package (bib-only)."""
+from .generator import ReportGenerator, EntryReport
+
+__all__ = ["ReportGenerator", "EntryReport"]

src/report/generator.py

@@ -0,0 +1,226 @@
+from dataclasses import dataclass
+from datetime import datetime
+from typing import Optional, List
+from pathlib import Path
+
+from ..parsers.bib_parser import BibEntry
+from ..analyzers.metadata_comparator import ComparisonResult
+from ..analyzers.duplicate_detector import DuplicateGroup
+
+
+@dataclass
+class EntryReport:
+    """Report for a single bib entry (bib-only: entry + comparison)."""
+    entry: BibEntry
+    comparison: Optional[ComparisonResult]
+
+
+class ReportGenerator:
+    """Generates bibliography-only markdown reports."""
+
+    def __init__(
+        self,
+        minimal_verified: bool = False,
+        check_preprint_ratio: bool = True,
+        preprint_warning_threshold: float = 0.50,
+    ):
+        self.entries: List[EntryReport] = []
+        self.duplicate_groups: Optional[List[DuplicateGroup]] = None
+        self.bib_files: List[str] = []
+        self.minimal_verified = minimal_verified
+        self.check_preprint_ratio = check_preprint_ratio
+        self.preprint_warning_threshold = preprint_warning_threshold
+
+    def add_entry_report(self, report: EntryReport):
+        self.entries.append(report)
+
+    def set_metadata(self, bib_files: str | List[str], tex_files: str | List[str] = None):
+        if isinstance(bib_files, str):
+            self.bib_files = [bib_files]
+        else:
+            self.bib_files = list(bib_files) if bib_files else []
+
+    def set_duplicate_groups(self, groups: List[DuplicateGroup]):
+        self.duplicate_groups = groups
+
+    def _is_verified(self, entry: EntryReport) -> bool:
+        return not self._has_issues(entry)
+
+    def _has_issues(self, entry: EntryReport) -> bool:
+        return bool(entry.comparison and entry.comparison.has_issues)
+
+    def _is_preprint(self, entry: BibEntry) -> bool:
+        preprint_keywords = [
+            "arxiv", "biorxiv", "medrxiv", "ssrn", "preprint",
+            "openreview", "techreport", "technical report", "working paper",
+        ]
+        if entry.entry_type.lower() in ["techreport", "unpublished", "misc"]:
+            text = " ".join([
+                entry.journal.lower(), entry.booktitle.lower(),
+                entry.publisher.lower(), entry.entry_type.lower(),
+            ])
+            if any(k in text for k in preprint_keywords):
+                return True
+        if entry.has_arxiv:
+            return True
+        venue = " ".join([entry.journal.lower(), entry.booktitle.lower(), entry.publisher.lower()])
+        return any(k in venue for k in preprint_keywords)
+
+    def get_summary_stats(self) -> dict:
+        """Return bibliography issue counts only (no LaTeX)."""
+        total = len(self.entries)
+        title_mismatches = author_mismatches = year_mismatches = unable_to_verify = 0
+        for e in self.entries:
+            if not e.comparison:
+                continue
+            if e.comparison.has_issues:
+                for issue in e.comparison.issues:
+                    if "Title mismatch" in issue:
+                        title_mismatches += 1
+                    elif "Author mismatch" in issue:
+                        author_mismatches += 1
+                    elif "Year mismatch" in issue:
+                        year_mismatches += 1
+                    elif "Unable to find" in issue:
+                        unable_to_verify += 1
+
+        stats = {}
+        if title_mismatches > 0:
+            stats["Title Mismatches"] = title_mismatches
+        if author_mismatches > 0:
+            stats["Author Mismatches"] = author_mismatches
+        if year_mismatches > 0:
+            stats["Year Mismatches"] = year_mismatches
+        if unable_to_verify > 0:
+            stats["Unable to Verify"] = unable_to_verify
+        if self.duplicate_groups:
+            stats["Duplicate Groups"] = len(self.duplicate_groups)
+        return stats
+
+    def _generate_issues_section(self) -> List[str]:
+        lines = ["## ⚠️ Critical Issues Detected", ""]
+        has_any = False
+
+        if self.duplicate_groups:
+            has_any = True
+            lines.append("### 🔄 Duplicate Entries")
+            for i, group in enumerate(self.duplicate_groups, 1):
+                lines.append(f"#### Group {i} (Similarity: {group.similarity_score:.0%})")
+                lines.append(f"**Reason:** {group.reason}")
+                lines.append("")
+                lines.append("| Key | Title | Year |")
+                lines.append("|-----|-------|------|")
+                for entry in group.entries:
+                    lines.append(f"| `{entry.key}` | {entry.title} | {entry.year} |")
+                lines.append("")
+
+        issue_entries = [e for e in self.entries if self._has_issues(e)]
+        if issue_entries:
+            has_any = True
+            lines.append("### ⚠️ Metadata Issues")
+            for report in issue_entries:
+                lines.extend(self._format_entry_detail(report, is_verified=False))
+
+        if not has_any:
+            lines.append("🎉 **No critical issues found!**")
+        return lines
+
+    def _generate_verified_section(self) -> List[str]:
+        lines = ["## ✅ Verified Entries", ""]
+        verified = [e for e in self.entries if self._is_verified(e)]
+        if not verified:
+            lines.append("_No verified entries found._")
+            return lines
+        lines.append(f"Found **{len(verified)}** entries with correct metadata.")
+        lines.append("")
+        lines.append("<details>")
+        lines.append("<summary>Click to view verified entries</summary>")
+        lines.append("")
+        for report in verified:
+            lines.extend(self._format_entry_detail(report, minimal=self.minimal_verified, is_verified=True))
+        lines.append("</details>")
+        return lines
+
+    def _format_entry_detail(self, report: EntryReport, minimal: bool = False, is_verified: bool = False) -> List[str]:
+        entry = report.entry
+        comp = report.comparison
+        lines = []
+        icon = "✅" if is_verified else "⚠️"
+        lines.append(f"#### {icon} `{entry.key}`")
+        lines.append(f"**Title:** {entry.title}")
+        lines.append("")
+        if comp:
+            status_icon = "✅" if comp.is_match else "❌"
+            lines.append(f"- **Metadata Status:** {status_icon} {comp.source.upper()} (Confidence: {comp.confidence:.1%})")
+            if comp.has_issues and not minimal:
+                lines.append("  - **Discrepancies:**")
+                for issue in comp.issues:
+                    if "Mismatch" in issue or "mismatch" in issue:
+                        lines.append(f"    - 🔴 {issue}")
+                        if "Title" in issue:
+                            lines.append(f"      - **Bib:** `{comp.bib_title}`")
+                            lines.append(f"      - **Fetched:** `{comp.fetched_title}`")
+                        elif "Author" in issue:
+                            lines.append(f"      - **Bib:** `{', '.join(comp.bib_authors)}`")
+                            lines.append(f"      - **Fetched:** `{', '.join(comp.fetched_authors)}`")
+                    else:
+                        lines.append(f"    - 🔸 {issue}")
+        lines.append("")
+        lines.append("---")
+        lines.append("")
+        return lines
+
+    def save_bibliography_report(self, filepath: str):
+        """Generate and save bibliography-only report."""
+        total = len(self.entries)
+        verified = sum(1 for e in self.entries if self._is_verified(e))
+        issues = sum(1 for e in self.entries if self._has_issues(e))
+        dup_str = str(len(self.duplicate_groups)) if self.duplicate_groups else "N/A"
+
+        preprint_str = "N/A"
+        preprint_warning = []
+        if self.check_preprint_ratio and self.entries:
+            preprint_count = sum(1 for e in self.entries if self._is_preprint(e.entry))
+            preprint_ratio = preprint_count / len(self.entries)
+            preprint_str = f"{preprint_count} ({preprint_ratio:.1%})"
+            if preprint_ratio > self.preprint_warning_threshold:
+                preprint_warning = [
+                    "",
+                    f"> ⚠️ **High Preprint Ratio:** {preprint_ratio:.1%} of entries are preprints.",
+                ]
+
+        bib_names = ", ".join([f"`{Path(f).name}`" for f in self.bib_files]) if self.bib_files else "N/A"
+        lines = [
+            "# Bibliography Validation Report",
+            "",
+            f"**Generated:** {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}",
+            "",
+            "| File Type | Filename |",
+            "|-----------|----------|",
+            f"| **Bib File(s)** | {bib_names} |",
+            "",
+            "> **⚠️ Disclaimer:** This report is generated by an automated tool. Please verify reported issues manually.",
+            "",
+            "## 📊 Summary",
+            "",
+            "| Metric | Count |",
+            "|--------|-------|",
+            f"| **Total Entries** | {total} |",
+            f"| ✅ **Verified (Clean)** | {verified} |",
+            f"| ⚠️ **With Issues** | {issues} |",
+            f"| 🔄 **Duplicate Groups** | {dup_str} |",
+            f"| 📄 **Preprints** | {preprint_str} |",
+            "",
+        ]
+        if preprint_warning:
+            lines.extend(preprint_warning)
+            lines.append("")
+        lines.extend(self._generate_issues_section())
+        lines.append("")
+        lines.extend(self._generate_verified_section())
+        lines.append("")
+        lines.append("---")
+        lines.append(f"Report generated by **CiteScan** on {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
+
+        with open(filepath, "w", encoding="utf-8") as f:
+            f.write("\n".join(lines))

src/utils/__init__.py

@@ -0,0 +1,5 @@
+"""Utilities package"""
+from .normalizer import TextNormalizer
+from .progress import ProgressDisplay
+
+__all__ = ['TextNormalizer', 'ProgressDisplay']

src/utils/fetched_bibtex.py

@@ -0,0 +1,118 @@
+"""
+Build BibTeX string from fetched metadata (ground truth).
+Single central builder: dispatches by source and formats one consistent style.
+"""
+from typing import Any, List, Optional
+import re
+
+
+def _escape(s: str) -> str:
+    """Escape BibTeX special chars: \\ { }"""
+    if not s:
+        return ""
+    return s.replace("\\", "\\\\").replace("{", "\\{").replace("}", "\\}")
+
+
+def _author_list_to_bibtex(authors: Any) -> str:
+    """Convert authors (list or str) to BibTeX author field (Name1 and Name2)."""
+    if isinstance(authors, str):
+        return _escape(authors.strip())
+    if isinstance(authors, list):
+        return " and ".join(_escape(str(a).strip()) for a in authors if a)
+    return ""
+
+
+def _first_author_last_name(authors: Any) -> str:
+    """Get last name (last word) of first author for key generation."""
+    if isinstance(authors, str):
+        parts = authors.strip().split()
+        return parts[-1] if parts else "unknown"
+    if isinstance(authors, list) and authors:
+        first = str(authors[0]).strip()
+        parts = first.split()
+        return parts[-1] if parts else "unknown"
+    return "unknown"
+
+
+def _bibtex_key(authors: Any, year: str) -> str:
+    """Generate a safe BibTeX key: LastNameYear."""
+    last = _first_author_last_name(authors)
+    # Alphanumeric only for key
+    last = re.sub(r"[^a-zA-Z0-9]", "", last)
+    y = (year or "nodate").strip()
+    y = re.sub(r"[^0-9]", "", y)[:4] if y else "nodate"
+    return f"{last}{y}" if last else f"ref{y}"
+
+
+def build_fetched_bibtex(source: str, result: Any) -> str:
+    """
+    Build a BibTeX entry string from fetched metadata.
+    source: 'arxiv' | 'crossref' | 'scholar' | 'semantic_scholar' | 'openalex' | 'dblp'
+    result: the fetcher result object (ArxivMetadata, CrossRefResult, etc.)
+    """
+    title = ""
+    authors: Any = []
+    year = ""
+    doi = ""
+    url = ""
+    venue = ""
+    entry_type = "misc"
+
+    if source == "arxiv":
+        title = getattr(result, "title", "") or ""
+        authors = getattr(result, "authors", []) or []
+        year = getattr(result, "year", "") or ""  # property
+        doi = getattr(result, "doi", "") or ""
+        url = getattr(result, "abs_url", "") or ""
+        venue = getattr(result, "journal_ref", "") or ""
+        entry_type = "article" if venue else "misc"
+    elif source == "crossref":
+        title = getattr(result, "title", "") or ""
+        authors = getattr(result, "authors", []) or []
+        year = getattr(result, "year", "") or ""
+        doi = getattr(result, "doi", "") or ""
+        url = getattr(result, "url", "") or ""
+        venue = getattr(result, "container_title", "") or ""
+        entry_type = "article"
+    elif source == "scholar":
+        title = getattr(result, "title", "") or ""
+        authors = getattr(result, "authors", "") or ""
+        year = getattr(result, "year", "") or ""
+        url = getattr(result, "url", "") or ""
+        entry_type = "misc"
+    elif source == "semantic_scholar":
+        title = getattr(result, "title", "") or ""
+        authors = getattr(result, "authors", []) or []
+        year = getattr(result, "year", "") or ""
+        url = getattr(result, "url", "") or ""
+        entry_type = "misc"
+    elif source == "openalex":
+        title = getattr(result, "title", "") or ""
+        authors = getattr(result, "authors", []) or []
+        year = getattr(result, "year", "") or ""
+        doi = getattr(result, "doi", "") or ""
+        url = getattr(result, "url", "") or ""
+        entry_type = "misc"
+    elif source == "dblp":
+        title = getattr(result, "title", "") or ""
+        authors = getattr(result, "authors", []) or []
+        year = getattr(result, "year", "") or ""
+        doi = getattr(result, "doi", "") or ""
+        url = getattr(result, "url", "") or ""
+        entry_type = "misc"
+    else:
+        return ""
+
+    key = _bibtex_key(authors, year)
+    author_str = _author_list_to_bibtex(authors)
+
+    lines = [f"  author = {{{author_str}}}", f"  title = {{{_escape(title)}}}", f"  year = {{{year or '?'}}}"]
+    if venue:
+        lines.append(f"  journal = {{{_escape(venue)}}}")
+    if doi:
+        lines.append(f"  doi = {{{_escape(doi)}}}")
+    if url:
+        lines.append(f"  url = {{{_escape(url)}}}")
+    lines.append(f"  note = {{Fetched from {source}}}")
+
+    return f"@{entry_type}{{{key},\n" + ",\n".join(lines) + "\n}"

src/utils/normalizer.py

@@ -0,0 +1,233 @@
+"""
+Text normalization utilities for comparing bibliography entries.
+"""
+import re
+import unicodedata
+from unidecode import unidecode
+
+
+class TextNormalizer:
+    """Utility class for normalizing text for comparison."""
+    
+    # LaTeX command patterns
+    LATEX_COMMANDS = [
+        (r'\\textbf\{([^}]*)\}', r'\1'),
+        (r'\\textit\{([^}]*)\}', r'\1'),
+        (r'\\emph\{([^}]*)\}', r'\1'),
+        (r'\\textrm\{([^}]*)\}', r'\1'),
+        (r'\\texttt\{([^}]*)\}', r'\1'),
+        (r'\\textsf\{([^}]*)\}', r'\1'),
+        (r'\\textsc\{([^}]*)\}', r'\1'),
+        (r'\\text\{([^}]*)\}', r'\1'),
+        (r'\\mathrm\{([^}]*)\}', r'\1'),
+        (r'\\mathbf\{([^}]*)\}', r'\1'),
+        (r'\\mathit\{([^}]*)\}', r'\1'),
+        (r'\\url\{([^}]*)\}', r'\1'),
+        (r'\\href\{[^}]*\}\{([^}]*)\}', r'\1'),
+    ]
+    
+    # LaTeX special character mappings
+    LATEX_CHARS = {
+        r'\&': '&',
+        r'\%': '%',
+        r'\$': '$',
+        r'\#': '#',
+        r'\_': '_',
+        r'\{': '{',
+        r'\}': '}',
+        r'\~': '~',
+        r'\^': '^',
+        r'``': '"',
+        r"''": '"',
+        r'`': "'",
+        r"'": "'",
+        r'--': '–',
+        r'---': '—',
+    }
+    
+    # LaTeX accent commands
+    LATEX_ACCENTS = [
+        (r"\\'([aeiouAEIOU])", r'\1'),  # acute
+        (r'\\`([aeiouAEIOU])', r'\1'),   # grave
+        (r'\\^([aeiouAEIOU])', r'\1'),   # circumflex
+        (r'\\"([aeiouAEIOU])', r'\1'),   # umlaut
+        (r'\\~([nNaAoO])', r'\1'),       # tilde
+        (r'\\c\{([cC])\}', r'\1'),       # cedilla
+        (r"\\'{([aeiouAEIOU])}", r'\1'),
+        (r'\\`{([aeiouAEIOU])}', r'\1'),
+        (r'\\^{([aeiouAEIOU])}', r'\1'),
+        (r'\\"{([aeiouAEIOU])}', r'\1'),
+        (r'\\~{([nNaAoO])}', r'\1'),
+    ]
+    
+    @classmethod
+    def normalize_latex(cls, text: str) -> str:
+        """Remove LaTeX formatting commands."""
+        if not text:
+            return ""
+        
+        result = text
+        
+        # Remove LaTeX commands
+        for pattern, replacement in cls.LATEX_COMMANDS:
+            result = re.sub(pattern, replacement, result)
+        
+        # Handle LaTeX accents
+        for pattern, replacement in cls.LATEX_ACCENTS:
+            result = re.sub(pattern, replacement, result)
+        
+        # Replace LaTeX special characters
+        for latex_char, normal_char in cls.LATEX_CHARS.items():
+            result = result.replace(latex_char, normal_char)
+        
+        # Remove remaining braces
+        result = re.sub(r'[{}]', '', result)
+        
+        return result
+    
+    @classmethod
+    def normalize_unicode(cls, text: str) -> str:
+        """Normalize Unicode characters to ASCII."""
+        if not text:
+            return ""
+        
+        # Normalize unicode
+        text = unicodedata.normalize('NFKD', text)
+        # Convert to ASCII
+        text = unidecode(text)
+        return text
+    
+    @classmethod
+    def normalize_whitespace(cls, text: str) -> str:
+        """Normalize whitespace."""
+        if not text:
+            return ""
+        
+        # Replace multiple whitespace with single space
+        text = re.sub(r'\s+', ' ', text)
+        # Strip leading/trailing whitespace
+        text = text.strip()
+        return text
+    
+    @classmethod
+    def remove_punctuation(cls, text: str) -> str:
+        """Remove punctuation for comparison."""
+        if not text:
+            return ""
+        
+        # Keep alphanumeric and spaces only
+        return re.sub(r'[^\w\s]', '', text)
+    
+    @classmethod
+    def normalize_for_comparison(cls, text: str) -> str:
+        """
+        Full normalization pipeline for text comparison.
+        
+        Steps:
+        1. Remove LaTeX formatting
+        2. Normalize Unicode to ASCII
+        3. Convert to lowercase
+        4. Normalize whitespace
+        5. Remove punctuation
+        """
+        if not text:
+            return ""
+        
+        text = cls.normalize_latex(text)
+        text = cls.normalize_unicode(text)
+        text = text.lower()
+        text = cls.normalize_whitespace(text)
+        text = cls.remove_punctuation(text)
+        return text
+    
+    @classmethod
+    def normalize_author_name(cls, name: str) -> str:
+        """
+        Normalize author name format.
+        Handles: "Last, First" and "First Last" formats.
+        Returns: normalized "first last" format.
+        """
+        if not name:
+            return ""
+        
+        name = cls.normalize_latex(name)
+        name = cls.normalize_unicode(name)
+        name = cls.normalize_whitespace(name)
+        
+        # Handle "Last, First" format
+        if ',' in name:
+            parts = name.split(',', 1)
+            if len(parts) == 2:
+                name = f"{parts[1].strip()} {parts[0].strip()}"
+        
+        name = name.lower()
+        name = cls.remove_punctuation(name)
+        return name
+    
+    @classmethod
+    def normalize_author_list(cls, authors: str) -> list[str]:
+        """
+        Parse and normalize a list of authors.
+        Handles "and" as separator and "Last, First" format.
+        """
+        if not authors:
+            return []
+        
+        # Split by " and "
+        author_list = re.split(r'\s+and\s+', authors, flags=re.IGNORECASE)
+        
+        # Normalize each author
+        normalized = []
+        for author in author_list:
+            normalized_name = cls.normalize_author_name(author.strip())
+            if normalized_name:
+                normalized.append(normalized_name)
+        
+        return normalized
+    
+    @classmethod
+    def similarity_ratio(cls, text1: str, text2: str) -> float:
+        """Calculate word-based Jaccard similarity ratio between two strings."""
+        if not text1 or not text2:
+            return 0.0
+        
+        words1 = set(text1.split())
+        words2 = set(text2.split())
+        
+        if not words1 and not words2:
+            return 1.0
+        if not words1 or not words2:
+            return 0.0
+        
+        intersection = words1 & words2
+        union = words1 | words2
+        
+        return len(intersection) / len(union)
+    
+    @classmethod
+    def levenshtein_similarity(cls, s1: str, s2: str) -> float:
+        """Calculate normalized Levenshtein similarity."""
+        if not s1 and not s2:
+            return 1.0
+        if not s1 or not s2:
+            return 0.0
+        
+        # Simple Levenshtein implementation
+        m, n = len(s1), len(s2)
+        dp = [[0] * (n + 1) for _ in range(m + 1)]
+        
+        for i in range(m + 1):
+            dp[i][0] = i
+        for j in range(n + 1):
+            dp[0][j] = j
+        
+        for i in range(1, m + 1):
+            for j in range(1, n + 1):
+                if s1[i-1] == s2[j-1]:
+                    dp[i][j] = dp[i-1][j-1]
+                else:
+                    dp[i][j] = min(dp[i-1][j], dp[i][j-1], dp[i-1][j-1]) + 1
+        
+        max_len = max(m, n)
+        distance = dp[m][n]
+        return 1.0 - (distance / max_len)

src/utils/progress.py

@@ -0,0 +1,169 @@
+from rich.console import Console
+from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn, TaskProgressColumn, TimeElapsedColumn
+from rich.panel import Panel
+from rich.table import Table
+from rich.live import Live
+from rich.layout import Layout
+from rich.text import Text
+from contextlib import contextmanager
+from dataclasses import dataclass, field
+from typing import Optional
+
+
+@dataclass
+class ProgressStats:
+    """Statistics for progress display."""
+    total_entries: int = 0
+    processed: int = 0
+    success: int = 0
+    warnings: int = 0
+    errors: int = 0
+    current_entry: str = ""
+    current_task: str = ""
+
+
+class ProgressDisplay:
+    """Rich terminal progress display."""
+    
+    def __init__(self):
+        self.console = Console()
+        self.stats = ProgressStats()
+        self._progress: Optional[Progress] = None
+        self._live: Optional[Live] = None
+        self._main_task = None
+    
+    def _create_stats_table(self) -> Table:
+        """Create a statistics table."""
+        table = Table(show_header=False, box=None, padding=(0, 2))
+        table.add_column("Label", style="dim")
+        table.add_column("Value", style="bold")
+        
+        table.add_row("📚 Total Entries", str(self.stats.total_entries))
+        table.add_row("✅ Success", f"[green]{self.stats.success}[/green]")
+        table.add_row("⚠️ Warnings", f"[yellow]{self.stats.warnings}[/yellow]")
+        table.add_row("❌ Errors", f"[red]{self.stats.errors}[/red]")
+        
+        return table
+    
+    def _create_display(self) -> Panel:
+        """Create the main display panel."""
+        layout = Layout()
+        
+        # Status text
+        status_text = Text()
+        status_text.append("Current: ", style="dim")
+        status_text.append(self.stats.current_entry or "N/A", style="cyan bold")
+        status_text.append("\n")
+        status_text.append("Task: ", style="dim")
+        status_text.append(self.stats.current_task or "Initializing...", style="white")
+        
+        return Panel(
+            status_text,
+            title="[bold blue]📖 Bibliography Checker[/bold blue]",
+            border_style="blue"
+        )
+    
+    @contextmanager
+    def progress_context(self, total: int, description: str = "Processing"):
+        """Context manager for progress display."""
+        self.stats.total_entries = total
+        
+        with Progress(
+            SpinnerColumn(),
+            TextColumn("[progress.description]{task.description}"),
+            BarColumn(bar_width=40),
+            TaskProgressColumn(),
+            TimeElapsedColumn(),
+            console=self.console,
+            transient=False
+        ) as progress:
+            self._progress = progress
+            self._main_task = progress.add_task(description, total=total)
+            try:
+                yield self
+            finally:
+                self._progress = None
+                self._main_task = None
+    
+    def update(self, entry_key: str = "", task: str = "", advance: int = 0):
+        """Update progress display."""
+        if entry_key:
+            self.stats.current_entry = entry_key
+        if task:
+            self.stats.current_task = task
+        
+        if self._progress and self._main_task is not None:
+            desc = f"[cyan]{entry_key}[/cyan] - {task}" if entry_key else task
+            self._progress.update(self._main_task, description=desc, advance=advance)
+            self.stats.processed += advance
+    
+    def mark_success(self):
+        """Mark current entry as successful."""
+        self.stats.success += 1
+    
+    def mark_warning(self):
+        """Mark current entry with warning."""
+        self.stats.warnings += 1
+    
+    def mark_error(self):
+        """Mark current entry as error."""
+        self.stats.errors += 1
+    
+    def print_header(self, title: str):
+        """Print a section header."""
+        self.console.print()
+        self.console.print(Panel(
+            f"[bold]{title}[/bold]",
+            border_style="blue",
+            expand=False
+        ))
+    
+    def print_status(self, message: str, style: str = ""):
+        """Print a status message."""
+        self.console.print(f"  {message}", style=style)
+    
+    def print_success(self, message: str):
+        """Print a success message."""
+        self.console.print(f"  [green]✓[/green] {message}")
+    
+    def print_warning(self, message: str):
+        """Print a warning message."""
+        self.console.print(f"  [yellow]⚠[/yellow] {message}")
+    
+    def print_error(self, message: str):
+        """Print an error message."""
+        self.console.print(f"  [red]✗[/red] {message}")
+    
+    def print_info(self, message: str):
+        """Print an info message."""
+        self.console.print(f"  [blue]ℹ[/blue] {message}")
+    
+    def print_detailed_summary(self, bib_stats: dict, output_dir: str):
+        """Print bibliography issue summary (bib-only)."""
+        self.console.print()
+        bib_table = Table(show_header=True, header_style="bold cyan", box=None, padding=(0, 1))
+        bib_table.add_column("📚 Bibliography Issues", style="white")
+        bib_table.add_column("Count", justify="right", style="bold red")
+        if bib_stats:
+            for label, value in bib_stats.items():
+                bib_table.add_row(label, str(value))
+        else:
+            bib_table.add_row("[green]No issues found[/green]", "0")
+        summary_panel = Panel(
+            bib_table,
+            title="[bold]📚 Bibliography Summary[/bold]",
+            border_style="blue",
+            padding=(1, 2),
+        )
+        self.console.print(summary_panel)
+        guide_table = Table(show_header=True, header_style="bold green", box=None, padding=(0, 2))
+        guide_table.add_column("File Name", style="cyan")
+        guide_table.add_column("Description", style="dim")
+        guide_table.add_row("bibliography_report.md", "Metadata verification and issues for each bib entry")
+        self.console.print(Panel(
+            guide_table,
+            title="[bold green]Output Directory[/bold green]",
+            subtitle=f"Location: [blue underline]{output_dir}[/blue underline]",
+            border_style="green",
+            padding=(1, 1),
+        ))