Leonnel1220 commited on
Commit
4293320
Β·
verified Β·
1 Parent(s): 7bfa9f0

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +0 -82
README.md CHANGED
@@ -27,88 +27,6 @@ The public subset contains 120 tasks across Global (English) and Chinese markets
27
  - πŸ’» Code: <https://github.com/ICBCBench/ICBCBench>
28
  - πŸ“Š Dataset: <https://huggingface.co/datasets/DeepFin-Intelligence/ICBCBench>
29
 
30
- ## Repository Layout
31
-
32
- ```
33
- β”œβ”€β”€ app.py # Gradio entry point
34
- β”œβ”€β”€ create_leaderboard.py # Main Gradio Blocks UI
35
- β”œβ”€β”€ tabs/
36
- β”‚ β”œβ”€β”€ leaderboard_tab.py # Leaderboard rendering
37
- β”‚ β”œβ”€β”€ data_viewer_tab.py # Single-model report viewer
38
- β”‚ β”œβ”€β”€ data_viewer_side_by_side_tab.py # Side-by-side comparison
39
- β”‚ └── shared_data.py # Lazy data loading
40
- β”œβ”€β”€ utils/
41
- β”‚ β”œβ”€β”€ rank_leaderboard.py # Aggregate raw results -> leaderboard.csv
42
- β”‚ └── merge_raw_data.py # Merge raw data + scores -> data_viewer.jsonl
43
- └── data/
44
- β”œβ”€β”€ leaderboard.csv # Aggregated leaderboard (generated)
45
- β”œβ”€β”€ data_viewer.jsonl # Per-sample viewer data (generated)
46
- β”œβ”€β”€ raw_data/ # Per-model prompts + articles
47
- └── raw_results/ # Per-model evaluation scores
48
- ```
49
-
50
- ## Data Format
51
-
52
- ### `data/leaderboard.csv`
53
-
54
- | Column | Description |
55
- |--------|-------------|
56
- | `model` | Model / system name |
57
- | `overall` | Weighted overall score across EN & ZH, Objective & Subjective |
58
- | `objective_en` | Objective score on Global (English) tasks |
59
- | `objective_zh` | Objective score on Chinese tasks |
60
- | `objective_avg` | Average objective score across both language tracks |
61
- | `subjective_en` | Subjective report score on Global (English) tasks |
62
- | `subjective_zh` | Subjective report score on Chinese tasks |
63
- | `subjective_avg` | Average subjective score across both language tracks |
64
- | `citation_score` | Citation consistency score (0-100) |
65
- | `source_quality` | Source authority & timeliness score (0-100) |
66
- | `rmsce` | Root Mean Square Calibration Error for objective confidence (lower is better) |
67
-
68
- ### `data/raw_results/<model>/results.jsonl`
69
-
70
- Each line should be a JSON object:
71
-
72
- ```json
73
- {
74
- "id": "1",
75
- "language": "en",
76
- "track": "objective",
77
- "score": 0.85,
78
- "correct": true,
79
- "confidence": 0.90
80
- }
81
- ```
82
-
83
- or for subjective tasks:
84
-
85
- ```json
86
- {
87
- "id": "41",
88
- "language": "zh",
89
- "track": "subjective",
90
- "score": 0.72,
91
- "expert_score": 0.74,
92
- "citation_score": 0.68,
93
- "source_quality_score": 0.70
94
- }
95
- ```
96
-
97
- Scores can be either 0-1 ratios or 0-100 percentages.
98
-
99
- ## Local Development
100
-
101
- ```bash
102
- pip install -r requirements.txt
103
- python utils/rank_leaderboard.py # generate data/leaderboard.csv
104
- python utils/merge_raw_data.py # generate data/data_viewer.jsonl
105
- python app.py
106
- ```
107
-
108
- > **Note**: The `data/raw_results/` directory in this repository currently contains legacy DeepResearch-Bench-format `race_result.txt` files. Before running `rank_leaderboard.py` for ICBCBench, replace these with ICBCBench `results.jsonl` files as described above. The checked-in `data/leaderboard.csv` is a sample ICBCBench leaderboard and will be overwritten if you rerun the script without updating `raw_results/`.
109
-
110
- Then open <http://localhost:7860>.
111
-
112
  ## Citation
113
 
114
  ```bibtex
 
27
  - πŸ’» Code: <https://github.com/ICBCBench/ICBCBench>
28
  - πŸ“Š Dataset: <https://huggingface.co/datasets/DeepFin-Intelligence/ICBCBench>
29
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  ## Citation
31
 
32
  ```bibtex