Update model card for Think-on-Graph 3.0 framework (RAG-Factory)

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +68 -36
README.md CHANGED
@@ -1,17 +1,21 @@
1
  ---
2
- license: apache-2.0
 
 
 
3
  language:
4
  - en
5
  - zh
6
- base_model:
7
- - Qwen/Qwen2-7B-Instruct
8
- pipeline_tag: text-generation
9
  library_name: transformers
 
 
10
  tags:
11
  - finance
12
  - text-generation-inference
13
- datasets:
14
- - IDEA-FinAI/Golden-Touchstone
 
 
15
  ---
16
 
17
  <!-- markdownlint-disable first-line-h1 -->
@@ -19,18 +23,22 @@ datasets:
19
  <!-- markdownlint-disable no-duplicate-header -->
20
 
21
  <div align="center">
22
- <img src="https://github.com/IDEA-FinAI/Golden-Touchstone/blob/main/assets/Touchstone-GPT-logo.png?raw=true" width="15%" alt="Golden-Touchstone" />
23
- <h1 style="display: inline-block; vertical-align: middle; margin-left: 10px; font-size: 2em; font-weight: bold;">Golden-Touchstone Benchmark</h1>
 
 
 
 
24
  </div>
25
 
26
  <div align="center" style="line-height: 1;">
27
- <a href="https://arxiv.org/abs/2411.06272" target="_blank" style="margin: 2px;">
28
- <img alt="arXiv" src="https://img.shields.io/badge/Arxiv-2411.06272-b31b1b.svg?logo=arXiv" style="display: inline-block; vertical-align: middle;"/>
29
  </a>
30
- <a href="https://github.com/IDEA-FinAI/Golden-Touchstone" target="_blank" style="margin: 2px;">
31
- <img alt="github" src="https://img.shields.io/github/stars/IDEA-FinAI/Golden-Touchstone.svg?style=social" style="display: inline-block; vertical-align: middle;"/>
32
  </a>
33
- <a href="https://huggingface.co/IDEA-FinAI/TouchstoneGPT-7B-Instruct" target="_blank" style="margin: 2px;">
34
  <img alt="datasets" src="https://img.shields.io/badge/🤗-Datasets-yellow.svg" style="display: inline-block; vertical-align: middle;"/>
35
  </a>
36
  <a href="https://huggingface.co/IDEA-FinAI/TouchstoneGPT-7B-Instruct" target="_blank" style="margin: 2px;">
@@ -38,33 +46,56 @@ datasets:
38
  </a>
39
  </div>
40
 
41
- # Golden-Touchstone
42
 
 
43
 
44
- Golden Touchstone is a simple, effective, and systematic benchmark for bilingual (Chinese-English) financial large language models, driving the research and implementation of financial large language models, akin to a touchstone. We also have trained and open-sourced Touchstone-GPT as a baseline for subsequent community research.
45
 
 
46
 
47
- ## Introduction
48
 
49
- The paper shows the evaluation of the diversity, systematicness and LLM adaptability of each open source benchmark.
 
 
50
 
51
- ![benchmark_info](https://github.com/IDEA-FinAI/Golden-Touchstone/blob/main/assets/benchmark_info.png?raw=true)
 
 
52
 
53
- By collecting and selecting representative task datasets, we built our own Chinese-English bilingual Touchstone Benchmark, which includes 22 datasets
 
 
 
 
54
 
55
- ![golden_touchstone_info](https://github.com/IDEA-FinAI/Golden-Touchstone/blob/main/assets/golden_touchstone_info.png?raw=true)
56
 
57
- We extensively evaluated GPT-4o, llama3, qwen2, fingpt and our own trained Touchstone-GPT, analyzed the advantages and disadvantages of these models, and provided direction for subsequent research on financial large language models
58
 
59
- ![evaluation](https://github.com/IDEA-FinAI/Golden-Touchstone/blob/main/assets/evaluation.png?raw=true)
 
 
60
 
61
- ## Evaluation of Touchstone Benchmark
62
 
63
- Please See our github repo [Golden-Touchstone](https://github.com/IDEA-FinAI/Golden-Touchstone)
64
 
65
- ## Usage of Touchstone-GPT
 
 
 
66
 
67
- Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
 
 
 
 
 
 
 
 
68
 
69
  ```python
70
  from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -77,7 +108,8 @@ model = AutoModelForCausalLM.from_pretrained(
77
  )
78
  tokenizer = AutoTokenizer.from_pretrained("IDEA-FinAI/TouchstoneGPT-7B-Instruct")
79
 
80
- prompt = "What is the sentiment of the following financial post: Positive, Negative, or Neutral?\nsees #Apple at $150/share in a year (+36% from today) on growing services business."
 
81
  messages = [
82
  {"role": "system", "content": "You are a helpful assistant."},
83
  {"role": "user", "content": prompt}
@@ -100,17 +132,17 @@ generated_ids = [
100
  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
101
  ```
102
 
103
-
104
-
105
  ## Citation
106
- ```
107
- @misc{wu2024goldentouchstonecomprehensivebilingual,
108
- title={Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Models},
109
- author={Xiaojun Wu and Junxi Liu and Huanyi Su and Zhouchi Lin and Yiyan Qi and Chengjin Xu and Jiajun Su and Jiajie Zhong and Fuwei Wang and Saizhuo Wang and Fengrui Hua and Jia Li and Jian Guo},
110
- year={2024},
111
- eprint={2411.06272},
 
 
112
  archivePrefix={arXiv},
113
  primaryClass={cs.CL},
114
- url={https://arxiv.org/abs/2411.06272},
115
  }
116
  ```
 
1
  ---
2
+ base_model:
3
+ - Qwen/Qwen2-7B-Instruct
4
+ datasets:
5
+ - IDEA-FinAI/Golden-Touchstone
6
  language:
7
  - en
8
  - zh
 
 
 
9
  library_name: transformers
10
+ license: apache-2.0
11
+ pipeline_tag: text-generation
12
  tags:
13
  - finance
14
  - text-generation-inference
15
+ - retrieval-augmented-generation
16
+ - rag
17
+ - graph-neural-networks
18
+ - llm-reasoning
19
  ---
20
 
21
  <!-- markdownlint-disable first-line-h1 -->
 
23
  <!-- markdownlint-disable no-duplicate-header -->
24
 
25
  <div align="center">
26
+
27
+ <div style="margin: 20px 0;al">
28
+ <img src="https://github.com/DataArcTech/RAG-Factory/blob/main/assets/logo.png?raw=true" width="120" height="120" alt="RAG-Factory Logo" style="border-radius: 20px; box-shadow: 0 8px 32px rgba(0, 217, 255, 0.3);">
29
+ </div>
30
+
31
+ # ✨ TouchstoneGPT-7B-Instruct: A Model for Think-on-Graph 3.0 via RAG-Factory
32
  </div>
33
 
34
  <div align="center" style="line-height: 1;">
35
+ <a href="https://huggingface.co/papers/2509.21710" target="_blank" style="margin: 2px;">
36
+ <img alt="Paper" src="https://img.shields.io/badge/Paper-2509.21710-b31b1b.svg?logo=arXiv" style="display: inline-block; vertical-align: middle;"/>
37
  </a>
38
+ <a href="https://github.com/DataArcTech/RAG-Factory" target="_blank" style="margin: 2px;">
39
+ <img alt="github" src="https://img.shields.io/github/stars/DataArcTech/RAG-Factory.svg?style=social" style="display: inline-block; vertical-align: middle;"/>
40
  </a>
41
+ <a href="https://huggingface.co/datasets/IDEA-FinAI/Golden-Touchstone" target="_blank" style="margin: 2px;">
42
  <img alt="datasets" src="https://img.shields.io/badge/🤗-Datasets-yellow.svg" style="display: inline-block; vertical-align: middle;"/>
43
  </a>
44
  <a href="https://huggingface.co/IDEA-FinAI/TouchstoneGPT-7B-Instruct" target="_blank" style="margin: 2px;">
 
46
  </a>
47
  </div>
48
 
49
+ This Hugging Face repository hosts the `TouchstoneGPT-7B-Instruct` model, an instance of a Large Language Model (LLM) based on `Qwen/Qwen2-7B-Instruct`. This model is suitable for integration within the **Think-on-Graph 3.0 (ToG-3)** framework, a novel approach to Retrieval-Augmented Generation (RAG) that enhances LLM reasoning on heterogeneous graphs. The ToG-3 framework is implemented and further detailed in the [RAG-Factory GitHub repository](https://github.com/DataArcTech/RAG-Factory).
50
 
51
+ ## Paper Abstract: Think-on-Graph 3.0
52
 
53
+ Retrieval-Augmented Generation (RAG) and Graph-based RAG has become the important paradigm for enhancing Large Language Models (LLMs) with external knowledge. However, existing approaches face a fundamental trade-off. While graph-based methods are inherently dependent on high-quality graph structures, they face significant practical constraints: manually constructed knowledge graphs are prohibitively expensive to scale, while automatically extracted graphs from corpora are limited by the performance of the underlying LLM extractors, especially when using smaller, local-deployed models. This paper presents Think-on-Graph 3.0 (ToG-3), a novel framework that introduces Multi-Agent Context Evolution and Retrieval (MACER) mechanism to overcome these limitations. Our core innovation is the dynamic construction and refinement of a Chunk-Triplets-Community heterogeneous graph index, which pioneeringly incorporates a dual-evolution mechanism of Evolving Query and Evolving Sub-Graph for precise evidence retrieval. This approach addresses a critical limitation of prior Graph-based RAG methods, which typically construct a static graph index in a single pass without adapting to the actual query. A multi-agent system, comprising Constructor, Retriever, Reflector, and Responser agents, collaboratively engages in an iterative process of evidence retrieval, answer generation, sufficiency reflection, and, crucially, evolving query and subgraph. This dual-evolving multi-agent system allows ToG-3 to adaptively build a targeted graph index during reasoning, mitigating the inherent drawbacks of static, one-time graph construction and enabling deep, precise reasoning even with lightweight LLMs. Extensive experiments demonstrate that ToG-3 outperforms compared baselines on both deep and broad reasoning benchmarks, and ablation studies confirm the efficacy of the components of MACER framework.
54
 
55
+ ## ✨ Features of RAG-Factory (Think-on-Graph 3.0 Implementation)
56
 
57
+ The [RAG-Factory](https://github.com/DataArcTech/RAG-Factory) framework, which implements the concepts of Think-on-Graph 3.0, provides a factory for building advanced RAG pipelines, including:
58
 
59
+ - Standard RAG implementations
60
+ - GraphRAG architectures
61
+ - Multi-modal RAG systems
62
 
63
+ <div align="center">
64
+ <img src="https://github.com/DataArcTech/RAG-Factory/blob/main/assets/knowledge_base_screenshot.png?raw=true" alt="Example Knowledge Base Screenshot of RAG-Factory" width="800">
65
+ </div>
66
 
67
+ Key features include:
68
+ - Modular design for easy customization
69
+ - Support for various knowledge graph backends
70
+ - Integration with multiple LLM providers
71
+ - Configurable pipeline components
72
 
73
+ ## Installation (for RAG-Factory)
74
 
75
+ To set up the RAG-Factory environment, clone the repository and install dependencies:
76
 
77
+ ```bash
78
+ pip install -e .
79
+ ```
80
 
81
+ ## Usage (for RAG-Factory)
82
 
83
+ You can run predefined RAG pipelines using the `RAG-Factory` framework:
84
 
85
+ ```bash
86
+ bash run.sh naive_rag/graph_rag/mm_rag
87
+ ```
88
+ or
89
 
90
+ ```bash
91
+ python main.py --config examples/graphrag/config.yaml
92
+ ```
93
+
94
+ For more examples and detailed configurations, please refer to the `examples/` directory in the [RAG-Factory GitHub repository](https://github.com/DataArcTech/RAG-Factory).
95
+
96
+ ## Usage of TouchstoneGPT-7B-Instruct
97
+
98
+ This `TouchstoneGPT-7B-Instruct` model is a `Qwen2-7B-Instruct`-based LLM that can be used for text generation tasks, either standalone or as a component within RAG frameworks like Think-on-Graph 3.0. Below is a code snippet using the `transformers` library to load the tokenizer and model and generate content.
99
 
100
  ```python
101
  from transformers import AutoModelForCausalLM, AutoTokenizer
 
108
  )
109
  tokenizer = AutoTokenizer.from_pretrained("IDEA-FinAI/TouchstoneGPT-7B-Instruct")
110
 
111
+ prompt = "What is the sentiment of the following financial post: Positive, Negative, or Neutral?
112
+ sees #Apple at $150/share in a year (+36% from today) on growing services business."
113
  messages = [
114
  {"role": "system", "content": "You are a helpful assistant."},
115
  {"role": "user", "content": prompt}
 
132
  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
133
  ```
134
 
 
 
135
  ## Citation
136
+ If you find our work on Think-on-Graph 3.0 useful for your research and applications, please consider citing the paper:
137
+
138
+ ```bibtex
139
+ @misc{wu2025ToG-3,
140
+ title={Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrieval},
141
+ author={Xiaojun Wu, Cehao Yang, Xueyuan Lin, Chengjin Xu, Xuhui Jiang, Yuanliang Sun, Hui Xiong, Jia Li, Jian Guo},
142
+ year={2025},
143
+ eprint={2509.21710},
144
  archivePrefix={arXiv},
145
  primaryClass={cs.CL},
146
+ url={https://arxiv.org/abs/2509.21710},
147
  }
148
  ```