SPARK2 / README.md

Add files using upload-large-folder tool

d80f4d0 verified 3 months ago

4.9 kB

---
language:
- ko
base_model:
- google/gemma-3-27b-it
- google/gemma-3-27b-pt
pipeline_tag: text-generation
---
## Usage Guide
개인은 자유롭게 사용할 수 있습니다.  
기업 및 기관은 비상업적 목적으로 이용해 주시기 바랍니다.  
또한, 추후 협업 및 네트워크 구축을 위해 기관 정보와 AI 모델 사용 담당자 정보를 메일로 보내주시면 연락드리겠습니다.  
## CONTACT : kistep_ax@kistep.re.kr
Individuals are free to use this without restrictions.  
For companies and institutions, please use it for non-commercial purposes.  
Additionally, to facilitate future collaboration and network building, please send us an email with your institution's information and the contact details of the person responsible for using the AI model. We will get in touch with you.
### 1. Description
**SPARK2** is a large language model developed by the Korea Institute of S&T Evaluation and Planning (KISTEP). As the successor to SPARK-RAG and SPARK-Report, SPARK2 unifies KISTEP's task-specific models into a single model. It is optimized for RAG (Retrieval-Augmented Generation) tasks and incorporates Chain of Thought (CoT) reasoning to enhance its response accuracy and performance.
### 2. Key Features
- **Enhanced Reliability through RAG:** Provides highly reliable responses by leveraging the organization's internal databases through Retrieval-Augmented Generation (RAG).
- **Transparent Reasoning:** Trained to demonstrate its reasoning process through Chain of Thought (CoT) inside `<think>` tags, clearly showing the information sources and logic behind each response with `<source>` citations.
- **Structured Output:** Responses in well-formatted markdown, including tables, text, and summaries for improved readability and clarity.
- **Base Model:** Built on Gemma-3-27b as the foundation model. Supervised Fine-Tuning (SFT) was performed on `google/gemma-3-27b-pt`, and the fine-tuned model was merged with `google/gemma-3-27b-it` using TIES (mergekit).
- **Training Method:** Trained with Supervised Fine-Tuning (SFT), using LoRA (rsLoRA, r=64) with FSDP2 and Context Parallelism.
- **Context Length:** The maximum context length for training data is 12,288.
### 3. Data
| source | KISTEP Documents<br>(synthetic QA) | KISTEP Documents<br>(evolved QA) |
| ------ | ---------------------------------- | -------------------------------- |
| count  | 7,534                              | 2,547                            |
- The training data generated from KISTEP documents consists of `(Q, CONTEXT, A)` format, with Chain of Thought (CoT) reasoning included in the answers.
- Evolved QA augments the base synthetic QA with multi-hop, keyword, unanswerable, and multi-turn variations.
- Train / validation split: 9,080 / 1,001.
### 4. Usage
- Please combine files into a single file using the command below before use. (When using ollama, you can utilize the GGUF file.)
```
cat SPARK2-bf16_part1.gguf SPARK2-bf16_part2.gguf > SPARK2-bf16.gguf
```
- Python code
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "kistepAI/SPARK2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model.eval()

messages = [
    {"role": "user", "content": "안녕하세요."}
]
input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<end_of_turn>")
]
outputs = model.generate(
    input_ids,
    max_new_tokens=2048,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.3,
    top_p=0.95,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
```
### 5. Prompt Template (RAG)
SPARK2 is trained with the following RAG prompt format. Place the user question in `<question>`, retrieved chunks in `<context>`, and previous conversation in `<history>`.
```
당신은 RAG 전문가입니다. <question>에는 사용자 질문, <context>에는 사용자 질문으로 검색한 결과, <history>에는 과거 대화 내역이 있습니다. <context>를 바탕으로 질문에 답변하세요.

<question>
{question}
</question>

<context>
{context}
</context>

<history>
{history}
</history>

**답변 형식:**

1. <think> 태그에 추론 과정 작성 (1 depth 개조식, 10줄 이하)
   - 질문 분석 → 출처 분석 (문서명, 날짜, 페이지, 최신성, 일치성) → 정보 종합 → 구조화 순서

2. 본문 답변 작성
   - 개조식 또는 표 형식으로 구조화
   - 최신 정보 우선, 시간순 비교 제시
   - 각 정보 뒤에 <source>[출처 + 인덱스]</source> 표기

이제 답변을 작성하세요.
```

	---
	language:
	- ko
	base_model:
	- google/gemma-3-27b-it
	- google/gemma-3-27b-pt
	pipeline_tag: text-generation
	---

	## Usage Guide
	개인은 자유롭게 사용할 수 있습니다.
	기업 및 기관은 비상업적 목적으로 이용해 주시기 바랍니다.
	또한, 추후 협업 및 네트워크 구축을 위해 기관 정보와 AI 모델 사용 담당자 정보를 메일로 보내주시면 연락드리겠습니다.
	## CONTACT : kistep_ax@kistep.re.kr
	Individuals are free to use this without restrictions.
	For companies and institutions, please use it for non-commercial purposes.
	Additionally, to facilitate future collaboration and network building, please send us an email with your institution's information and the contact details of the person responsible for using the AI model. We will get in touch with you.

	### 1. Description
	SPARK2 is a large language model developed by the Korea Institute of S&T Evaluation and Planning (KISTEP). As the successor to SPARK-RAG and SPARK-Report, SPARK2 unifies KISTEP's task-specific models into a single model. It is optimized for RAG (Retrieval-Augmented Generation) tasks and incorporates Chain of Thought (CoT) reasoning to enhance its response accuracy and performance.

	### 2. Key Features
	- Enhanced Reliability through RAG: Provides highly reliable responses by leveraging the organization's internal databases through Retrieval-Augmented Generation (RAG).
	- Transparent Reasoning: Trained to demonstrate its reasoning process through Chain of Thought (CoT) inside `<think>` tags, clearly showing the information sources and logic behind each response with `<source>` citations.
	- Structured Output: Responses in well-formatted markdown, including tables, text, and summaries for improved readability and clarity.
	- Base Model: Built on Gemma-3-27b as the foundation model. Supervised Fine-Tuning (SFT) was performed on `google/gemma-3-27b-pt`, and the fine-tuned model was merged with `google/gemma-3-27b-it` using TIES (mergekit).
	- Training Method: Trained with Supervised Fine-Tuning (SFT), using LoRA (rsLoRA, r=64) with FSDP2 and Context Parallelism.
	- Context Length: The maximum context length for training data is 12,288.

	### 3. Data
	\| source \| KISTEP Documents<br>(synthetic QA) \| KISTEP Documents<br>(evolved QA) \|
	\| ------ \| ---------------------------------- \| -------------------------------- \|
	\| count \| 7,534 \| 2,547 \|

	- The training data generated from KISTEP documents consists of `(Q, CONTEXT, A)` format, with Chain of Thought (CoT) reasoning included in the answers.
	- Evolved QA augments the base synthetic QA with multi-hop, keyword, unanswerable, and multi-turn variations.
	- Train / validation split: 9,080 / 1,001.

	### 4. Usage
	- Please combine files into a single file using the command below before use. (When using ollama, you can utilize the GGUF file.)
	```
	cat SPARK2-bf16_part1.gguf SPARK2-bf16_part2.gguf > SPARK2-bf16.gguf
	```
	- Python code
	```python
	import torch
	from transformers import AutoTokenizer, AutoModelForCausalLM


	model_id = "kistepAI/SPARK2"

	tokenizer = AutoTokenizer.from_pretrained(model_id)
	model = AutoModelForCausalLM.from_pretrained(
	model_id,
	torch_dtype=torch.bfloat16,
	device_map="auto",
	)

	model.eval()

	messages = [
	{"role": "user", "content": "안녕하세요."}
	]

	input_ids = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	return_tensors="pt"
	).to(model.device)

	terminators = [
	tokenizer.eos_token_id,
	tokenizer.convert_tokens_to_ids("<end_of_turn>")
	]

	outputs = model.generate(
	input_ids,
	max_new_tokens=2048,
	eos_token_id=terminators,
	do_sample=True,
	temperature=0.3,
	top_p=0.95,
	)

	response = outputs[0][input_ids.shape[-1]:]
	print(tokenizer.decode(response, skip_special_tokens=True))
	```

	### 5. Prompt Template (RAG)
	SPARK2 is trained with the following RAG prompt format. Place the user question in `<question>`, retrieved chunks in `<context>`, and previous conversation in `<history>`.
	```
	당신은 RAG 전문가입니다. <question>에는 사용자 질문, <context>에는 사용자 질문으로 검색한 결과, <history>에는 과거 대화 내역이 있습니다. <context>를 바탕으로 질문에 답변하세요.

	<question>
	{question}
	</question>

	<context>
	{context}
	</context>

	<history>
	{history}
	</history>

	답변 형식:

	1. <think> 태그에 추론 과정 작성 (1 depth 개조식, 10줄 이하)
	- 질문 분석 → 출처 분석 (문서명, 날짜, 페이지, 최신성, 일치성) → 정보 종합 → 구조화 순서

	2. 본문 답변 작성
	- 개조식 또는 표 형식으로 구조화
	- 최신 정보 우선, 시간순 비교 제시
	- 각 정보 뒤에 <source>[출처 + 인덱스]</source> 표기

	이제 답변을 작성하세요.
	```