ZERO-Kin1

ZERO-Kin1 (Kin = 金, money/finance - "ZERO" for the zero-cost build: consumer hardware, no paid APIs; 1 = first release) is a QLoRA adapter for Qwen/Qwen2.5-3B-Instruct fine-tuned for Japanese financial table QA and numerical reasoning (jfinqa-style tasks: DuPont decomposition, YoY change, consistency arithmetic, trend classification over EDINET securities-report tables).

• jfinqa (official harness, 1,000 questions, zero-shot, temperature 0): overall 81.6%
  • Numerical 75.5 / Consistency 94.5 / Temporal 84.8 (baseline Qwen2.5-3B-Instruct zero-shot: 39.6 overall)
• Internal holdout (100 companies reserved from training): 89%

Provenance

Fine-tuned only on 2,117 synthesized jfinqa-format QA pairs with worked-solution completions, generated from EDINET-Bench filings (structured summary/PL/BS/CF of Japanese securities reports). No jfinqa test data was used for training: all 104 EDINET codes appearing in the official jfinqa test set were excluded from the training pool (0 overlapping companies, 0 id collisions, 0 identical question texts - programmatically asserted).

Training

• QLoRA: 4-bit NF4 base (double quant, fp16 compute), LoRA r=16 / α=32 / dropout 0.05 on attention q_proj, k_proj, v_proj, o_proj - 7,372,800 trainable params (0.24%)
• 2 epochs, lr 2e-4, effective batch 8 (1 × grad-accum 8), max seq 1280, completion-only loss, gradient checkpointing - single RTX 3060 12GB
• Completions are short arithmetic chains ending with the final answer on the last line

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

base_id = "Qwen/Qwen2.5-3B-Instruct"
adapter_id = "745H1N/ZERO-Kin1"  # this repo
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.float16,
                         bnb_4bit_use_double_quant=True)
tok = AutoTokenizer.from_pretrained(base_id)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb,
                                         device_map="auto"),
    adapter_id)
model.eval()

prompt = (
    "以下の財務データ(表)と文章を読み、質問に答えてください。\n"
    "計算は正確に行い、最終的な答えのみを出力してください（説明や単位の補足は最小限）。\n\n"
    + pre_text + "\n[表]\n" + table + "\n" + post_text + "\n"
    + "[質問] " + question + "\n[答え] "
)
inputs = tok(prompt, return_tensors="pt", truncation=True, max_length=1280).to(model.device)
out = model.generate(**inputs, max_new_tokens=160, do_sample=False,
                     pad_token_id=tok.eos_token_id)
text = tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
answer = text.strip().split("\n")[-1]   # the final answer is the LAST line

The model emits a short arithmetic working before the answer; consumers should take the last line (the answer is in the official jfinqa answer format: 1-decimal percentages, comma-grouped amounts, はい/いいえ・増収/減収 categoricals).

Data attribution

• Base model: Qwen/Qwen2.5-3B-Instruct (Apache-2.0)
• Training data synthesized from EDINET-Bench (Japanese EDINET filings; benchmark data Apache-2.0 / PDL 1.0)
• Evaluation benchmark: jfinqa - Japanese Financial Numerical Reasoning QA Benchmark (Apache-2.0) by ajtgjmdjp: dataset · GitHub · PyPI · leaderboard. Used for evaluation only, never for training. jfinqa has no arXiv paper - cite via the repository.