SEC 10-K Analyst 2B ยท GGUF Q4_K_M

A 2B model that beats Qwen3.5-27B at answering questions about SEC 10-K filings, at about 1/25 of its size.

Results

SEC 10-K Analyst 2B Qwen3.5-27B
Correct on 87 held-out questions 84 72
Parameters 2B 27B
Quantization 4-bit (Q4_K_M) base + F16 LoRA adapters none (BF16, 16-bit)
Weights on disk 2.2 GB 54 GB
GPU memory, full pipeline 11โ€“17 GB (all nine adapters + label picker; 17 GB with the published settings, 11 GB in the smallest, same answers) over 54 GB for weights alone
Runs on one GPU with 12 GB or more (tested: RTX 4090, RTX A4000, DGX Spark), or the processor alone (slow) an 80 GB-class datacenter GPU

Run it in one step. Tell your coding agent: "Set up https://huggingface.co/APEXE3HQ/sec10k-analyst-2b by following its AGENTS.md." It downloads the model and runs bash run.sh, which installs everything, starts the model, checks it and opens the web app at http://127.0.0.1:7860. Prefer to do it yourself? See How to run.

A specialised 4-bit 2B model beats a general 27B model running at full BF16 precision.

  • Same pipeline: both models answered through the same question-answering app; only the model differs.
  • Held out: none of the 87 questions or their filings was used in training. Questions, ground truth, rubric and scorer are public: usmane3/sec10k-heldout-87.
  • Graded blind: four independent reviewers with a fixed rubric, the answers unlabeled.
  • Sealed test: on 150 further questions that nobody reads, scored once per version, v1.2 answers 135 correctly.

What it does

  • Exact reported figures: revenue, net income, cash flow, debt, leases, deferred revenue, taxes and buybacks.
  • Splits and look-alikes: current / noncurrent / total splits, and easily confused pairs such as tax expense vs taxes paid.
  • Derived figures: differences, margins and growth, computed and checked.
  • Financial-health verdicts backed by the figures.
  • Natural questions: follow-ups, misspelled company names, "FY25"-style years and omitted years.
  • Declines what a 10-K can't answer, such as live prices or future periods, and says why.
  • Citations: every figure is cited to its source.

Label picker vs Jev 1.14

Its label picker, which finds the exact filing line a question asks about, beat Jev 1.14, a commercial decision model, on a sealed test scored once under pre-registered rules: 96.2% vs 94.8% on 3,075 verified questions, at 4-bit.

Label picker vs Jev 1.14

How we built it

A specialist base. We started from Qwen3.5-2B and fine-tuned it to read SEC filings, first with SFT, then with DPO. The fine-tune is merged into the weights and quantized to 4-bit Q4_K_M.

One base, nine specialists. Each step of answering a question has its own LoRA adapter on that shared base:

  • Planner. Trained with CE (cross-entropy) on 3,072 verified plans: the 1,643 of v1.0 (1,426 built directly from the filings' ground truth, plus 217 plans from Qwen3.5-27B that passed every check) and 1,429 new ones built from the ground truth. It learns to:

    • split current, noncurrent and total;
    • plan derived figures, margins, changes between years and financial-health checks;
    • tell look-alike pairs apart, such as cash paid for taxes vs tax expense;
    • follow a conversation;
    • decline what no 10-K can answer.

    The plan validator rejected all of the roughly 22,700 deliberately broken plans it was tested against.

  • Writer. Trained with CE (cross-entropy) + KL distillation from Qwen3.5-27B on 1,042 checked answers: the 485 audited answers of v1.0 plus 557 new ones (491 written by the model itself, 66 by Qwen3.5-27B), each kept only if it passed the strict checker. The student outscores its teacher, 84 vs 72.

  • Label picker. Four adapters (first pass, tournament, two judges) find the exact line in the filing. It beat Jev 1.14, a commercial decision model, on a sealed test.

  • Three text roles, trained with SFT.

No wrong answer in the training data. Every planner and writer training example passed a strict automatic checker. For each figure, the checker finds the right row and the right year in the cited source. Two independent audits tried to break it, and the final checker:

  • rejects all 140 deliberately wrong answers written to fool it;
  • accepts all 1,530 correct reference answers.

Tested blind. The test set is 87 held-out questions, never used in training. Both models ran in the same app, and independent reviewers graded the unlabeled answers without ever seeing the automatic scores: 84 for this model, 72 for Qwen3.5-27B. By the original automatic checker, the untrained Qwen3.5-2B scores 6 of 87 in that app and this model 81. The questions, ground truth, rubric and scorer are public: usmane3/sec10k-heldout-87.

Files

File Size Role
base-only-Q4_K_M.gguf 1.31 GB the fine-tuned base (Qwen3.5-2B + SFT + DPO, merged), 4-bit; shared by every role
adapters/router-F16-lora.gguf 67 MB planner
adapters/writer-F16-lora.gguf 67 MB writer and calculator
adapters/router_legacy_v8-F16-lora.gguf 67 MB figure lookups
adapters/prose_v6-F16-lora.gguf 67 MB text answers
adapters/sections_v9-F16-lora.gguf 67 MB section reading
adapters/picker/*-F16-lora.gguf 4 ร— 135 MB label picker: first pass, tournament, two judges
data/ 3.8 GB S&P 500 10-K snapshot: verified figures, XBRL facts, passages
app/ โ€” the pipeline that connects the roles
run.sh โ€” one-command setup: checks, installs, starts the model and the web app
AGENTS.md โ€” setup instructions for AI coding agents
start.sh, serving/ โ€” the launcher: loads the base once and serves all nine adapters
ask.py โ€” ask questions from the command line
webapp.py โ€” browser chat with cited sources (Gradio)
requirements.txt โ€” Python packages

How to run

This model is a pipeline: nine adapters on one base, plus the SEC filing data in data/. It is not a single chat model, so run it with the included scripts rather than loading the GGUF into a chat app.

pip install -U huggingface_hub brotli
hf download APEXE3HQ/sec10k-analyst-2b --local-dir sec10k-analyst-2b     # 6.3 GB, includes data/
cd sec10k-analyst-2b
bash run.sh

run.sh does all of this, and is safe to rerun:

  1. checks the machine;
  2. installs everything into .venv (the first run builds llama.cpp, which takes a few minutes);
  3. starts the model;
  4. answers a check question;
  5. starts the web app.

It ends with:

READY. SEC 10-K Analyst 2B is running.
    Web app:        http://127.0.0.1:7860
    Command line:   .venv/bin/python ask.py "What was Apple's net income in fiscal 2024?"
    Stop:           bash run.sh stop

The web app is a chat with follow-ups. Each answer comes with its cited sources and the plan it followed.

On a rented GPU or a remote server the web app listens on that machine only. From your own computer, run ssh -L 7860:127.0.0.1:7860 <user>@<server> and open http://127.0.0.1:7860. Other ways are in HOW_TO_RUN.md.

From the command line:

$ .venv/bin/python ask.py "What was ResMed's total revenue for fiscal 2025?"

ResMed, fiscal 2025 (filing period ended 2025-06-30). Total revenue: 5146.327 USD millions [S1].

Sources:
  [S1] Verified figures table: revenue
        https://www.sec.gov/Archives/edgar/data/943819/000094381925000035/

Manual steps, settings and troubleshooting are in HOW_TO_RUN.md.

Updating from an earlier version: run the same hf download command again (it fetches only the changed files; v1.1 to v1.2 is about 70 MB), then bash run.sh stop and bash run.sh.

Platforms

  • Tested: Linux with an NVIDIA RTX 4090 (24 GB) in every memory setting, and with an RTX A4000 (16 GB, which uses 12.4 GB), on Python 3.11 and 3.12.
  • DGX Spark (GB10, Arm): tested; run.sh picks the 11 GB setting, and questions take 7โ€“55 s.
  • Several copies on one GPU: tested on one RTX 4090 (24 GB): two copies in the smallest setting (21.5 GB together) answered 5.1 questions a minute against 3.9 for one copy, with shorter waits. A GPU with more memory fits more copies.
  • No GPU: DEVICE=cpu bash run.sh gives the same figures, slowly: on 16 server cores about a minute for a simple question and 9โ€“15 minutes when the label picker is needed. See HOW_TO_RUN.md.
  • Phones: on 6 Arm cores of the kind used in 2025 flagship phones (2 Cortex-X925 + 4 Cortex-A725, on a DGX Spark), a question takes about 2โ€“3 minutes, or about 20 minutes when the label picker is needed, in about 5 GB of memory. For quick answers on a phone today, run the model on a GPU and open the web app in the phone's browser. An Android app that runs everything on the phone is in development.
  • Other GPUs give the same figures, but the wording of an answer and its list of cited sources can differ slightly. On the RTX A4000, 9 of 12 sampled answers matched the RTX 4090's word for word; the other 3 had the same figures.
  • Apple Silicon Macs with 24 GB or more should work through Metal, but are not yet tested end-to-end. On a 24 GB Mac, run.sh picks the 11 GB setting.

Changelog

v1.2 (20 November 2026)

An independent engineering team evaluated v1.0 on their own hardware, with their own questions checked against SEC data. v1.2 addresses each issue they raised:

Issue raised In v1.2
Margins came back as net income Fixed. A margin asked by name is computed from the filing's own figures (Apple's fiscal 2024 gross margin: 46.2%)
Growth was refused Fixed. Change between two years, and growth in one named year, measured from the year before
Cash taxes paid answered with tax expense Fixed for cash paid for taxes and for interest, however it is asked
A unit 1,000ร— off on casual wording Fixed (since v1.1)
Declines said only "I cannot complete this request." Declines now say why; a clarification says what is missing
One copy of the model per GPU INSTANCE=2 bash run.sh runs another copy beside the first; see several copies
No way to run without a GPU DEVICE=cpu bash run.sh runs on the processor alone; slow, see no GPU
Shell errors on DGX Spark Fixed, tested on a real DGX Spark: it picks the 11 GB setting and runs
The 87 test questions were not public Now public: usmane3/sec10k-heldout-87

Also new: it installs on Python 3.11, works on rented GPUs and remote servers, and run.sh checks that the download is complete.

Test v1.1 v1.2
87 held-out questions, independent reviewers 83 84
Sealed test of 150 questions, original automatic checker 128 135
Sealed test of 150 questions, strict automatic checker 127 135
The question types they raised: 53 new questions on the 7 companies in their report 9 53
  • The 87: four independent reviewers graded v1.1, v1.2 and Qwen3.5-27B together, with the answers unlabeled. v1.2 fixed 3 answers and broke 2, both clarification questions where an answer was due (no wrong figure).
  • The sealed test: 150 questions written after v1.0 from question templates never used in training. Nobody reads its questions or answers, and each version is scored on it exactly once. Question by question, v1.2 fixed 12 answers and broke 5 (original checker).
  • The 53: written for this round from their examples and scored automatically (the right number must appear in the answer), so it shows those question types are fixed rather than serving as an independent test.
  • What changed in the files: the planner adapter, retrained with 1,429 new verified plans; the app's checks; the setup scripts. The base model and the other eight adapters are unchanged; an existing download fetches about 70 MB.

Earlier versions stay available: add --revision v1.1 (or v1.0) to the download command.

All versions

Version Date What changed
v1.2 20 Nov 2026 Retrained planner (margins, growth, cash paid); clearer declines; several copies per GPU; CPU mode; DGX Spark, Python 3.11 and remote-server fixes; the 87 questions public
v1.1 20 Nov 2026 New writer adapter; answer tidy-up step; results on a new sealed test
v1.0 19 Nov 2026 First public version

Built on

Qwen3.5-2B (Apache-2.0), fine-tuned as described above ยท SEC EDGAR filings.

Downloads last month
212
GGUF
Model size
2B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

4-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for APEXE3HQ/sec10k-analyst-2b

Finetuned
Qwen/Qwen3.5-2B
Adapter
(420)
this model