thinkingdbx/kautilyaa-9b-finance

Kautilyaa 9B (MoE): Financial Analysis Model

Developer: ThinkingDBx (Pvt. Ltd.) · Created by Mallesh Madapathi Model type: Sparse Mixture-of-Experts, decoder-only causal LM Status: Research preview

Disclaimer. Kautilyaa is a research and analysis tool. Nothing it produces constitutes financial advice, an investment recommendation, or a solicitation to trade. It has no knowledge of live markets on its own. See Grounding.

---

At a glance

Property	Value
Total parameters	9B
Active parameters per token	2.6B
Experts / routing	8 experts, top-2
Layers / hidden / heads	32 / 2048 / 16 (4 KV heads, GQA)
Context window	4,096 tokens
Vocabulary	49,152 (SentencePiece, trained from scratch)
Precision	bfloat16
Prompt format	ChatML
Base architecture	Mixtral-compatible (MixtralForCausalLM)

Pre-trained from scratch on SEC EDGAR filings and FNSPID (financial news + price data), then instruction-tuned. It is not a fine-tune of an existing open-weight model.

---

What it is for

Kautilyaa is built to sit behind a retrieval layer that injects live market data into its context. Its job is to read those numbers and reason about them, not to recall prices from memory, which it cannot do.

Works well:

• Reading a [LIVE MARKET DATA] block and answering questions from it
• Quoting prices, multiples, margins, ranges accurately from context
• Ranking / screening a basket of instruments on a supplied metric
• Declining to quote an instrument it wasn't given data for
• Correcting a user who asserts a figure contradicting the supplied data
• SEC filing and financial-news register

Does not work:

• More than one instrument per context block. See the hard constraint below.
• Arithmetic. Do not ask it to compute percentages, ratios, or differences. It will produce plausible, wrong numbers. Compute derived values upstream and put them in the context block.
• Recalling live or historical prices from memory
• General-purpose assistant tasks outside finance
• Long documents. The context window is 4,096 tokens

---

Hard constraint: one instrument per context block

The model does not select an instrument from the question. It anchors on the first instrument in the context block and answers using that one's numbers, whichever instrument was actually asked about.

Measured on 32 hand-written questions with disjoint numeric bands per instrument, so every figure is unambiguously attributable:

Instruments in block	Correct attribution	Cross-talk errors	Fully clean answers
1	100%	0	8/8
2	50%	20	4/8
3	36%	27	2/8
5	22%	26	2/8

At one instrument the model is exact: perfect attribution, no cross-talk, no invented numbers. With two or more it reliably answers about the wrong one. For example, asked about HDFCBANK.NS it returns Tesla's price, carried over from the first block entry and rendered with a rupee sign.

Integrations must inject exactly one instrument per request. If a user names several, issue one call per instrument and merge the results. This is not a soft recommendation; it is the difference between 100% and 50% correctness.

---

Measured performance

Perplexity, identical texts, before and after instruction tuning:

Text type	Base	Instruction-tuned
General prose	2.40	2.39
SEC filing language	6.60	6.64
Financial news	9.90	9.33
SQL (out of domain)	4.40	4.50

Instruction tuning caused no measurable capability regression.

Grounding behaviour, 150 held-out cases with context blocks containing values absent from training. Hallucinated numbers counts numeric tokens in the answer that do not appear in the supplied context.

Category	n	Hallucinated numbers / answer	Clean answers	Citation recall
Quote readout	18	0.00	100%	78%
52-week position	17	0.00	100%	65%
Valuation	20	0.00	100%	30%
Fundamentals	13	0.00	100%	60%
Comparison	18	0.00	100%	93%
Screening / ranking	16	0.00	100%	86%
Correcting a wrong figure	16	0.00	100%	100%
Absent instrument	14	0.00	100%	n/a
Personal advice request	18	0.00	100%	89%
All	150	0.00	100%	67%

• Declines to quote an instrument absent from context: 100%
• Declines to give personalised investment advice: 89%
• Emits <|im_end|> correctly: 100% (given an adequate token budget)
• Includes a risk disclaimer: 87%

Zero hallucinated numeric values across 150 held-out cases. Every figure in every answer traces to the supplied context block. Citation recall is the share of available values the answer chose to cite. Lower is not an error; it reflects answer brevity.

---

Prompt format

ChatML. A chat template ships in tokenizer_config.json, so:

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("kautilyaa-9b")
model = AutoModelForCausalLM.from_pretrained("kautilyaa-9b", dtype="bfloat16")

msgs = [
    {"role": "system", "content": SYSTEM_PROMPT + "\n\n" + live_market_block},
    {"role": "user", "content": "Where does INFY.NS sit in its 52-week range?"},
]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=256, suppress_tokens=[0, 1, 3])

suppress_tokens is required

Token IDs 0 (<unk>), 1 (<s>) and 3 (<pad>) have near-zero embeddings but above-average lm_head norms. They were never targets during pre-training, so they never received the negative gradient that shaped real tokens. Under uncertainty they win, and generation collapses into repeating them. Always pass suppress_tokens=[0, 1, 3]. It ships in generation_config.json.

Do not prepend BOS. The <s> embedding is dead (norm 0.003) and prepending it roughly doubles loss (9.13 vs 4.70 measured). add_bos_token is set to false.

Serve with vLLM or transformers. Ollama's Modelfile cannot express suppress_tokens, so the collapse mitigation does not survive that path.

---

Known limitations and provenance

Documented plainly, because they affect how the model behaves:

Weight decay damaged the network during pre-training. The MaxText config left adamw_mask at its default [], which applies AdamW decay to every parameter including RMSNorm gains and the embedding table. Over 576,000 steps at weight_decay=0.1 this drove early-layer norm gains toward zero. Layer 0's input_layernorm survived at +0.025 against an initialisation of 1.0, and the network compensated by inflating the final layer. Practical effects:

• Effective depth is lower than 32 layers implies
• 8,181 embedding rows decayed to near zero (tokens unseen in training)
• Special-token embeddings are dead, hence suppress_tokens

The model was trained and converged with this damage, so it is internally consistent and usable, but it is weaker than its parameter count suggests. A modern open-weight 7–8B model will outperform it on general tasks.

Arithmetic is unreliable and this is not fixable by further tuning at this scale. Instruction tuning explicitly trains the model to quote raw values and make ordinal comparisons rather than compute. Architect around it.

Context is 4,096 tokens. Earlier internal documentation claimed 32,768; that was never true of this checkpoint. No RoPE scaling has been applied or validated.

---

Training

Stage	Detail
Pre-training	MaxText on TPU, 576,000 steps, SEC EDGAR + FNSPID
Instruction tuning	LoRA r=32, α=64, attention-only (q,k,v,o), weight_decay=0.0
SFT data	1,007 curated finance examples + 3,200 synthetic grounded examples
Hardware	1× A100 40GB (spot)

LoRA targets attention only: the capability being taught is retrieval from context, and attention is the mechanism that performs it. The MoE expert FFNs already carry the domain knowledge.

Synthetic grounding examples deliberately use values decoupled from real-world figures, so the model learns to read the context block rather than recall a prior. It therefore quotes what it is given and should never be trusted as a source of market facts on its own.

---

License

Apache-2.0. See LICENSE. Free for commercial use, modification and redistribution, with attribution and no warranty.

Citation

@misc{kautilyaa2026,
  title  = {Kautilyaa: A Mixture-of-Experts Model for Financial Analysis},
  author = {Madapathi, Mallesh},
  year   = {2026},
  note   = {ThinkingDBx (Pvt. Ltd.)}
}