Gro97/customer-concentration-risk-llama4-scout-lora

Model Card: Customer Concentration Risk Analysis — Llama 4 Scout 17B LoRA

Model Details

Model Description

This is a LoRA adapter fine-tuned on top of Llama-4-Scout-17B-16E-Instruct to analyze customer/segment concentration risk from SEC Form 10-K risk factor disclosures. Given a company's disclosed customer or segment concentration percentage across two consecutive fiscal years, the model classifies the trend (worsening, improving, or stable -- a change of ≤1.5 percentage points is treated as stable) and determines whether the disclosure crosses the standard U.S. GAAP/SEC materiality threshold (10% of net sales or receivables) between the two periods, along with a short risk interpretation.

This targets a genuine reasoning gap: correctly classifying a concentration trend and threshold crossing requires actually computing the percentage-point change and comparing both values against a fixed threshold, not just recognizing that "customer concentration" is a risk topic -- a distinction base LLMs often blur, treating the presence of risk language as equivalent to a correctly quantified trend.

Training labels were generated by construction: given the two disclosed percentages, trend_classification and crossed_materiality_threshold follow deterministically from the stated ≤1.5pp-stable rule and the 10% threshold comparison. Every one of the 3,130 rows was independently re-verified against this rule with zero mismatches found.

Submitted to the AutoScientist Challenge (Market Analysis & News track) by Adaption Labs.

• Developed by: Grold Otieno Mboya
• Shared by: Grold Otieno Mboya
• Model type: LoRA adapter for causal language modeling (financial disclosure / concentration-risk reasoning)
• Language(s): English
• License: Llama 4 Community License Agreement (inherited from the base model)
• Finetuned from model: meta-llama/Llama-4-Scout-17B-16E-Instruct (trained via the 4-bit quantized variant togethercomputer/Llama-4-Scout-17B-16E-Instruct_bnb_4bit, per this adapter's own adapter_config.json)

Model Sources

Platform	Dataset	Model
Hugging Face	https://huggingface.co/datasets/Gro97/adaption-customer-concentration-risk-10k	customer-concentration-risk-llama4-scout-lora
Kaggle	https://www.kaggle.com/datasets/groldotieno/adaption-customer-concentration-risk-10k	customer-concentration-risk-llama4-scout-lora

Live Demo: https://riskscreener.adaptionlabs.app/

Uses

Direct Use

Screening a company's customer/segment concentration disclosures for trend direction and materiality-threshold crossings across two fiscal years -- useful for equity/credit analysts doing quick disclosure triage, portfolio managers monitoring concentration exposure across a watchlist, and researchers building datasets from 10-K risk factor sections.

Downstream Use

Could be extended to more than two fiscal years (multi-period trend analysis), other GAAP materiality thresholds (e.g. receivables concentration, geographic concentration), or integrated into a larger disclosure-monitoring pipeline that flags filings for analyst review.

Out-of-Scope Use

Not a substitute for reading the actual 10-K filing or consulting a financial professional before making investment decisions. Not evaluated for general capability, factuality, or safety beyond the concentration-risk task described here. The model reasons only over the two percentages it is given -- it does not verify that those figures are accurately transcribed from a real filing, so output quality depends entirely on the accuracy of the input data supplied to it.

Bias, Risks, and Limitations

The training data combines 30 rows sourced from a real company's actual disclosed filings (Allison Transmission Holdings Inc's disclosed concentration with Daimler AG and Navistar International Corporation / Traton SE, plus a small number of anonymized entries labeled by SEC CIK number) with 3,100 synthetic rows using invented company and customer names but realistic percentage ranges. Every row's trend_classification and crossed_materiality_threshold label was independently re-derived from the two raw percentages using the stated rule and confirmed to match exactly across all 3,130 rows, with zero inconsistencies found.

Known limitation -- real-company representation: only one real company (Allison Transmission Holdings Inc) is meaningfully represented across the 30 real rows; this is not a broad, diverse sample of real SEC filings, and performance on real-world filings from other companies or industries has not been separately measured.

Known limitation -- no source verification at inference time: the model reasons only over whatever percentages it is given in the prompt. It does not check those figures against a real filing, so it cannot catch a transcription error or a fabricated input -- output correctness depends on the accuracy of the numbers supplied by the user.

Known limitation -- single fixed threshold: the model is trained specifically around the 10% GAAP/SEC materiality convention and the ±1.5 percentage-point stability rule; it has not been evaluated against company- or industry-specific materiality thresholds that may differ from this default.

Recommendations

Treat outputs as a fast triage/screening aid, not a substitute for reading the underlying filing. Verify any input figures against the actual 10-K before relying on the trend classification, particularly for real-world use outside the training distribution's single real-company anchor.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model_id = "meta-llama/Llama-4-Scout-17B-16E-Instruct"
adapter_id = "Gro97/customer-concentration-risk-llama4-scout-lora"

bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id, quantization_config=bnb_config, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

system_prompt = (
    "You are a financial analyst specializing in SEC disclosure analysis. "
    "Given a company's disclosed customer or segment concentration-risk "
    "percentage across two consecutive fiscal years, and the standard U.S. "
    "GAAP materiality threshold for concentration disclosure (10% of net "
    "sales or receivables), determine whether the risk is worsening, "
    "improving, or stable, and whether it crosses the materiality threshold "
    "between the two periods. A change of 1.5 percentage points or less "
    "should be classified as stable. Respond only with strict JSON with "
    "exactly these keys: company, customer_or_segment, "
    "disclosed_materiality_threshold_pct, period_a_year, period_a_value_pct, "
    "period_b_year, period_b_value_pct, trend_classification ('worsening', "
    "'improving', or 'stable'), crossed_materiality_threshold (boolean), "
    "risk_interpretation. Base your answer only on the two reported values given."
)
user_prompt = (
    "ExampleCo's Form 10-K risk factors disclose customer concentration "
    "risk under the standard SEC/GAAP materiality threshold of 10.0% of "
    "net sales. For fiscal year 2022, ExampleCo reported that Customer X "
    "accounted for 8.2% of net sales. For fiscal year 2023, ExampleCo "
    "reported that Customer X accounted for 12.7% of net sales."
)

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_prompt},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=300)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details

Training Data

Custom-built dataset customer_concentration_risk_10k: 3,130 rows.

Component	Rows	Description
Real (cited)	30	Allison Transmission Holdings Inc's actual disclosed concentration with Daimler AG / Navistar-Traton, plus anonymized SEC-CIK-referenced entries
Synthetic (realistic)	3,100	Invented company/customer names, realistic percentage ranges

Average prompt length 58 words, average completion length 51 words. Every row's trend_classification and crossed_materiality_threshold label was independently recomputed from the two disclosed percentages and confirmed to match exactly across all 3,130 rows.

• Hugging Face: https://huggingface.co/Gro97/customer-concentration-risk-llama4-scout-lora
• Kaggle: https://www.kaggle.com/models/groldotieno/customer-concentration-risk-llama4-scout-lora

Training Procedure

Fine-tuned via Adaption Labs' AutoScientist platform using LoRA.

Training Hyperparameters

• Training regime: 4-bit quantized base model (bnb_4bit) with LoRA adapters trained in higher precision (QLoRA-style)
• LoRA rank / alpha / dropout / target modules / epochs: see adapter_config.json and trainer_state.json in this repository for the exact configuration used for this run -- not independently re-confirmed in this write-up.

Speeds, Sizes, Times

Not independently benchmarked by the author; training was run on Adaption Labs' hosted infrastructure. Exported adapter size: ~850MB (adapter_model.safetensors).

Evaluation

Testing Data, Factors & Metrics

Testing Data

Two evaluation sets, as reported by Adaption Labs' AutoScientist platform: (1) a held-out split of this project's own dataset, (2) a broader set of unseen tasks from Adaption's internal Market Analysis & News category benchmark (labeled "Corporate & Business" in the platform's own domain taxonomy -- the same evaluation category, reported under Adaption's internal domain tag rather than the challenge's category name).

Metrics

Pairwise preference win rate against the un-adapted base model, as reported by Adaption Labs' AutoScientist platform (judge methodology not disclosed to the end user).

Results

Evaluation set	Base model win rate	Adapted model win rate
This dataset's held-out samples	36%	64%
Broader Market Analysis & News category	28%	72%

Summary

The adapter shows a substantial improvement over the base model both on its own training distribution (64% win rate) and, more notably, an even larger improvement on the broader category benchmark (72% win rate) -- the broader-category gain exceeding the own-dataset gain is a distinct pattern from this author's other submissions, and is worth noting rather than assuming it simply generalizes better: it may also reflect that the broader category's baseline (28%) started weaker than this dataset's own baseline (36%), giving more room for improvement rather than necessarily indicating stronger transfer.

Environmental Impact

• Hardware Type: Not disclosed by the training platform for this run.
• Hours used: Not disclosed.
• Cloud Provider: Adaption Labs' hosted infrastructure.
• Compute Region: Not disclosed.
• Carbon Emitted: Not calculated.

Technical Specifications

Model Architecture and Objective

LoRA adapter applied to meta-llama/Llama-4-Scout-17B-16E-Instruct (a Mixture-of-Experts architecture, trained here via a 4-bit quantized variant), trained via supervised fine-tuning to classify customer concentration risk trends and materiality-threshold crossings from SEC 10-K disclosure data.

Compute Infrastructure

Software

• PEFT 0.15.1
• Trained via Adaption Labs' AutoScientist platform

Citation

BibTeX:

@misc{mboya2026concentrationrisk,
  author = {Mboya, Grold Otieno},
  title = {Customer Concentration Risk Analysis: A LoRA Adapter for
           SEC 10-K Disclosure Trend Classification},
  year = {2026},
  howpublished = {AutoScientist Challenge submission, Adaption Labs},
  url = {https://huggingface.co/Gro97/customer-concentration-risk-llama4-scout-lora}
}

Model Card Contact

See author's Hugging Face profile: https://huggingface.co/Gro97

Framework versions

• PEFT 0.15.1