uaytug/fumea-f-dense

FUMEA-F Dense: Frontier Unified Multi-Expert Agent — Financial (Dense)

FUMEA-F Dense is an 8-billion parameter language model created by merging four domain-specialized models using the DARE+TIES method. It consolidates financial analysis, marketing intelligence, market trend detection, and chain-of-thought financial reasoning capabilities into a single dense transformer — suitable for deployment on consumer hardware without MoE routing overhead.

Architecture

Property	Value
Parameters	~8B
Architecture	Decoder-only Transformer (Dense)
Merge Method	DARE+TIES
Density	0.75
Normalization	Enabled
int8 Mask	Enabled
Context Window	131,072 tokens
Positional Encoding	RoPE with YaRN scaling (factor 4.0, base 32,768)
Precision	bfloat16
Vocabulary Size	151,936

Merge Configuration

Four 8B-parameter models were merged with the following weight distribution, biased toward financial domains:

Expert Domain	Weight	Role
Marketing Intelligence	0.30 (base)	Brand strategy, campaign analysis, market positioning
Financial Forecasting	0.25	Time-series prediction, technical indicators, quantitative modeling
Market Trends	0.20	E-commerce analytics, consumer behavior, competitive pricing
Financial Reasoning	0.25	Multi-step chain-of-thought, valuation logic, regulatory analysis

The marketing model served as the base for the merge due to its broad domain coverage, providing the structural foundation onto which specialized financial and trend capabilities were integrated.

Capabilities

Financial Analysis

• Fundamental metrics: EBITDA margin, P/E ratio, PEG ratio, DCF frameworks
• Technical analysis: candlestick interpretation, support/resistance identification
• Risk assessment: portfolio theory, diversification strategies, stress testing concepts

Marketing Intelligence

• Campaign performance analysis and ROI attribution
• Competitive landscape and positioning assessment
• Market sizing, segmentation, and go-to-market strategy

Market Trend Detection

• E-commerce pricing dynamics and conversion analysis
• Consumer behavior pattern recognition
• Emerging market opportunity identification

Financial Reasoning

• Step-by-step derivation of financial conclusions
• Multi-factor valuation with explicit reasoning chains
• Regulatory compliance analysis with structured argumentation

Tool Use Supports structured tool calling via the chat template, including:

• analyze_ohlcv — OHLCV time-series analysis with configurable indicators
• web_search — External information retrieval
• code_executor — Python code execution
• file_reader — Structured file parsing

Extended Context 128K token context window for processing lengthy financial documents, earnings transcripts, and large datasets in a single pass.

Usage

Basic Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "uaytug/fumea-f-dense",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("uaytug/fumea-f-dense", trust_remote_code=True)

messages = [
    {"role": "system", "content": "You are a financial analysis assistant."},
    {"role": "user", "content": "Explain the PEG ratio and when it is most useful for stock valuation."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

output = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
    repetition_penalty=1.1,
)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Tool Use

tools = [
    {
        "name": "analyze_ohlcv",
        "description": "Analyze OHLCV time-series data for pattern recognition and trend detection",
        "parameters": {
            "type": "object",
            "properties": {
                "symbol": {"type": "string", "description": "Trading symbol"},
                "ohlcv_data": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "timestamp": {"type": "string"},
                            "open": {"type": "number"},
                            "high": {"type": "number"},
                            "low": {"type": "number"},
                            "close": {"type": "number"},
                            "volume": {"type": "number"}
                        }
                    }
                },
                "indicators": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "Technical indicators: RSI, MACD, BB, EMA_20, SMA_50"
                },
                "prediction_horizon": {"type": "integer"}
            },
            "required": ["symbol", "ohlcv_data"]
        }
    }
]

messages = [
    {"role": "user", "content": "Run a technical analysis on TSLA with RSI and Bollinger Bands."}
]

text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)

Quantized Versions

GGUF quantizations from F32 to IQ2_M are available at uaytug/fumea-f-dense-gguf for use with Ollama, LM Studio, llama.cpp, and other GGUF-compatible runtimes.

Quantization	Recommended Use
Q8_0	Maximum quality, high-end hardware
Q5_K_M	Strong quality with moderate resource requirements
Q4_K_M	Best balance of quality and efficiency (recommended)
Q3_K_M	Reduced quality, constrained environments
IQ2_M	Experimental, extreme compression

Hardware Requirements

Configuration	Minimum VRAM / RAM
Full precision (bfloat16)	16 GB VRAM
Q8_0 GGUF	8.71 GB VRAM
Q4_K_M GGUF	5.03 GB VRAM
Q3_K_S GGUF	3.77 GB VRAM
Q2_K GGUF	3.28 GB VRAM

Dense vs. MoE

This is the dense variant of the FUMEA-F family. Compared to the MoE version:

Property	FUMEA-F Dense	FUMEA-F MoE
Parameters	~8B	24B (16B active)
Experts	1 (unified)	4 (top-2 routing)
Min VRAM (bf16)	16 GB	48 GB
Min VRAM (Q4_K_M)	6 GB	8 GB
Inference Speed	Faster (no routing)	Slower (expert selection overhead)
Specialization	Blended across all domains	Dynamic routing to domain experts

Choose Dense for resource-constrained deployments or when uniform cross-domain performance is preferred. Choose MoE for maximum capability when hardware allows.

Generation Defaults

Parameter	Value
temperature	0.6
top_p	0.9
repetition_penalty	1.1
max_new_tokens	8192

Build Process

1. Four 8B-parameter domain-specialized models were merged using DARE+TIES with density 0.75 and normalization enabled
2. Weight distribution was biased toward financial domains (marketing 0.30, forecasting 0.25, trends 0.20, reasoning 0.25)
3. Context window was extended to 128K via YaRN RoPE scaling
4. Tool-use chat template was injected for structured function calling
5. GGUF quantization ladder was generated using llama.cpp with CUDA acceleration

Related Models

• uaytug/fumea-f-dense-gguf — Quantized GGUF versions of this model
• uaytug/fumea-f — 24B MoE version (4 experts, top-2 routing)
• uaytug/fumea-f-gguf — Quantized GGUF versions of the MoE model

Limitations

• This is a merged model, not fine-tuned on curated financial datasets. Output quality reflects the combined capabilities of the source models.
• Financial predictions and analysis should not be used as sole basis for investment decisions.
• Tool-use capability depends on the inference framework supporting structured function calling.
• Extended context performance has not been systematically benchmarked beyond the base model's tested range.
• As a dense merge, domain-specific performance may be diluted compared to the individual specialist models or the MoE variant.

License

Apache 2.0

Citation

@misc{FUMEA-F-Dense,
  author = {Umut Aytuğ Semerci},
  title = {FUMEA-F Dense: Dense model of Financial Frontier Unified Multi-Expert Agent},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/uaytug/fumea-f-dense}
}