---
license: mit
base_model: hpcai-tech/openmoe-base
tags:
  - finance
  - mixture-of-experts
  - openmoe
  - umt5
language:
  - en
---

# Meridian.AI

Meridian.AI is an experimental finance-focused sparse Mixture-of-Experts (MoE) causal language model trained via continual updates.

## Intended use

Use this model for:

- Financial Q&A style prompting
- Basic quantitative finance explanations
- Summarization/classification-style finance text tasks

## Base model + tokenizer

- **Base model weights**: `hpcai-tech/openmoe-base`
- **Tokenizer**: `google/umt5-small` (256k vocab, SentencePiece/umT5)

This repo includes a working umT5 tokenizer at the root so `AutoTokenizer.from_pretrained("MeridianAlgo/MeridianAI")` works.

## How to use (Transformers)

The model weights are stored under the `checkpoint/` subfolder.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "MeridianAlgo/MeridianAI"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    subfolder="checkpoint",
    trust_remote_code=True,
    torch_dtype=torch.float32,
    low_cpu_mem_usage=True,
    ignore_mismatched_sizes=True,
)
model.eval()

prompt = """### Instruction:
Explain what a P/E ratio is and how it is used.

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(
    **inputs,
    max_new_tokens=128,
    do_sample=True,
    temperature=0.85,
    top_p=0.92,
    repetition_penalty=1.25,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id,
)

print(tokenizer.decode(out[0], skip_special_tokens=True))
```

## Training data

Training uses streaming mixes of finance datasets (FinanceMTEB family) plus optional larger corpora depending on environment configuration.

## Limitations

- This is a continually trained experimental model and may exhibit repetition.
- Not financial advice.
- Outputs may be incorrect or outdated.

## Source code

Training pipeline and scripts live in the GitHub repo: https://github.com/MeridianAlgo/FinAI