File size: 2,205 Bytes
56dbf84 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 | ---
license: mit
base_model: hpcai-tech/openmoe-base
tags:
- finance
- mixture-of-experts
- openmoe
- umt5
language:
- en
---
# Meridian.AI
Meridian.AI is an experimental finance-focused sparse Mixture-of-Experts (MoE) causal language model trained via continual updates.
## Intended use
Use this model for:
- Financial Q&A style prompting
- Basic quantitative finance explanations
- Summarization/classification-style finance text tasks
## Base model + tokenizer
- **Base model weights**: `hpcai-tech/openmoe-base`
- **Tokenizer**: `google/umt5-small` (256k vocab, SentencePiece/umT5)
This repo includes a working umT5 tokenizer at the root so `AutoTokenizer.from_pretrained("MeridianAlgo/MeridianAI")` works.
## How to use (Transformers)
The model weights are stored under the `checkpoint/` subfolder.
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "MeridianAlgo/MeridianAI"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
repo_id,
subfolder="checkpoint",
trust_remote_code=True,
torch_dtype=torch.float32,
low_cpu_mem_usage=True,
ignore_mismatched_sizes=True,
)
model.eval()
prompt = """### Instruction:
Explain what a P/E ratio is and how it is used.
### Response:
"""
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(
**inputs,
max_new_tokens=128,
do_sample=True,
temperature=0.85,
top_p=0.92,
repetition_penalty=1.25,
no_repeat_ngram_size=3,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(out[0], skip_special_tokens=True))
```
## Training data
Training uses streaming mixes of finance datasets (FinanceMTEB family) plus optional larger corpora depending on environment configuration.
## Limitations
- This is a continually trained experimental model and may exhibit repetition.
- Not financial advice.
- Outputs may be incorrect or outdated.
## Source code
Training pipeline and scripts live in the GitHub repo: https://github.com/MeridianAlgo/FinAI
|