README.md

---
license: apache-2.0
library_name: ecoaco
pipeline_tag: text-generation
language:
  - en
tags:
  - ecoaco
  - rurtech
  - mixture-of-experts
  - moe
  - sparse-moe
  - banking
  - finance
  - fintech
  - causal-lm
  - india
model_name: ECOACO
---

# ECOACO — the rurtech.ai Mixture-of-Experts LLM for banking

**ECOACO** (Sanskrit: *wealth, prosperity, purpose*) is rurtech.ai's vertical
**Mixture-of-Experts** language model for banking and finance. It is a real,
from-scratch sparse-MoE causal transformer — each block routes every token to
the **top-2 of 8 expert FFNs**, so the model holds far more capacity than it
activates per token.

Repo id: **`rurtech010101/ECOACO`** · search Hugging Face for **“ECOACO MoE”**.

> This checkpoint is a **reference / demonstration** model trained at small scale.
> The architecture is production-faithful — scaling up is a config change
> (`dim`, `n_layers`, `n_experts`, `vocab_size`) plus a real pretraining corpus
> and GPU budget. It is **not** a production banking model and must not be used
> for real financial decisions.

## Load it in one line (Python)

```python
# pip install torch safetensors huggingface_hub
from modeling_ecoaco import EcoacoForCausalLM

model, tok = EcoacoForCausalLM.from_pretrained("rurtech010101/ECOACO")
print(model.chat(tok, "The Layer-0 action gate", max_new_tokens=60))
```

`from_pretrained` accepts a local folder or a Hugging Face repo id. The
`modeling_ecoaco.py` and `tokenizer.py` files ship in the model repo, so the
class is available wherever the weights are.

## Architecture

| | |
|---|---|
| Name | **ECOACO 1.0** |
| Type | Sparse Mixture-of-Experts, decoder-only |
| Total parameters | ~20.7M (reference checkpoint) |
| Active parameters / token | ~6.5M (top-2 of 8 experts) |
| Layers | 6 · Hidden 256 · Heads 8 |
| Experts / layer | 8, top-2 routed |
| Positional encoding | Rotary (RoPE) |
| Normalization | RMSNorm |
| FFN | SwiGLU experts |
| Router aux loss | Switch-Transformer load balancing |
| Tokenizer | Byte-level BPE (no `<unk>`) |
| Context length | 512 |
| `model_type` | `ecoaco` |

## Files

```
config.json               HF-style config (model_type: ecoaco)
model.safetensors         weights
tokenizer.json            byte-level BPE tokenizer
generation_config.json    default sampling params
modeling_ecoaco.py         the model definition (+ from_pretrained)
tokenizer.py              the tokenizer implementation
```

## Ollama

```bash
python export_gguf.py --artifacts artifacts --out ecoaco.gguf
ollama create ecoaco -f Modelfile
ollama run ecoaco
```

## Reproduce training

```bash
python train.py --steps 400        # byte-BPE + AdamW on the banking corpus
python generate.py --prompt "Safety before autonomy"
```

The corpus is the rurtech.ai platform's own documentation, so ECOACO learns to
speak about the runtime, the Layer-0 action gate, guardrails, and sovereign
deployment.

## Intended use and limitations

- **Intended:** demonstrating the ECOACO MoE architecture and the
  train → package → publish (Hugging Face / git / Ollama) pipeline; a base for
  scaled-up domain pretraining.
- **Not intended:** any real banking, credit, compliance, or customer-facing
  decision. In the rurtech.ai platform every model output flows through the
  Layer-0 action gate and human approval.

## Citation

```
@software{ecoaco_moe_2026,
  title  = {ECOACO: a Mixture-of-Experts LLM for banking},
  author = {rurtech.ai},
  year   = {2026},
  url    = {https://huggingface.co/rurtech010101/ECOACO}
}
```

## License

Apache-2.0.