ECOACO — the rurtech.ai Mixture-of-Experts LLM for banking

ECOACO (Sanskrit: wealth, prosperity, purpose) is rurtech.ai's vertical Mixture-of-Experts language model for banking and finance. It is a real, from-scratch sparse-MoE causal transformer — each block routes every token to the top-2 of 8 expert FFNs, so the model holds far more capacity than it activates per token.

Repo id: rurtech010101/ECOACO · search Hugging Face for “ECOACO MoE”.

This checkpoint is a reference / demonstration model trained at small scale. The architecture is production-faithful — scaling up is a config change (dim, n_layers, n_experts, vocab_size) plus a real pretraining corpus and GPU budget. It is not a production banking model and must not be used for real financial decisions.

Load it in one line (Python)

# pip install torch safetensors huggingface_hub
from modeling_ecoaco import EcoacoForCausalLM

model, tok = EcoacoForCausalLM.from_pretrained("rurtech010101/ECOACO")
print(model.chat(tok, "The Layer-0 action gate", max_new_tokens=60))

from_pretrained accepts a local folder or a Hugging Face repo id. The modeling_ecoaco.py and tokenizer.py files ship in the model repo, so the class is available wherever the weights are.

Architecture

Name	ECOACO 1.0
Type	Sparse Mixture-of-Experts, decoder-only
Total parameters	~20.7M (reference checkpoint)
Active parameters / token	~6.5M (top-2 of 8 experts)
Layers	6 · Hidden 256 · Heads 8
Experts / layer	8, top-2 routed
Positional encoding	Rotary (RoPE)
Normalization	RMSNorm
FFN	SwiGLU experts
Router aux loss	Switch-Transformer load balancing
Tokenizer	Byte-level BPE (no <unk>)
Context length	512
model_type	ecoaco

Files

config.json               HF-style config (model_type: ecoaco)
model.safetensors         weights
tokenizer.json            byte-level BPE tokenizer
generation_config.json    default sampling params
modeling_ecoaco.py         the model definition (+ from_pretrained)
tokenizer.py              the tokenizer implementation

Ollama

python export_gguf.py --artifacts artifacts --out ecoaco.gguf
ollama create ecoaco -f Modelfile
ollama run ecoaco

Reproduce training

python train.py --steps 400        # byte-BPE + AdamW on the banking corpus
python generate.py --prompt "Safety before autonomy"

The corpus is the rurtech.ai platform's own documentation, so ECOACO learns to speak about the runtime, the Layer-0 action gate, guardrails, and sovereign deployment.

Intended use and limitations

• Intended: demonstrating the ECOACO MoE architecture and the train → package → publish (Hugging Face / git / Ollama) pipeline; a base for scaled-up domain pretraining.
• Not intended: any real banking, credit, compliance, or customer-facing decision. In the rurtech.ai platform every model output flows through the Layer-0 action gate and human approval.

Citation

@software{ecoaco_moe_2026,
  title  = {ECOACO: a Mixture-of-Experts LLM for banking},
  author = {rurtech.ai},
  year   = {2026},
  url    = {https://huggingface.co/rurtech010101/ECOACO}
}

License

Apache-2.0.