---
license: apache-2.0
library_name: transformers
tags:
- distributed-training
- volunteer-computing
- mixture-of-experts
- diffusion
- llm
- agillm
---

# AGILLM4.1 — community-trained open LLM

AGILLM4.1 is an open ~1.1B-parameter Transformer trained **continuously across a volunteer compute network**.
Architecture: **DiffusionBlocks** (4 blocks × 7 layers = 28 layers), **MoE** FFN (2 experts, top-1, 4× MLP),
tied embeddings, AR/SAT/NAT heads, sub-linear attention (1024–2048 ctx). The DiffusionBlock split is what makes
distributed training/inference across ordinary internet links practical — each node owns a block.

## Join the network — contribute CPU **or** GPU

The worker is **outbound-HTTPS only and sandboxed**: it pulls a layer-block lease, trains it locally, and submits
the result to a **quarantine pool that is validated server-side** before it can touch the live checkpoint.
**No account, no SSH, no access to anyone else's machine.** Lease size **auto-adapts to your hardware** (VRAM/RAM).

```bash
git clone https://github.com/Marxist-Leninist/AGILLM4.1.git
cd AGILLM4.1
python -m venv .venv && . .venv/bin/activate
python -m pip install --upgrade pip torch     # CUDA build for GPU
python public_join/agillm41_join_worker.py \
  --coordinator-url https://join.opentransformers.online --loop
#   --device auto   (default: detects CUDA / DirectML / CPU)
#   add --device cuda to force GPU
```

A single GPU contributor outweighs dozens of CPU ones — GPUs train ~1024–2048-token context at batch 4–24
sized to their VRAM; CPUs contribute smaller blocks sized to their RAM.

## Contribution points → distributed inference

Validated contributions earn **points**, redeemable for **distributed inference of the latest model**:
- Your balance: `https://join.opentransformers.online/api/v1/points/<your-participant-id>`
- Leaderboard: `https://join.opentransformers.online/api/v1/leaderboard`
- Live network monitor (nodes / stages / economy): `https://monitor.opentransformers.online`

Points are credited **only after server-side validation** of your submitted update (finite, norm-bounded,
structurally sane); junk earns nothing and can never execute on the coordinator.

## Links
- Code + worker: https://github.com/Marxist-Leninist/AGILLM4.1
- Coordinator: https://join.opentransformers.online  ·  Monitor: https://monitor.opentransformers.online