| --- |
| license: apache-2.0 |
| library_name: transformers |
| tags: |
| - distributed-training |
| - volunteer-computing |
| - mixture-of-experts |
| - diffusion |
| - llm |
| - agillm |
| --- |
| |
| # AGILLM4.1 — community-trained open LLM |
|
|
| AGILLM4.1 is an open ~1.1B-parameter Transformer trained **continuously across a volunteer compute network**. |
| Architecture: **DiffusionBlocks** (4 blocks × 7 layers = 28 layers), **MoE** FFN (2 experts, top-1, 4× MLP), |
| tied embeddings, AR/SAT/NAT heads, sub-linear attention (1024–2048 ctx). The DiffusionBlock split is what makes |
| distributed training/inference across ordinary internet links practical — each node owns a block. |
|
|
| ## Join the network — contribute CPU **or** GPU |
|
|
| The worker is **outbound-HTTPS only and sandboxed**: it pulls a layer-block lease, trains it locally, and submits |
| the result to a **quarantine pool that is validated server-side** before it can touch the live checkpoint. |
| **No account, no SSH, no access to anyone else's machine.** Lease size **auto-adapts to your hardware** (VRAM/RAM). |
|
|
| ```bash |
| git clone https://github.com/Marxist-Leninist/AGILLM4.1.git |
| cd AGILLM4.1 |
| python -m venv .venv && . .venv/bin/activate |
| python -m pip install --upgrade pip torch # CUDA build for GPU |
| python public_join/agillm41_join_worker.py \ |
| --coordinator-url https://join.opentransformers.online --loop |
| # --device auto (default: detects CUDA / DirectML / CPU) |
| # add --device cuda to force GPU |
| ``` |
|
|
| A single GPU contributor outweighs dozens of CPU ones — GPUs train ~1024–2048-token context at batch 4–24 |
| sized to their VRAM; CPUs contribute smaller blocks sized to their RAM. |
|
|
| ## Contribution points → distributed inference |
|
|
| Validated contributions earn **points**, redeemable for **distributed inference of the latest model**: |
| - Your balance: `https://join.opentransformers.online/api/v1/points/<your-participant-id>` |
| - Leaderboard: `https://join.opentransformers.online/api/v1/leaderboard` |
| - Live network monitor (nodes / stages / economy): `https://monitor.opentransformers.online` |
|
|
| Points are credited **only after server-side validation** of your submitted update (finite, norm-bounded, |
| structurally sane); junk earns nothing and can never execute on the coordinator. |
|
|
| ## Links |
| - Code + worker: https://github.com/Marxist-Leninist/AGILLM4.1 |
| - Coordinator: https://join.opentransformers.online · Monitor: https://monitor.opentransformers.online |
|
|
