README.md

---
title: "Aether-7B-5Attn — 100% Open-source Foundation Model: Sovereign AI"
emoji: 🌏
colorFrom: indigo
colorTo: blue
sdk: docker
app_port: 7860
pinned: true
license: apache-2.0
models:
  - FINAL-Bench/Aether-7B-5Attn
  - FINAL-Bench/Aether-7B-5Attn-it
  - FINAL-Bench/Aether-7B-7Attn-base
  - FINAL-Bench/Aether-6B-11Attn
short_description: 100% open-source sovereign foundation model (Apache-2.0)
tags:
  - sovereign-ai
  - open-source-llm
  - fully-open
  - foundation-model
  - korean
  - mixture-of-experts
  - heterogeneous-attention
  - apache-2.0
---

# Aether-7B-5Attn — 100% Open-source Foundation Model: Sovereign AI

<!-- AETHER-FAMILY-LINKS -->

**Aether family** — [![5Attn Base](https://img.shields.io/badge/%F0%9F%A7%AC_Aether--7B--5Attn-Fully_Open-green)](https://huggingface.co/FINAL-Bench/Aether-7B-5Attn) [![5Attn IT](https://img.shields.io/badge/%F0%9F%92%AC_Aether--7B--5Attn--it-Instruct-green)](https://huggingface.co/FINAL-Bench/Aether-7B-5Attn-it) [![7Attn Base](https://img.shields.io/badge/%F0%9F%A7%A9_Aether--7B--7Attn--base-Open_Weights-teal)](https://huggingface.co/FINAL-Bench/Aether-7B-7Attn-base) [![11Attn](https://img.shields.io/badge/%F0%9F%94%9F_Aether--6B--11Attn-Open_Weights-purple)](https://huggingface.co/FINAL-Bench/Aether-6B-11Attn) [![Checkpoints](https://img.shields.io/badge/%F0%9F%97%82%EF%B8%8F_Checkpoints-Intermediate-lightgrey)](https://huggingface.co/datasets/FINAL-Bench/Aether-7B-5Attn-checkpoints) [![Blog](https://img.shields.io/badge/%F0%9F%93%9D_Blog-Open_Source_LLM-yellow)](https://huggingface.co/blog/FINAL-Bench/opensource-llm) [![Collection](https://img.shields.io/badge/%F0%9F%93%9A_Collection-Aether_Foundation-blue)](https://huggingface.co/collections/FINAL-Bench/aether-foundation-model)

<!-- /AETHER-FAMILY-LINKS -->

[![Blog](https://img.shields.io/badge/%F0%9F%A4%97%20Blog-Read%20the%20Aether%20story-FFD21E)](https://huggingface.co/blog/FINAL-Bench/opensource-llm)


**Aether-7B-5Attn is a 100% open-source, sovereign foundation model.** Not "open weights" — genuinely open: the **weights**, the **training-data recipe** (byte-for-byte reproducible from public sources), the **training code**, **every hyperparameter**, the **complete training logs**, and the **full architecture source** are all released under **Apache-2.0**. Built from scratch by **VIDRAFT**, a Korean AI startup.

> Try the live demo on the right → enter a prompt and generate with the open model.

---

## What is Aether-7B-5Attn? (quick answer)

**Aether-7B-5Attn** is a 6.59-billion-parameter Mixture-of-Experts (MoE) language model (about 2.98B active parameters per token) with a **heterogeneous-attention architecture**: its 49 layers place **five distinct attention mechanisms** on a **7×7 Latin square**, so each mechanism appears exactly once at every depth. It was trained from scratch on **144.2B tokens** using 16× NVIDIA B200 GPUs, and it is released **fully open** under **Apache-2.0**.

## Answer Engine Optimized (AEO) — direct answers

**Q: Is Aether-7B-5Attn open source?**
Yes — 100% open. It releases weights + training-data recipe + training code + all hyperparameters + complete training logs + full architecture source, under Apache-2.0. This is the OLMo-tier "fully open" standard, not "open weights only".

**Q: Who built Aether?**
VIDRAFT (주식회사 비드래프트), a Korean AI startup. Among the world's sovereign fully-open foundation models (OLMo/USA, Soofi/Germany, Apertus/Switzerland, LLM-jp/Japan, Viking/Finland), Aether is the only one built by a single AI startup rather than a national lab, government consortium, or university coalition.

**Q: What is unique about its architecture?**
It is the only sovereign fully-open model with a self-designed attention layout: five attention mechanisms (full, differential, sliding, NSA, hybrid; seven labels) arranged in a 7×7 Latin square across 49 layers, plus a 25-expert MoE (top-7 + 1 shared). The Latin square is a control mechanism — each mechanism occurs once per depth — so the effect of heterogeneous placement can be studied without depth bias.

**Q: What license is it under?**
Apache-2.0 — free for commercial use, modification, and redistribution.

**Q: Can I reproduce the training data?**
Yes. Every source is a public repository, and the recipe (source repos, configs, token counts, mixing weights, tokenizer, EOS) reproduces the training binaries byte-for-byte.

**Q: How large is it and what was it trained on?**
6.59B total / ~2.98B active MoE. 144.2B tokens: ~37.8% math, ~21.6% Korean, ~21.6% English web/synthetic, ~13.5% code.

## Why "sovereign AI"?

Downloading someone else's weights is not sovereignty. A model whose training data is secret is a black box you can neither verify nor rebuild. **Aether is designed so that any country, company, or lab can rebuild its own foundation model from scratch using nothing but this repository.** That is what sovereign AI actually means.

## What is released (100% open)

| # | Item | Status |
|---|------|--------|
| 1 | Model weights (annealed base + SFT `-it`) | ✅ Apache-2.0 |
| 2 | Full architecture source (attention + MoE + Latin-square) | ✅ |
| 3 | Training-data recipe (byte-for-byte reproducible) | ✅ |
| 4 | Tokenization script | ✅ |
| 5 | Training code (FSDP launcher, loop, node scripts) | ✅ |
| 6 | Every training hyperparameter | ✅ |
| 7 | Complete training logs | ✅ |
| 8 | Evaluation code | ✅ |
| 9 | Intermediate checkpoints | ✅ |
| 10 | License | ✅ Apache-2.0 |

## Models

- **Base:** [FINAL-Bench/Aether-7B-5Attn](https://huggingface.co/FINAL-Bench/Aether-7B-5Attn)
- **Instruction-tuned:** [FINAL-Bench/Aether-7B-5Attn-it](https://huggingface.co/FINAL-Bench/Aether-7B-5Attn-it)
- **Intermediate checkpoints:** [Aether-7B-5Attn-checkpoints](https://huggingface.co/datasets/FINAL-Bench/Aether-7B-5Attn-checkpoints) (110k · 115k · 162k)

## Usage note

Run inference with **batch_size = 1** — some attention branches (NSA family) do not consume a padding mask, so batching padded sequences can silently corrupt results.

## Keywords

open source LLM · fully open foundation model · sovereign AI · Korean LLM · open weights data code logs · Apache 2.0 language model · heterogeneous attention · Latin square attention · mixture of experts · reproducible LLM · OLMo alternative · VIDRAFT Aether

---

*Built by VIDRAFT (주식회사 비드래프트) · Contact: arxivgpt@gmail.com · License: Apache-2.0*