Update README.md
Browse files
README.md
CHANGED
|
@@ -1,37 +1,53 @@
|
|
| 1 |
---
|
| 2 |
title: README
|
| 3 |
-
emoji:
|
| 4 |
colorFrom: indigo
|
| 5 |
colorTo: pink
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
-
#
|
| 11 |
-
> Chase the SOTA pipeline, not the MMLU slop.
|
| 12 |
-
>
|
| 13 |
-
Meet i3, a state-of-the-are AI model that can be trained in a few hours on a NVIDIA Quadro P100 at the level of LLMs that need days of running on giant GPU farms.
|
| 14 |
|
| 15 |
-
|
| 16 |
-
- Repo to the current open-sourced code of i3: https://github.com/FlameF0X/open-i3
|
| 17 |
-
- Discord server: https://discord.gg/qtXApjpaJF
|
| 18 |
|
| 19 |
-
|
| 20 |
-
1. [ ] Train `i3-lab/i3-Ethan-it`
|
| 21 |
-
3. [ ] Train `i3-lab/i3-1B`
|
| 22 |
-
2. [ ] Train `i3-lab/i3-7B-A1.6B`
|
| 23 |
|
| 24 |
-
|
| 25 |
-
|
|
|
|
|
|
|
| 26 |
|
| 27 |
-
|
|
|
|
|
|
|
| 28 |
|
| 29 |
-
|
| 30 |
-
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
-
|
|
|
|
| 33 |
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
---
|
| 36 |
|
| 37 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
title: README
|
| 3 |
+
emoji: ⚡
|
| 4 |
colorFrom: indigo
|
| 5 |
colorTo: pink
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
+
# Welcome to i3-lab
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
+
**"Chase the SOTA pipeline, not the MMLU slop."**
|
|
|
|
|
|
|
| 13 |
|
| 14 |
+
i3-lab is dedicated to extreme efficiency in LLM architecture. We develop the **i3** model family—state-of-the-art architectures designed to reach high performance levels in hours on consumer-grade hardware (like the NVIDIA Quadro P100) that typically require days on massive GPU clusters.
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
+
## i3: High-Efficiency Training
|
| 19 |
+
We specialize in hybrid architectures, specifically **RWKV-Attention**, to bypass the quadratic scaling bottlenecks of traditional Transformers.
|
| 20 |
|
| 21 |
+
* **Fast Iteration:** Trainable in hours, not weeks.
|
| 22 |
+
* **Accessible SOTA:** High performance on legacy/mid-range hardware.
|
| 23 |
+
* **Open Research:** Push the boundaries of what is possible with limited compute.
|
| 24 |
|
| 25 |
+
### Quick Links
|
| 26 |
+
* **Source Code:** [FlameF0X/open-i3](https://github.com/FlameF0X/open-i3)
|
| 27 |
+
* **Community:** [Join our Discord](https://discord.gg/qtXApjpaJF)
|
| 28 |
+
|
| 29 |
+
---
|
| 30 |
|
| 31 |
+
## Roadmap / TODO
|
| 32 |
+
We are currently scaling our architecture through the following milestones:
|
| 33 |
|
| 34 |
+
- [ ] **i3-Ethan-it** — Specialized instruction-tuned variant.
|
| 35 |
+
- [ ] **i3-1B** — Our first major scale-up.
|
| 36 |
+
- [ ] **i3-7B-A1.6B** — Mixture of Experts / Sparsity testing.
|
| 37 |
|
| 38 |
---
|
| 39 |
|
| 40 |
+
## Usage & Attribution
|
| 41 |
+
The `open-i3` codebase is licensed under **Apache 2.0**. We believe in open-source, but we value attribution.
|
| 42 |
+
|
| 43 |
+
If you use our architecture (RWKV-Attention) or our weights, you are required per **Section 4(b)** and **4(d)** to:
|
| 44 |
+
1. Carry prominent notices of any modifications.
|
| 45 |
+
2. Include a readable copy of the attribution notices from our **NOTICE** file.
|
| 46 |
+
|
| 47 |
+
> [!IMPORTANT]
|
| 48 |
+
> You **must** include the attribution link found in the [open-i3 GitHub](https://github.com/FlameF0X/open-i3) in your documentation or model card.
|
| 49 |
+
|
| 50 |
+
---
|
| 51 |
+
<p align="center">
|
| 52 |
+
Made with ❤️ and <b>DETERMINATION</b> by Daniel.
|
| 53 |
+
</p>
|