Quazim0t0 commited on
Commit
915f224
ยท
verified ยท
1 Parent(s): e343b1c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +67 -1
README.md CHANGED
@@ -7,4 +7,70 @@ sdk: static
7
  pinned: false
8
  ---
9
 
10
- Edit this `README.md` markdown file to author your organization card.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  pinned: false
8
  ---
9
 
10
+ # ๐ŸŒผ DaisyChainAI
11
+
12
+ **Modular minds, not monoliths.** We build capable systems by *daisy-chaining* a handful of
13
+ small, sharp specialists behind a learned router โ€” instead of training one giant model to do
14
+ everything. Each specialist is cheap, swappable, and crisp on its own domain; chained together,
15
+ they behave like one model at a fraction of the active compute.
16
+
17
+ ---
18
+
19
+ ## ๐Ÿ”— What "daisy-chaining" means
20
+
21
+ A **daisy chain** links independent units in series so a signal can flow from one to the next,
22
+ each unit handling what it's good at and passing the rest along. That's exactly how our systems work:
23
+
24
+ - **Each link is one small specialist** โ€” a dense ~74M model trained on a *single* domain. It is
25
+ excellent at its own data and (deliberately) surprised by everything else.
26
+ - **The router is the connector between links.** When an input arrives, it travels down the chain:
27
+ every specialist reports how *surprised* it is (bits/base) and exposes its hidden state, and a tiny
28
+ learned router hands the work to the link that's most at home with it.
29
+ - **The chain grows link by link.** Because the specialists are trained *separately*, you can chain a
30
+ new domain on without retraining the others โ€” add a link, extend the router, done. Remove or upgrade
31
+ a single link the same way.
32
+ - **One link runs per query.** Only the routed specialist computes, so a chain of four ~74M experts
33
+ costs ~74M of compute per token โ€” roughly **7ร— cheaper** than a 500M monolith of comparable scope.
34
+
35
+ So "DaisyChain" is both the brand and the mechanism: **a chain of specialists, connected by routing,
36
+ that you extend one flower at a time.**
37
+
38
+ ---
39
+
40
+ ## ๐Ÿ› ๏ธ How the models are built
41
+
42
+ Each specialist is grown by **interleaving two steps**, per domain:
43
+
44
+ 1. **Continued pretraining** โ€” next-token training on *only* that domain's data, so the specialist
45
+ becomes genuinely crisp on its home distribution (and the router can tell the links apart).
46
+ 2. **Per-domain distillation** โ€” the specialist is distilled from a larger teacher foundation model
47
+ *restricted to its own domain* (soft-target KD, plus a factorized per-nucleotide variant where the
48
+ teacher supports it). It learns the teacher's behavior on its slice without ever becoming a generic
49
+ clone โ€” the specialization is what makes routing work.
50
+
51
+ We iterate those two steps until each link is as strong as its capacity allows, then train the
52
+ **router**: a small head that reads every specialist's surprise plus a compressed view of its hidden
53
+ state and predicts the home domain โ€” recovering bias-corrections a plain "lowest-perplexity-wins" rule
54
+ misses.
55
+
56
+ This is, in lineage, a **cluster Branch-Train-Merge (cBTM) mixture of domain experts** โ€” independent
57
+ experts + perplexity-aware routing โ€” with iterative distillation from a larger teacher layered on top.
58
+
59
+ ---
60
+
61
+ ## ๐Ÿงฌ Current project โ€” DaisyChain Genomics
62
+
63
+ Four DNA/RNA specialists (**eukaryote ยท prokaryote ยท mRNA ยท mRNA-splice**, ~74M each, **โ‰ˆ295M total โ€”
64
+ under 500M**), each distilled per-domain from a 500M genomic foundation model, behind a learned router.
65
+
66
+ | | |
67
+ |---|---|
68
+ | **Routing accuracy** (held-out) | **94.8%** |
69
+ | **Active params / query** | ~74M (one specialist) |
70
+ | **vs the 500M teacher** | within ~6% likelihood; closing with training |
71
+
72
+ - ๐Ÿ“ฆ **Model:** [`DaisyChainAI/daisychain-genomics`](https://huggingface.co/DaisyChainAI/daisychain-genomics)
73
+ - ๐ŸŽฎ **Live demo:** [`Daisychain-Genomics-Demo`](https://huggingface.co/spaces/DaisyChainAI/Daisychain-Genomics-Demo) โ€” paste DNA, watch the chain light up specialist-by-specialist and route in real time.
74
+
75
+ More links on the chain โ€” and more chains โ€” coming. ๐ŸŒผ
76
+