File size: 4,848 Bytes
159b866
 
 
 
 
 
 
 
1b3a633
159b866
72e6bec
159b866
 
 
6d1e5b6
159b866
 
 
72e6bec
159b866
e1eb9ff
72e6bec
e1eb9ff
159b866
 
72e6bec
 
 
 
 
 
 
 
1b3a633
72e6bec
 
 
 
1b3a633
72e6bec
 
1b3a633
72e6bec
 
 
 
1b3a633
72e6bec
 
 
1b3a633
72e6bec
 
 
 
1b3a633
72e6bec
 
1b3a633
72e6bec
 
1b3a633
72e6bec
 
1b3a633
72e6bec
 
 
 
 
159b866
 
 
e1eb9ff
159b866
 
 
e1eb9ff
 
 
 
 
 
 
 
159b866
 
 
72e6bec
159b866
72e6bec
159b866
72e6bec
 
 
 
 
 
 
159b866
 
 
72e6bec
1b3a633
72e6bec
159b866
 
 
 
 
72e6bec
159b866
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
<div align="center">

<img src="https://raw.githubusercontent.com/flatseek/flatseek/main/logo.svg" alt="Flatseek" width="72">

# Flatseek

### Open Knowledge Infrastructure for AI

**Search. Retrieve. Run. Train.**

Open-source infrastructure for building search systems, AI applications, and language models from your own knowledge.

</div>

---

## About

Flatseek is an open-source ecosystem for modern AI infrastructure.

Instead of treating search, retrieval, model training, inference, and model storage as isolated problems, the ecosystem provides modular projects that work independently or together as a complete AI pipeline.

Index your knowledge once, then reuse it across search, semantic retrieval, Retrieval-Augmented Generation (RAG), model training, inference, analytics, and deployment.

```text
                        Knowledge
                            β”‚
        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
        β”‚                                       β”‚
        β–Ό                                       β–Ό

    Flatseek                              Flatvec
 Keyword Search                       Semantic Search

        β”‚                                       β”‚
        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                            β”‚
                            β–Ό

                        Flatask
                       RAG Runtime

                            β”‚
        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
        β”‚                                       β”‚
        β–Ό                                       β–Ό

      Flattune                           Flatbuild
 Dataset Generation                  Train Models
  & Fine-tuning                    From Scratch

        β”‚                                       β”‚
        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                            β”‚
                            β–Ό

                       Flatweight
                    Model Storage Layer

                            β”‚
                            β–Ό

                        Flatrun
                   Streaming Inference

                            β”‚
                            β–Ό

                        Flatlens
                  Analytics & Exploration
```

---
## Projects

| Project | Description |
|---------|-------------|
| **[Flatseek](https://github.com/flatseek/flatseek)** | Disk-first full-text search engine for portable datasets and production-scale search. An alternative to Elasticsearch, OpenSearch, and Meilisearch. |
| **[Flatvec](https://github.com/flatseek/flatvec)** | Disk-first vector search engine for semantic retrieval and RAG. An alternative to FAISS, Milvus, Pinecone, Qdrant, and Chroma. |
| **[Flatask](https://github.com/flatseek/flatask)** | Retrieval-Augmented Generation (RAG) runtime that connects keyword search, semantic search, and language models into a unified retrieval pipeline. |
| **[Flattune](https://github.com/flatseek/flattune)** | Dataset generation and fine-tuning pipeline for creating high-quality training corpora from existing knowledge. |
| **[Flatbuild](https://github.com/flatseek/flatbuild)** | Framework for training transformer language models from scratch, including tokenizer training, chat templates, checkpoint export, and GGUF conversion. |
| **[Flatweight](https://github.com/flatseek/flatweight)** | Storage-native model format and tooling for organizing, converting, and managing model weights for efficient deployment and streaming inference. |
| **[Flatrun](https://github.com/flatseek/flatrun)** | Streaming inference runtime that executes GGUF, SafeTensors, and MLX models directly from storage with minimal memory usage. |
| **[Flatlens](https://github.com/flatseek/flatlens)** | Analytics and visualization interface for exploring search indexes, datasets, retrieval pipelines, and AI workflows. |

---

# Design Principles

The Flat ecosystem is built around a few simple ideas:

- Build knowledge once and reuse it everywhere.
- Keep data portable and storage-native.
- Prefer open formats over proprietary services.
- Make every project useful on its own.
- Compose projects into larger AI workflows when needed.
- Train, deploy, and share models with reproducible pipelines.
- Run efficiently on consumer hardware whenever possible.

---

# License

Unless otherwise noted, projects and published artifacts are released under the Apache License 2.0.

---

<div align="center">

### Build once. Search anywhere. Train from scratch. Run everywhere.

</div>