README.md

---
title: Knowme
emoji: 🍄
colorFrom: red
colorTo: indigo
sdk: gradio
sdk_version: 6.16.0
python_version: '3.13'
app_file: app.py
pinned: true
license: mit
short_description: Stop losing ideas. Capture, connect, recall.
tags:
  - build-small-hackathon
  - backyard-ai
  - tiny-titan
  - nvidia-nemotron
  - off-brand
  - off-the-grid
  - field-notes
  - track:backyard
  - sponsor:nvidia
  - achievement:offgrid
  - achievement:offbrand
  - achievement:fieldnotes
---

# Mycelium — Personal Knowledge Agent

> Capture fast. Think later. Let the system surface what matters.

Mycelium is a local-first AI knowledge companion that closes the loop between saving something and actually learning from it. No more screenshot graveyards or forgotten browser tabs.


## The problem

Everyone has the same graveyard: saved links, screenshots, notes-to-self — all gone dark in a week. The capture habit exists. The recall loop doesn't.

Mycelium fixes the loop.

## What it does

- **Capture** notes, URLs, and images — each processed into a structured summary with intent classification (`learn` / `act` / `reference` / `ephemeral`) and semantic tags
- **ASK** — semantic search across your knowledge base with LLM synthesis, follow-up questions, and Feynman self-testing
- **BRIEF** — daily digest of what you saved, with synthesis across captures and a weekly thread
- **REVIEW** — spaced repetition (SM-2) targeting specific claims from your own notes, not generic flashcards
- **GRAPH** — visual map of how your ideas connect via embedding similarity

## How it works

1. You capture a note, URL, or image
2. **NVIDIA Nemotron-Mini-4B** extracts the core insight, classifies intent, generates tags and recall questions
3. **Qwen2.5-VL-7B** handles image captures — describe a whiteboard, diagram, or screenshot
4. **BGE-base-en-v1.5** embeds summaries into a 768-dim vector space
5. Related captures link automatically via cosine similarity
6. The surface engine resurfaces what you should revisit, weighted by intent and time

## Tech

- **LLM**: `nvidia/Nemotron-Mini-4B-Instruct` via HF Transformers + ZeroGPU
- **Vision**: `Qwen/Qwen2.5-VL-7B-Instruct` for image capture
- **Embeddings**: `BAAI/bge-base-en-v1.5` (768-dim, top MTEB retrieval)
- **Backend**: FastAPI + SQLite (persistent at `/data/mind.db`)
- **Frontend**: React + TypeScript + Tailwind CSS



##
# Mycelium

**A local-first personal knowledge agent that captures, connects, and surfaces what matters — when it matters.**

![Mycelium UI](assets/image.png)

🚀 **[Live demo](https://huggingface.co/spaces/build-small-hackathon/mycelium)** · 📹 **[Demo video](https://www.youtube.com/watch?v=Kr7LxRm0JBs)** · 📓 **[Field Notes](https://huggingface.co/blog/build-small-hackathon/mycelium)**

---

## The Problem

Everyone saves things. Nobody revisits them. Screenshots, bookmarks, notes-to-self — all gone dark in a week. The capture habit exists. The recall loop doesn't.

Mycelium fixes the recall loop.

---

## What It Does

- **Capture** — notes, links, images. Each processed by a local LLM into a structured summary with intent classification (`learn` / `act` / `reference` / `ephemeral`) and semantic tags.
- **ASK** — semantic search across your knowledge base with LLM synthesis, gap analysis, Feynman self-testing, and learning arc.
- **BRIEF** — daily digest with synthesis across captures and a weekly thread.
- **REVIEW** — spaced repetition (SM-2) targeting specific claims from your own notes.
- **GRAPH** — visual map of how your ideas connect via embedding similarity.

---

## Quick Start

### Prerequisites

- Python 3.11+
- Node 18+
- [LM Studio](https://lmstudio.ai) — for local LLM inference

### 1. Clone and install

```bash
git clone https://github.com/ajit3259/Mycelium
cd Mycelium
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt
```

### 2. Set up LM Studio

1. Download [LM Studio](https://lmstudio.ai)
2. Load a **chat model** — recommended: `google/gemma-3-4b-it` or any instruction-tuned model under 8B
3. Load an **embedding model** — recommended: `BAAI/bge-base-en-v1.5` (search it in LM Studio's Discover tab, download the GGUF version)
4. Start the Local Server (default: `http://localhost:1234`)


Repace LM with

Set up GGUF Llama Server

1. Start your GGUF chat model server

Example:

llama-server \
  -m model.gguf \
  --host 0.0.0.0 \
  --port 8080

API:
http://localhost:8080/v1

2. (Optional) Start an embedding model server

If embeddings are required, run a BGE embedding GGUF:

llama-server \
  -m bge-base-en-v1.5-q8_0.gguf \
  --embedding \
  --port 8081

Embedding API:
http://localhost:8081/v1

3. Update application settings

Chat API:
http://localhost:8080/v1

Embedding API:
http://localhost:8081/v1

### 3. Initialize the database

```bash
python -c "from db import init_db; init_db()"
```

Optionally seed with 22 example captures to explore the UI:
```bash
python seed.py
```

### 4. Start the backend

```bash
uvicorn main:app --port 8000
```

### 5. Build the frontend

```bash
cd frontend
npm install
npm run build   # outputs to ../static/, served by FastAPI
```

Open [http://localhost:8000](http://localhost:8000).

> **Dev mode (hot reload):** `npm run dev` in the frontend folder — runs on port 5173 with proxy to backend.

---

## Configuration

All config is in `config.py` and can be overridden with environment variables:

| Variable | Default | Description |
|---|---|---|
| `LM_STUDIO_URL` | `http://localhost:1234/v1` | LM Studio server URL |
| `LM_MODEL` | auto-detected | Chat model name (leave blank to use whatever is loaded) |
| `EMBED_MODEL` | `BAAI/bge-base-en-v1.5` | Sentence-transformer embedding model |
| `DB_PATH` | `./mind.db` | SQLite database path |
| `UPLOADS_DIR` | `./uploads` | Directory for uploaded images |

---

## How It Works

```
Capture → LLM enrichment → Embed → Connect → Surface → Review
```

1. You capture a note, URL, or image
2. LM Studio extracts summary, tags, intent, claims, and a recall question
3. `BAAI/bge-base-en-v1.5` embeds the summary into a 768-dim vector
4. Related captures link automatically via cosine similarity (rank-based, top 2)
5. Surface engine resurfaces captures by intent + recency scoring
6. SM-2 spaced repetition schedules review of what you should remember

---

## Project Structure

```
.
├── main.py          # FastAPI app — all API endpoints
├── lm.py            # Unified LLM interface (LM Studio + HF Transformers)
├── db.py            # SQLite schema + all queries
├── surface.py       # Intent-weighted surfacing + scoring
├── config.py        # Config — auto-detects HF Spaces vs local
├── seed.py          # Optional: seed 22 example captures
├── requirements.txt
├── FIELD_NOTES.md   # Build post-mortem
├── frontend/        # React + TypeScript + Vite + Tailwind CSS
│   └── src/
│       ├── App.tsx
│       ├── api.ts
│       ├── types.ts
│       └── components/
│           ├── CaptureBar.tsx      # Note / Link / Image capture
│           ├── Feed.tsx            # Recent captures
│           ├── AskScreen.tsx       # Semantic search + synthesis
│           ├── BriefScreen.tsx     # Daily digest
│           ├── ReviewScreen.tsx    # Spaced repetition
│           ├── GraphView.tsx       # Knowledge graph
│           └── Card.tsx            # Shared capture card
└── static/          # Built frontend (served by FastAPI)
```

---