upload_instructions.md

# Hugging Face Hub Upload Instructions — CortexFM

Step-by-step instructions to push the CortexFM checkpoint and model card to the Hugging Face Hub after you obtain a user access token.

## 0. Prerequisites

- An active Hugging Face account at <https://huggingface.co>.
- A **write-scope** user access token from <https://huggingface.co/settings/tokens>.
- WSL Ubuntu 24.04 (or any POSIX shell) with `git` and `python` ≥ 3.10.
- Locally available checkpoint at `~/cortex-fm/runs/pretrain_v1/ckpt/epoch28-0.2599.ckpt` (58 MB; LaTeX reports 60.7 MB — both refer to the same file, MB vs MiB rounding).
- This `src/release/hf/` directory containing `README.md`, `model_card.md`, `LICENSE`, `.gitattributes`, and `upload_instructions.md`.

## 1. Install Hugging Face CLI and Git LFS

```bash
pip install --upgrade "huggingface_hub[cli]"
sudo apt-get install -y git-lfs   # or: brew install git-lfs (macOS)
git lfs install --skip-repo       # initialize LFS hooks for the user
```

Verify:

```bash
huggingface-cli --version   # expect: huggingface_hub version ...
git lfs version             # expect: git-lfs/3.x ...
```

## 2. Authenticate

```bash
huggingface-cli login
```

When prompted, paste the **write-scope** token. Choose `n` when asked "Add token as git credential" if you prefer not to store it in `~/.git-credentials`; in that case you will be prompted to enter the token again at the first `git push`.

## 3. Create the model repository

**Option A — via CLI:**

```bash
huggingface-cli repo create CortexFM --type model
# repo URL appears as: https://huggingface.co/<your-username>/CortexFM
```

**Option B — via web UI:** Go to <https://huggingface.co/new>, choose "Model", name it `CortexFM`, leave the rest at defaults, and click "Create model".

Replace `<USERNAME>` in the next step with your actual Hugging Face username (or organization).

## 4. Clone the empty repository

```bash
cd ~
git clone https://huggingface.co/<USERNAME>/CortexFM
cd CortexFM
```

If you skipped the git-credential store in step 2, git will now prompt for your username and the token (used as the password).

## 5. Configure LFS tracking

```bash
git lfs install
git lfs track "*.ckpt"
git lfs track "*.pt"
git lfs track "*.safetensors"
# .gitattributes is now updated with the LFS patterns
```

Or simply copy the prepared `.gitattributes` from this `hf/` directory in step 6.

## 6. Copy release files into the repository

Assuming this project lives at `/mnt/e/projects/school\ 1/` (typical WSL mount of `E:\projects\school 1\`):

```bash
HF_SRC="/mnt/e/projects/school 1/src/release/hf"
cp "$HF_SRC/README.md"            .
cp "$HF_SRC/model_card.md"        .
cp "$HF_SRC/LICENSE"              .
cp "$HF_SRC/.gitattributes"       .
cp "$HF_SRC/upload_instructions.md" .   # optional — uploaders may keep this for reference

# Optional FALCON wrapper reference
mkdir -p benchmark_wrapper
cp "$HF_SRC/benchmark_wrapper/"*.py benchmark_wrapper/ 2>/dev/null || true
```

## 7. Copy the checkpoint

```bash
cp ~/cortex-fm/runs/pretrain_v1/ckpt/epoch28-0.2599.ckpt .
```

Verify the file is LFS-tracked **before** committing:

```bash
git lfs status
# expect: "epoch28-0.2599.ckpt" listed under "Objects to be committed"
git check-attr filter epoch28-0.2599.ckpt
# expect: epoch28-0.2599.ckpt: filter: lfs
```

If the file is **not** showing `filter: lfs`, re-run `git lfs track "*.ckpt"` and `git add .gitattributes` before adding the checkpoint.

## 8. Compute and record the checksum (recommended)

```bash
sha256sum epoch28-0.2599.ckpt > epoch28-0.2599.ckpt.sha256
```

Add the hash to the model card or commit it alongside the weights for downstream integrity verification.

## 9. Commit and push

```bash
git add .gitattributes README.md model_card.md LICENSE
git add epoch28-0.2599.ckpt epoch28-0.2599.ckpt.sha256
git add upload_instructions.md 2>/dev/null || true
git add benchmark_wrapper/      2>/dev/null || true

git commit -m "Initial CortexFM release (pretrain_v1, epoch28-0.2599.ckpt)"
git push origin main
```

The first push uploads the checkpoint to LFS storage; expect a few seconds to a few minutes depending on bandwidth.

## 10. Verify on the Hub

Open `https://huggingface.co/<USERNAME>/CortexFM` in a browser:

- The model card (README.md) should render with the YAML frontmatter parsed into a sidebar (tags, license, datasets, metrics).
- The checkpoint should show "Stored with Git LFS" and a download icon.
- The `LICENSE` file should be linked from the sidebar.
- Tags `brain-computer-interface`, `foundation-model`, `neural-decoding`, `transformer`, `cross-modal` should appear.

## 11. (Optional) Smoke-test the public download

From a clean directory:

```bash
pip install huggingface_hub
python - <<'PY'
from huggingface_hub import hf_hub_download
path = hf_hub_download(repo_id="<USERNAME>/CortexFM", filename="epoch28-0.2599.ckpt")
print("Downloaded to:", path)

# Optional: load the Lightning module if cortex_fm is on PYTHONPATH
# import torch
# from cortex_fm.training import CortexFMPretrainModule
# m = CortexFMPretrainModule.load_from_checkpoint(path, map_location="cpu", strict=True)
# print("Params:", sum(p.numel() for p in m.parameters()))   # expect 5,044,994
PY
```

## 12. Post-upload housekeeping

- Add the Hugging Face Hub link to the thesis Chapter 5 §5.5 "Public checkpoint" section, replacing the DOI placeholder `10.xxxx/cortexfm-pretrain-v1`.
- Mirror to Zenodo for a permanent DOI if desired (Hugging Face → Zenodo "Mint a DOI" feature is automated for organizations only; for personal accounts, manually upload to <https://zenodo.org/deposit/new>).
- Announce the release (optional): tag the Hugging Face card on `r/MachineLearning`, post on the FALCON Discord, or link from the project's GitHub.

---

## Troubleshooting

**Checkpoint pushed but shows as raw file (not LFS):**
You added the checkpoint before configuring LFS tracking. Run:

```bash
git rm --cached epoch28-0.2599.ckpt
git lfs track "*.ckpt"
git add .gitattributes epoch28-0.2599.ckpt
git commit -m "Move checkpoint to Git LFS"
git push origin main
```

**`git push` rejected with "this repository is over its data quota":**
Hugging Face free accounts have generous quotas but very large checkpoints may need plan upgrades or splitting. CortexFM at 60.7 MB is well within all free-tier limits.

**`huggingface-cli login` fails with `401 Unauthorized`:**
The token must have **write** scope. Re-issue at <https://huggingface.co/settings/tokens> with role = "Write".

**LFS object missing on download:**
Run `git lfs fetch && git lfs checkout` inside the clone. If the issue persists, check the repository's LFS files page on the Hub.

---

## Summary checklist

- [ ] HF account + write token
- [ ] `huggingface-cli login` succeeds
- [ ] Repository `CortexFM` created on Hub
- [ ] Local clone has `.gitattributes` with `*.ckpt filter=lfs ...`
- [ ] `README.md`, `model_card.md`, `LICENSE` copied
- [ ] Checkpoint copied and `git check-attr filter` reports `lfs`
- [ ] SHA-256 checksum recorded (recommended)
- [ ] Initial commit pushed
- [ ] Browser verification: card renders, weights downloadable, license visible
- [ ] Thesis Chapter 5 §5.5 DOI placeholder updated with HF URL