Hugging Face's logo

newempire1101
/
CortexFM

Feature Extraction
PyTorch
English
Korean
brain-computer-interface
foundation-model
neural-decoding
transformer
cross-modal
bci
electrophysiology
emg
spike-trains
Model card Files
xet
 Community
CortexFM / upload_instructions.md
newempire1101's picture
newempire1101
Add upload_instructions.md
f312faa verified
preview code
|
Raw
History Blame Contribute Delete
7.21 kB

Hugging Face Hub Upload Instructions — CortexFM

Step-by-step instructions to push the CortexFM checkpoint and model card to the Hugging Face Hub after you obtain a user access token.

0. Prerequisites

  • An active Hugging Face account at https://huggingface.co.
  • A write-scope user access token from https://huggingface.co/settings/tokens.
  • WSL Ubuntu 24.04 (or any POSIX shell) with git and python ≥ 3.10.
  • Locally available checkpoint at ~/cortex-fm/runs/pretrain_v1/ckpt/epoch28-0.2599.ckpt (58 MB; LaTeX reports 60.7 MB — both refer to the same file, MB vs MiB rounding).
  • This src/release/hf/ directory containing README.md, model_card.md, LICENSE, .gitattributes, and upload_instructions.md.

1. Install Hugging Face CLI and Git LFS 

pip install --upgrade "huggingface_hub[cli]"
sudo apt-get install -y git-lfs   # or: brew install git-lfs (macOS)
git lfs install --skip-repo       # initialize LFS hooks for the user

Verify:

huggingface-cli --version   # expect: huggingface_hub version ...
git lfs version             # expect: git-lfs/3.x ...

2. Authenticate 

huggingface-cli login

When prompted, paste the write-scope token. Choose n when asked "Add token as git credential" if you prefer not to store it in ~/.git-credentials; in that case you will be prompted to enter the token again at the first git push.

3. Create the model repository

Option A — via CLI:

huggingface-cli repo create CortexFM --type model
# repo URL appears as: https://huggingface.co/<your-username>/CortexFM

Option B — via web UI: Go to https://huggingface.co/new, choose "Model", name it CortexFM, leave the rest at defaults, and click "Create model".

Replace <USERNAME> in the next step with your actual Hugging Face username (or organization).

4. Clone the empty repository 

cd ~
git clone https://huggingface.co/<USERNAME>/CortexFM
cd CortexFM

If you skipped the git-credential store in step 2, git will now prompt for your username and the token (used as the password).

5. Configure LFS tracking 

git lfs install
git lfs track "*.ckpt"
git lfs track "*.pt"
git lfs track "*.safetensors"
# .gitattributes is now updated with the LFS patterns

Or simply copy the prepared .gitattributes from this hf/ directory in step 6.

6. Copy release files into the repository

Assuming this project lives at /mnt/e/projects/school\ 1/ (typical WSL mount of E:\projects\school 1\):

HF_SRC="/mnt/e/projects/school 1/src/release/hf"
cp "$HF_SRC/README.md"            .
cp "$HF_SRC/model_card.md"        .
cp "$HF_SRC/LICENSE"              .
cp "$HF_SRC/.gitattributes"       .
cp "$HF_SRC/upload_instructions.md" .   # optional — uploaders may keep this for reference

# Optional FALCON wrapper reference
mkdir -p benchmark_wrapper
cp "$HF_SRC/benchmark_wrapper/"*.py benchmark_wrapper/ 2>/dev/null || true

7. Copy the checkpoint 

cp ~/cortex-fm/runs/pretrain_v1/ckpt/epoch28-0.2599.ckpt .

Verify the file is LFS-tracked before committing:

git lfs status
# expect: "epoch28-0.2599.ckpt" listed under "Objects to be committed"
git check-attr filter epoch28-0.2599.ckpt
# expect: epoch28-0.2599.ckpt: filter: lfs

If the file is not showing filter: lfs, re-run git lfs track "*.ckpt" and git add .gitattributes before adding the checkpoint.

8. Compute and record the checksum (recommended) 

sha256sum epoch28-0.2599.ckpt > epoch28-0.2599.ckpt.sha256

Add the hash to the model card or commit it alongside the weights for downstream integrity verification.

9. Commit and push 

git add .gitattributes README.md model_card.md LICENSE
git add epoch28-0.2599.ckpt epoch28-0.2599.ckpt.sha256
git add upload_instructions.md 2>/dev/null || true
git add benchmark_wrapper/      2>/dev/null || true

git commit -m "Initial CortexFM release (pretrain_v1, epoch28-0.2599.ckpt)"
git push origin main

The first push uploads the checkpoint to LFS storage; expect a few seconds to a few minutes depending on bandwidth.

10. Verify on the Hub

Open https://huggingface.co/<USERNAME>/CortexFM in a browser:

  • The model card (README.md) should render with the YAML frontmatter parsed into a sidebar (tags, license, datasets, metrics).
  • The checkpoint should show "Stored with Git LFS" and a download icon.
  • The LICENSE file should be linked from the sidebar.
  • Tags brain-computer-interface, foundation-model, neural-decoding, transformer, cross-modal should appear.

11. (Optional) Smoke-test the public download

From a clean directory:

pip install huggingface_hub
python - <<'PY'
from huggingface_hub import hf_hub_download
path = hf_hub_download(repo_id="<USERNAME>/CortexFM", filename="epoch28-0.2599.ckpt")
print("Downloaded to:", path)

# Optional: load the Lightning module if cortex_fm is on PYTHONPATH
# import torch
# from cortex_fm.training import CortexFMPretrainModule
# m = CortexFMPretrainModule.load_from_checkpoint(path, map_location="cpu", strict=True)
# print("Params:", sum(p.numel() for p in m.parameters()))   # expect 5,044,994
PY

12. Post-upload housekeeping

  • Add the Hugging Face Hub link to the thesis Chapter 5 §5.5 "Public checkpoint" section, replacing the DOI placeholder 10.xxxx/cortexfm-pretrain-v1.
  • Mirror to Zenodo for a permanent DOI if desired (Hugging Face → Zenodo "Mint a DOI" feature is automated for organizations only; for personal accounts, manually upload to https://zenodo.org/deposit/new).
  • Announce the release (optional): tag the Hugging Face card on r/MachineLearning, post on the FALCON Discord, or link from the project's GitHub.

Troubleshooting

Checkpoint pushed but shows as raw file (not LFS): You added the checkpoint before configuring LFS tracking. Run:

git rm --cached epoch28-0.2599.ckpt
git lfs track "*.ckpt"
git add .gitattributes epoch28-0.2599.ckpt
git commit -m "Move checkpoint to Git LFS"
git push origin main

git push rejected with "this repository is over its data quota": Hugging Face free accounts have generous quotas but very large checkpoints may need plan upgrades or splitting. CortexFM at 60.7 MB is well within all free-tier limits.

huggingface-cli login fails with 401 Unauthorized: The token must have write scope. Re-issue at https://huggingface.co/settings/tokens with role = "Write".

LFS object missing on download: Run git lfs fetch && git lfs checkout inside the clone. If the issue persists, check the repository's LFS files page on the Hub.

Summary checklist

  • HF account + write token
  • huggingface-cli login succeeds
  • Repository CortexFM created on Hub
  • Local clone has .gitattributes with *.ckpt filter=lfs ...
  • README.md, model_card.md, LICENSE copied
  • Checkpoint copied and git check-attr filter reports lfs
  • SHA-256 checksum recorded (recommended)
  • Initial commit pushed
  • Browser verification: card renders, weights downloadable, license visible
  • Thesis Chapter 5 §5.5 DOI placeholder updated with HF URL