gitmigration.md

HuggingFace Spaces Git Migration: LFS → Xet Storage

Problem

HuggingFace Spaces deprecated Git LFS and now requires Xet storage for binary files (images, audio, models, etc.).

When pushing to HF Spaces with LFS-tracked files:

remote: Your push was rejected because it contains binary files.
remote: Please use https://huggingface.co/docs/hub/xet to store binary files.

Why: HF's pre-receive hook scans git history and rejects any commits with binary files routed via LFS. It requires Xet protocol instead.

---

⚠️ Important: Problematic Commits to Clean Up

The following commits attempted to add PNG files before git-xet was properly installed and configured. These caused the rejection error:

• d9520f4 - "feat: replace Unsplash cover URLs with local assets/covers/ PNG paths"
• f423e05 - "feat: replace Unsplash cover URLs with local assets/covers/ PNG paths" (duplicate)

Status: ✅ These commits were cleaned up via git-filter-repo and PNG files were restored via proper git-xet workflow.

For future reference: Always ensure git-xet is installed and configured before committing large binary files. If this happens again:

# Remove problematic commits from history
git filter-repo --path assets/covers/ --invert-paths --force

# Restore from backup or re-add with git-xet properly configured
# Then push

---

The Solution: Use git-xet

Store large binary files in git using Xet protocol instead of deprecated LFS.

Why git-xet:

• ✅ Binary files remain in git history (version-controlled)
• ✅ Standard git workflow (no special handling)
• ✅ Automatic deduplication & fast transfers
• ✅ HF officially supports and recommends it
• ✅ Works with existing .gitattributes configuration

---

How We Fixed It (Step-by-Step)

Step 1: Verify git-xet is Installed

git xet --version
# Output: git-xet 0.2.1

If not installed:

# macOS/Linux
brew install git-xet
git xet install

# Or download from: https://github.com/huggingface/xet-core/releases

Step 2: Verify .gitattributes Configuration

Your .gitattributes should route binary files through LFS (git-xet intercepts):

*.png filter=lfs diff=lfs merge=lfs -text
*.jpg filter=lfs diff=lfs merge=lfs -text
*.jpeg filter=lfs diff=lfs merge=lfs -text
*.wav filter=lfs diff=lfs merge=lfs -text
*.mp3 filter=lfs diff=lfs merge=lfs -text

With git-xet installed, these files automatically route through Xet protocol instead of legacy LFS.

Step 3: Remove from .gitignore

If binaries are in .gitignore, remove them so they can be tracked:

# Edit .gitignore and remove any lines excluding your binaries
# Example: remove "assets/covers/" line

Step 4: Add and Commit Binaries

git add assets/covers/
git commit -m "feat: add story cover images via Xet storage"

Step 5: Push to HF Space

git push steve-hf main

What happens:

• ✅ Git detects binary files routed via filter=lfs
• ✅ git-xet intercepts and uses Xet protocol (not legacy LFS)
• ✅ Files upload via chunk-deduplication
• ✅ HF pre-receive hook accepts them (no "binary files rejected" error)

Verification:

# Verify files are tracked in git
git ls-files | grep -i png

# Verify push succeeded
git log --oneline -2

---

Key Takeaways

Aspect	Before	After
Problem	HF rejected LFS-tracked binary files	✅ Resolved with git-xet
Git history	Contained PNG LFS pointers	Properly tracked via Xet
PNG files	Attempted LFS upload (rejected)	Committed via Xet protocol
Storage backend	Legacy HF LFS (no longer supported)	HF Xet (official standard)
Push status	❌ "binary files rejected" hook error	✅ Successful push

---

Future Workflows

Adding New Binary Files

Once git-xet is installed and .gitattributes is configured, just use normal git:

git add new_file.png
git commit -m "Add new image"
git push origin main
# Files automatically routed through Xet protocol

Cloning the Repository

With git-xet installed, cloning and pulling works normally:

git clone git@hf.co:spaces/sshtel/Huggingface_Hack
cd Huggingface_Hack
# Binary files auto-downloaded via Xet deduplication

Updating Existing Binary Files

# Modify a PNG file
# (e.g., replace cover image)

git add assets/covers/changed_image.png
git commit -m "Update cover image"
git push steve-hf main
# Only changed chunks uploaded via Xet (efficient)

---

Files Changed

• Added: assets/covers/*.png (10 story cover images, 19 MB total)
  • Tracked in git via Xet protocol
  • Routed through filter=lfs in .gitattributes
• Created: gitmigration.md (this file)
• Kept: .gitattributes (LFS syntax, git-xet intercepts)
• Cleaned: .gitignore (removed assets/covers/ exclusion)

---

References

• HuggingFace Docs: Xet Storage with Git
• Git Xet Repository
• HF Docs: Upload Files
• Xet Protocol: Chunk Deduplication

Troubleshooting

"Your push was rejected because it contains binary files"

Solution: Ensure git-xet is installed and .gitattributes is configured:

git xet --version          # Verify installation
git config --global --list | grep xet   # Verify config
cat .gitattributes         # Verify routing rules

Large file uploads are slow

Solution: git-xet has adaptive concurrency. For high-bandwidth machines:

export HF_XET_HIGH_PERFORMANCE=1
git push

Need to revert git-xet

git xet uninstall
# Files in repos still work via LFS bridge (slower, but compatible)