td-toolkit / td_fuse /__init__.py
td-builder's picture
Fixed code: vocab mismatch fix for cross-arch merging (Llama/Falcon)
5d61448 verified
"""
TD Fuse β€” Transport and Merge pipeline for Time Dilation project.
Merges 5 different-architecture 7B models into Qwen3-8B using
optimal transport (Transport and Merge, arxiv 2602.05495).
Architecture:
td_fuse/
β”œβ”€β”€ __init__.py ← This file
β”œβ”€β”€ config.py ← Model configs, merge order, hyperparameters
β”œβ”€β”€ canary.py ← Canary injection + testing ("brain surgery")
β”œβ”€β”€ transport.py ← Wrapper around official T&M code
β”œβ”€β”€ techniques.py ← Advanced techniques (Theseus, ARM, OTMF, RAM, Mergeability)
β”œβ”€β”€ merge.py ← Sequential merge orchestrator
β”œβ”€β”€ validate.py ← Post-merge validation (canary, perplexity, benchmarks)
β”œβ”€β”€ heal.py ← QLoRA healing fine-tune via Unsloth
└── run.py ← Main entry point
Usage:
python -m td_fuse.run --config default --stage all
python -m td_fuse.run --config default --stage demo # Dad demo (DeepSeek only)
"""
__version__ = "0.1.0"
__author__ = "Milan (TD Project)"