| --- |
| license: mit |
| language: |
| - en |
| tags: |
| - peft |
| - low-rank-adaptation |
| - efficient-finetuning |
| - basis |
| --- |
| |
| # BASIS: [这里填写你的论文全名或副标题] |
|
|
| > Storage cleanup note (2026-04-14): this repository currently preserves the directory layout and model card only. Checkpoint artifacts were removed from the Hub to reclaim storage and can be re-uploaded later if needed. |
|
|
| ## 📖 Model Overview |
| This repository hosts the trained checkpoints and projection matrices for **BASIS**, an efficient and structurally unified fine-tuning framework. BASIS optimizes low-rank adaptation by aligning the trainable subspace with the intrinsic data distribution. |
|
|
| These checkpoints represent various base models adapted using the BASIS methodology, achieving competitive performance under strict rank and parameter budgets. |
|
|
| ## 🚀 Methodology Quick Glance |
| BASIS operates through a meticulously decoupled pipeline to maximize both local and global parameter efficiency: |
|
|
| 1. **Calibration-Aware Basis Construction:** We perform robust symmetric eigendecomposition on the empirical covariance of activations, scaling the base weights to construct a subspace that preserves the most critical directions. |
| 2. **Cross-Layer Rank Allocation:** Instead of uniform rank distribution, we dynamically allocate rank budgets across different layers. This is guided by Fisher information (for sensitivity) and managed via a streaming approach to ensure scalability for 70B+ models. |
| 3. **In-Subspace Adaptation:** The final fine-tuning occurs strictly within this optimized, calibration-aware subspace. |
|
|
| ## 📂 Available Checkpoints |
| *Please refer to the specific sub-directories or branches for different base models and rank configurations.* |
|
|
| | Base Model | Rank Budget | Target Task/Domain | Download Link | |
| |---|---|---|---| |
| | [e.g., Llama-2-7b] | [e.g., R=32] | [e.g., Commonsense QA] | `[Link]` | |
| | [e.g., Mistral-7B] | [e.g., R=64] | [e.g., Math Reasoning] | `[Link]` | |
|
|
| ## 💻 How to Use |
| To load and use these checkpoints for inference or further evaluation, you can use the following snippet: |
|
|
| ```python |
| # [TODO: 提供一段你们官方代码库中加载 BASIS 权重的 Python 伪代码或真实代码] |
| # Example: |
| # from basis import BASISModel |
| # model = BASISModel.from_pretrained("Zishan-Shao/BASIS", subfolder="llama-2-7b-r32") |
| |
