| --- |
| language: |
| - en |
| - zh |
| license: apache-2.0 |
| library_name: diffusers |
| pipeline_tag: text-to-image |
| tags: |
| - unidflow-conti |
| - flow-matching |
| - text-to-image |
| --- |
| |
| # FlowShift |
|
|
| **From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models with FlowShift** |
|
|
| **Accepted at WACV 2027** |
|
|
| Gayatri Deshmukh, Onkar Susladkar, Halil Kilicoglu, and Ismini Lourentzou |
|
|
| Publication metadata is supplied by the authors; DOI and page numbers are pending. |
|
|
| [Inference code](https://github.com/PLAN-Lab/UniDFlowConti) |
|
|
| ## Checkpoint scope |
|
|
| These are converted pretrained text-to-image inference components: a local |
| mixture-of-experts text feature encoder, learned queries, text projection, |
| continuous-flow denoiser, tokenizer, scheduler, and Flux2 VAE. |
|
|
| Checkpoint preparation removes unused parameters and updates architecture |
| metadata. These files have undergone no additional training or FlowShift |
| adaptation. Original checkpoint provenance and required third-party notices |
| are recorded in [NOTICE](NOTICE). The VAE architecture is unchanged. |
|
|
| The tensor files occupy approximately 45.17 GB in BF16. Use the accompanying |
| `unidflow_conti` package to load them; a generic Diffusers pipeline resolver |
| alone does not provide the local model classes. |
|
|
| ## Inference |
|
|
| Install the code repository, then download the checkpoint using an account |
| that has access to this private model: |
|
|
| ```bash |
| python -m pip install -e /path/to/UniDFlowConti |
| unidflow-download --repo onkarsus13/UniDFlowConti \ |
| --cache-dir /data/onkar2/hf --output /data/onkar2/hf/UniDFlowConti |
| ``` |
|
|
| ```python |
| import torch |
| from unidflow_conti import UniDFlowContiPipeline |
| |
| pipeline = UniDFlowContiPipeline.from_pretrained( |
| "/data/onkar2/hf/UniDFlowConti", torch_dtype=torch.bfloat16, device="cuda:1", |
| ) |
| image = pipeline( |
| "A cinematic photograph of a red fox standing in fresh snow", |
| height=1024, width=1024, num_inference_steps=50, guidance_scale=5, |
| generator=torch.Generator("cuda:1").manual_seed(42), |
| ).images[0] |
| image.save("fox.png") |
| ``` |
|
|
| Set `cpu_offload=True` when loading for reduced GPU memory use. Image dimensions |
| must be positive multiples of 16. The Python implementation is loaded from the |
| installed package; no remote checkpoint Python is executed. |
|
|
| ## Validation |
|
|
| Two complete 50-step GPU runs passed: 1024 × 1024 generation and 1024 × 768 |
| with CPU offload and negative prompting. The code repository includes eight |
| passing automated tests and a validation report. |
|
|
|  |
|
|
|  |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{deshmukh2027flowshift, |
| title = {From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models with {FlowShift}}, |
| author = {Deshmukh, Gayatri and Susladkar, Onkar and Kilicoglu, Halil and Lourentzou, Ismini}, |
| booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, |
| year = {2027} |
| } |
| ``` |
|
|
| ## License |
|
|
| [Apache License 2.0](LICENSE). Attribution is retained in [NOTICE](NOTICE). |
|
|
