Update README.md

e937d3c verified 14 days ago

3.07 kB

language:
  - en
  - zh
license: apache-2.0
library_name: diffusers
pipeline_tag: text-to-image
tags:
  - unidflow-conti
  - flow-matching
  - text-to-image

FlowShift

From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models with FlowShift

Accepted at WACV 2027

Gayatri Deshmukh, Onkar Susladkar, Halil Kilicoglu, and Ismini Lourentzou

Publication metadata is supplied by the authors; DOI and page numbers are pending.

Inference code

Checkpoint scope

These are converted pretrained text-to-image inference components: a local mixture-of-experts text feature encoder, learned queries, text projection, continuous-flow denoiser, tokenizer, scheduler, and Flux2 VAE.

Checkpoint preparation removes unused parameters and updates architecture metadata. These files have undergone no additional training or FlowShift adaptation. Original checkpoint provenance and required third-party notices are recorded in NOTICE. The VAE architecture is unchanged.

The tensor files occupy approximately 45.17 GB in BF16. Use the accompanying unidflow_conti package to load them; a generic Diffusers pipeline resolver alone does not provide the local model classes.

Inference

Install the code repository, then download the checkpoint using an account that has access to this private model:

python -m pip install -e /path/to/UniDFlowConti
unidflow-download --repo onkarsus13/UniDFlowConti \
  --cache-dir /data/onkar2/hf --output /data/onkar2/hf/UniDFlowConti

import torch
from unidflow_conti import UniDFlowContiPipeline

pipeline = UniDFlowContiPipeline.from_pretrained(
    "/data/onkar2/hf/UniDFlowConti", torch_dtype=torch.bfloat16, device="cuda:1",
)
image = pipeline(
    "A cinematic photograph of a red fox standing in fresh snow",
    height=1024, width=1024, num_inference_steps=50, guidance_scale=5,
    generator=torch.Generator("cuda:1").manual_seed(42),
).images[0]
image.save("fox.png")

Set cpu_offload=True when loading for reduced GPU memory use. Image dimensions must be positive multiples of 16. The Python implementation is loaded from the installed package; no remote checkpoint Python is executed.

Validation

Two complete 50-step GPU runs passed: 1024 × 1024 generation and 1024 × 768 with CPU offload and negative prompting. The code repository includes eight passing automated tests and a validation report.

Citation

@inproceedings{deshmukh2027flowshift,
  title     = {From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models with {FlowShift}},
  author    = {Deshmukh, Gayatri and Susladkar, Onkar and Kilicoglu, Halil and Lourentzou, Ismini},
  booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  year      = {2027}
}

License

Apache License 2.0. Attribution is retained in NOTICE.