Fibo-lite / README.md

mokady

Update README.md

3ab510d verified 10 days ago

preview code

raw

history blame contribute delete

9.82 kB

metadata

language:
  - en
base_model: briaai/FIBO
pipeline_tag: text-to-image
library_name: diffusers
license: cc-by-nc-4.0
license_name: bria-fibo
license_link: https://creativecommons.org/licenses/by-nc/4.0/deed.en
tags:
  - lora
  - guidance-distillation
  - scfm
  - shortcutting-flow-matching
  - text-to-image
  - fast-inference
  - few-step
  - bria
  - fibo
extra_gated_heading: Access FIBO Lite
extra_gated_description: >-
  This model is an extension of Bria AI's FIBO model. Weights are open source
  for non-commercial use only, per the provided
  [license](https://creativecommons.org/licenses/by-nc/4.0/deed.en).
extra_gated_fields:
  Name: text
  Email: text
  Company/Org name: text
  Company Website URL: text
  Discord user: text
  I agree to BRIA’s Privacy policy, Terms & conditions, and acknowledge Non commercial use to be Personal use / Academy / Non profit (direct or indirect): checkbox

FIBO Lite Hero

Accelerate FIBO inference with a two-stage distillation approach.
This model combines CFG distillation with Shortcutting Flow Matching (SCFM) for ultra-efficient few-step generation.

🚀 Overview

This is a two-stage distilled model for the FIBO text-to-image model, combining:

CFG Distillation: First, we distill classifier-free guidance into the model, enabling inference with Guidance Scale = 1.0 (skipping the negative prompt pass).
SCFM (Shortcutting Flow Matching): On top of the CFG-distilled merged model, we apply velocity field self-distillation to enable efficient few-step sampling.

This two-stage approach allows FIBO Lite to generate high-quality images in significantly fewer inference steps while maintaining the benefits of CFG distillation.

🔑 Key Benefits

Two-Stage Distillation: Combines CFG distillation with SCFM for maximum efficiency—the SCFM was trained on top of the already CFG-distilled merged model.
Few-Step Generation: SCFM enables efficient sampling in significantly fewer inference steps.
No CFG Overhead: Running at guidance_scale=1 means calculating the noise prediction only once per step instead of twice.
Quality Tradeoff: As a distillation approach, there is a slight quality degradation compared to the full model at CFG=5, but the speed gains make it ideal for rapid iteration and production workflows.
Drop-in Replacement: Works seamlessly with existing FIBO workflows—just set guidance_scale=1.0.
Memory Efficient: Minimal additional GPU memory overhead.

📊 Comparison & Examples

Left: Regular FIBO (Base Model, CFG=5) | Right: FIBO Lite (Distilled LoRA, CFG=1)
Comparison

🖼️ More Example Outputs from FIBO Distilled LoRA:

Feature	Base FIBO	FIBO Lite
Guidance Scale	5.0 (Typical)	1.0 (Distilled)
Compute per Step	2x (Cond + Uncond)	1x (Cond Only)
Speed	Baseline	~10x Faster
Quality	Full Quality	Slight Degradation

🛠️ Usage

Requirements

pip install git+https://github.com/huggingface/diffusers torch torchvision  boltons ujson sentencepiece accelerate transformers

for using gemini api, you need to install google-genai as well

Quick Start

import os
import json
import torch
from diffusers import BriaFiboPipeline
from diffusers.modular_pipelines import ModularPipeline

# -------------------------------
# Load the VLM pipeline
# -------------------------------
torch.set_grad_enabled(False)

# Using local VLM
vlm_pipe = ModularPipeline.from_pretrained("briaai/FIBO-VLM-prompt-to-JSON", trust_remote_code=True)


# Using Gemini API, requires GOOGLE_API_KEY environment variable
# assert os.getenv("GOOGLE_API_KEY") is not None, "GOOGLE_API_KEY environment variable is not set"
# vlm_pipe = ModularPipeline.from_pretrained("briaai/FIBO-gemini-prompt-to-JSON", trust_remote_code=True)



# Load the FIBO pipeline
pipe = BriaFiboPipeline.from_pretrained(
    "briaai/Fibo-lite",
    torch_dtype=torch.bfloat16,
)
pipe.to("cuda")

# Generate with guidance_scale=1.0 (2x faster!)
output = vlm_pipe(
    prompt="A hyper-detailed, ultra-fluffy owl sitting in the trees at night, looking directly at the camera with wide, adorable, expressive eyes. Its feathers are soft and voluminous, catching the cool moonlight with subtle silver highlights. The owl's gaze is curious and full of charm, giving it a whimsical, storybook-like personality."
)
json_prompt_generate = output.values["json_prompt"]

results_generate = pipe(
    prompt=json_prompt_generate, num_inference_steps=8, guidance_scale=1
)
results_generate.images[0].save("image_generate.png")
with open("image_generate_json_prompt.json", "w") as f:
    f.write(json_prompt_generate)

Key Parameters

guidance_scale=1.0: This is the magic setting! With the LoRA loaded, you get the quality of CFG=5 at the speed of CFG=1.
num_inference_steps=8: Standard for FIBO Lite. Adjust based on your quality/speed tradeoff.
No negative prompt needed: The distillation handles this internally.

🧠 Training Details

This model was created using a two-stage distillation process:

Stage 1: CFG Distillation

Method: Guidance Distillation (Distilling the CFG effect into the model weights via LoRA)
Base Model: briaai/FIBO
Trainable Parameters: 471,472,128
Precision: bfloat16
Teacher Configuration: CFG=5.0 (standard FIBO setting)
Student Configuration: CFG=1.0 (target deployment)
Training Objective: Minimize KL divergence between teacher and student outputs

The CFG distillation LoRA was then merged into the base model to create a CFG-free foundation.

Stage 2: SCFM (Shortcutting Flow Matching)

Building on the merged CFG-distilled model, we applied Shortcutting Flow Matching (SCFM):

Method: Velocity Field Self-Distillation
Base Model: CFG-distilled merged FIBO (from Stage 1)
Key Innovation: SCFM imparts a shortcut mechanism to standard flow matching models, enabling flexible trajectory-skipping without requiring step-size embedding
Training Approach: Works on the velocity field rather than sample space, learning rapidly from self-guided distillation in an online manner
Result: Efficient few-step flows without compromising quality

Why Two Stages?

Stage 1 (CFG Distillation) eliminates the need for dual forward passes (conditional + unconditional), halving compute per step.
Stage 2 (SCFM) enables the model to take larger steps through the diffusion trajectory, dramatically reducing the number of required inference steps.

Together, these stages compound to deliver ultra-efficient generation: fewer steps × less compute per step = significantly faster inference.

🤝 Community & Support

GitHub: FIBO Repository
Hugging Face: FIBO Model Card
Commercial Licensing: Contact Bria AI

📚 Citation

If you use this model in your research or project, please cite:

@misc{gutflaish2025generating,
  title         = {Generating an Image From 1,000 Words: Enhancing Text-to-Image With Structured Captions},
  author        = {Gutflaish, Eyal and Kachlon, Eliran and Zisman, Hezi and Hacham, Tal and Sarid, Nimrod and Visheratin, Alexander and Huberman, Saar and Davidi, Gal and Bukchin, Guy and Goldberg, Kfir and Mokady, Ron},
  year          = {2025},
  eprint        = {2511.06876},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  doi           = {10.48550/arXiv.2511.06876},
  url           = {https://arxiv.org/abs/2511.06876}
}

@misc{cai2025scfm,
  title         = {Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch},
  author        = {Cai, Xu and Wu, Yang and Chen, Qianli and Wu, Haoran and Xiang, Lichuan and Wen, Hongkai},
  year          = {2025},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}

📄 License

Source-Code & Weights

The model is open source for non-commercial use with this license
For commercial use Click here.