carddeframe-klein4b / README.md
pearsonkyle's picture
Link companion dataset repo
6a3fe94 verified
|
Raw
History Blame Contribute Delete
4.62 kB
metadata
license: apache-2.0
base_model:
  - black-forest-labs/FLUX.2-klein-base-4B
tags:
  - lora
  - image-to-image
  - image-editing
  - flux2
  - trading-cards
  - ai-toolkit
library_name: diffusers
pipeline_tag: image-to-image

Card De-frame LoRA (FLUX.2-Klein 4B)

An instruction-editing LoRA that strips the frame, text, and UI elements from trading-card images (Magic: The Gathering, Pokémon, Yu-Gi-Oh!, Digimon) and extends the artwork to a seamless full-bleed illustration.

File Base model Recommended sampling Time/image*
carddeframe_klein4b_v6.safetensors FLUX.2-klein-base-4B 20–25 steps, CFG 4.0, empty negative ~115 s

*measured at 848×1184 on an RTX 4060 Ti 16 GB with layer offloading.

Usage

The input card is passed as the control/reference image (kontext-style edit conditioning) and the prompt is the edit instruction. The instruction is game-specific — use the exact phrasing the LoRA was trained with (shown with each example below).

FLUX.2-klein-base is not guidance-distilled: sample with standard CFG ≈ 4.0 and an empty negative prompt. Quality saturates at 20 steps; 12 steps is usable (95%).

Python example (MTG)

import torch
from diffusers import Flux2KleinPipeline
from diffusers.utils import load_image

pipe = Flux2KleinPipeline.from_pretrained(
    "black-forest-labs/FLUX.2-klein-base-4B", torch_dtype=torch.bfloat16
)
pipe.load_lora_weights("carddeframe_klein4b_v6.safetensors")
pipe.enable_model_cpu_offload()  # fits in 16 GB VRAM

card = load_image("mtg_card.png")
prompt = (
    "Remove the title, mana cost, type line, rules text box, power and toughness, "
    "and set symbol. Paint over those areas with a natural continuation of the "
    "existing artwork."
)

image = pipe(
    image=[card],
    prompt=prompt,
    guidance_scale=4.0,   # true CFG with an empty negative prompt
    num_inference_steps=20,
    height=1184,
    width=848,
    generator=torch.Generator("cpu").manual_seed(42),
).images[0]
image.save("full_bleed.png")

For the other games, swap the prompt for the matching instruction below.

Before / after examples

All examples are held-out validation cards (not in the training set), generated at 20 steps, CFG 4.0, empty negative, 848×1184.

Magic: The Gathering

Prompt:

Remove the title, mana cost, type line, rules text box, power and toughness, and set symbol. Paint over those areas with a natural continuation of the existing artwork.

MTG before/after

Pokémon

Prompt:

Remove the card name, HP, energy type icons, attack names and damage, weakness, resistance, retreat cost, set number, and illustrator credit. Paint over those areas with a natural continuation of the existing artwork.

Pokémon before/after

Yu-Gi-Oh!

Prompt:

Remove the card name, attribute icon, level stars, card type line, ATK and DEF values, effect text box, and outer card border frame. Paint over those areas with a natural continuation of the existing artwork.

Yu-Gi-Oh! before/after

Digimon

Prompt:

Remove the card name, level indicator, attribute icon, type bar, DP value, play cost, digivolve costs, effect text box, and card border. Paint over those areas with a natural continuation of the existing artwork.

Digimon before/after

Training

  • Dataset: 547 curated (input card → de-framed full-bleed) pairs across the four games, teacher-generated with Qwen-Image-Edit-2511 and manually curated. Captions are the edit instructions above. The full training set is published at pearsonkyle/carddeframe-dataset.
  • Network: LoRA rank 64, alpha 32, on the transformer only.
  • Recipe: trained with ostris/ai-toolkit at lr 1e-4, bf16, flow-matching with weighted timestep sampling. The 600-step checkpoint was selected by held-out evaluation — later checkpoints did not improve.

Known limitations

  • Cards with non-standard layouts (full-art EX / promo Pokémon cards, some Yu-Gi-Oh! frames) may keep their inner border or art box instead of extending to full bleed (~3 of 24 held-out cards).
  • Output has a mild painterly softening versus the original art crop.
  • Card art is owned by the respective game publishers; this LoRA is intended for research and personal use on imagery you have rights to process.