Update README.md

901c1eb verified 22 days ago

10.1 kB

base_model:
  - Lightricks/LTX-2.3
base_model_relation: adapter
license: other
license_name: ltx-2-community-license
license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE
language:
  - en
tags:
  - ltx-video
  - ic-lora
  - ltx-2.3
  - video-to-video
  - decompression
  - video-restoration
pipeline_tag: video-to-video
extra_gated_description: >-
  By clicking "Agree and Access" you acknowledge the [Privacy
  Policy](https://static.lightricks.com/legal/Privacy%20Policy%20-%20LTX%20Platform.pdf) 
  and consent to receive offers and updates. You can unsubscribe at any time.
extra_gated_button_content: Agree and Access
widget:
  - text: >-
      Reference shows a wide field of blooming sunflowers bordered by green
      trees, heavily compressed with visible macroblocking, chroma bleed, and
      ringing artifacts. Edited shows the same scene restored to high quality
      with sharp detail, clean edges, and no compression artifacts. ENHANCE
      QUALITY a wide field of blooming sunflowers bordered by green trees.
      Subject identity, framing, and background geometry are identical to the
      reference; only compression artifacts and image quality differ between
      reference and edited.
    output:
      url: examples/Examples_2__27_sunflower_field_sbs.mp4
  - text: >-
      Reference shows an aerial twilight view of a classical dome and
      illuminated city buildings, heavily compressed with visible macroblocking,
      chroma bleed, and ringing artifacts. Edited shows the same scene restored
      to high quality with sharp detail, clean edges, and no compression
      artifacts. ENHANCE QUALITY an aerial twilight view of a classical dome and
      illuminated city buildings. Subject identity, framing, and background
      geometry are identical to the reference; only compression artifacts and
      image quality differ between reference and edited.
    output:
      url: examples/Examples_2__07_rome_dome_aerial_sbs.mp4
  - text: >-
      Reference shows an aerial view of a vehicle driving on a desert dirt
      trail, heavily compressed with visible macroblocking, chroma bleed, and
      ringing artifacts. Edited shows the same scene restored to high quality
      with sharp detail, clean edges, and no compression artifacts. ENHANCE
      QUALITY an aerial view of a vehicle driving on a desert dirt trail.
      Subject identity, framing, and background geometry are identical to the
      reference; only compression artifacts and image quality differ between
      reference and edited.
    output:
      url: examples/Examples_2__05_desert_trail_aerial_sbs.mp4

LTX-2.3 22B IC-LoRA Decompression

This is a Decompression IC-LoRA trained on top of LTX-2.3-22B, which removes compression artifacts from low bit-rate / heavily-compressed video — macroblocking, chroma bleed, ringing, and banding — while preserving subject identity, framing, and geometry.

It is based on the LTX-2.3 foundation model.

Prompt: Reference shows a wide field of blooming sunflowers bordered by green trees, heavily compressed with visible macroblocking, chroma bleed, and ringing artifacts. Edited shows the same scene restored to high quality with sharp detail, clean edges, and no compression artifacts. ENHANCE QUALITY a wide field of blooming sunflowers bordered by green trees. Subject identity, framing, and background geometry are identical to the reference; only compression artifacts and image quality differ between reference and edited.

Prompt: Reference shows an aerial twilight view of a classical dome and illuminated city buildings, heavily compressed with visible macroblocking, chroma bleed, and ringing artifacts. Edited shows the same scene restored to high quality with sharp detail, clean edges, and no compression artifacts. ENHANCE QUALITY an aerial twilight view of a classical dome and illuminated city buildings. Subject identity, framing, and background geometry are identical to the reference; only compression artifacts and image quality differ between reference and edited.

Prompt: Reference shows an aerial view of a vehicle driving on a desert dirt trail, heavily compressed with visible macroblocking, chroma bleed, and ringing artifacts. Edited shows the same scene restored to high quality with sharp detail, clean edges, and no compression artifacts. ENHANCE QUALITY an aerial view of a vehicle driving on a desert dirt trail. Subject identity, framing, and background geometry are identical to the reference; only compression artifacts and image quality differ between reference and edited.

Model Files

ltx-2.3-22b-ic-lora-decompression-0.9.safetensors — the single released checkpoint (used for the published inference samples).

Model Details

Base Model: LTX-2.3-22B Video
Training Type: IC-LoRA (video-to-video)
Control Type: Video-to-video — a heavily-compressed, low bit-rate input (reference) video drives a clean, artifact-free output video.
Reference Downscale Factor: 1 (the reference is encoded at 1× the output resolution).
Pipeline details: No special pre/post transform — the compressed reference video is VAE-encoded as the control signal and the model predicts the restored result.

Intended Use & Out-of-Scope

Intended use: Improving low bit-rate / heavily-compressed video — removing compression artifacts such as macroblocking, chroma bleed, ringing, and banding from low-bitrate sources, restoring sharp detail and clean edges while keeping the original subject and composition.

Out of scope: This is not a general restoration model. It is specifically not designed for: out-of-focus / defocus blur, motion blur, noise / grain removal, super-resolution / upscaling, or colorization. It targets compression-artifact removal only; content far outside the training distribution may underperform.

Control Signal Requirements

Control signal type: Low bit-rate / heavily-compressed source video (visible macroblocking, chroma bleed, ringing).
Expected input: A reference video (.mp4 / .mov / .mkv / .webm / .avi).
Preprocessing: None required — the compressed reference is VAE-encoded directly. The reference is used at 1× the output resolution (downscale factor 1).
Alignment: The output matches the reference frame count, FPS, resolution, and aspect ratio. Best results at the 960×544×121 @ 24fps training bucket (both landscape 960×544 and portrait 544×960 were seen in training).

How It Works

The model is conditioned on both the compressed reference video latents and a text prompt that describes the compressed source and the desired clean result. The prompt convention learned in training is:

Reference shows {scene}, heavily compressed with visible macroblocking, chroma bleed, and ringing artifacts. Edited shows the same scene restored to high quality with sharp detail, clean edges, and no compression artifacts. ENHANCE QUALITY {scene}. Subject identity, framing, and background geometry are identical to the reference; only compression artifacts and image quality differ between reference and edited.

Representative prompt from a real run:

Reference shows a small reddish-brown wild rabbit sitting upright between large moss-covered grey boulders, with a fallen log and blurred autumn foliage behind it under soft natural daylight, heavily compressed with visible macroblocking, chroma bleed, and ringing artifacts. Edited shows the same scene restored to high quality with sharp detail, clean edges, and no compression artifacts. ENHANCE QUALITY a small reddish-brown wild rabbit sitting upright between large moss-covered grey boulders, with a fallen log and blurred autumn foliage behind it under soft natural daylight. Subject identity, framing, and background geometry are identical to the reference; only compression artifacts and image quality differ between reference and edited.

Usage

🔌 ComfyUI

Copy the LoRA weights into models/loras.
Load the LTX-2.3-22B base model and add ltx-2.3-22b-ic-lora-decompression-0.9.safetensors as the LoRA.
Start at strength 1.0 and adjust to taste.
Use an IC-LoRA (video-to-video) workflow from the LTX-2 ComfyUI repository, which already wires the reference-video control nodes. Connect your compressed clip as the reference video and write the prompt using the ENHANCE QUALITY convention above. Because the reference downscale factor is 1, a generic reference encode at output resolution is correct.
Start at or near the 960×544 training bucket; generating far above it can weaken artifact removal on high-frequency detail.

Recommended Settings

LoRA strength / weight: 1.0 (the published samples used strength 1.0).
Resolution & frames: Trained and validated at 960×544×121 @ 24fps (frames satisfy (frames-1) % 8 == 0). Both landscape (960×544) and portrait (544×960) were in training. Start near this bucket for the strongest, most consistent effect.
Prompting: Follow the Reference shows … heavily compressed … ENHANCE QUALITY … only compression artifacts and image quality differ structure documented in How It Works. Describe the same scene in both halves; keep identity/framing/geometry language intact so the model only removes artifacts.
Production inference recipe (what we used): Run via the distilled ltx_pipelines.ic_lora pipeline with the identity-safe, stage-1-only native hi-res recipe — render on a 2× canvas with --skip-stage-2 --tile-reference-encode (stage 2 is skipped so the reference stays anchored for the whole denoise), LoRA strength 1.0, seed 42, 121 frames @ 24fps. The distilled checkpoint uses fixed sigmas, so there is no CFG / guidance scale and no negative prompt. A dev-trained LoRA loads cleanly on the distilled checkpoint.

References

Code: GitHub Repository
Inference Pipeline: ltx_pipelines.ic_lora (LTX-2 distilled IC-LoRA pipeline)

Tips & Troubleshooting

Residual artifacts at very high resolution: the model generates stage-1 at the full output resolution, which is well above the 960×544 training bucket. If macroblocking or ringing persists, lower the generation/output resolution toward the training bucket.
Wrong tool for the job: if the source suffers from defocus/motion blur, noise/grain, or low resolution rather than compression, this LoRA will not fix it — it only targets compression artifacts.
Identity drift: keep the stage-1-only recipe (do not use the two-stage path for identity-critical clips) so the reference stays attached for the entire denoise.

Dataset

The model was trained using a proprietary dataset.

Training

Technique: IC-LoRA (rank 128, alpha 128, dropout 0.05) on the DiT transformer; target modules attn1.to_q, attn1.to_k, attn1.to_v, attn1.to_out.0, ff.net.0.proj, ff.net.2.
Hyperparameters: bf16 mixed precision, AdamW optimizer, learning rate 1.5e-4, cosine scheduler, gradient checkpointing on, batch size 1.
Steps: 1000 total training steps. Released checkpoint: ltx-2.3-22b-ic-lora-decompression-0.9.safetensors.
Infrastructure: LTX-2 Community Trainer (single-node, 8 GPU).

License

See the LTX-2-community-license for full terms.

Acknowledgments

Base model by Lightricks
Training infrastructure: LTX-2 Community Trainer