README.md

---
license: mit
tags:
  - research
  - weight-compression
  - image-codecs
---

# PixelModel-v4 Image Codec Experiments

> [!NOTE]
> This repository documents experiments conducted on **PixelModel-v4** only.
> PixelModel-v5 was released after these experiments and uses a substantially
> different architecture. The observations reported here should not be assumed
> to transfer directly to v5.

This repository is a research archive for measuring how image codecs affect a
PixelModel-v4 weight image and the model reconstructed from it. It is not the
main PixelModel-v4 model repository and is not a replacement model card.

## Scope/Provenance

The upstream model card and model assets are licensed under MIT by
[bench-labs/PixelModel-v4](https://huggingface.co/bench-labs/PixelModel-v4). The
upstream model revision used by the experiment is
`80878fdd6e130d229c2581424a3456b47b979d4b`.

The archive records a fixed comparison: 100 prompts, 50 sampling steps, CFG 6,
Apple MPS, and identical prompt, seed, initial latent, and settings across
conditions. The archive is intended for auditability and re-running the
experiment, not for making claims about codecs outside these recorded settings.
The `samples/` tree contains one artifact for every successful generation row
(800 artifacts in the full 17-condition run), while failed rows retain null
sample provenance.

## Motivation

PixelModel-v4 stores trained weights as image data, which makes ordinary image
storage and transport tools relevant to model serialization. Earlier work used
PNG as the lossless weight container. This repository explores what happens when
the same fp16 weight image is passed through modern lossless and lossy codecs,
then reconstructed with several byte-level methods. This is exploratory research:
the goal is to record the observed relationship between codec output,
reconstructed weight values, and generation behavior without proposing a
general-purpose compression result.

## Experimental Setup

- Model: PixelModel-v4 DiT with an fp16 weight image, using the upstream revision above.
- Weight representation: each fp16 word is serialized across image bytes; the
  high byte contains the sign, exponent, and upper mantissa bits.
- Prompts: 100 prompts.
- Sampling: 50 steps, CFG 6.
- Controls: identical prompt, seed, initial latent, and inference settings.
- Device: Apple MPS for the archived full run.
- Raw reconstruction is accepted only when the decoded weight tensor is finite.
  A NaN count by itself is not called “nonfinite”; the gate checks the complete
  tensor and reports NaN, +Inf, and -Inf separately.
- FID is N/A for this archive. CLIP and confidence intervals are not included
  unless they are present in the archived CSV.

## Codecs Evaluated

| Codec | Fixed setting | Role |
|---|---:|---|
| PNG | lossless | baseline |
| WebP | lossless | lossless control |
| JPEG | Q100 | lossy codec |
| JPEG | Q80 | lossy codec |
| WebP | Q80 | lossy codec |
| AVIF | Q70 | lossy codec |
| JPEG XL | distance 1 (`d1`) | lossy codec |

The protected representation keeps the high byte as PNG data and stores the low
byte through the lossy codec. The code also records the raw and finite-repair
paths separately.

## Reconstruction Methods

| Method | Description |
|---|---|
| Raw | Decode the codec payload directly into the weight image representation. |
| Finite Repair Zero | Replace decoded NaN, +Inf, and -Inf values with zero while leaving finite values unchanged. |
| Protected High Byte | Preserve the high byte losslessly and decode only the low byte through the lossy codec. |

## Experiment Flow

```mermaid
flowchart TD
    A[PixelModel-v4 fp16 weights] --> B[Serialize weights into image bytes]
    B --> C[Encode with PNG, WebP, JPEG, AVIF, or JPEG XL]
    C --> D[Decode codec payload]
    D --> E{Reconstruction method}
    E --> F[Raw]
    E --> G[Finite Repair Zero]
    E --> H[Protected High Byte]
    F --> I[Validate reconstructed tensor]
    G --> I
    H --> I
    I --> J{Finite-weight gate passes?}
    J -- Yes --> K[Run inference]
    J -- No --> L[Record blocked condition and diagnostics]
    K --> M[Record generation, weight, storage, and quality metrics]
    L --> M
```

## Representative Results

The following links are relative archive paths. A failed cell is rendered as an
explicit “NO SAMPLE” panel; it is not replaced with a black image or an inferred
metric.

![Representative JPEG Q100 comparison table](images/visual_table_jpeg_q100.png)

*Caption: JPEG Q100 table showing raw, finite-repair, and protected cells for representative prompts.*

![Representative JPEG Q80 comparison table](images/visual_table_jpeg_q80.png)

*Caption: JPEG Q80 table. Raw and Finite Repair Zero cells have no generated sample; Protected High Byte cells show the archived outputs.*

![Representative WebP Q80 comparison table](images/visual_table_webp_q80.png)

*Caption: lossy WebP Q80 table with explicit failed cells and archived Protected High Byte outputs.*

![Representative AVIF Q70 comparison table](images/visual_table_avif_q70.png)

*Caption: AVIF Q70 table with the same prompts, seeds, and inference settings as the controls.*

![Representative JPEG XL d1 comparison table](images/visual_table_jxl_d1.png)

*Caption: JPEG XL distance 1 table. Failed conditions are marked `NO SAMPLE`; no replacement image is used.*

## Results table from available CSV only

The values below are transcribed from the archived CSV outputs for the full
100-prompt run. They are counts and payload measurements, not estimates.

| Condition | Raw generation | Finite Repair Zero | Protected High Byte | Raw NaN / +Inf / -Inf | Protected exponent mismatch | Reliable payload saving vs PNG |
|---|---:|---:|---:|---:|---:|---:|
| PNG | 100/100 | N/A | N/A | 0 / 0 / 0 | 0 | 0.0000% |
| WebP lossless | 100/100 | N/A | N/A | 0 / 0 / 0 | 0 | 24.2799% |
| JPEG Q100 | 0/100 | 100/100 | 100/100 | 1,257 / 0 / 0 | 0 | -6.8825% |
| JPEG Q80 | 0/100 | 0/100 | 100/100 | 144,962 / 289 / 0 | 0 | 36.5568% |
| WebP Q80 | 0/100 | 0/100 | 100/100 | 1,097,350 / 0 / 0 | 0 | 37.6701% |
| AVIF Q70 | 0/100 | 0/100 | 100/100 | 445,041 / 59 / 0 | 0 | 39.2740% |
| JPEG XL d1 | 0/100 | 0/100 | 100/100 | 59,017 / 13 / 0 | 0 | 37.2180% |

The lossless WebP payload divided by the PNG payload is
`0.7572009918694315`. The saving percentages use PNG baseline payload bytes as
the denominator; Protected High Byte payloads include the high-byte PNG and the
codec-specific low-byte payload. If an
archive CSV is absent or differs, the CSV is authoritative and this table must
not be treated as an additional measurement.

## Main Observations

- PNG and WebP lossless completed 100/100 generations.
- All five lossy raw conditions completed 0/100 generations.
- JPEG was not uniquely fragile in this matrix: every tested lossy Raw setting
  introduced non-finite fp16 values and was blocked before generation.
- Finite Repair Zero completed 100/100 only for JPEG Q100; the other four lossy
  conditions recorded non-finite generation-stage failures.
- Protected High Byte completed 100/100 for all five lossy codecs.
- The protected rows recorded zero exponent mismatches.
- The raw diagnostic counts differ substantially across codecs; NaN, +Inf, and
  -Inf are kept as separate fields.

These observations motivated future work on bit-level importance analysis
rather than further broad codec comparisons.

## Discussion

Facts are the counts, byte diagnostics, exponent-mismatch counts, and payload
measurements recorded in the CSV files. They describe this run and do not by
themselves establish a causal mechanism.

A hypothesis consistent with the observations is that the fp16 high byte—which
contains the sign, exponent, and upper mantissa bits—is substantially more
important to the recorded model behavior than the lower byte. Protected High
Byte preserves that high byte while allowing low-byte compression to alter less
significant bits. The successful protected runs demonstrate the outcome for this
matrix, but they do not establish the proposed bit-importance mechanism as a
general causal fact. Protected High Byte is also a hybrid representation, not a
pure lossy weight artifact. Codec implementation choices, decoder behavior,
tensor layout, and the MPS runtime may contribute.

## Repository Contents

```text
images/                         Archived figures and representative images
results/                        CSV results, reports, and archive_manifest.json
samples/                        800 successful sample artifacts for all successful generation rows
weights/                        Archived payloads and weights/manifest.csv
scripts/cross_codec_report.py   Report generation
scripts/render_visual_tables.py Portable fail-closed visual-table renderer
scripts/verify_archive.py       Hash, byte-count, containment, and symlink verifier
scripts/pixelmodel_robustness/  11 packaged robustness modules
scripts/dit.py                  Packaged DiT implementation
scripts/config/                 Experiment configuration and prompt manifests
tests/                          Focused codec, repair, report, renderer, and archive tests
```

## Reproducibility

The full runner may require Apple MPS because that is the archived runtime. The
following commands are generic and intentionally do not promise universal
hardware reproducibility.

```bash
uv sync --locked
uv run --locked pytest
PYTHONPATH=scripts uv run --locked python scripts/verify_archive.py --root .
PYTHONPATH=scripts uv run --locked python scripts/render_visual_tables.py \
  --run-dir . \
  --prompt-ids-manifest results/representative_prompt_ids.json \
  --output scratch/rendered-tables
```

The complete archived experiment uses the packaged runner and the archived
configuration. Start with a small smoke test before attempting the full run.
The full implementation may require MPS and the source assets.

## How to verify/re-run

1. Verify the archive before reading results:

   ```bash
   PYTHONPATH=scripts uv run --locked python scripts/verify_archive.py --root .
   ```

2. Run the focused tests:

   ```bash
   uv run --locked pytest
   ```

3. Inspect the packaged report and experiment command surfaces:

   ```bash
   PYTHONPATH=scripts uv run --locked python scripts/cross_codec_report.py --help
   PYTHONPATH=scripts uv run --locked python -m pixelmodel_robustness.cross_codec_experiment --help
   ```

4. For a fresh source checkout, obtain the upstream assets at the exact
   revision. This command is a prerequisite only; it does not alter this
   archive:

   ```bash
   hf download bench-labs/PixelModel-v4 \
     --revision 80878fdd6e130d229c2581424a3456b47b979d4b
   ```

5. Use the packaged runner's `--help` output to select the input paths and
   output run directory, then run on a machine with the required model runtime.
   Preserve the runner manifest, encoded payloads, materialized weight images,
   samples, resource log, and raw rows together. `samples/` contains all 800
   successful sample artifacts, so raw sample provenance is complete even though
   the compact archive intentionally deduplicates payloads. Complete
   report regeneration should target a fresh canonical run directory with
   materialized weight images; the compact archive layout is not sufficient for
   a fully fresh report regeneration.

For AVIF, Pillow must have AVIF support. JPEG XL requires an ImageMagick build
with the libjxl delegate. These codec prerequisites are recorded by the
experiment where available.

## Weights archive explanation

The `weights/` directory is an archive of experiment inputs and codec payloads,
not a new or fine-tuned model release. `weights/manifest.csv` records the path,
byte count, and SHA-256 for each listed payload. Git LFS is configured for this
directory. The archive verifier checks that paths stay inside the repository,
that listed files exist with the recorded bytes and hashes, that paths are not
duplicated, and that no symlink is used in the archived trees.

## Limitations

- The experiment covers one model version, PixelModel-v4 at the pinned revision.
- Only the listed codec settings were evaluated; the setting matrix is limited.
- No result here establishes that one codec is universally superior.
- The full run uses one fixed prompt/seed/latent/settings protocol and one
  Apple MPS runtime.
- The results do not establish codec performance for other models, tensor
  layouts, image dimensions, devices, Pillow versions, or libjxl versions.
- Generation success is a gate outcome, not a quality ranking.
- No FID result is reported; FID is N/A here.
- Missing or failed samples are not imputed.
- The archive does not claim that protected high-byte storage preserves every
  aspect of a model outside the recorded experiment.
- These observations should not be generalized beyond this experiment without
  additional models, settings, runtimes, and independently specified evaluations.

## Future Work

- Perform bit-level importance analysis with pre-registered corruption masks.
- Evaluate exponent-only corruption.
- Evaluate mantissa-only corruption at controlled bit positions.
- Test additional codecs and codec settings.
- Investigate learned, weight-aware compression methods.
- Repeat the protocol across additional model revisions, tensor layouts, prompt
  sets, devices, and decoder versions.

## License and acknowledgment

This archive is released under the MIT License. It acknowledges
[bench-labs/PixelModel-v4](https://huggingface.co/bench-labs/PixelModel-v4) and
its MIT-licensed upstream model card. This repository is an experimental archive
built from the robustness-study source; it is not the upstream model repository.