README.md

---
license: cc-by-4.0
base_model: laion/CLIP-ViT-B-32-laion2B-s34B-b79K
pipeline_tag: zero-shot-classification
library_name: pointcal-c
tags:
- 3d
- point-cloud
- modelnet40-c
- clip
- calibration
- selective-prediction
- uncertainty
- zero-shot
---

# PointCal-C — selective zero-shot 3D recognition under corruption

A post-hoc **calibration and abstention head** on top of a frozen OpenCLIP
ViT-B/32, for zero-shot 3D point-cloud recognition under corruption, plus the
complete evaluation artifacts backing it.

Code: <https://github.com/crimsonKn1ght/pointcal-c>

## What the model is

The backbone is **never trained**. `laion/CLIP-ViT-B-32-laion2B-s34B-b79K` is
loaded frozen, in eval mode, with `requires_grad_(False)`.

The fitted model is **four scalars**, in `runs/full/results/calibration.json`,
fit on clean calibration objects only — corrupted labels, corruption identity
and severity never touch a fitted parameter:

| parameter | value | role |
|---|---|---|
| `temperature` | 1.2309 | divides logits before softmax |
| `bias` | -1.5723 | combined-score intercept |
| `weight_confidence` | +1.7071 | on `logit(p_calibrated)` |
| `weight_disagreement` | +4.9158 | on cross-view disagreement `d` |

Combined score: `sigmoid(bias + w_conf * logit(p_cal) + w_dis * d)`, where `d`
is `1 -` mean pairwise Jensen-Shannon divergence across the six views.

Converged, unclamped, not degenerate. Fit in 0.94 s on CPU.

## How inference works

```
point cloud (N,1024,3)
  -> centroid-center, unit-radius normalize
  -> 6 orthographic cameras  [front right back left top bottom]
  -> 64x64 splat raster + nearest-depth z-buffer -> upsample to 224x224
  -> frozen OpenCLIP ViT-B/32 image encoder
  -> per-view logits vs a fixed 8-template prompt ensemble   <-- cached here
  -> mean over views  ->  the prediction
  -> four confidence scores over that one prediction
```

All four confidence methods (`msp`, `temperature`, `disagreement`, `combined`)
rank the **same** predictions. A confidence method that changed a prediction
would be a different classifier, and `assert_predictions_unchanged` raises if
one ever does.

## Results

All 75 corrupted conditions pooled against clean, 1975 evaluation objects,
95% bootstrap intervals resampled over base object IDs:

| | accuracy | ECE (MSP) | ECE (combined) | AURC (MSP) | AURC (combined) |
|---|---|---|---|---|---|
| clean | 0.2896 | 0.1160 | 0.0245 | 0.4339 | 0.4147 |
| corrupted | 0.2354 | 0.1416 | 0.0229 | 0.5217 | 0.5008 |

Accuracy degrades under corruption and MSP calibration degrades with it. The
clean-fit combined score holds ECE near **0.023** under shift and lowers AURC
by 0.021.

The ECE improvement is large and consistent. **The AURC gain is small** and
should be read against the intervals in `results.csv`, not as a headline. This
is a reliability audit and a bounded abstention baseline, not a
state-of-the-art claim.

Accuracy by severity: 0.2896 (clean), 0.2593, 0.2515, 0.2388, 0.2252, 0.2023 —
monotonically non-increasing.

### Pre-registered hypotheses

Of the four pre-registered hypotheses, three are mechanically checked. H2 and H3
resolve True. **H1 does not hold strictly:** accuracy, NLL, Brier and AURC all
worsen monotonically with severity, but ECE dips at severities 3→4 and again
from clean→1. A paired object-grouped bootstrap places both reversals within
noise of zero, so H1 is supported in direction but not as strict monotonicity.
H4 is read off the family-scope rows rather than reduced to a verdict.

## Repository contents

The expensive artifact is `runs/*/logits/` — per-view logits for every
condition under both prompt modes. Every metric, ablation, bootstrap interval
and figure here was computed on CPU from those caches, so **any re-analysis is
free**: a different aggregation, disagreement statistic, coverage level or
bootstrap depth costs no GPU time, because CLIP never runs again.

```
runs/<tier>/
  logits/<condition>.npz     per-view logits, both prompt modes (~1.4 MB each)
  results/calibration.json   the fitted scalars above
  results/results.csv|.json  metrics table with intervals
  results/ablations.csv      view-count, prompt-mode, disagreement ablations
  results/predictions.npz    per-sample confidences and correctness
  results/results_summary.md auto-generated tables + hypothesis checks
  figures/fig1..fig4         accuracy, ECE, risk-coverage, cost
  provenance/                data manifest, run manifest, split audit
  ledger_inference.json      measured GPU-hours, dollars, throughput, memory
artifacts/split.json         the frozen calibration/evaluation object split
env/requirements.lock.txt    exact resolved environment (171 packages)
```

Tiers: `xs` (5 conditions, 100 objects), `s` (13 conditions), `full` (all 76 =
clean + 15 corruptions x 5 severities).

## Training / evaluation data

Evaluated on **ModelNet40-C** (2468 objects, 40 classes, 15 corruption types,
5 severities). The calibration scalars are fit on the clean split of 493
calibration objects; 1975 evaluation objects are held out.

The split is over **base object IDs**, not samples, and every corruption array
is verified row-aligned with the clean array, so holding an object out of
calibration holds it out under all 76 conditions. Audited: `overlap: 0`,
0 row-misaligned conditions, all 40 classes present on both sides.

## Compute

| tier | conditions | GPU-hours | USD | views/s | peak VRAM |
|---|---|---|---|---|---|
| xs | 5 | 0.001 | 0.0003 | 1091 | 2.5 GB |
| s | 13 | 0.015 | 0.004 | 2816 | 12.1 GB |
| full | 76 | 0.089 | 0.025 | 2808 | 12.1 GB |

**0.105 GPU-hours and about $0.03** of inference in total, on a single Runpod
RTX 4000 Ada at $0.28/hr (2026-08-16).

## Known defects

See **[ERRATUM.md](ERRATUM.md)** for the full list. In short: the `ledger` block
embedded in `runs/full/provenance/run_manifest.json` is wrong — it records a
cache-hit re-invocation, not the real run. **Trust `ledger_inference.json`,**
which every figure and the compute table above are derived from. `git_dirty` in
that manifest is not meaningful either, because the dirtiness check counts
expected untracked artifacts.

## Limitations

- `full` uses **200** bootstrap replicates, not the preregistered 1000. The CPU
  bootstrap dominated runtime (~2 h at 1000 vs ~5 min of GPU work). Deeper
  intervals recompute from the cached logits at no GPU cost.
- NLL and Brier require a distribution over all 40 classes, so they are reported
  for `msp` and `temperature` only. For the two ranking scores they are `null`,
  not a lookalike substitute.
- `delta_*_vs_clean` rows in `results.csv` carry `NaN` intervals: degradation is
  reported as point estimates without paired bootstrap intervals. Method
  differences should likewise not be read from the marginal intervals alone.
- Cross-view disagreement is prior art, and calibration degrading under 3D
  corruption is a confirmation rather than a discovery.
- Absolute accuracy is low (~0.29 clean). This is zero-shot transfer of a 2D
  model to depth renders, with no 3D training whatsoever.
- Severity is reported **1-indexed**, matching the on-disk release.

## Provenance

- Frozen spec hash: `ca487597c978ab17217bd640ab8bf95c8a21db0eaacd51ae5ba4b00618dcc820`
- Backbone checkpoint SHA-256:
  `1bd3c7172de5b207ceac554f5ab5266166f3b9baccc9af5989bc801016d080ad`
- Split fingerprint: `cc5ed4b3ce7108154039f497d474f3ac6bdbfe7eb7dbfe4d0f90785e1829bb31`

## Licence and attribution

Cached logits and metrics are **derived from ModelNet40-C** and redistributed
under **CC BY 4.0**, as the source record declares.

- ModelNet40-C — Zenodo [6017834](https://zenodo.org/records/6017834),
  DOI `10.5281/zenodo.6017834`, CC BY 4.0
- Generation code — <https://github.com/jiachens/ModelNet40-C>, BSD-3-Clause
- Underlying ModelNet40 carries its own original terms
- OpenCLIP — MIT · PointCal-C code — MIT

The multi-view depth-projection approach follows the PointCLIP paper (CVPR
2022); the projector is an independent reimplementation from the paper text,
with no PointCLIP repository code used.