Release evaluated Forma3D Tiny 2 continuation

1648582 verified 1 day ago

8.24 kB

# Forma3D Tiny 2 on an M4 MacBook Air with 16 GB RAM

This is a proposed weights-only continuation of `Lucasgitguy-co/forma3d-tiny`,
named `Lucasgitguy-co/forma3d-tiny-2`. It preserves the small FP32 architecture:
306,321 parameters, grid 16 cubed, decoder width 64. It aims to improve coarse
proxy reconstruction through more object diversity. No v2 training has been run
as part of preparing this package, and improvement is not guaranteed.

## Run the commands

Extract the kit into a local folder. Open macOS Terminal in that folder. Use a
folder stored locally rather than an offloaded iCloud folder. Keep the Mac on
power and allow cooling during a sustained run.

```bash
# If uv is not installed, install it with Homebrew first.
brew install uv
# From the extracted forma3d-tiny-2-kit folder:
bash posttrain.sh setup
bash posttrain.sh prepare
bash posttrain.sh train
bash posttrain.sh evaluate
```

Run each command only after the previous one succeeds. Setup downloads the source
and current v1 weights at the fixed revision
`e80b8d895cc2e405894473abe4ce40b3472eaca7`. It installs Python 3.11 and the
publisher's tested dependency lock in an isolated environment under
`~/Library/Application Support/Forma3D-tiny-2/venv`. Setup does not train or upload.
`image3d doctor` must report Metal available. These public source files execute
locally during preparation/training; inspect the scripts before running them.

Preparation requests 128 eligible Scenario meshes plus 128 CC0 Objaverse meshes,
four views each. It excludes known v1 asset IDs, paths, source groups, and exact
mesh hashes. It keeps every view of each related object in the same group. A
separate 15 percent test split is frozen before training. Validation uses 20
percent of the remaining development groups. Counts depend on duplicate groups;
read `data/forma3d-v2/preparation.json` and `v2-audit.json` for the realized counts.

The mesh-download budget is 4,096 MiB per preparation invocation, excluding dataset
metadata and Python packages. It reserves 8 GiB of free disk while fetching meshes.
Have comfortably more than 12 GiB available before setup and preparation; this is
a planning allowance, not a measured total. RAM and duration have not been
benchmarked for this exact v2 recipe. No pretrained teacher is loaded.

Training initializes from **v1 model.pt**, rather than its original ellipsoid
checkpoint. It uses fresh AdamW, batch size 2, learning rate 0.00005, 8,000 optimizer
steps, weight decay 0.0001, gradient clipping at norm 1, seed 43, and validation
every 200 steps. This lower learning rate is a proposed conservative setting, not
a measured optimum. The best validation loss selects `best.pt`.

The underlying training command is:

```bash
PY="$HOME/Library/Application Support/Forma3D-tiny-2/venv/bin/python"
caffeinate -i "$PY" -m image3d_student.cli train \
  --manifest data/forma3d-v2/manifest.jsonl \
  --init-from upstream/model.pt \
  --out runs/forma3d-v2 --device mps --width 64 \
  --steps 8000 --batch-size 2 --lr 0.00005 \
  --eval-every 200 --validation-fraction 0.2 --seed 43
```

Do not substitute grid 32 in this recipe: it increases query cells eightfold and
changes the evaluation representation. A width change is incompatible with the
starting learned weights.

## Inspect and optionally publish

Evaluation compares v1 and the selected v2 checkpoint on **the same new test
objects**, requiring lower test loss and higher test occupancy IoU, plus lower
validation loss, a completed run, source separation, and nonempty qualitative
predictions. It writes local comparisons to `runs/forma3d-v2/inspection/` and
stages `release/forma3d-tiny-2/` only if the gates pass. Inspect these PNGs.
The release card uses the actual run measurements and correctly records v1 as
its parent. No scores are prefilled.

The test set is for final assessment. If a release gate fails, do not repeatedly
tune hyperparameters against that test set. Develop on validation data and reserve
another genuinely unused object set for a later final claim.

To publish after inspection:

```bash
"$HOME/Library/Application Support/Forma3D-tiny-2/venv/bin/hf" auth login
bash posttrain.sh publish
```

Use your own Hugging Face write token in Terminal, not chat. This command creates
`Lucasgitguy-co/forma3d-tiny-2` privately, uploads the evaluated package, and makes
it public after the upload succeeds. It refuses to overwrite an existing repo.
A failed upload may leave a private partial repo; inspect that repo before
attempting recovery. Only the staged release folder is uploaded. Raw meshes,
download caches, credentials, and optimizer state are excluded.

## Generate a Gaussian PLY

Use a centered object cutout on black. Replace the example path with your image:

```bash
PY="$HOME/Library/Application Support/Forma3D-tiny-2/venv/bin/python"
"$PY" -m image3d_student.cli generate \
  --checkpoint runs/forma3d-v2/best.pt --image /path/to/your-cutout.png \
  --out results/object.ply --device mps
"$PY" -m image3d_student.cli preview \
  --ply results/object.ply --out results/preview.png
```

This model outputs Gaussian PLY, not a GLB or triangle mesh. It has no text
conditioner, automatic background remover, or verified natural-photo training.
The training images are renders of mesh-derived Gaussian proxies. The metadata
flag `contains_synthetic: false` means no procedural smoke-test fixtures; it does
not mean the input images are photographs or that all upstream meshes are human
made. Scenario includes generated 3D assets.

## Recovery and adjustments

Dataset preparation can resume: repeat `bash posttrain.sh prepare`. If there are
not enough eligible assets, edit `--objaverse-shards 16` to 32 in `posttrain.sh`
and repeat preparation; the upstream parser permits up to 160. If the download
budget is exhausted, the completed objects remain saved; rerun or increase the
budget after checking free space. Budgets apply per invocation.

Training requires an empty run directory. Ctrl-C saves completed steps to
`last.pt`, but the trainer does not implement exact optimizer-state resume.
The provided release pipeline intentionally expects initialization from pinned
v1, so a run restarted from an interrupted v2 checkpoint requires a separately
implemented evaluation/release path and is not supported here. Preserve the old
run and start a new output directory for a fresh attempt.

For memory pressure, close other large apps and use batch size 1 in a **fresh**
run. Keep training/evaluation devices consistent. Do not disable the MPS memory
limit. Do not promise a training duration: the fanless M4 Air may slow under load.
If evaluation fails, its diagnostic report remains in the run directory and
nothing is uploaded. Every evaluation staging attempt needs a fresh release path.

## Files and validation

`prepare_v2.py` wraps the pinned upstream preprocessing without changing its
renderer or labels. `audit_data.py` checks known v1 overlap, cache hashes, and
split separation. `stage_v2.py` reuses upstream evaluation and writes correct v2
provenance. `publish_v2.py` verifies a complete SHA256 inventory before upload.

The kit was syntax-checked; its exclusion and data-audit logic was tested with
small local fixtures. Flag compatibility was checked against the pinned parser.
The full dataset download, MPS training, and Hugging Face publication were not
executed. This is a runnable recipe requiring validation on your Mac, not a
completed training result.

## Primary sources

- [Pinned model and code](https://huggingface.co/Lucasgitguy-co/forma3d-tiny/tree/e80b8d895cc2e405894473abe4ce40b3472eaca7)
- [Original model commands](https://huggingface.co/Lucasgitguy-co/forma3d-tiny/blob/e80b8d895cc2e405894473abe4ce40b3472eaca7/COMMANDS.md)
- [Scenario dataset](https://huggingface.co/datasets/scenario-labs/3d-collection/tree/55b4c002b4c1529fc48da3dff98516c3069bfa3f)
- [Objaverse dataset](https://huggingface.co/datasets/allenai/objaverse/tree/556637099bf4fa79ea7b239d0d9c328b8a2e9ac8)

Preserve per-asset attributions. This package uses the existing Apache 2.0
implementation. Its source/data filtering additions are provided under Apache
2.0; the original NOTICE and third-party/data license terms still apply.

	# Forma3D Tiny 2 on an M4 MacBook Air with 16 GB RAM

	This is a proposed weights-only continuation of `Lucasgitguy-co/forma3d-tiny`,
	named `Lucasgitguy-co/forma3d-tiny-2`. It preserves the small FP32 architecture:
	306,321 parameters, grid 16 cubed, decoder width 64. It aims to improve coarse
	proxy reconstruction through more object diversity. No v2 training has been run
	as part of preparing this package, and improvement is not guaranteed.

	## Run the commands

	Extract the kit into a local folder. Open macOS Terminal in that folder. Use a
	folder stored locally rather than an offloaded iCloud folder. Keep the Mac on
	power and allow cooling during a sustained run.

	```bash
	# If uv is not installed, install it with Homebrew first.
	brew install uv

	# From the extracted forma3d-tiny-2-kit folder:
	bash posttrain.sh setup
	bash posttrain.sh prepare
	bash posttrain.sh train
	bash posttrain.sh evaluate
	```

	Run each command only after the previous one succeeds. Setup downloads the source
	and current v1 weights at the fixed revision
	`e80b8d895cc2e405894473abe4ce40b3472eaca7`. It installs Python 3.11 and the
	publisher's tested dependency lock in an isolated environment under
	`~/Library/Application Support/Forma3D-tiny-2/venv`. Setup does not train or upload.
	`image3d doctor` must report Metal available. These public source files execute
	locally during preparation/training; inspect the scripts before running them.

	Preparation requests 128 eligible Scenario meshes plus 128 CC0 Objaverse meshes,
	four views each. It excludes known v1 asset IDs, paths, source groups, and exact
	mesh hashes. It keeps every view of each related object in the same group. A
	separate 15 percent test split is frozen before training. Validation uses 20
	percent of the remaining development groups. Counts depend on duplicate groups;
	read `data/forma3d-v2/preparation.json` and `v2-audit.json` for the realized counts.

	The mesh-download budget is 4,096 MiB per preparation invocation, excluding dataset
	metadata and Python packages. It reserves 8 GiB of free disk while fetching meshes.
	Have comfortably more than 12 GiB available before setup and preparation; this is
	a planning allowance, not a measured total. RAM and duration have not been
	benchmarked for this exact v2 recipe. No pretrained teacher is loaded.

	Training initializes from v1 model.pt, rather than its original ellipsoid
	checkpoint. It uses fresh AdamW, batch size 2, learning rate 0.00005, 8,000 optimizer
	steps, weight decay 0.0001, gradient clipping at norm 1, seed 43, and validation
	every 200 steps. This lower learning rate is a proposed conservative setting, not
	a measured optimum. The best validation loss selects `best.pt`.

	The underlying training command is:

	```bash
	PY="$HOME/Library/Application Support/Forma3D-tiny-2/venv/bin/python"
	caffeinate -i "$PY" -m image3d_student.cli train \
	--manifest data/forma3d-v2/manifest.jsonl \
	--init-from upstream/model.pt \
	--out runs/forma3d-v2 --device mps --width 64 \
	--steps 8000 --batch-size 2 --lr 0.00005 \
	--eval-every 200 --validation-fraction 0.2 --seed 43
	```

	Do not substitute grid 32 in this recipe: it increases query cells eightfold and
	changes the evaluation representation. A width change is incompatible with the
	starting learned weights.

	## Inspect and optionally publish

	Evaluation compares v1 and the selected v2 checkpoint on **the same new test
	objects**, requiring lower test loss and higher test occupancy IoU, plus lower
	validation loss, a completed run, source separation, and nonempty qualitative
	predictions. It writes local comparisons to `runs/forma3d-v2/inspection/` and
	stages `release/forma3d-tiny-2/` only if the gates pass. Inspect these PNGs.
	The release card uses the actual run measurements and correctly records v1 as
	its parent. No scores are prefilled.

	The test set is for final assessment. If a release gate fails, do not repeatedly
	tune hyperparameters against that test set. Develop on validation data and reserve
	another genuinely unused object set for a later final claim.

	To publish after inspection:

	```bash
	"$HOME/Library/Application Support/Forma3D-tiny-2/venv/bin/hf" auth login
	bash posttrain.sh publish
	```

	Use your own Hugging Face write token in Terminal, not chat. This command creates
	`Lucasgitguy-co/forma3d-tiny-2` privately, uploads the evaluated package, and makes
	it public after the upload succeeds. It refuses to overwrite an existing repo.
	A failed upload may leave a private partial repo; inspect that repo before
	attempting recovery. Only the staged release folder is uploaded. Raw meshes,
	download caches, credentials, and optimizer state are excluded.

	## Generate a Gaussian PLY

	Use a centered object cutout on black. Replace the example path with your image:

	```bash
	PY="$HOME/Library/Application Support/Forma3D-tiny-2/venv/bin/python"
	"$PY" -m image3d_student.cli generate \
	--checkpoint runs/forma3d-v2/best.pt --image /path/to/your-cutout.png \
	--out results/object.ply --device mps
	"$PY" -m image3d_student.cli preview \
	--ply results/object.ply --out results/preview.png
	```

	This model outputs Gaussian PLY, not a GLB or triangle mesh. It has no text
	conditioner, automatic background remover, or verified natural-photo training.
	The training images are renders of mesh-derived Gaussian proxies. The metadata
	flag `contains_synthetic: false` means no procedural smoke-test fixtures; it does
	not mean the input images are photographs or that all upstream meshes are human
	made. Scenario includes generated 3D assets.

	## Recovery and adjustments

	Dataset preparation can resume: repeat `bash posttrain.sh prepare`. If there are
	not enough eligible assets, edit `--objaverse-shards 16` to 32 in `posttrain.sh`
	and repeat preparation; the upstream parser permits up to 160. If the download
	budget is exhausted, the completed objects remain saved; rerun or increase the
	budget after checking free space. Budgets apply per invocation.

	Training requires an empty run directory. Ctrl-C saves completed steps to
	`last.pt`, but the trainer does not implement exact optimizer-state resume.
	The provided release pipeline intentionally expects initialization from pinned
	v1, so a run restarted from an interrupted v2 checkpoint requires a separately
	implemented evaluation/release path and is not supported here. Preserve the old
	run and start a new output directory for a fresh attempt.

	For memory pressure, close other large apps and use batch size 1 in a fresh
	run. Keep training/evaluation devices consistent. Do not disable the MPS memory
	limit. Do not promise a training duration: the fanless M4 Air may slow under load.
	If evaluation fails, its diagnostic report remains in the run directory and
	nothing is uploaded. Every evaluation staging attempt needs a fresh release path.

	## Files and validation

	`prepare_v2.py` wraps the pinned upstream preprocessing without changing its
	renderer or labels. `audit_data.py` checks known v1 overlap, cache hashes, and
	split separation. `stage_v2.py` reuses upstream evaluation and writes correct v2
	provenance. `publish_v2.py` verifies a complete SHA256 inventory before upload.

	The kit was syntax-checked; its exclusion and data-audit logic was tested with
	small local fixtures. Flag compatibility was checked against the pinned parser.
	The full dataset download, MPS training, and Hugging Face publication were not
	executed. This is a runnable recipe requiring validation on your Mac, not a
	completed training result.

	## Primary sources

	- [Pinned model and code](https://huggingface.co/Lucasgitguy-co/forma3d-tiny/tree/e80b8d895cc2e405894473abe4ce40b3472eaca7)
	- [Original model commands](https://huggingface.co/Lucasgitguy-co/forma3d-tiny/blob/e80b8d895cc2e405894473abe4ce40b3472eaca7/COMMANDS.md)
	- [Scenario dataset](https://huggingface.co/datasets/scenario-labs/3d-collection/tree/55b4c002b4c1529fc48da3dff98516c3069bfa3f)
	- [Objaverse dataset](https://huggingface.co/datasets/allenai/objaverse/tree/556637099bf4fa79ea7b239d0d9c328b8a2e9ac8)

	Preserve per-asset attributions. This package uses the existing Apache 2.0
	implementation. Its source/data filtering additions are provided under Apache
	2.0; the original NOTICE and third-party/data license terms still apply.