design_docs/post_reduction_meshing.md

# Design Doc: Post-Reduction Meshing for Clean Surface Export

**Author:** Brian Clark  
**Last Updated:** 2025-11-07  
**Target Component:** Optional stage after `predictions_to_glb` point processing  
**Goal:** Convert cleaned point clouds into lightweight surface meshes for users who prefer shaded geometry over splatty point clouds.

---

## 1. Overview

After confidence filtering, voxel reduction, and denoising we hold a sparse, high-confidence point cloud.  
This design adds an optional meshing stage (ball-pivoting, Poisson, marching cubes) to produce a triangle mesh that can be exported alongside or instead of the GLB point cloud.

---

## 2. Use Cases

- Interactive viewers requiring solid surfaces (lighting, shadowing).
- Downstream pipelines that expect meshes (CAD tools, 3D printing previews).
- Scenes where splatty points look sparse even after reinflation.

We do **not** aim for watertight printable models—just visually continuous surfaces.

---

## 3. Meshing Strategies

| Method | Pros | Cons | Dependencies |
| ------ | ---- | ---- | ------------ |
| **Ball-Pivoting (BPA)** | Stable for uneven sampling, preserves detail | Needs normals, parameter tuning | Open3D |
| **Poisson Reconstruction** | Smooth surfaces, fills gaps | Blurs thin structures, more compute | Open3D |
| **Marching Cubes on fused depth** | Works without normals | Requires voxel grid, might need extra fusion step | Open3D / custom |

Initial plan: start with Open3D’s **Ball-Pivoting**, fall back to Poisson when BPA fails.

---

## 4. Pipeline Integration

1. Existing point-cleaning (voxel, support filtering, optional reinflation).
2. Normal estimation (Open3D `estimate_normals`, with KD-tree search radius tied to voxel size).
3. Mesh reconstruction (BPA default, Poisson fallback).
4. Mesh simplification (`simplify_quadric_decimation`) to hit target face count.
5. Export as GLB/OBJ/PLY; attach materials/colors using per-vertex colors (sampled from original points).

### Configuration options
| Option | Default | Notes |
| ------ | ------- | ----- |
| `meshing_enabled` | `False` | Opt-in feature |
| `meshing_method` | `"ball_pivoting"` | `"poisson"` available |
| `bpa_radii` | `[voxel_size, 2×, 4×]` | Radii list for BPA |
| `poisson_depth` | 8 | Tree depth, controls detail |
| `target_face_count` | 200k | Post-simplification triangle budget |
| `keep_point_cloud` | `True` | Export both mesh + original cloud |

---

## 5. Implementation Plan

### 5.1 Helper module
`stream3r/utils/mesh_utils.py`
```python
def build_surface_mesh(points, colors, *, config) -> trimesh.Trimesh:
    # open3d conversions, normal estimation
    # reconstruction via BPA/Poisson
    # color baking (nearest neighbor)

5.2 Export integration

In _generate_core_outputs after point cloud saved:

if settings.meshing_enabled:
    mesh = build_surface_mesh(vertices_3d, colors_rgb, config)
    mesh_url = _save_mesh(runtime, scene_id, mesh, temp_dir)
    artifacts["mesh_url"] = mesh_url

5.3 Storage

• Upload under models/meshes/scene_mesh.glb
• Optionally provide OBJ + MTL for compatibility.

---

6. Validation

1. Compare GLB mesh size vs. original point cloud.
2. Manual visual QA (Viewer, Blender) to spot holes or artifacts.
3. Automated checks:
   • Mesh exists and contains triangles.
   • Vertex count under budget.
   • Color channels preserved (mean deviation < ε).
4. Benchmark runtime per scene and ensure it fits within job timeouts.

---

7. Risks & Mitigations

Risk	Mitigation
Thin structures lost	Tune BPA radii; detect failure and revert to point cloud
Open3D dependency bloat	Gate meshing behind pip install open3d; log when unavailable
Runtime overhead	Make stage optional; expose meshing_timeout
Large meshes	Apply decimation & optional texture baking

---

8. Deliverables

1. Helper module for meshing + tests.
2. Scene artifacts update (mesh export, metadata).
3. New config flags (STREAM3R_MESHING_ENABLED, etc.).
4. Documentation/tutorial for users toggling the mesh output.

---

Outcome: An optional mesh artifact that gives viewers a solid-looking scene without fully abandoning the point-based pipeline.