Spaces:
Configuration error
Configuration error
File size: 4,255 Bytes
31dbf53 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 |
```markdown
# Design Doc: Post-Reduction Meshing for Clean Surface Export
**Author:** Brian Clark
**Last Updated:** 2025-11-07
**Target Component:** Optional stage after `predictions_to_glb` point processing
**Goal:** Convert cleaned point clouds into lightweight surface meshes for users who prefer shaded geometry over splatty point clouds.
---
## 1. Overview
After confidence filtering, voxel reduction, and denoising we hold a sparse, high-confidence point cloud.
This design adds an optional meshing stage (ball-pivoting, Poisson, marching cubes) to produce a triangle mesh that can be exported alongside or instead of the GLB point cloud.
---
## 2. Use Cases
- Interactive viewers requiring solid surfaces (lighting, shadowing).
- Downstream pipelines that expect meshes (CAD tools, 3D printing previews).
- Scenes where splatty points look sparse even after reinflation.
We do **not** aim for watertight printable models—just visually continuous surfaces.
---
## 3. Meshing Strategies
| Method | Pros | Cons | Dependencies |
| ------ | ---- | ---- | ------------ |
| **Ball-Pivoting (BPA)** | Stable for uneven sampling, preserves detail | Needs normals, parameter tuning | Open3D |
| **Poisson Reconstruction** | Smooth surfaces, fills gaps | Blurs thin structures, more compute | Open3D |
| **Marching Cubes on fused depth** | Works without normals | Requires voxel grid, might need extra fusion step | Open3D / custom |
Initial plan: start with Open3D’s **Ball-Pivoting**, fall back to Poisson when BPA fails.
---
## 4. Pipeline Integration
1. Existing point-cleaning (voxel, support filtering, optional reinflation).
2. Normal estimation (Open3D `estimate_normals`, with KD-tree search radius tied to voxel size).
3. Mesh reconstruction (BPA default, Poisson fallback).
4. Mesh simplification (`simplify_quadric_decimation`) to hit target face count.
5. Export as GLB/OBJ/PLY; attach materials/colors using per-vertex colors (sampled from original points).
### Configuration options
| Option | Default | Notes |
| ------ | ------- | ----- |
| `meshing_enabled` | `False` | Opt-in feature |
| `meshing_method` | `"ball_pivoting"` | `"poisson"` available |
| `bpa_radii` | `[voxel_size, 2×, 4×]` | Radii list for BPA |
| `poisson_depth` | 8 | Tree depth, controls detail |
| `target_face_count` | 200k | Post-simplification triangle budget |
| `keep_point_cloud` | `True` | Export both mesh + original cloud |
---
## 5. Implementation Plan
### 5.1 Helper module
`stream3r/utils/mesh_utils.py`
```python
def build_surface_mesh(points, colors, *, config) -> trimesh.Trimesh:
# open3d conversions, normal estimation
# reconstruction via BPA/Poisson
# color baking (nearest neighbor)
```
### 5.2 Export integration
In `_generate_core_outputs` after point cloud saved:
```python
if settings.meshing_enabled:
mesh = build_surface_mesh(vertices_3d, colors_rgb, config)
mesh_url = _save_mesh(runtime, scene_id, mesh, temp_dir)
artifacts["mesh_url"] = mesh_url
```
### 5.3 Storage
- Upload under `models/meshes/scene_mesh.glb`
- Optionally provide OBJ + MTL for compatibility.
---
## 6. Validation
1. Compare GLB mesh size vs. original point cloud.
2. Manual visual QA (Viewer, Blender) to spot holes or artifacts.
3. Automated checks:
- Mesh exists and contains triangles.
- Vertex count under budget.
- Color channels preserved (mean deviation < ε).
4. Benchmark runtime per scene and ensure it fits within job timeouts.
---
## 7. Risks & Mitigations
| Risk | Mitigation |
| ---- | ---------- |
| Thin structures lost | Tune BPA radii; detect failure and revert to point cloud |
| Open3D dependency bloat | Gate meshing behind `pip install open3d`; log when unavailable |
| Runtime overhead | Make stage optional; expose `meshing_timeout` |
| Large meshes | Apply decimation & optional texture baking |
---
## 8. Deliverables
1. Helper module for meshing + tests.
2. Scene artifacts update (mesh export, metadata).
3. New config flags (`STREAM3R_MESHING_ENABLED`, etc.).
4. Documentation/tutorial for users toggling the mesh output.
---
**Outcome:** An optional mesh artifact that gives viewers a solid-looking scene without fully abandoning the point-based pipeline.
```
|