harphool17/tumtraf-coopdet3d-pedboost

TUMTraf CoopDet3D Ped-Boost — 3rd Place @ DriveX 2026

🌟 Why This Model is Used & Its Benefits

This model uses Sensor Fusion (combining Cameras and LiDAR). By looking at both pictures (Camera) and 3D distance maps (LiDAR), this model is incredibly accurate at detecting large vehicles like Cars, Trucks, Vans, and Trailers.

• Benefit: It captures rich visual context (like the shape and color of a truck) while also knowing exactly how far away it is.
• Our Customization: We heavily modified the training data to force the AI to pay 25x more attention to Pedestrians, fixing a common weakness in self-driving AI.

🛠️ Training Process & Data Pipeline

We trained this model using the MMDetection3D / CoopDet3D framework on the TUMTraf-V2X dataset.

1. Data Processing

• V2X Coordinate Transformation (VehiclePointsToInfraCoords):
  • What it is: Projects all vehicle-side LiDAR and Camera data into the infrastructure's coordinate system.
  • Why we chose this: In V2X (Vehicle-to-Everything), data comes from different physical locations. If we didn't mathematically align them perfectly, the AI would see duplicate "ghost" cars.
• Multi-Sweep Merging:
  • What it is: Accumulating historical LiDAR sweeps.
  • Why we chose this: A single LiDAR scan is often sparse. Stacking historical sweeps makes the 3D point cloud much denser, helping the AI see the solid outline of vehicles.

2. Copy-Paste Augmentation (The "Ped-Boost")

• Object Paste (ObjectPasteCoop):
  • What it is: The AI cuts out 3D objects from a database and pastes them into empty areas of the road in new frames.
  • Why we chose this (The 25x Ped-Boost): The natural dataset has thousands of cars but very few pedestrians. Without intervention, the AI achieves high overall accuracy by simply ignoring pedestrians. By overriding the config (PEDESTRIAN: 25), we forced it to paste 25 Pedestrians into every training frame, forcing the AI to learn them.

3. Training Hyperparameters

• Voxelization ([0.075, 0.075, 0.2]):
  • Why we chose this: This is an extremely fine-grained, high-resolution voxel size. If voxels are too large, small objects like pedestrians and bicycles get swallowed up and vanish. Fine voxels preserve precise shapes.
• Camera Augmentations (GridMaskCoop at 50%):
  • Why we chose this: GridMask randomly places black squares over the camera image. In the real world, cameras get blocked by dirt, rain, or glare. By blinding the camera 50% of the time during training, we forced the AI to rely heavily on the LiDAR backup, making the final model highly robust to sensor failure.
• 3D LiDAR Augmentations (Scale, Rotate, Translate):
  • Why we chose this: Prevents overfitting by ensuring the AI never sees a vehicle from the exact same angle or distance twice.

4. How Accuracy Was Tracked

• We evaluated the model at the end of every epoch using the official BEV AP@IoU0.5 metric on the validation split.
• Why we early-stopped at Epoch 18: Despite training being scheduled longer, we observed that Epoch 18 achieved the highest validation precision for large vehicles. Pushing past Epoch 18 caused the model to overfit on the heavily augmented Pedestrian class, hurting vehicle accuracy. We strategically locked in Epoch 18 for our submission.

📂 Files

File	Size	Description
coopdet3d_pedboost_ep18.pth	256MB	Best checkpoint — used for CAR, TRUCK, VAN, TRAILER
coopdet3d_pedboost_ep60.pth	258MB	Ep60 checkpoint — used for CAR, BICYCLE & PEDESTRIAN union

🏆 Results (CodaBench — Final Submission)

Class	Precision	Recall	AP
TRUCK	1.000	0.941	0.941
VAN	1.000	0.950	0.950
TRAILER	0.994	0.931	0.925
CAR	0.967	0.902	0.872