You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

1. Overview

We, Team Geggle Sprints, has developed a Car Accident Scenario Prediction model fine-tuning the PaliGemma.

[One Take, Report the Scenario]
This service allows users to share a car accident scenario with just a single photo taken at the accident scene. Many drivers are overwhelmed when faced with an unexpected car accident. Moreover, in most car-sharing services, the process to report the accident is often confusing, as drivers must take unnecessary steps to connect with the insurance provider via the platform’s customer service.

By simply uploading a photo of the accident scene to our Car Accident Scenario Prediction model, it generates a precise description of the accident in natural language. This helps drivers swiftly file insurance claims and focus on handling the accident scene.

The model has been fine-tuned on Google’s VLM, PaliGemma, using our custom dataset to generate human-level car accident scenarios. We created 110 detailed annotations based on official terms for vehicle parts and formats from government-issued accident reports. The model was trained by iteratively adjusting the prompts for optimal accuracy.

[What’s next: AI Customer Service]
Our next step is to train the model on insurance companies' accident response guidelines and integrate Text-to-Speech technology. This will enable the service to provide detailed on-scene response instructions via AI voice assistance, all triggered by a single photo.

Team Geggle Sprints, supported by Google Machine Learning Bootcamp 2024 #GemmaSprint

Model Architecture

We employed the Paligemma model, a large-scale image and text processing model, pre-trained on a vast dataset and fine-tuned for this specific task.

The fine-tuned Paligemma model showed notable improvements in predicting accident scenarios compared to the base model. (Please insert actual metrics here once evaluated).

Key Model Details:

Vocab Size: 257,152
Image Variant: So400m/14
Pooling Type: None
Data Type: Float16

Training Process

The model was trained using a cosine learning rate schedule with the following configuration:

Batch Size: 8
Learning Rate: 0.03
Total Training Steps: 64
Evaluation Steps: 16

During each evaluation, the model’s predictions were validated using unseen images in a domain different from the training data.

Results

2. Compare results

2.1 Before apply Car datasets

Train

Prediction

2.2 After applying Car Dataset

Train

Prediction

Conclusion

This project demonstrates the capability of Paligemma in handling complex prediction tasks such as car accident scenario prediction. Fine-tuning the model significantly improved its performance, suggesting potential use cases in autonomous driving and safety analysis.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support