Spaces:

QaziAbsaar
/

ML-Project

Sleeping

Kaggle Account
- Sign up at https://www.kaggle.com
- Free account is sufficient
Kaggle API Setup

Install kaggle CLI:
```
pip install kaggle
```

Step-by-Step Setup

1. Get Your Kaggle Credentials

Go to https://www.kaggle.com/settings/account
Click "Create New API Token"
This downloads kaggle.json

2. Place Credentials

On Linux/Mac:

mkdir -p ~/.kaggle
cp kaggle.json ~/.kaggle/
chmod 600 ~/.kaggle/kaggle.json

On Windows:

Place kaggle.json in: C:\Users\<YourUsername>\.kaggle\

3. Download the Dataset

Method A: Using the provided script

cd ML_Pro
python download_dataset.py

This:

Authenticates with Kaggle
Downloads the official dataset
Extracts to data/ directory
Validates file structure

Method B: Using Kaggle CLI directly

kaggle competitions download -c nyc-taxi-fare-prediction
unzip -d data/ nyc-taxi-fare-prediction.zip

Method C: Manual download

Visit: https://www.kaggle.com/c/nyc-taxi-fare-prediction/data
Click "Download All"
Extract contents to data/ folder

Verify Download

After download, verify the dataset:

# Check file exists
ls -lh data/

# Check CSV structure
head -5 data/train.csv

# Check row count (takes 1-2 minutes)
wc -l data/train.csv

Expected output:

-rw-r--r--  1 user  staff  2.6G  May  7 12:34 train.csv

Option 2: Generate Synthetic Data (Quick Testing)

For quick testing without downloading the large dataset:

python generate_synthetic_data.py

This creates: data/nyc_taxi_fare.csv (100,000 records, ~6 MB)

Advantages:

✓ Instant generation (< 30 seconds)
✓ No large download required
✓ Realistic data structure
✓ Good for prototyping

Disadvantages:

✗ Not real historical data
✗ Smaller dataset (100K vs 55M records)
✗ For assignment validation only

Option 3: Sample Dataset (Medium Size)

For intermediate testing with real data:

# Download and sample first 500K rows
head -500001 data/train.csv > data/train_sample.csv

# Or use Python
python -c "
import pandas as pd
df = pd.read_csv('data/train.csv', nrows=500000)
df.to_csv('data/train_sample.csv', index=False)
"

Dataset Format

Columns (Expected)

Column	Type	Description
`key`	string	Unique identifier
`fare_amount`	float	Taxi fare ($) - TARGET
`pickup_datetime`	string	Pickup timestamp
`pickup_longitude`	float	Pickup location longitude
`pickup_latitude`	float	Pickup location latitude
`dropoff_longitude`	float	Dropoff location longitude
`dropoff_latitude`	float	Dropoff location latitude
`passenger_count`	int	Number of passengers

Example Row

key,fare_amount,pickup_datetime,pickup_longitude,pickup_latitude,dropoff_longitude,dropoff_latitude,passenger_count
2015-03-25 07:30:35.000000107,14.5,2015-03-25 07:30:35 UTC,-73.999619,40.734462,-73.988403,40.733444,1

Data Characteristics

Statistics

Fare Range: $0 - $500
Passenger Range: 1 - 8
Time Span: 2009-2015 (mostly 2015)
Records: ~55 million

Quality Issues (Handled by preprocessing)

Missing Values: Some records have NaN coordinates
Outliers: Some fares exceed typical NYC prices
Duplicates: May exist in raw data
Invalid Coordinates: Out of NYC bounds

Our taxi_fare.py preprocessing handles all these issues.

Troubleshooting

Issue: Kaggle authentication fails

Error: Kaggle API not installed
Solution: pip install kaggle

Error: ~/.kaggle/kaggle.json not found
Solution: Download from https://www.kaggle.com/settings/account

Issue: Dataset download is slow

Solution:

Kaggle isn't always fast; patience required
Alternative: Try Method C (manual download)
Or use the synthetic data (generate_synthetic_data.py)

Issue: Disk space insufficient

Full dataset: ~2.6 GB
After processing: ~500 MB
Ensure you have 5 GB free space

Solution if low on space:

# Use only first 100K rows for training
python train.py --sample-size 100000

Issue: File corruption during download

Try re-downloading:

rm data/train.csv
python download_dataset.py

File Paths (Important)

The training script expects the dataset at:

data/nyc_taxi_fare.csv  ← This is the expected location

If your file has a different name:

# Rename it
mv data/train.csv data/nyc_taxi_fare.csv

# Or specify in training
python train.py --data data/train.csv

Assignment Submission Note

IMPORTANT: For assignment submission, ensure:

✓ The dataset used is NYC Taxi Fare Prediction
✓ Source: Kaggle (https://www.kaggle.com/c/nyc-taxi-fare-prediction)
✓ Documentation of which dataset split was used
✓ Reproducible preprocessing

This ensures compliance with assignment requirements.

Additional Resources

Kaggle Competition: https://www.kaggle.com/c/nyc-taxi-fare-prediction
Kaggle API Documentation: https://docs.kaggle.com/api
NYC Taxi Data Blog: https://chriswhong.com/open-data/

Quick Start Checklist

Choose Option 1, 2, or 3 above
Download and extract dataset
Place at data/nyc_taxi_fare.csv
Run: python train.py
Check: artifacts/taxi_fare_ann_model.joblib created
Success: Metrics displayed in console

Last Updated: May 7, 2026
Status: ✓ All options validated