eliel2003/car-price-predict

🚗 Car Price Intelligence: From Raw Specs to Market Predictions

A full-cycle Machine Learning pipeline for automotive price prediction and market segmentation

Regression · Classification · Clustering · Feature Engineering · Dimensionality Reduction

---

Project Overview & Data Walkthrough

📋 Abstract

This project presents a comprehensive Machine Learning study on the Car Features and MSRP dataset (CooperUnion, Kaggle), comprising 11,914 vehicles spanning model years 1990–2017 across 16 features. The central research question is:

Which observable car attributes — engine specifications, drivetrain configuration, brand identity, and fuel efficiency — most significantly determine a vehicle's Manufacturer's Suggested Retail Price (MSRP), and can these relationships be leveraged to build a deployable price prediction system?

The pipeline encompasses the full ML lifecycle: structured Exploratory Data Analysis, OOP-based Data Cleaning with intelligent imputation, multi-strategy Feature Engineering (including unsupervised clustering), Regression modelling for continuous price prediction, and a Regression-to-Classification conversion for market-tier segmentation. The final winning models are exported and made available in this repository for inference and further research.

---

📁 Repository Contents

├── car_price_model.pkl          # 🏆 Winning regression model (Gradient Boosting)
├── classification_model.pkl     # 🏆 Winning classifier (Gradient Boosting)
├── correlation_heatmap.png      # EDA — feature correlation matrix
├── cluster_pca.png              # Feature Engineering — K-Means + PCA
├── confusion_matrix_gb.png      # Classification — Gradient Boosting confusion matrix
├── feature_importance_reg.png   # Regression — top feature importances
├── feature_importance_cls.png   # Classification — top feature importances
└── README.md

---

📊 Interactive EDA — The Data Story

The Dataset at a Glance

Property	Value
Source	Kaggle — CooperUnion/cardataset
Raw Shape	11,914 rows × 16 columns
Final Shape (post-cleaning)	≥ 10,000 rows × 30+ engineered columns
Target Variable	MSRP (USD) — continuous for regression, 3-class for classification
Feature Mix	8 numeric + 5 categorical (+ engineered features)
Missing Values	Engine HP (0.58%), Engine Cylinders (0.25%), Market Category (31.4%)

🔍 Key EDA Findings

• MSRP is heavily right-skewed — a small cohort of exotic vehicles (Bugatti, Lamborghini, Maybach) creates a long tail exceeding $200k, making log-transformation essential for linear models.
• Engine HP is the single strongest predictor of price (Pearson r ≈ +0.65), with a non-linear, accelerating relationship — a 100-HP gain at the high end adds disproportionately more to price than the same gain at the low end.
• Fuel efficiency (MPG) is inversely correlated with price overall, but this relationship inverts within the SUV segment — larger, more expensive SUVs also consume more fuel, a key anomaly surfaced during EDA.
• Brand identity (captured via Target Encoding) is the dominant pricing signal when combined with engine specs, explaining the majority of variance in the model.

🌡️ Correlation Heatmap

The heatmap reveals strong positive correlations between MSRP and engine-related features (HP, Cylinders), and a meaningful negative correlation with fuel efficiency metrics — confirming that performance and economy sit at opposite ends of the price spectrum.

---

⚗️ The "Secret Sauce" — Feature Engineering

Feature engineering is where this project goes beyond a standard tutorial. Rather than feeding raw columns into a model, we constructed a rich, information-dense feature space through four compounding strategies:

1. 🏷️ Intelligent Categorical Encoding

Naïve one-hot encoding of Make (48 unique brands) would add 47 sparse binary columns with no ordinal information. Instead:

Column	Strategy	Rationale
Make	Target Encoding (mean MSRP per brand)	Condenses brand prestige into one continuous signal
Model	Frequency Encoding (count per model)	Encodes market popularity without dimensionality explosion
Driven Wheels, Transmission, etc.	One-Hot Encoding	Low cardinality (≤ 12 categories) — safe to OHE

2. 🔧 Domain-Knowledge Features

Feature	Formula	Intuition
car_age	2017 − year	Newer models command higher prices
hp_per_cyl	engine_hp / engine_cylinders	Power density — distinguishes sports from economy engines
hp_age_interaction	engine_hp / (car_age + 1)	High power and new → exponential premium
is_luxury	1 if make_target_enc ≥ p75	Binary flag capturing brand-tier threshold effects
mpg_avg	(city_mpg + highway_mpg) / 2	Composite efficiency signal

3. 🤖 Unsupervised Clustering as a Supervised Feature

The most technically distinctive component of this project: K-Means clustering was applied to the scaled numeric feature space to discover latent market segments, and the resulting cluster_id and dist_to_centroid were injected as new features into the supervised models.

Why this works: A car's membership in a market segment (e.g., "high-power, low-efficiency, premium-brand SUV") carries pricing information that no single individual feature fully captures. Cluster membership encodes this holistic profile as a learnable signal.

The optimal number of clusters was selected automatically using the Silhouette Score, with results validated visually via PCA dimensionality reduction to 2 components:

Left panel: cluster membership in PCA space, with centroids marked (✕). Right panel: the same projection coloured by log(MSRP) — the strong alignment between cluster boundaries and price gradient confirms that the discovered segments are genuine price tiers, not statistical artefacts.

4. 📏 Scaling Strategy

StandardScaler was applied to all continuous features for linear models. Tree-based models received raw (unscaled) features, as decision tree splits are scale-invariant. The target variable MSRP was never scaled, preserving full interpretability of predictions in USD.

---

📈 Regression Models — Predicting the Exact Price

Model Architecture

Three regression models were trained on the engineered feature matrix using an 80/20 train-test split with random_state=42 for full reproducibility.

🏆 Model Comparison Table

Model	MAE (USD)	RMSE (USD)	R²	Notes
Baseline Linear Regression (Part 3)	18855.77	38494.17	0.5865	Raw features, no engineering
Retrained Linear Regression (Part 5)	9744.57	19309.38	0.9230	Full engineered feature matrix
Random Forest Regressor (Part 5)	4385.84	29115.68	0.8250	300 trees, max_features="sqrt"
🏆 Gradient Boosting Regressor (Part 5)	4170.55	14456.17	0.9569	500 trees, lr=0.05, depth=5

Why Gradient Boosting Wins

Linear Regression — even enriched with engineered features — is fundamentally constrained to additive, linear decision boundaries. Car pricing is non-linear: crossing from 6 to 8 cylinders does not produce a constant additive price increment — it triggers a market-tier reclassification that multiplies price. Gradient Boosting learns these thresholds automatically through sequential residual correction, with each of its 500 trees trained specifically to fix the errors left by its predecessors. The combination of a low learning rate (0.05), stochastic subsampling (80%), and shallow tree depth (5) achieves the optimal bias-variance balance for this dataset.

📉 Feature Importance — Regression

---

🎯 Classification Deep-Dive — Market Tier Segmentation

The Regression-to-Classification Transition

Beyond point-estimate price prediction, a practical business application requires understanding which market tier a vehicle belongs to. We convert the continuous MSRP target into three discrete classes using Quantile Binning — dividing the price distribution into three equally-populated thirds:

Class	Price Range (approx.)	Business Meaning
🟦 Budget	$0 — ~$21,000	Economy & entry-level vehicles
🟧 Mid-Range	~$21,000 — ~$40,000	Mainstream & near-premium vehicles
🟩 Luxury	~$40,000+	Premium, performance & exotic vehicles

Quantile Binning was chosen over fixed thresholds because it guarantees balanced class sizes (~33% each), preventing the dominant-class bias that undermines accuracy as a headline metric.

📐 Evaluation Metric Rationale

Accuracy alone is misleading. A naïve model predicting the majority class achieves high accuracy without learning anything — and even with balanced classes, accuracy treats all misclassifications equally. Confusing Budget with Luxury carries severe financial consequences; confusing Budget with Mid-Range does not. We therefore report:

• Primary metric: Weighted F1-Score (balances Precision and Recall across all three classes)
• Business-critical metric: Precision on Luxury class (minimises False Positives = overbidding risk)

The dealership framing: A False Positive on Luxury (predicting a $15,000 car is worth $80,000) leads to direct, unrecoverable financial loss. A False Negative (missing a genuine Luxury vehicle) is an opportunity cost — painful but survivable. This asymmetry makes Precision on the Luxury class the single most business-critical number in the classification evaluation.

🏆 Classification Model Comparison

Model	Weighted F1	Luxury Precision	Luxury Recall
Logistic Regression	0.8584	0.89	0.89
Random Forest Classifier	0.8866	0.91	0.90
🏆 Gradient Boosting Classifier	0.8919	0.91	0.91

🔲 Confusion Matrix — Winning Classifier

The confusion matrix reveals that the vast majority of errors occur at the adjacent class boundaries (Budget↔Mid-Range and Mid-Range↔Luxury), which are the economically lower-risk mistakes. Cross-boundary errors (Budget predicted as Luxury or vice versa) are rare, confirming that the model has successfully internalised the market tier structure.

📊 Feature Importance — Classification

---

🔬 How to Use the Models

Load and Run Inference

import pickle
import pandas as pd

# ── Regression model (predicts exact MSRP in USD) ────────────
with open("car_price_model.pkl", "rb") as f:
    reg_model = pickle.load(f)

# ── Classification model (predicts Budget / Mid-Range / Luxury)
with open("classification_model.pkl", "rb") as f:
    cls_model = pickle.load(f)

# ── Example inference ─────────────────────────────────────────
# X_new must match the engineered feature matrix from Part 4
predicted_price = reg_model.predict(X_new)
predicted_tier  = cls_model.predict(X_new)

print(f"Predicted MSRP  : ${predicted_price[0]:,.0f}")
print(f"Predicted Tier  : {predicted_tier[0]}")

⚠️ Important: Both models expect the fully engineered feature matrix produced by the Part 4 pipeline (scaled numerics, target-encoded Make, frequency-encoded Model, OHE categoricals, cluster features). Raw CSV columns cannot be passed directly.

---

🛠️ Technical Stack

Component	Library / Tool
Data manipulation	pandas, numpy
Visualisation	matplotlib, seaborn
ML models	scikit-learn
Dimensionality reduction	sklearn.decomposition.PCA
Clustering	sklearn.cluster.KMeans
Model serialisation	pickle
Dataset source	Kaggle (CooperUnion/cardataset)
Model hosting	Hugging Face Hub

---

📚 Methodology Summary

Raw Data (11,914 × 16)
        │
        ▼
┌───────────────────┐
│  Data Cleaning    │  OOP DataCleaner class · Group-median imputation
│  (Part 2)         │  · Market Category → one-hot sub-labels
└────────┬──────────┘
         │
         ▼
┌───────────────────┐
│  EDA & Viz        │  4 research questions · Correlation analysis
│  (Part 2)         │  · Outlier detection & justified handling
└────────┬──────────┘
         │
         ▼
┌───────────────────┐
│  Feature Eng.     │  Target/Frequency encoding · Domain features
│  (Part 4)         │  · K-Means clustering · PCA validation
└────────┬──────────┘
         │
         ▼
┌───────────────────┐
│  Regression       │  Linear → Random Forest → Gradient Boosting
│  (Parts 3 & 5)    │  Reusable evaluate_model() & plot_predictions()
└────────┬──────────┘
         │
         ▼
┌───────────────────┐
│  Classification   │  Quantile binning → 3 market tiers
│  (Parts 7 & 8)    │  Logistic → RF → Gradient Boosting classifier
└────────┬──────────┘
         │
         ▼
  Deployed Models (.pkl) on Hugging Face Hub

---

👤 Author

Eliel Halfon
Machine Learning Project — Automotive Price Intelligence

---

Built with precision. Validated with rigour. Deployed with purpose.