Upload 3 files

4c2ae51 verified 4 months ago

15.4 kB

📦 DataCo Smart Supply Chain: Profit Prediction and Loss Detection

Author: Ido Yaaran Assignment: Assignment 2, Classification, Regression, Clustering and Evaluation Dataset: DataCo Smart Supply Chain via Kaggle Repository: idoyaaran/dataco-supply-chain-model

🎥 Presentation

📌 Project Overview

This project builds a full machine learning pipeline on the DataCo Smart Supply Chain dataset, a real-world logistics dataset covering 180,000+ supply chain orders across customers, shipping modes, product categories, geographies, and profit margins.

Two prediction goals:

Regression: Given order details available at placement time, how profitable will this order be?

Classification: Will this order result in a financial loss?

Why it matters: Nearly 1 in 5 orders in this dataset loses money. If a model can flag likely-loss orders before dispatch, the logistics team has a window to act. That could mean rerouting, absorbing cost early, or notifying the customer proactively before the damage is done.

🗺️ Full Project Workflow

Raw Dataset (180,516 rows x 53 features)
    |
Part 2: EDA and Cleaning
  |-- Column audit: drop PII, constants, leaky columns, redundant columns
  |-- Fix data types (dates, categoricals)
  |-- Drop 3 rows with missing zip codes
  |-- Confirm 0 duplicate rows
  |-- Final: 180,513 rows x 36 columns
  |-- 6 research questions with Plotly visualizations
    |
Part 3: Baseline Regression
  |-- Target: Benefit per order (USD profit/loss)
  |-- Linear Regression, default parameters, no feature engineering
  |-- Result: R2 = 0.016 (floor to beat)
    |
Part 4: Feature Engineering and Clustering
  |-- 10 new features: date parts, ratios, binary flags
  |-- KMeans k=4 (elbow method), PCA visualization
  |-- Add cluster label and cluster distance as features
    |
Part 5: Three Improved Regression Models
  |-- Linear Regression (engineered), Random Forest, Gradient Boosting
  |-- Random Forest wins: R2 = 0.481, MAE = 47.79 USD
  |-- Feature importance visualization for all models
    |
Part 6: Export Regression Winner
  |-- regression_model.pkl uploaded to HuggingFace
    |
Part 7: Regression to Classification
  |-- Business rule threshold: profit < 0 = Loss, profit >= 0 = Profitable
  |-- Class split: 81.2% Profitable / 18.8% Loss
    |
Part 8: Three Classification Models
  |-- Logistic Regression, Random Forest, Gradient Boosting
  |-- Evaluated on F1 and recall for Loss class (not accuracy)
  |-- Logistic Regression wins: Macro F1 = 0.50
  |-- classification_model.pkl uploaded to HuggingFace
    |
Upload notebook + models + dataset + README to HuggingFace

📂 Repository Contents

File	Description
`DataCo_assignment_2(final edition).ipynb`	Full notebook with all parts and outputs
`regression_model.pkl`	Winning regression model (Random Forest)
`classification_model.pkl`	Winning classification model (Logistic Regression)
`DataCoSupplyChainDataset.csv`	Full dataset, load directly in the notebook
`charts/`	All visualizations exported as PNG
`README.md`	This file

📊 Dataset Overview

Property	Value
Source	Kaggle, DataCo Smart Supply Chain
Raw size	180,516 rows x 53 features
After cleaning	180,513 rows x 36 features
Regression target	`Benefit per order` (continuous USD profit/loss)
Classification target	Binary: Loss (profit below 0) / Profitable (profit at or above 0)
Class distribution	81.2% Profitable / 18.8% Loss
Domain	Global e-commerce supply chain operations

🔍 Part 2: EDA and Key Findings

Cleaning Summary

Step	Action	Reason
PII columns	Dropped (email, name, password, street)	No predictive value, privacy
`Product Description`	Dropped	100% empty
`Order Zipcode`	Dropped	86% missing values
`Product Status`	Dropped	Constant column, all zeros
Duplicate columns	Dropped	`Order Profit Per Order`, `Order Item Total`, `Product Price` were exact copies of other columns
ID columns	Dropped	Arbitrary numbers, no signal
`Customer Zipcode`	3 rows dropped	Only 3 rows missing, negligible
Date columns	Converted to datetime	Required for extracting time features

Two columns were kept for EDA but excluded from modeling because they are leaky: Delivery Status and Days for shipping (real). Both only exist after the order is complete and would not be available at prediction time.

Q1: Which shipping mode has the highest late delivery rate?

First Class sits at 95.3% late delivery rate. Standard Class is 38.1%. Customers paying more for premium shipping are getting the worst service. This is a systemic operational failure, not a random fluctuation.

Q2: How has order volume changed over time?

Volume is flat at around 5,000 orders per month throughout the dataset period. No growth, no decline, no seasonal spike. Profit per month tracks the same flat line.

Q3: Which countries have the highest late delivery risk?

Late delivery risk is between 50-70% almost everywhere globally. Geography is not the driver of late deliveries. The problem is internal to operations.

Q4: Does a higher discount lead to lower profit?

No relationship. Loss orders appear at every discount level including zero discount. Whatever causes losses, discounting is not it.

Q5: How many orders are actually losses?

18.7% of all orders are losses. The distribution has a long left tail reaching -4,000 USD on individual shipments. These are not rounding errors, they are real substantial losses mixed in with profitable orders.

Q6: How well does each shipping mode stick to its schedule?

First Class has almost no spread in delivery gap: it is always exactly one day late. Second Class has the widest spread, making its delivery timing the least predictable for customers.

⚙️ Part 4: Feature Engineering

The raw features do not give tree models much to differentiate orders. These features were added before modeling:

Feature	How it was built	Signal
`order_month`	`.dt.month` from order date	Seasonal shipping patterns
`order_year`	`.dt.year` from order date	Year over year trends
`order_dayofweek`	`.dt.dayofweek` from order date	Monday vs Friday ordering
`order_quarter`	`.dt.quarter` from order date	Q4 holiday rush
`order_hour`	`.dt.hour` from order date	Time of day patterns
`revenue_per_item`	Sales divided by quantity	Average item value in the order
`discount_to_price_ratio`	Discount divided by product price	Relative discount size
`is_high_discount`	1 if discount rate above 15%	Binary flag for aggressive discounts
`is_urgent`	1 if scheduled days at or below 2	Binary flag for tight shipping windows
`is_weekend`	1 if day of week is Saturday or Sunday	Binary flag for weekend orders
`cluster`	KMeans (k=4) label	Which order segment this belongs to
`cluster_distance`	Euclidean distance to cluster centroid	How atypical this order is

The cluster_distance feature is worth highlighting. Orders far from their cluster centroid are unusual in some way. Unusual orders tend to be the ones that are hardest to predict and most likely to affect profit in unexpected ways.

🔵 Part 4: Clustering

KMeans was run with k=4, chosen from the elbow plot where the inertia curve flattens. Clusters were visualized in 2D using PCA.

Three clusters overlapped in PCA space, mixed order types with similar overall profiles. Cluster 3 sat completely isolated in the top corner. It turned out to be almost entirely high-ticket Dell Laptop orders: a single product category with a completely different price, discount, and profit profile from everything else in the dataset.

Cluster	Profile
0	Mixed standard orders
1	Mixed standard orders
2	Mixed standard orders
3	High-ticket Dell Laptop orders, isolated in PCA space

📈 Part 3: Baseline Regression

Linear Regression with no feature engineering gave R2 = 0.016. The model explained almost nothing. This chart shows how scattered the predictions are around the actual profit values.

📈 Part 5: Regression Models

Results

Model	MAE (USD)	RMSE (USD)	R2
Baseline Linear Regression	53.77	103.30	0.016
Linear Regression (engineered features)	57.36	86.00	0.318
Random Forest	47.79	75.05	0.481
Gradient Boosting	52.86	93.25	0.198

Winner: Random Forest (R2 = 0.481)

The most important result in the entire regression section is not which model won. It is the jump from R2 = 0.016 to R2 = 0.318 using the exact same Linear Regression algorithm but with engineered features. Better inputs beat a better algorithm.

Random Forest won because order profitability is driven by non-linear interactions between shipping mode, product category, customer segment, and discount that no linear model can capture. Gradient Boosting underperformed at default parameters. It is more sensitive to tuning than Random Forest and would likely improve significantly with hyperparameter search.

An R2 of 0.481 means the model explains roughly half of what drives per-order profit. For a logistics operation handling 180,000+ orders, that is actionable.

Feature Importance

🏷️ Part 7: Regression to Classification

The continuous profit target was converted to binary using a business rule threshold:

Class	Label	Rule	Proportion
0	Loss	Profit below 0 USD	18.8%
1	Profitable	Profit at or above 0 USD	81.2%

This threshold was chosen because it is the most operationally meaningful cutoff. The question it answers is not "is this order in the bottom half of profit?" but "will this order cost the company money?" That is a question a logistics team can act on.

The 81/19 class split means accuracy is a misleading metric. A model predicting "Profitable" for every single order would score 81% accuracy without learning anything. F1 score and recall for the Loss class are the metrics that matter here.

🧠 Part 8: Classification Models

Results

Model	Loss Recall	Loss F1	Macro F1
Logistic Regression	0.05	0.10	0.50
Random Forest	0.04	0.08	0.49
Gradient Boosting	0.01	0.01	0.46

Winner: Logistic Regression

All three models hit around 82% accuracy. All three also largely learned to predict "Profitable" for almost every order. That is the class imbalance problem in practice.

Gradient Boosting had perfect precision for Loss orders: whenever it predicted Loss, it was correct. But it only made that prediction 74 times out of 6,664 actual loss orders. A model that catches 1% of the cases it is supposed to catch is not useful.

Logistic Regression won with the best macro F1 and best recall for Loss orders. The simplest model outperformed both tree models, which is a reminder that complexity does not automatically mean better results. Fixing this problem properly would require class weighting or oversampling techniques to force the model to pay more attention to the minority class.

💡 Key Takeaways

Feature engineering matters more than model choice. The same Linear Regression algorithm improved R2 by 20x just from better input features.

Accuracy is the wrong metric for imbalanced data. 82% accuracy sounds reasonable until you realise a trivial baseline also scores 82%.

Class imbalance is a data problem, not a model problem. Switching from Logistic Regression to Random Forest to Gradient Boosting did not fix it. The fix requires changing how the training data represents the minority class.

First Class shipping is broken. A 95% late delivery rate for the most expensive shipping option is not a statistical artifact. It is an operational problem that the data makes impossible to ignore.

🚀 How to Load the Models

import pickle
import urllib.request

reg_url = 'https://huggingface.co/idoyaaran/dataco-supply-chain-model/resolve/main/regression_model.pkl'
clf_url = 'https://huggingface.co/idoyaaran/dataco-supply-chain-model/resolve/main/classification_model.pkl'

urllib.request.urlretrieve(reg_url, 'regression_model.pkl')
urllib.request.urlretrieve(clf_url, 'classification_model.pkl')

with open('regression_model.pkl', 'rb') as f:
    reg_model = pickle.load(f)

with open('classification_model.pkl', 'rb') as f:
    clf_model = pickle.load(f)

# Regression: predicted profit in USD
y_profit = reg_model.predict(X_new)

# Classification: 0 = Loss, 1 = Profitable
y_class = clf_model.predict(X_new)
y_proba = clf_model.predict_proba(X_new)

📦 Requirements

pandas>=1.3
numpy>=1.21
scikit-learn>=1.0
matplotlib>=3.4
plotly>=5.0
scipy>=1.7

📋 Assignment Structure

Part	Description	Output
Part 1	Dataset selection	DataCo Smart Supply Chain chosen
Part 2	EDA and cleaning	180,513 rows x 36 columns, 6 research questions
Part 3	Baseline regression	R2 = 0.016 floor established
Part 4	Feature engineering and clustering	12 new features, KMeans k=4
Part 5	Three improved regression models	Random Forest wins, R2 = 0.481
Part 6	Export regression winner	`regression_model.pkl`
Part 7	Regression to classification	81/19 class split, business rule threshold
Part 8	Three classification models	Logistic Regression wins, Macro F1 = 0.50
Part 9	Presentation video	Link above

✅ Submission Checklist

Submit to Moodle only one link: the link to this HuggingFace Model Repository.

Item	Status
`README.md`	✅
Python Notebook (`.ipynb`)	✅
Regression model (`regression_model.pkl`)	✅
Classification model (`classification_model.pkl`)	✅
Video Presentation (`dataco-supply-chain-model.mp4`)	✅

Assignment 2, Intro to Data Science, May 2026