{% extends "layout.html" %} {% block content %}

Linear Regression Explained

Understand the fundamentals of Linear Regression, its computational flow, and how it makes predictions.

What is Linear Regression?

Linear Regression is a fundamental supervised learning algorithm used for predicting a continuous outcome variable (dependent variable) based on one or more input features (independent variables). It models the relationship between the variables by fitting a linear equation to the observed data.

For a simple linear regression with one input feature, the equation used by our model is:
Predicted Score = (20 x Hours Studied) + 15

  • Slope (m=20): Represents how much the predicted outcome changes for every one-unit increase in the input feature. It indicates the strength and direction of the relationship.
  • Intercept (b=15): Represents the predicted outcome when all input features are zero. It's the baseline value.

Why Slope (m) is 20

The slope of 20 means each hour of studying contributes 20 points to your exam score. For example, if you study one more hour, your predicted score increases by 20 points.

Why Intercept (b) is 15

The intercept of 15 represents points earned regardless of study time. This could account for:

  • Class attendance and participation
  • Homework assignments
  • Quizzes and in-class activities
  • Base marks for attempting the exam

Computational Flow (Input to Output):

The following steps illustrate how our Linear Regression model makes predictions:

  1. Input Data: You (the user) provide a value for 'Hours Studied'.
  2. Load Model: The Flask application loads the pre-trained `supervised_model.pkl`. This model contains the learned parameters: a slope (m) of 20 and an intercept (b) of 15.
  3. Calculate: The model computes the predicted score using its linear equation:

    Predicted Score = (20 * Input Hours) + 15

    This is a simple multiplication and addition operation.
  4. Result: The calculated 'Predicted Score' is returned by the model.
  5. Display: The Flask application then renders this predicted score on the web page for you to see.

Our Training Data:

The model was trained on the following data points to learn the relationship between 'Hours Studied' and 'Score' using the equation `Score = 20 * Hours + 15`:

Hours Studied X Score Y
1 35
2 55
3 75
4 95
5 115

Cost Function Quantifying Error

When a linear regression model is being trained, it doesn't just randomly draw a line. It evaluates how good its current line is by using a Cost Function. The goal of training is to find the line i.e. the specific m and b values that minimizes this cost.

A common cost function for linear regression is the Mean Squared Error MSE. It calculates the average of the squared differences between the actual observed values $y_i$ and the values predicted by the model $\hat{y_i}$.

Mean Squared Error (MSE) Formula:

\[ \text{MSE} = \frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y}_i)^2 \]

  • N: The total number of data points.
  • \( y_i \): The actual score for data point i.
  • \( \hat{y}_i \): The predicted score for data point i, calculated as \( m \times x_i + b \).

Squaring the differences ensures that all errors are positive and penalizes larger errors more heavily. The model continuously adjusts its m and b to make this MSE value as small as possible.

Gradient Descent Learning the Best Line

Gradient Descent is an optimization algorithm used by linear regression and many other machine learning models to find the values of m and b that minimize the cost function like MSE. Imagine the cost function as a landscape with hills and valleys, and the goal is to find the lowest point (the minimum cost).

  1. Start Randomly: The algorithm starts with some initial, often random, values for m and b.
  2. Calculate Gradient: It calculates the gradient of the cost function with respect to m and b. The gradient is like a vector that points in the direction of the steepest ascent on the cost landscape.
  3. Take a Step: To minimize the cost, the algorithm takes a small step in the opposite direction of the gradient (downhill). The size of this step is controlled by a parameter called the learning rate.
  4. Repeat: Steps 2 and 3 are repeated iteratively, with m and b being updated in each iteration. With each step, the model gets closer to the optimal m and b values that minimize the cost.
  5. Convergence: This process continues until the algorithm converges, meaning the cost function stops decreasing significantly, indicating it has found the minimum or a very good approximation of it.

So, when `model.fit(X, y)` is called, behind the scenes, an optimization algorithm like Gradient Descent is tirelessly working to find the m and b that best fit your data by minimizing the prediction errors.

Visualizing the Regression Line

Slope (m): , Intercept (b):

Make a Prediction:

Predicted Score:

{% if prediction is not none %} {{ prediction | round(2) }} {% else %} --.-- {% endif %}

This is the score predicted by the linear regression model for the hours you entered.

{% endblock %}