Machine Learning Evaluation Metrics

Understanding Errors, Bias, Variance & Model Performance

Learn how machine learning models make mistakes, how underfitting and overfitting occur, and how professional data scientists evaluate models.

Start Learning Explore Metrics

Why Evaluation Matters?

  • Measure Model Quality
  • Detect Overfitting
  • Detect Underfitting
  • Improve Accuracy
  • Select Best Algorithm
  • Increase Generalization

Accuracy

Overall Correctness

Precision

Correct Positives

Recall

Find All Positives

F1 Score

Balanced Metric

Learning Roadmap

Topics Covered in this Module

Error

Understand actual values, predicted values, and model mistakes.

Bias

Wrong assumptions causing underfitting.

Variance

Learning noise and overfitting.

Underfitting

Model too simple.

Overfitting

Model too complex.

Tradeoff

Find the optimal model.

Understanding Error

Every Machine Learning model makes mistakes. These mistakes are called Errors.

What is Error?

Error is the difference between the Actual Value and the Predicted Value.

Error = Actual Value − Predicted Value


Actual Marks Predicted Marks Error
80 75 5
90 85 5
70 78 -8

Interactive Error Calculator


Result Appears Here

Understanding Bias

Bias occurs when the model is too simple and makes wrong assumptions.

What is Bias?

Bias is the error caused by incorrect assumptions in the learning algorithm.

  • Model is too simple
  • Cannot learn real patterns
  • Produces poor predictions
  • Causes Underfitting

Student Exam Example

A student studies only one chapter and assumes:

"All exam questions will come from this chapter."

Exam contains questions from many chapters.

Result: Poor Marks

Underfitting

High Bias = Underfitting

Model fails to learn patterns from training data.

  • Too Simple
  • High Training Error
  • High Testing Error
  • Poor Accuracy

Machine Learning Example 

y = x²

Using

Linear Regression

y = mx + c

Straight line cannot fit a curve.

Underfitting Occurs

Understanding Variance

Variance occurs when the model learns noise and memorizes data.

What is Variance?

Variance occurs when a model becomes too complex.

  • Learns Noise
  • Memorizes Data
  • Fails on New Data
  • Causes Overfitting

Student Example

Student memorizes all homework answers.

Learns answers but not concepts.

Exam contains new questions.

Result: Poor Performance

Overfitting

High Variance = Overfitting

Model memorizes training data instead of learning patterns.

  • Low Training Error
  • High Testing Error
  • Too Complex
  • Poor Generalization

Machine Learning Example 

Decision Tree

Depth = 50

Model memorizes every record.

Training Accuracy = 100% Testing Accuracy = Low

Overfitting Occurs

Bias vs Variance Comparison

Feature Bias Variance
Meaning Wrong Assumptions Learning Noise
Problem Underfitting Overfitting
Model Too Simple Too Complex
Training Error High Low
Testing Error High High
Example Linear Regression on Curve Deep Decision Tree

Quick Revision

Error

Difference between Actual and Predicted Value

Bias

Too Simple → Underfitting

Variance

Too Complex → Overfitting

Goal

Learn Pattern, Not Noise

Bias & Variance Dartboard Visualization

The Dartboard Analogy is one of the easiest ways to understand Bias and Variance.

High Bias

Predictions are grouped together but far from the target.

High Variance

Predictions are spread everywhere.

Low Bias

Predictions are close to actual value.

Ideal Model

Accurate and Consistent Predictions.

Model Complexity

Understanding how complexity affects learning.

Simple Model

High Bias
  • Too Few Features
  • Cannot Learn Pattern
  • Underfitting
  • Poor Accuracy

Balanced Model

Optimal Model
  • Good Generalization
  • Captures Pattern
  • High Accuracy
  • Best Performance

Complex Model

High Variance
  • Memorizes Data
  • Learns Noise
  • Overfitting
  • Poor Generalization

Bias-Variance Tradeoff

Finding the Sweet Spot Between Underfitting and Overfitting

Real Machine Learning Examples

Linear Regression on Curve Data

Dataset: y = x²

Linear Regression can only draw a straight line.

Result: Underfitting

Deep Decision Tree

Tree Depth = 50

Memorizes every record.

Result: Overfitting

Polynomial Degree = 15

Curve passes through every point.

Learns Noise

Polynomial Degree = 2

Captures actual trend.

Good Generalization

Interview Questions

Bias is the error caused due to overly simplified assumptions in a model, leading to underfitting.

Variance is the error caused when a model learns noise from training data, leading to overfitting.

Underfitting occurs when the model is too simple. Overfitting occurs when the model is too complex.

Pruning, Regularization, Dropout, More Data, Cross Validation.

Key Takeaway

# High Bias Underfitting

# High Variance Overfitting

# Balanced Complexity Best Model

Confusion Matrix

The foundation of Classification Evaluation Metrics

Predicted Positive Predicted Negative
Actual Positive TP
(True Positive) 		FN
(False Negative)
Actual Negative FP
(False Positive) 		TN
(True Negative)

TP

Actual Positive and Predicted Positive.

Cancer Patient correctly detected.

TN

Actual Negative and Predicted Negative.

Healthy Person correctly identified.

FP

Actual Negative but Predicted Positive.

Healthy Person wrongly classified as Cancer.

FN

Actual Positive but Predicted Negative.

Cancer Patient missed by model.

Accuracy

What is Accuracy?

Percentage of total correct predictions.

Accuracy = (TP + TN) --- TP + TN + FP + FN

Example

TP = 40 TN = 35 FP = 10 FN = 15

Accuracy = 75%

Precision

When model predicts Positive, how often is it correct?

Precision = TP --- TP + FP

Spam Email Example: Out of 100 emails predicted as spam, 80 were actually spam.

Precision = 80%

Recall (Sensitivity)

Out of all actual positives, how many did the model find?

Recall = TP --- TP + FN

Cancer Detection Example 100 Cancer Patients Model Finds 90

Recall = 90%

F1 Score

Balance Between Precision and Recall

F1 Score = 2 × Precision × Recall --- Precision + Recall

Used when classes are imbalanced.

Good balance between Precision and Recall

ROC Curve

ROC = Receiver Operating Characteristic

  • Y Axis → True Positive Rate
  • X Axis → False Positive Rate
  • Higher Curve = Better Model

AUC Score

Area Under ROC Curve

AUC Performance
1.0 Perfect
0.9+ Excellent
0.8+ Good
0.7+ Fair
0.5 Random Guess

Metric Calculator

Results Will Appear Here

Mini Quiz

Which metric is most important for Cancer Detection?

Accuracy
Recall
Precision
AUC

Regression Evaluation Metrics

Used when the target variable is continuous (numbers).

MAE

Average Error

MSE

Squared Error

RMSE

Root Error

R²

Model Fit

MAE

Mean Absolute Error

Average of absolute prediction errors.

MAE = Σ |Actual − Predicted| --------------------- n

Example: Actual = [100,120,150] Predicted = [90,125,145] Errors = [10,5,5] MAE = (10+5+5)/3 MAE = 6.67

Interpretation

On average, the model is making an error of 6.67 units.

MSE

Mean Squared Error

Squares the error before averaging.

MSE = Σ (Actual − Predicted)² ----------------------- n

Errors = [10,5,5] Squared Errors 100,25,25 MSE = 150 / 3 MSE = 50

Large mistakes are penalized heavily.
RMSE

Root Mean Squared Error

Square root of MSE.

RMSE = √MSE

RMSE = √50 RMSE = 7.07

Most widely used regression metric.
R² SCORE

Coefficient of Determination

Measures how well the model explains variance in the data.

R² = 1 − (SSR / SST)

Example: R² = 0.92

92% of the variation is explained by the model.
R² Meaning
1.0 Perfect
0.9+ Excellent
0.8+ Good
0.5+ Average
0 No Learning
< 0 Worse than Guessing

Regression Metric Calculator

Enter Actual and Predicted values separated by commas.

Results will appear here

Classification vs Regression

Feature Classification Regression
Output Category Numeric Value
Example Pass / Fail Salary Prediction
Metrics Accuracy, Precision, Recall MAE, MSE, RMSE, R²
Algorithms Logistic Regression, SVM Linear Regression

Complete Model Evaluation Dashboard

Accuracy

Classification

Precision

Classification

Recall

Classification

F1 Score

Classification

ROC-AUC

Classification

MAE

Regression

RMSE

Regression

R² Score

Regression

Final Revision Sheet

Classification Metrics

  • Accuracy → Overall Correctness
  • Precision → Correct Positives
  • Recall → Find All Positives
  • F1 Score → Balance Precision & Recall
  • ROC Curve → TPR vs FPR
  • AUC → Area Under ROC

Regression Metrics

  • MAE → Average Error
  • MSE → Squared Error
  • RMSE → Root Error
  • R² → Explained Variance

High Bias = Underfitting

High Variance = Overfitting

Balanced Model = Best Performance