Skip to content

Learn / Supervised

Supervised learning

Learning a mapping from inputs to a known answer. Regression predicts numbers, classification predicts categories.

On this page
  1. Intuition
  2. What the data looks like
  3. The two tasks
  4. Bias and variance
  5. How the data is prepared
  6. Pitfalls

Intuition

You have examples where the answer is known: past transactions marked fraud or not, customers who churned or stayed, flats and the prices they sold for. Supervised learning finds a function ff such that f(x)≈yf(x) \approx y on those examples and, more importantly, on new ones it has never seen.

f^=arg⁡min⁡f∈F  1n∑i=1nL(yi,f(xi))+λ Ω(f)\hat{f} = \arg\min_{f \in \mathcal{F}} \; \frac{1}{n}\sum_{i=1}^{n} L\big(y_i, f(x_i)\big) + \lambda\,\Omega(f)

Every supervised algorithm is a choice of three things: the family of functions F\mathcal{F}, the loss LL that defines “wrong”, and the penalty Ω\Omega that discourages needlessly complicated answers.

What the data looks like

Supervised data is a table where every training row already has the answer filled in. The columns you know at prediction time are the inputs; the column you want to predict is the answer.

Churn prediction: one row per customer One row = one customer on the snapshot date. Answer: churned in the next 30 days (yes / no).
rowcustomerinputlogins_30dinputbalance_leiinputtenure_monthsinputsalary_cardanswerchurned_next_30d
C-104123107noyes
C-1042261840054yesno
C-10430953noyes
C-104414520022yesno

Supervised learning needs the answer column for every training row. The model learns to fill it in for new customers whose answer is not known yet.

Example rows are invented for illustration.

The two tasks

RegressionClassification
TargetA numberA category
Typical lossSquared errorLog-loss (cross-entropy)
OutputA valueA probability per class
ExampleNext month’s transaction volumeWill this customer churn?
  • Regression: linear models, trees, boosting and a small neural net fitting curves.
  • Classification: decision boundaries, from logistic regression to random forests.

Bias and variance

A model that is too simple misses the pattern (underfitting, high bias). A model that is too flexible memorises the noise in its training set (overfitting, high variance). You see the difference by comparing two numbers:

  • training loss keeps falling while test loss rises → overfitting
  • both are high and close together → underfitting

Every playground on this site plots the two curves side by side for exactly this reason.

How the data is prepared

Labels, splits, leakage, scaling, encoding and class imbalance are covered on the data preparation page. The short version: split first, fit preprocessing on the training set only, and evaluate on data that looks like production.

Pitfalls

  • Choosing the model by its score on the test set. That turns the test set into training data.
  • Optimising a metric the business does not care about.
  • Assuming a good offline score survives a change in the data (drift). Monitor after deployment.