Classification and regression

A bank needs the answer "will repay the loan or not", a realtor needs an apartment's price. Both tasks are supervised learning, but their answers differ in nature. The first answer is picked from two options, the second can be any number. This decides which model to use and how to measure its error.

Classification is a task where the target takes one value from a finite set of categories. The categories are called classes: spam and not spam, cat, dog and bird. There can be two classes, as with spam, or dozens, as with dog breeds in photos. Regression is a task where the target is a number on a continuous scale: a price, a temperature, a delivery time.

In regression, errors come in different sizes: a miss by a thousand rubles is better than a miss by a million. That is why a regression error is measured in the target's units. In classification, an answer is either right or wrong, and the share of wrong answers is counted.

The target decides, not the name

Only the target defines the task type. Classes are often stored as numbers: 0 is a cat, 1 a dog, 2 a bird. A numeric column does not make the task regression.

Check whether an average of two answers makes sense. The average price of two apartments is still a price, but there is no average of a cat and a bird.

The model's name does not define the task type either. Logistic regression, despite its name, solves a classification task: it predicts a sample's class.

Common mistake

Class numbers are treated as ordinary numbers:

classes = ["cat", "dog", "bird"]
y = [0, 2, 0, 2]
average = sum(y) / len(y)  # ✗
print(average)  # 1.0
print(classes[int(average)])  # dog

Python computes this without an error, but the answer is meaningless. The average of cats and birds turned out to be a dog.

The number 1 sits between 0 and 2 only because the classes were numbered that way. The distance between class numbers means nothing. A regression model trained on such a column makes the same error: it predicts "in-between" classes.

If the target is a category, the task stays classification, however the classes are stored.