3.5 Classification, types of classification, and matrix of confusion

Classification

Types of classification

At the very beginning, it is important to emphasize that not all classifications are the same. Therefore, we will first find out what classifications exist and what characterizes them. Examples of sorting pieces of paper or sorting mail are examples of binary classification because we have only two groups: pieces of paper to be thrown away and pieces of paper to be stored, i.e. desirable and undesirable mail. Groups in the world of machine learning are called classes so we will continue to stick to that date. In order to be able to distinguish between classes, we associate them with names that approximate what they actually contain. For example, "slips of paper" and "junk mail" are clear enough names. Names are often specified by labels, which appear in the dataset to which the classification task is applied.

 If we have more than two classes, we are talking about a multiclass classification task. For example, such is the task of sorting photos by events where each event can represent one class. We can create three directories, i.e. three classes, give them the names "excursion", "aunt's birthday" and "trip", and then assign each of the photos to one of these classes by putting it in the appropriate directory.

 We can think about different types of classification based on the criteria of affiliation. For example, one e-mail can be either desirable or undesirable, it cannot belong to both the class of desirable and undesirable e-mails at the same time. It is similar with the photographs and classes that we have introduced. On the other hand, one newspaper article can be simultaneously on the topic of culture, travel and food, so we can associate it with a greater number of classes - the one that represents culture, the one that represents travel and the one that represents food. Since in this case the instances have more features, i.e. labels, we call this type of classification multilabelary classification . Although it is very interesting and useful, we will not cover the multilabelar classification with further contents, but we will focus on binary and multiclass classification.

Classification from the point of view of machine learning

When we think about the classification task from the point of view of machine learning, we are interested in discrete mappings, i.e. mappings that can assign one of finite values to input variables. Most often, the number of classes is smaller, expressed in a single digit number, but you can also recall the ImageNet  set and the image classification competition in which 1000 classes are used. Variables that can take a finite number of values are called categorical, so we can talk about classification It's like a mapping that is characterized by a categorical target variable

 \( F(x) = \begin{cases} 0 & \text{if } x < 0 ,\\ \frac{1}{2} & \text{if } 0 \leq x < 1 ,\\ 1 & \text{if } x \geq 1 \end{cases} \)

An example of a discrete function.

Since classification is a supervised machine learning task, the dataset used to train the model contains pairs of inputs and expected outputs. The inputs can be images, text messages, or tabular data, and all the guidelines discussed in the lesson on preparing a dataset apply to their preparation. The output is always the name of the class. Although we introduced class names in order to make it easier to follow the classification task, When we get to the part of preparing the data, we need to transform it into numerical values as well. Here we can be guided by the preparations that we have discussed for working with categorical attributes: mapping a set of values or  one-hot coding.

In the case of binary classification, we usually map the class names to the values 0 and 1. For example, the occurrence of the class name "junk mail" is replaced by the value 0, and the occurrence of the name "junk mail" by the value 1. Often, instances that have a label of 0 are said to belong to  a negative class, and instances that have a label of 1 are said to belong to a positive class .

 When it comes to multiclass classification, we use one-hot coding to prepare the target variable . For example, for the task of sorting photos by events, we will transform the outputs into vectors of length three because we have exactly three classes: "excursion", "aunt's birthday", and "trip". Next, we will assign to each of these values a vector that has a one at the appropriate position, and zero at all remaining positions. That will, In order, be the values of (1, 0, 0), (0, 1, 0) and (0, 0, 1). Here it is important to consistently adhere to the chosen order of classes.

Below, we will get to know the two algorithms used for the binary classification task. The problem of multiclass classification can be solved through specially designed algorithms, but also through several associated binary classifiers. We will bring closer two such techniques called "one against all" and "one on one".

 Let's imagine that we have three classes: red, green and blue. The "one against all" approach implies that we need to learn three binary classifiers: one that distinguishes the green class from the rest (unions of the red and blue classes), one that distinguishes the blue class from the rest (unions of the green and red classes), and one that distinguishes the red class from the rest (unions of the green and blue classes). When we need to classify a new instance, we run each of the three binary classifiers and apply the principle of highest confidence to the results obtained: the instance joins the class whose classifier is the safest. We'll see soon how the safety of the classifier is assessed.

One-vs-All classification 

Again, imagine that we have three classes: red, green, and blue. The one-to-one approach involves training binary classifiers that can distinguish between each of the class pairs: red and green, green and blue, and red and blue. In general, if we have n classes, the number of binary classifiers we need to train is \(\frac{n ⋅ (n−1)}{2}\) . When we need to classify a new instance, we run each of the learned classifiers and apply the principle of majority voting to the results obtained: the instance joins the class for which the largest number of classifiers vote.

One-vs-One classification