2.3 Basic concepts of machine learning

The concepts that we will introduce in this lesson are the basic concepts of machine learning. They will help you follow the topics discussed below, and you will learn more about each of them.

In terms of machine learning, the collection of data we have is called a data set. It can be some fine tabular records, similar to those we encounter in databases or Excel files, but also some group of satellite images or audio clips. One specific element of a data set is called an instance . So one particular row in a log table or one particular satellite image are examples of instances. The number of instances in a data set can determine the choice of learning algorithm because some algorithms require more data than others.

 In instances there are attributes, properties that we use to describe the data. If we imagine that it is a tabular record of earthquake occurrences, the date and time of occurrence, latitude, longitude, earthquake strength, level of destruction and other important data can appear as attributes. Attributes are equally called features. A little later you will learn what all kinds of attributes exist and what we need to take care of. The attributes on the basis of which we need to learn how to solve a task are called input variables (inputs), and those that need to be taught output variables. Thus, the date and time of an earthquake, its geo-coordinates and its magnitude can be input variables in the task of determining the devastation of an earthquake. The devastation of an earthquake is also present as an attribute in the data set, so it would be an output variable. Sometimes we will use less formal terms such as input and output . It is important to note that it is the task that dictates what will be input variables and what will be output variables.

Show answer

We have said that the goal of machine learning algorithms is to determine the mapping of given inputs to given outputs. Now we can be more precise and say that the goal of machine learning is to determine the mappings of given input variables to given output variables. These mappings are called  models.

Bottom of Form

The concept that we associate with mapping is a function. In math class, you've heard a lot about functions such as input-to-output mappings. For example, the function of one variable \(y = 2x + 4\) maps the input \(x = 5\) to the value \(y = 14\), while the function of multiple variables \(y = 2x_1-3x_2+x_3+5\) maps the input \((x_1,x_2,x_3)=(1,-1,3)\) to the value \(y = 13\). The variables that appear in the functions are related to the values of the attribute. Thus, x in the first function can represent the square footage of the property, while \(x_1, x_2, x_3\) in the second function can represent the values of attributes such as latitude, longitude, and earthquake strength. In math class, you've heard that there are different classes of functions (linear, polynomial, trigonometric, exponential, logarithmic), and that each of them is characterized by some special properties such as continuity, monotony, or convexity. All this knowledge is welcome when looking for the right model.

The complexity of  a function is something that we will not formally introduce. You will understand that some functions are simpler than other "sausages". Simple functions are more rewarding for work and easier to understand, but they do not give us much freedom to describe some of the more unusual relationships between the attributes themselves and the outputs. On the other hand, complex functions are complex for a reason, so it can be difficult for us to keep track of some of their mathematical behaviors that can affect learning. We try to strike a balance between complexity and what we know about data and what we want to learn.

 In models, as we have seen in the introductory example of real estate pricing, parameters such as k and n can appear. Such models are called parametric models and the task of determining the right model is reduced to the task of determining the best values of the parameters. In the linear model, only two parameters appeared in the real estate pricing task, while modern models, Those that are based on neural networks have millions or billions of parameters. We will see that there are also slightly different non-parametric models , whose forms are expressed differently.

The process of finding a model is called model training. If there are unknown parameters in the model, we need to determine their values during training. That is our goal.

In the dataset used to train models, inaccurate, or contradictory values can also be found. That is why models are never absolutely correct. This brings us to another important concept in machine learning theory: the loss function. The error function tells us how wrong the model is. We actively use its values during model training and strive for those model configurations that lead us to the lowest value of the error function. In the case of parametric models, as was the case in the introductory example with real estate, the goal is to determine those parameter values for which the value of the error function is the lowest.

When we train a machine learning model, we need to assess how good it actually is for application in practice. This is what the so-called quality measures serve us - each of them is adapted to a specific learning task and the domain in which the model will be applied. It is important to emphasize that, in general, the error function and quality measures differ. Both aim to give us information about how good the model is, The error function does this during model training, while quality measures do this after model training. The error function is closely tied to the model, while quality measures are designed so that they can be understood by both users and domain experts. If the correct quality values are not obtained, the model must be repaired. Below, we'll talk about what that means and how it can be achieved. The whole process of testing the quality of a model and calculating its quality measures is called model testing.

Typically, the values calculated and generated by a trained model are called predictions. Thus, the price of a new property or the assessment of the devastation of an earthquake are examples of model predictions. This is why we are talking about predictions in the world of artificial intelligence. It is clear to you that these predictions are by no means random, but very well-founded and data-based. The application of the model itself is also called inference.

All the terms that are emphasized are important concepts of machine learning and are always present in the literature on machine learning and its applications. That is why it is important that they are clear to you and that you understand what role they play in the development of a model.