2.6 Data in Machine Learning

Today, almost all activity domains generate large amounts of data: information about videos we've watched online, products we've purchased, friends we've connected with on social media, as well as information about doctor visits, weather conditions in our city, or traffic conditions recorded by relevant institutions. All this data can be used to better understand the environment in which it is generated.

Just like in the story of databases you encountered last year, in machine learning, we describe important entities and events whose behaviors we want to model using attributes (also called features). For example, a movie can be described by its title, genre, year of release, production company, budget, profit, synopsis, director's name, and main actors' names. Choosing the right attributes to track and record when collecting data is not an easy task because we don't know in advance which attributes will be most useful for the task we want to solve in the future. For instance, if we want to use data to predict a movie's profit (a regression task), information about actors and the production company might be more useful, while for determining the movie's genre (a classification task), the synopsis might be more helpful. In more complex domains, these choices come with even more dilemmas and challenges.

Due to the need to use data for a wide range of applications, we might consider collecting as many attribute values as possible. While this idea is valid in some situations, generally, we must remember that large amounts of data require appropriate storage, hardware to support their processing, and a team of experts with the necessary skills and knowledge to perform these tasks. Therefore, such choices can be costly and require special planning. It's also important to note that analyzing and understanding large amounts of data is challenging and requires appropriate technical competencies, such as data visualization techniques. Additionally, many domains involving private and sensitive data must consistently follow regulations and ethical guidelines on data collection, which imposes further restrictions on attribute selection and storage possibilities. Thus, the task of collecting data and creating high-quality datasets is challenging and demanding, requiring careful organization.

In the upcoming lessons, we will see that each attribute is defined by its type and value set, and these properties affect how we prepare the data. Ultimately, machine learning algorithms can only be applied to numerical values. The number of attributes and their properties also influence the choice of machine learning algorithm.

Advanced machine learning algorithms, such as neural networks, can identify important attributes for solving a task on their own. This relieves us from thinking about attribute selection and combinations. This is particularly useful when working with complex data like images or textual content, where defining and extracting attributes is not always intuitive. These algorithms can work with raw data.

• What do you find challenging about data collection in the domain that interests you? It could be sports, a scientific discipline, a social phenomenon, or anything else.
• Do you have any concerns or reservations about data collection and processing?
• What is most important to you personally in the data collection process?

Popular datasets

It may surprise you, but datasets can be popular too! Some of them are known for being used in the first machine learning tasks, while some have achieved their popularity through persistent community engagements to expand and complement them. As different datasets track different domains of AI, here we will use this as a criterion for grouping and displaying them. Namely, we will get to know sets that contain images, textual data, audio archives and videos. A large number of libraries used in the field of machine learning make it possible to quickly and easily load the sets we are going to discuss.

Computer vision

MNIST

 Certainly one of the most popular sets in the field of computer vision is   MNIST, a set of images of handwritten numerals. Its development was started by the US National Institute of Standards and Technology (eng.  National Institute of Standards and Technology (NIST)  back in 1998. All images are 28x28 pixels, black and white, and there are a total of 70,000 of them: 60,000 images make up the training set and 10,000 images make the test set. In the image you can see some of the digits from this data set.

Some of the figures of the MNIST conference

The MNIST set is used to train multiclass classifiers, most often in combination with convolutional neural networks, which you will hear more about later in the course.

For each digit of the MNIST set, one class is provided. Think about which digits are potentially problematic to distinguish (for example, the digits 1 and 7 may resemble each other), and then try to find some examples on the web.

ImageNet

 The images in the ImageNet  set represent images of general objects: computers, windows, airplanes, seedlings, tropical animals and various other entities. Interestingly, these images are organized into related groups (so-called synsets) between which the parent-child relationship applies. For example, all sailing ships belong to one group (one synset), in the hierarchy below them there are groups of gliders and trimarans, while in the hierarchy above there are groups of watercraft,  vessels and vehicles. In the picture, in the bottom row, you can trace this hierarchy: at the bottom of it are the trimarans, and at the top of the vehicle. In the top row are synsets that refer to dogs and some of their categorizations.

Example of an ImageNet image

The collection currently contains about 14 million images and over 21,000 synsets. It is used in a variety of image classification and object detection tasks in images.

 The official website of the ImageNet conference is   https://www.image-net.org/index.php  . Researchers from Stanford and Princeton universities are actively working on its development.

Try to find out which group a computer belongs to in the ImageNet set and which groups are in the hierarchy below and above

COCO

 The   COCO   dataset (acronym for Common Objects in Context  ) is used in tasks of object detection, image segmentation, and automatic association of titles to images. It was created by Microsoft and shared with the community in 2015.

An image of the COCO set with marked recognized objects: planes, trucks and cars

 The set can be viewed interactively on the official website: for each image there is a URL from which the image was taken, several titles associated with the image, and then a series of icons corresponding to the recognized objects. The number of images in the dataset is 330,000 and contains 80 categories of objects with over 1.5 million instances. The link to the search section on the site is https://cocodataset.org/#explore .

Natural Language Processing

IMDB

 If you like to watch movies and TV shows, you will be interested  in  the IMDB  dataset, which contains user reviews from the popular IMDB platform. For each view in this dataset, it is also known whether it is positive or negative, i.e. whether it primarily contains something praiseworthy and good about the film or some criticism and objection. When it comes to datasets that contain textual content,  It is always important to emphasize in which language they are written. The IMDB dataset contains views that are in English with a total of 50,000 views, 25,000 positive and 25,000 negative views. Below you can see a positive and a negative entry in this data set.

Review	Sentiment (0-negative, 1-positive)

Examples of positive and negative reviews of the IMDB set

The IMDB dataset is used in sentiment analysis tasks - recall that these are tasks in which it is necessary to recognize an emotion or attitude present in the text. Since the set contains only information whether the review is positive or negative, the task of sentiment analysis in the IMDB set is approached as a problem of binary classification. In general, the sentiment scale can be finer and includes ratings such as very positive, positive, neutral, negative, or very negative.

Sound processing

AudioSet

 An AudioSet   is a dataset that contains 10-second snippets of video from YouTube. Each of these snippets is associated with the characteristics of the sounds heard in them. The set was created by Google and contains over 2 million clips with a total duration of 5.8 thousand hours.

An example of a video clip with the associated audio annotations it contains

 The official website of the conference provides an overview of examples and insight into the organization of the conference. 632 different categories are used, such as the sounds of musical instruments, the sound of the wind, the sound of man, noise, etc. You can visit the address https://research.google.com/audioset/index.html and listen to some more examples. The conference itself was created with the idea of supporting the development of sound recognition algorithms.

Video Processing

Moments in Time

 Moments in Time   is a dataset that is being developed with the idea of helping artificial intelligence systems learn to recognize actions and events. This set currently contains one million videos of 3 seconds in length in which activities are marked. The videos contain people, animals, objects and natural phenomena. Just some of the events that are covered are dancing,  exercise , climbing a tree, jumping into the water and sleeping.

 The Moments in Time gathering is being developed by a team from the Massachusetts Institute of Technology (MIT) and on the official website of the project you can see some more examples of videos and recognized actions. The link to the official website is http://moments.csail.mit.edu/ .

A video in which it is recognized that a man feeds a rabbit