Classifying all major Machine Learning algorithms

There are two common ways of classifying machine learning algorithms. One is based on learning style, the other is based on the function (of each algorithm).

1. Classifying based on learning method

By learning style, Machine Learning algorithms are usually divided into four groups:

• Supervised learning

• Unsupervised learning

• Semi-supervised learning

• Reinforcement learning

There are a number of subgroups that do not have Semi-supervised learning or Reinforcement learning.

Supervised Learning

Supervised learning is an algorithm that predicts the outcome of a new input based on known input, outcome . This data pair is also called ( data, label ).

Supervised learning is the most common one in the Machine Learning algorithms.

Example 1: In handwriting recognition, we have pictures of thousands of examples of each digit written by different people. We put the images into an algorithm and show it to each picture with a digit. After the algorithm creates a model, ie a function where the input is an image and the output is a digit, when it receives a new image that the model has not seen before , it will predict the image. That picture contains the digits.

MNIST : handwritten digits database. (Source: Simple Neural Network Implementation in Ruby)

This example is quite similar to the way people learned when they were young. We give the alphabet to a child and show them the letter A, this is the letter B. After a few times taught, the child can recognize the letter A, the letter B in a book. They have never seen it.

Example 2: Detecting faces in a photo has been developed for a long time. Initially, facebook used this algorithm to indicate faces in a photo and asked the user to tag friends - labeling each face. The greater the number of pairs of data ( faces, names ), the greater the accuracy of subsequent tags .

Example 3: The algorithm for detecting faces in an image is also an algorithm for supervised learning with training data of thousands of pairs ( images, faces ) and ( images, not human faces). ) is included. Note that this data only distinguishes human faces and not human faces regardless of the faces of different people.

The supervised learning algorithm is further broken down into two main categories:

Classification

A problem is called classification if the labels of the input data are divided into finite groups. For example, Gmail determines whether an email is spam; Creditors determine if a customer is capable of paying off a debt. The three examples above are divided into this category.

Regression

If the label is not split into groups that are a specific net value. For example, a house of x m 2 x m2there yybedroom and city center z km z kilometer how much will it cost?

Microsoft recently has a sex-based and age-based application . Sex prediction can be considered as a classification algorithm , the age prediction can be considered as the Regression algorithm . Note that the age prediction can also be considered Classification if we consider age to be a positive integer no greater than 150, we will have 150 different classes.

Unsupervised Learning

In this algorithm, we do not know the outcome or label but only the input data. The unsupervised learning algorithm will rely on the structure of the data to perform certain tasks, such as clustering or dimension reduction for convenient storage and calculation. .

In a mathematical way, unsupervised learning is when we only have data into XXdo not know the label YY corresponding.

These algorithms are called unsupervised learning because unlike supervised learning, we do not know the exact answer for each input. Like when we learn, how teachers can not only tell us that the letter A or B. The term word unattended named in this sense.

Unsupervised learning problems are further broken down into two categories:

Clustering

A problem that groups the whole data XXSmall groups are based on the relevance of the data in each group. For example, grouping customers based on purchase behavior. This is like giving a child a lot of pieces with different shapes and colors, such as triangles, squares, rounds with blue and red, then ask them to divide them into groups. . Although children do not know which piece corresponds to which image or color, they are most likely to be able to classify the pieces by color or shape.

Association

It is a problem when we want to discover a rule based on given data. For example, men who buy clothes often tend to buy watches or belts; Spider-Man moviegoers tend to watch more Batman films, based on which create a Recommendation System, which promotes shopping.

Semi-Supervised Learning

Problems when we have large amounts of data XXBut only part of them are labeled Semi-Supervised Learning. The problems in this group lie between the two groups mentioned above.

A typical example of this group is that only part of the image or text is labeled (for example, a photo of a person, a animal, or a text of science or politics) and most of the other pictures / texts. Unassigned labels are collected from the internet. In fact, many of the Machine Learning problems belong to this group because labeling data is time consuming and expensive. Many types of data even require a new expert to label (medical images, for example). In contrast, unlabeled data can be collected at low cost from the internet.

Reinforcement Learning

Reinforcement learning is a mathematical problem that enables a system to automatically determine behavior based on circumstances to maximize the performance. At present, Reinforcement learning is mainly applied to Game Theory, algorithms need to determine the next move to achieve the highest score.

AlphaGo played go with Lee Sedol. AlphaGo is an example of Reinforcement learning. (Source: AlphaGo AI Defeats Sedol Again, With 'Near Perfect Game')

Example 1: AlphaGo is now famous for playing chess against human beings . Go is considered to be extremely complex with a total of approximately 10^761, compared to chess is 10^120 And the total number of atoms in the universe is about 10^80!! So, the algorithm has to choose the optimal one among billions of billions of choices, and of course, it can not apply the same algorithm as IBM Deep Blue (IBM Deep Blue has won the human chess. 20 years ago). Basically, AlphaGo includes algorithms in both Supervised learning and Reinforcement learning. In the Supervised learning section, data from human-made games is put into training. However, AlphaGo's ultimate goal is not to play as humans, but to even win over humans. So after learning the game of human beings, AlphaGo plays itself with millions of games to find new moves better. Algorithm in this self-play is classified as Reinforcement learning. (See alsoGoogle DeepMind's AlphaGo: How it works).

Example 2: Training for the Mario gaming computer. This is an interesting program teaching computer game Mario. This game is simpler than Go, because at one point, the player only has to press a small number of buttons (move, jump, fire) or no button press. At the same time, the reaction of the machine is simpler and repeatable at each play (at a specific time there will be a fixed obstacle in a fixed position). The input of the algorithm is the diagram of the screen at the present time, the task of the algorithm is with that input, which key combination should be pressed. This training is based on how far far the score is for moving in the game, the further and faster you get the higher the bonus (this bonus is not the point of the game but the point of the the programmer himself created). Through training, the algorithm will find an optimal way to maximize points,

Train for the Mario gaming computer

2. Classifying based on functions

There is a second categorization based on the function of algorithms. In this section, I would just list the algorithms. Specific information will be presented in other articles at this blog. During the writing process, I might add some algorithms.

Regression Algorithms

1. Linear Regression

2. Logistic Regression

3. Stepwise Regression

Classification Algorithms

1. Linear Classifier

2. Support Vector Machine (SVM)

3. Kernel SVM

4. Sparse Representation-based classification (SRC)

Instance-based Algorithms

1. k-Nearest Neighbor (kNN)

2. Learning Vector Quantization (LVQ)

Regularization Algorithms

1. Ridge Regression

2. Least Absolute Shrinkage and Selection Operator (LASSO)

3. Least-Angle Regression (LARS)

Bayesian Algorithms

1. Naive Bayes

2. Gaussian Naive Bayes

Clustering Algorithms

1. k-means clustering

2. k-Medians

3. Expectation Maximization (EM)

Artificial Neural Network Algorithms

1. Perceptron

2. Softmax Regression

3. Multi-layer Perceptron

4. Back-Propagation

Dimensionality Reduction Algorithms

1. Principal Component Analysis (PCA)

2. Linear Discriminant Analysis (LDA)

Ensemble Algorithms

1. Boosting

2. AdaBoost

3. Random Forest

And many other algorithms.

3. References

• A Tour of Machine Learning Algorithms

• Take advantage of modern Machine Learning algorithms

If you have any question, please leave a comment below so I can answer it soon.