[Review] Learning algorithms for classification-a comparison on handwritten digit recognition
[Review] Learning algorithms for classification-a comparison on handwritten digit recognition

Keywords: LeNet, Convolutional Neural Networks, Handwritten Digit Recognition All the figures in this post come from ‘Learning algorithms for classification-a comparison on handwritten digit recognition’1 Basic Works Convolutional network2 Inspiration Raw accuracy, training time, recognition time and memory requirements should be considered in classification. From experiments and comparison, the results can illuminate which one is... » read more

[Combining Models] Boosting and AdaBoost
[Combining Models] Boosting and AdaBoost

The committee has an equal weight for every prediction from all models, and it gives little improvement than a single model. Then boosting was built for this problem. Boosting is a technique for combining multiple 'base' classifiers to produce a form of the committee that

[Combining Models] Committees
[Combining Models] Committees

The committee is a native inspiration for how to combine several models(or we can say how to combine the outputs of several models). For example, we can combine all the models by

[Mixture Models] EM Algorithm
[Mixture Models] EM Algorithm

Maximizing likelihood could not be used to the Gaussian mixture model directly, for its severe defects that we have come across at 'Maximum Likelihood of Gaussian Mixtures'. By the inspiration of K-means, a two-step algorithm was developed.

[Mixture Models] Mixtures of Gaussians
[Mixture Models] Mixtures of Gaussians

We have introduced a mixture distribution in the post 'An Introduction to Mixture Models'. And the example in that post was just two components Gaussian Mixture. However, in this post, we would like to talk about Gaussian mixtures formally. And it severs to motivate the expectation-maximization(EM) algorithm.

[Mixture Models] K-means Clustering
[Mixture Models] K-means Clustering

Original form K-Means algorithm might be one of the most accessible algorithms in machine learning. And many books and courses started with it. However, if we convert the task which K-means dealt with into a more mathematical form, there would be more interesting aspects coming to us.