##### [Combining Models] Boosting and AdaBoost

The committee has an equal weight for every prediction from all models, and it gives little improvement than a single model. Then boosting was built for this problem. Boosting is a technique for combining multiple 'base' classifiers to produce a form of the committee that

##### [Combining Models] Committees

The committee is a native inspiration for how to combine several models(or we can say how to combine the outputs of several models). For example, we can combine all the models by

##### [Combining Models] Bayesian Model Averaging(BMA) and Combining Models

Bayesian model averaging(BMA) is another wildly used method which is very like a combining model. However, the difference between BMA and combining models is significant.

##### [Combining Models] An Introduction to Combining Models

The mixture of Gaussians had been discussed in the post 'Mixtures of Gaussians'. It can not only be used to introduce 'EM algorithm' but contain a strategy to improve model performance.

##### [Mixture Models] Maximum Likelihood of Gaussian Mixtures

Gaussian mixtures had been discussed in 'Mixtures of Gaussians'. And once we have training data and a certain hypothesis, what we should do next is estimating the parameters of the model. Both kinds of parameters from a mixture of Gaussians

##### [Mixture Models] Mixtures of Gaussians

We have introduced a mixture distribution in the post 'An Introduction to Mixture Models'. And the example in that post was just two components Gaussian Mixture. However, in this post, we would like to talk about Gaussian mixtures formally. And it severs to motivate the expectation-maximization(EM) algorithm.

##### [Mixture Models] An Introduction to Mixture Models

We have discussed many machine learning algorithms, including linear regression, linear classification, neural network models and e.t.c, till now. However, most of them are supervised learning, which means a teacher is leading the models to bias to a certain task

##### [Linear Classification] Logistic Regression

Logistic sigmoid function(logistic function for short) had been introduced in post 'An Introduction to Probabilistic Generative Models for Linear Classification'.

##### [Linear Classification] An Introduction to Probabilistic Generative Models

The generative model used for making decisions contains the inference step and decision step

##### [Linear Classification] Fisher Linear Discriminant(LDA)

'Least-square method' in classification can only deal with a small set of tasks. That is because it was built for the regression task. However, we want a method to solve linear classification especially.