##### [Linear Classification] Logistic Regression

Logistic sigmoid function(logistic function for short) had been introduced in post 'An Introduction to Probabilistic Generative Models for Linear Classification'.

Logistic sigmoid function(logistic function for short) had been introduced in post 'An Introduction to Probabilistic Generative Models for Linear Classification'.

The generative model used for making decisions contains the inference step and decision step

'Least-square method' in classification can only deal with a small set of tasks. That is because it was built for the regression task. However, we want a method to solve linear classification especially.

Least-squares for linear regression had been talked in 'Simple Linear Regression'. And in this post, we want to find out whether this powerful algorithm can be used in classification.

The discriminant function or discriminant model is on the other side of 'the generative model'. So we, here, have a look at the behave of discriminant function in linear classification

In the posts 'Introduction to Linear Regression', 'Simple Linear Regression' and 'Polynomial Regression and Features-Extension of Linear Regression', we had discussed the regression task. The goal of regression is to find out a function or hypothesis that given an input $\boldsymbol{x}$, the hypothesis can make a prediction $\hat{y}$ which should be as close to the target $y$ as possible.

In this post, we talk about how to use linear regression and its extention polynomial regression fit data

To any input $\boldsymbol{x}$, our goal in a regression task is to give a prediction $\hat{y}=y(\boldsymbol{x})$ to approximate target $t$ where the function $y$ is the chosen hypothesis. And the difference between $t$ and $\hat{y}$ can be called 'error' or more precisely 'loss'.

Squares of the difference between the output of a predictor and the target are wildly used loss function especially in regression problems