[Machine Learning]: #7 Classification Introduction

[Machine Learning] #7 Classification Introduction

****************************************

[What is Classification Problem]:

if or not problem/ yes or no problem.

Is it a spam email? Are you a good boy? Do you like this girl?...... 

[Logistic Regression Model]:

Linear Regression performs badly in the classification problems. because the classification problem is not 'Continuous'. Intuitively, it also doesn't make sense for ${H_\theta }(x)$ which is larger than 1 or smaller than 0 we when knowing the $y = \{ 0,1\}$, $y$ is just 0 or 1.

So we try to make the value of  ${H_\theta }(x)$ between 0 and 1, how can we do that? Yes, Logistic Function.

$${H_\theta }(x) = g({\theta ^T}x)$$

$$z = {\theta ^T}x$$

$$g(z) = {1 \over {1 + {e^{ - z}}}}$$
And the following image shows that what the function looks like:
[What does the funtion means]:
${H_\theta }(x)$ gives us the probability that the output is 1.
For example.  ${H_\theta }(x)=0.8$ means that we have 80% probability that the output is 1, on the contrary, the output is 0 's probability is 0.3. We use $P(y = 1|x;\theta ) = 0.8$ to represent that the probabilty of $y=1$ is 0.8 in the case of $x$ and $\theta$.

[Decision Bounary]:

In order to get our discrete 0 or 1 classification, we can translate the output of the hypothesis function ${H_\theta }(x)$ as follows:

when, 

$$\eqalign{   & {H_\theta }(x) \ge 0.5 -  > class1  \cr   & {H_\theta }(x) < 0.5 -  > class2 \cr}$$
So, what does it means?

$${H_\theta }(x) = g({\theta ^T}x) \ge 0.5 \to class1$$

which means

$${\theta ^T}x \ge 0 \to class1$$

From the above statments we can say that:
$$\eqalign{   & {\theta ^T}x \ge 0 \to class1  \cr   & {\theta ^T}x < 0 \to class2 \cr}$$
The Decision Boundary is the line to seperate the area where $y=0$ and $y=1$.

[An Example]:

In this case, if we consider that when $${\theta ^T}x \ge 0 \to class1$$

then  $${x_1}^2 + x_2^2 \ge 1$$

[How to find the Decision Boundary ?]:

Please check the next [Machine Learning]#8