[Machine Learning]: #7 Classification Introduction

[Machine Learning] #7 Classification Introduction

****************************************

[What is Classification Problem]:

if or not problem/ yes or no problem.

Is it a spam email? Are you a good boy? Do you like this girl?...... 

[Logistic Regression Model]:

Linear Regression performs badly in the classification problems. because the classification problem is not 'Continuous'. Intuitively, it also doesn't make sense for \({H_\theta }(x)\) which is larger than 1 or smaller than 0 we when knowing the \(y = \{ 0,1\}\), \(y\) is just 0 or 1.

So we try to make the value of  \({H_\theta }(x)\) between 0 and 1, how can we do that? Yes, Logistic Function.

$${H_\theta }(x) = g({\theta ^T}x)$$

$$z = {\theta ^T}x$$

$$g(z) = {1 \over {1 + {e^{ - z}}}}$$
And the following image shows that what the function looks like:
[What does the funtion means]:
\({H_\theta }(x)\) gives us the probability that the output is 1.
For example.  \({H_\theta }(x)=0.8\) means that we have 80% probability that the output is 1, on the contrary, the output is 0 's probability is 0.3. We use \(P(y = 1|x;\theta ) = 0.8\) to represent that the probabilty of \(y=1\) is 0.8 in the case of \(x\) and \(\theta\).

[Decision Bounary]:

In order to get our discrete 0 or 1 classification, we can translate the output of the hypothesis function \({H_\theta }(x)\) as follows:

when, 

$$\eqalign{
  & {H_\theta }(x) \ge 0.5 -  > class1  \cr
  & {H_\theta }(x) < 0.5 -  > class2 \cr} $$
So, what does it means?

$${H_\theta }(x) = g({\theta ^T}x) \ge 0.5 \to class1$$

which means

$${\theta ^T}x \ge 0 \to class1$$

From the above statments we can say that:
$$\eqalign{
  & {\theta ^T}x \ge 0 \to class1  \cr
  & {\theta ^T}x < 0 \to class2 \cr} $$
The Decision Boundary is the line to seperate the area where \(y=0\) and \(y=1\).

[An Example]:

In this case, if we consider that when $${\theta ^T}x \ge 0 \to class1$$

then $${x_1}^2 + x_2^2 \ge 1$$

[How to find the Decision Boundary ?]:

Please check the next [Machine Learning]#8