[Machine Learning]: #2 Model and Cost Function

[Machine Learning]: Model and Cost Function

Outline:
1) Linear Regression Model
2) Cost Function
3) Math Model

• Linear Regression Model

First, we give an example of the linear regression model, the example as follows, we are given the house size and house price, we want to know the "relation" between the house price and house size. What should we do?
          House Size(\(x\)):  \([2104, 2000, 1216, 1234, 852, 737, 469, 220]\);
          House Price(\(y\)):  \([460, 300, 232, 315, 178, 177, 140, 100]\);

To describe the supervised learning problem slightly more formally, our goal is, given a training set, to learn a function H (H is the Hypothesis Fucntion): X → Y so that h(x) is a “good” predictor for the corresponding value of y. For historical reasons, this function h is called a hypothesis. Seen pictorially, the process is therefore like this:

         -> input:  \(x_{i}\)            
         -> output:  \(y_{i}\)
           

When \(y\) is continuous, we call the learning problem as a regression problem, when \(y\) is discrete values, we call the learning problem as a classification problem.

--------------------------------------------------------------------------------------------------------------------------

• Cost Function : \({\theta _i}\)

Now, let's consider \(H\) is a Linear Function, and suppose \({H_\theta }\left( x \right) = {\theta _0} + {\theta _1}x\), the problem become how to choose \({\theta _i}\)'s.

How to do this? Here is an idea: choose \({\theta _i}\)'s so that the hypothesis function as close as to real value \(y\). 

Now, we define a Cost Function: \(J({\theta _0},{\theta _1})\), where

$$J({\theta _0},{\theta _1}) = {1 \over {2m}}\sum\nolimits_{i = 1}^m {{{(\hat y_i^{} - {y_i})}^2}}  = {1 \over {2m}}\sum\nolimits_{i = 1}^m {{{({H_\theta }({x_i}) - {y_i})}^2}} $$

And our Goal is to minimize the Cost Function !!!

--------------------------------------------------------------------------------------------------------------------------
Now we get the mathematic model for this question

• Math Model

         Hypothesis:                              \({H_\theta }(x) = {\theta _0} + {\theta _1}x\)

         Parameters:                              \({\theta _0}, {\theta _1}\)

         Cost Function:                         \(J({\theta _0},{\theta _1}) = {1 \over {2m}}\sum\nolimits_{i = 1}^m {{{(\hat y_i^{} - {y_i})}^2}}  = {1 \over {2m}}\sum\nolimits_{i = 1}^m {{{({H_\theta }({x_i}) - {y_i})}^2}} \)

         Goal: minimize:                        \(J({\theta _0},{\theta _1})\)

How can we solve this problem?

Please continue read another Blog [Machine Learning]: Parameter Learning