Cost Function

The measurement of accuracy of a hypothesis function. The accuracy is given as an average difference of all the results of the hypothesis from the inputs ($x$’s) to the outputs ($y$’s).

$$\begin{equation} J(\Theta_{0},\Theta_{1})=\frac{1}{2m}\sum_{i=1}^{m}(h_{\Theta}(x_{i}) - y_{i})^{2} \end{equation}$$

where $m$ is the number of inputs (e.g. training examples)

This function is also known as the squared error function or mean squared error. The $\frac{1}{2}$ is a convenience for the cancellation of the 2 which will be present due to the squared term being derived (see gradient descent).

The basic idea of the cost function is to choose a $\Theta_{0}$ and $\Theta_{1}$ such that the $h_{\Theta}(x)$ is as close to $y$, as possible, for our training examples $(x,y)$.

In an ideal world, the cost function would have a value of 0 (i.e. $J(\Theta_{0},\Theta_{1}) = 0$), which would imply we have a straight line which passes through each of our data points and that we can, with perfect accuracy, predict any new data point which may come into our set.