I am trying to write the cost function for regularised linear regression. (I am doing exercise 5 from the Stanford Coursera course on machine learning.) In the code below I also include a print of the data so that you can copy it into a .txt file if you want to try it yourself.
Solved
Silly mistake: make sure that you don't make the following mistake when dividing in python:
1 / 2*m is not the same as 1 / (2*m)
import numpy as np from scipy.io import loadmat data = loadmat('ex5data1.mat') X = data['X'] X = np.insert(X, 0, 1, axis=1) y = data['y'] theta = np.ones((2, 1)) print X print y print ' ' def cost_function(theta, X, y, reg_param): theta = np.matrix(theta) X = np.matrix(X) y = np.matrix(y) m = y.shape h = X * theta #This is an mxn matrix with an nx1 matrix to give h being mx1 error = np.power((h - y), 2) #Elementwise subtraction of y(mx1) from h(mx1) term1 = np.sum(error) / 2*y.shape[0] #Non-regularisation term in the cost function reg = (reg_param * np.sum(np.power(theta[1:, :], 2))) / 2*y.shape[0] #Regularisation term in the cost function return term1 + reg print cost_function(X, y, theta, 1)
Output:[[ 1. -15.93675813]
[ 1. -29.15297922]
[ 1. 36.18954863]
[ 1. 37.49218733]
[ 1. -48.05882945]
[ 1. -8.94145794]
[ 1. 15.30779289]
[ 1. -34.70626581]
[ 1. 1.38915437]
[ 1. -44.38375985]
[ 1. 7.01350208]
[ 1. 22.76274892]]
[[ 2.13431051]
[ 1.17325668]
[ 34.35910918]
[ 36.83795516]
[ 2.80896507]
[ 2.12107248]
[ 14.71026831]
[ 2.61418439]
[ 3.74017167]
[ 3.73169131]
[ 7.62765885]
[ 22.7524283 ]]
43775.0196797
The correct output should be ~303, so mine is waaaay off, meaning that I must be making some big mistake in my implementation.Solved
Silly mistake: make sure that you don't make the following mistake when dividing in python:
1 / 2*m is not the same as 1 / (2*m)