Python Forum
What is wrong with this implementation of the cost function for linear regression?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
What is wrong with this implementation of the cost function for linear regression?
#1
I am trying to write the cost function for regularised linear regression. (I am doing exercise 5 from the Stanford Coursera course on machine learning.) In the code below I also include a print of the data so that you can copy it into a .txt file if you want to try it yourself.

import numpy as np
from scipy.io import loadmat

data = loadmat('ex5data1.mat')
X = data['X']
X = np.insert(X, 0, 1, axis=1)
y = data['y']
theta = np.ones((2, 1))

print X
print y
print ' '

def cost_function(theta, X, y, reg_param):
	theta = np.matrix(theta)
	X = np.matrix(X)
	y = np.matrix(y)
	m = y.shape
	h = X * theta #This is an mxn matrix with an nx1 matrix to give h being mx1
	error = np.power((h - y), 2) #Elementwise subtraction of y(mx1) from h(mx1)
        term1 = np.sum(error) / 2*y.shape[0] #Non-regularisation term in the cost function
	reg = (reg_param * np.sum(np.power(theta[1:, :], 2))) / 2*y.shape[0] #Regularisation term in the cost function
	return term1 + reg
	
print cost_function(X, y, theta, 1)
Output:
[[ 1. -15.93675813] [ 1. -29.15297922] [ 1. 36.18954863] [ 1. 37.49218733] [ 1. -48.05882945] [ 1. -8.94145794] [ 1. 15.30779289] [ 1. -34.70626581] [ 1. 1.38915437] [ 1. -44.38375985] [ 1. 7.01350208] [ 1. 22.76274892]] [[ 2.13431051] [ 1.17325668] [ 34.35910918] [ 36.83795516] [ 2.80896507] [ 2.12107248] [ 14.71026831] [ 2.61418439] [ 3.74017167] [ 3.73169131] [ 7.62765885] [ 22.7524283 ]] 43775.0196797
The correct output should be ~303, so mine is waaaay off, meaning that I must be making some big mistake in my implementation.

Solved

Silly mistake: make sure that you don't make the following mistake when dividing in python:

1 / 2*m is not the same as 1 / (2*m)
Reply
#2
(Dec-22-2017, 07:24 PM)JoeB Wrote: make sure that you don't make the following mistake when dividing in python


well, not only python. They are not same also in 'standard' math calculations :-)
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Simple linear regression with interaction summary table Andrzej_Andrzej 0 227 Feb-21-2024, 07:44 AM
Last Post: Andrzej_Andrzej
  Linear regression doubt - Urgent kumarants 6 3,053 May-05-2020, 04:11 PM
Last Post: kumarants
  prediction using linear regression (extrapolation?) in a loop karlito 0 3,181 Feb-05-2020, 10:56 AM
Last Post: karlito
  Linear Regression on Time Series karlito 5 3,843 Jan-28-2020, 10:02 AM
Last Post: buran
  How to build linear regression by implementing Gradient Descent using only linear alg PythonSpeaker 1 2,159 Dec-01-2019, 05:35 PM
Last Post: Larz60+
  Linear Regression Python3 code giving weird solutions deepsen 0 1,396 Nov-01-2019, 12:06 PM
Last Post: deepsen
  something that returns a linear function compatible with lognorm.pdf and lognorm.cdf? Afterdarkreader 2 3,184 Dec-19-2017, 10:42 PM
Last Post: Afterdarkreader
  help on exponential smoothing and linear regression hkyeung 1 3,106 Sep-02-2017, 09:31 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020