Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Linear Regression Python3 code giving weird solutions
#1
Dear Members,
Welcome to another noob question. I am learning to code in Python for scientific research.
I wrote a python code following the math for linear regression. When I use the dataset used in the website from where I referred the math all my intermediate steps match the solution provided in the worked-out example in the website. I checked by plotting the data on excel that everything matches.
Next, I used a small subset of a second data set and the Root Mean Square Error value comes out in the 200s while an excel plot says the RMSE should be 0.99, however, the slope and intercept value I calculate matches with the values in excel. I cannot track what is going on with my RMSE calculation.
Both data sets are available in my code.
Data set 1 between lines 22-24 and is commented out.
Data set 2 (the one causing the problem) can be found between lines 32-34
The RMSE code can be found between lines 105 to 140.

Any insight would be greatly appreciated.

'''
 Created on Fri Nov 01 2019 1:15:01 PM

  2019 Deep Sen
'''
'''
Linear Regression Math steps:
https://machinelearningmastery.com/simple-linear-regression-tutorial-for-machine-learning/

I wrote the python code so that I can understand every step. This is version 1.
Version 2 will have functions and user can enter data.
Version 3 will allow user to point to an csv file with x,y data.
Version 4 will have grahical user interface
'''
# Simple Linear regression

import matplotlib.pyplot as plt
import numpy as np

# Data

# Trial data in the link given above
#depth_x = np.array([1, 2, 4, 3, 5])
#age_y = np.array([1, 3, 3, 2, 5])

# When I use the data above, all my intermidiate solutions match all the intermidiate step solutions provided in the link above. An excel plot of the data gives the same slope, intercept and RMSE value so the mathematics shown is correct.

# But when I use the data from Waltham see data block below I get a really weird RMSE value in the two hundreds! What am I missing? The slope value and intercept value matches with an excel plot of the data below.

# Data in Mathematics Tool for Geologist by David Waltham, pg 21, Table 2.2

depth_x = np.array([0.5, 1.3, 2.47, 4.9, 8.2])
age_y = np.array([1020, 2376, 5008, 10203, 15986])

'''
Simple linear regression model y = b0 + b1*x
where x is the independent variable, y is the dependent variable, b0 is the intercept and b1 is the slope.
'''
#######################################

### Estimating b1, which represents the slope
'''
1) b1 can be estimated by :

b1 = sum((xi-mean(x)) * (yi-mean(y))) / sum((xi – mean(x))**2)

b1 = (ss_xy/ss_xx)**2

where xi and yi are the ith value of x and y in an array or a list.

'''
# Mean of depth_x and age_y

average_x = np.mean(depth_x)
average_y = np.mean(age_y)
print('Average Depth: ', average_x, 'm' '\nAverage Age:', average_y, 'yr')

# Calculating difference between depth_xi and average_x

diff_x = depth_x - average_x

#print('(xi - mean x): ',diff_x)

# Calculating difference between age_yi and average_y
diff_y = age_y - average_y

#print('(yi - mean y): ', diff_y)

# Products of the differences

p_xy = diff_x * diff_y

#print('Product of difference: ', p_xy)

# Sum of products

sp_xy = np.sum(p_xy)

#print('Sum of Products: ', sp_xy)

# Sum of the difference between xi and mean x

sp_xx = np.sum(diff_x**2)

#print('Sum of difference (xi - mean x): ', sp_xx)


# Calculating b1 = (sp_xy / sp_xx)**2

b1 = (sp_xy/sp_xx)

print('Slope: ', b1)

#####################################

# Estimating b0 which represents the intercept

'''
2) b0 can be estimated by:

b0 = mean(y) – b1 * mean(x)

'''
b0 = average_y - b1 * average_x

print('Intercept: ', b0)

######################################

# Calculaing the Root Mean Square Error

# Calculating predicted value of y

pred_y = b0 + (b1 * depth_x)

#print('Predicted y value: ', pred_y)

# RMSE = sqrt( sum( (pred_y – yi)^2 )/n )

# Square of difference between pred_y and yi

sqdiff_y = (pred_y - age_y)**2

#print('Square of (pred_y - yi): ', sqdiff_y)

#Sum of the Square of difference between pred_y and yi

s_sqdiff_y = np.sum(sqdiff_y)

#print('Sum of square of (pred_y - yi): ', s_sqdiff_y)

# Average of the Sum of the Square of difference between pred_y and yi

av_s_sqdiff_y = s_sqdiff_y / np.size(age_y)

#print('Average of the Sum of the Square of (pred_y - yi): ', av_s_sqdiff_y)

# Square root of Average of the Sum of the Square of difference between pred_y and yi

rmse = np.sqrt(av_s_sqdiff_y)

print('Root Mean Square Error: ', rmse)

##################################

# Plotting scatter plot of data

plt.scatter(depth_x, age_y, color='m', marker = 'o', s = 30)

# Plotting Linear fit

plt. plot(depth_x, pred_y, color='r')

plt.show()
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Machine Learning Polynomial Regression braveYug 0 79 Nov-13-2019, 11:41 AM
Last Post: braveYug
  Regression Event navigates 1 130 Oct-01-2019, 07:27 PM
Last Post: Larz60+
  Evaluate dataset with logistic regression chisox721 6 415 Jun-06-2019, 03:01 PM
Last Post: chisox721
  sklearn regression to excel punksnotdead 1 374 Apr-14-2019, 12:32 PM
Last Post: punksnotdead
  Logistic regression with Tensorflow kiton 1 612 Nov-28-2018, 07:34 PM
Last Post: kiton
  Cannot use pypy on Python3 code ErnestTBass 11 2,752 May-24-2018, 02:01 PM
Last Post: ErnestTBass
  What is wrong with this implementation of the cost function for linear regression? JoeB 1 1,194 Dec-23-2017, 10:05 AM
Last Post: buran
  help on exponential smoothing and linear regression hkyeung 1 1,134 Sep-02-2017, 09:31 PM
Last Post: Larz60+

Forum Jump:


Users browsing this thread: 1 Guest(s)