Implementing a Lowpass Filter

[Madridista]

I have data from a IMU Sensor. I have file an Excelfile samplesdata where the data of the IMU Sensor is export. In samplesdata There is a column Euclidian Norm and i want to filter the noisy signal. I research and found the Low pass filter to filter the noisy signal from my data. I write this code in Jupyter Notebook:

import numpy as np
from scipy.signal import butter, lfilter, freqz
import matplotlib.pyplot as plt


def butter_lowpass(cutoff, fs, order=5):
    nyq = fs/2
    normal_cutoff = cutoff / nyq
    b, a = butter(order, normal_cutoff, btype='low', analog=False)
    return b, a


def butter_lowpass_filter(data, cutoff, fs, order=5):
    b, a = butter_lowpass(cutoff, fs, order=order)
    y = lfilter(b, a, data)
    return y


data=pd.read_csv('samplesdata.csv',sep=";", decimal=",",encoding='latin-1')
sensor_data=data[['Euclidian Norm']]
sensor_data=np.array(sensor_data)

# Filter requirements.
order = 10
fs = 100  # sample rate, Hz
cutoff = 30 

y = butter_lowpass_filter(sensor_data, cutoff, fs, order)
b, a = butter_lowpass(cutoff, fs, order)

plt.figure()
plt.subplot(211)
plt.plot(sensor_data, color='green')
plt.grid(color='green', linestyle='--', linewidth=0.5)
plt.title('original signal')

plt.subplot(212)
plt.plot(y, color='red')
plt.title('Filtered')
plt.grid(color='green', linestyle='--', linewidth=0.5)
plt.show(block=True)

The problem is that it only plot the unfiltered data twice.

https://ibb.co/PCNsrs1

In this Link you can see that it only shows the unfiltered data two times.

Does anyone has a idea what in this code could be wrong?

[deanhystad]

Worked fine for me. I don't have your data, so I made a noisy sine wave.

import numpy as np
from scipy.signal import butter, lfilter
import matplotlib.pyplot as plt

def butter_lowpass_filter(data, cutoff, fs, order=5):
    nyq = fs/2
    normal_cutoff = cutoff / nyq
    b, a = butter(order, normal_cutoff)
    return lfilter(b, a, data)

x = np.linspace(0, 10, 1000)
noisy = np.sin(x * np.pi + 10) + np.sin(x * np.pi * 117) * 0.1  # sin noise
# noisy = np.sin(x * np.pi + 10) + np.random.normal(size=x.shape)  # random noise
filtered = butter_lowpass_filter(noisy, 30, 100, order=10)

plt.figure()
plt.subplot(211)
plt.plot(x, noisy, color='green')
 
plt.subplot(212)
plt.plot(x, filtered, color='red')
plt.show(block=True)

Run it and other than some startup weirdness it looks great.

Then I tried some filtering some random noise. You can do this by moving the comment in the noisy generator. This didn't work out as well. There is filtering, but it is not easy to see the difference between filtered and unfiltered signal.

Next I played with changing the cutoff frequency. Changing it to 10 Hz started to reveal the underlying sine wave. Setting it to 2 cleared up all the noise but warps the underlying shape.

I think you just have a lot of noise relative to your signal. Can you boost the sampling rate?

[Madridista]

I changed the Parameters, but it didnt changed anything.

What is x in your code?

Maybe there is a problem what i plot? Because in Jupyter Notebook if i do a tab in the code it change a lot. Should i do a tab somewhere that Jupyter knows that i use the the method butter_lowpass_filter or something like that?

My data in Excel looks like that:

https://ibb.co/s2ZSfB8

[deanhystad]

x is time, in seconds. It made it easier for me to make sine waves of different frequencies if I have a time base to work from. Sorry, but I have no experience with Jupyter notebooks. This thread might be a better fit in the Data Science forum.

[Madridista]

Thank you, I will post in the Data Science forum. Because the issue is not maybe the code. Maybe its the code conventions.

[Madridista]

Now my post is in the Data Science Forum, because i was directed here. I hope someone can help me here :)