Python Forum
Drop rows from data with zero value - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Data Science (https://python-forum.io/forum-44.html)
+--- Thread: Drop rows from data with zero value (/thread-14958.html)



Drop rows from data with zero value - Devilish - Dec-26-2018

I know it's probably a real simple answer... I only have 2 columns of data, 1 being the Timedate Index, and the 2nd is my values. Most of the values are actually ZERO... I'm wanting to drop all rows with a ZERO value so it is only analyzing data that has a value. I'm running it through a percent change function looking for percent change between all non-zero data... I MUCH APPRECIATE any help


RE: Drop rows from data with zero value - stullis - Dec-26-2018

Line 12 below will do it for you.

data = [
    ["2018-01-01", 0],
    ["2018-01-02", 1],
    ["2018-01-03", 0],
    ["2018-01-04", 0],
    ["2018-01-05", 0],
    ["2018-01-06", 31],
    ["2018-01-07", 0],
    ["2018-01-08", 0]
]

data2 = [entry for entry in data if entry[1] != 0]
print(data2)



RE: Drop rows from data with zero value - micseydel - Dec-27-2018

If stullis' code doesn't answer your question, I highly recommend providing your code so that we have some context about what you're working with. Ideally it would be minimal code (5-10 lines) that shows the problem.


RE: Drop rows from data with zero value - Devilish - Dec-27-2018

(Dec-27-2018, 12:15 AM)micseydel Wrote: If stullis' code doesn't answer your question, I highly recommend providing your code so that we have some context about what you're working with. Ideally it would be minimal code (5-10 lines) that shows the problem.

It actually didn't work... maybe it's because my data actually has headers... here is my basic code

import pandas as pd
from pandas import Series, DataFrame
import numpy as np
import os
import io

# import 
df = pd.read_csv ('NQH7.txt', sep=', ', engine='python') #FOR ME it is importing the file with a space after the comma so having to specify that it is more than just a comma...
                                        # Without specifying it produces column names that begin with a space and that creates issues all over the place
# clean whitespaces if they exist 
df = df.applymap(lambda x: x.strip() if isinstance(x, str) else x)

# combine date and time into single column
df['datetime'] = pd.to_datetime(df.Date + ' ' + df.Time, format="%Y/%m/%d %H:%M:%S.%f")
df.set_index("datetime", inplace = True)
df = df.drop(columns=['Date', 'Time', 'Open', 'High', 'Low', 'Last', '# of Trades', 'Volume', 'OHLC Avg', 'HLC Avg', 
                     'HL Avg', 'Bid Volume', 'Ask Volume', 'Zig Zag Mid-Point', 'Extension Lines']) # drop combined columns
When I print I get...
Output:
Zig Zag Text Labels datetime 2017-01-02 17:00:00.000 5026.25 10.0 2017-01-02 17:00:00.220 5027.66 0.0 2017-01-02 17:00:00.400 5029.07 0.0 2017-01-02 17:00:07.100 5030.48 0.0 2017-01-02 17:00:08.700 5031.89 0.0 2017-01-02 17:00:08.170 5033.30 0.0 2017-01-02 17:00:09.000 5034.70 0.0 2017-01-02 17:00:11.400 5036.11 0.0 2017-01-02 17:00:18.500 5037.52 0.0 2017-01-02 17:00:22.200 5038.93 0.0 2017-01-02 17:17:08.000 5040.34 0.0 2017-01-02 17:19:45.700 5041.75 12.0 2017-01-02 18:02:02.100 5040.22 0.0 2017-01-02 18:04:02.300 5038.69 0.0 2017-01-02 18:05:57.000 5037.16 0.0 2017-01-02 18:14:54.000 5035.63 0.0 2017-01-02 18:18:41.300 5034.09 0.0 2017-01-02 18:35:01.100 5032.56 0.0 2017-01-02 18:36:25.000 5031.03 0.0 2017-01-02 18:37:25.000 5029.50 3.0 2017-01-02 18:45:40.000 5035.75 1.0
I'm wanting to basically append the df or create a new one that drops all rows where the 3rd column is a ZERO

THANKS BUT I GOT IT WORKING!!!