Using chunk size issue - Printable Version

Using chunk size issue - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Using chunk size issue (/thread-16299.html)

Using chunk size issue - sandy - Feb-21-2019

Hi ,

My inputs

query_string= sel * from table ( 340k rows)
I am using

df = pd.read_sql(query_string ,conn_bmg,chunksize=50000)

and working on

using this code

total=0
for each in df:
    df1= each[each.Column.str.contains("VENMO")]
    df1['column']='VENMO'


df1.to_csv(r"C:\Users\xxxx\Desktop\venmo_output8.CSV", index=False)

I need to get 340k rows.. since all 340k rows have VENMO. BUt I am getting only 27k rows.
I don't know why. Do I am using any thing wrong in code?

RE: Using chunk size issue - sandy - Feb-25-2019

Any help on this?