Python Forum

Full Version: Using chunk size issue
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi ,

My inputs

query_string= sel * from table ( 340k rows)
I am using
df = pd.read_sql(query_string ,conn_bmg,chunksize=50000)
and working on

using this code
total=0
for each in df:
    df1= each[each.Column.str.contains("VENMO")]
    df1['column']='VENMO'


df1.to_csv(r"C:\Users\xxxx\Desktop\venmo_output8.CSV", index=False)
I need to get 340k rows.. since all 340k rows have VENMO. BUt I am getting only 27k rows.
I don't know why. Do I am using any thing wrong in code?
Any help on this?