Python Forum

Hello

I'm working on Jupyter Notebook with a dataframe. When manipulating the dataframe using functions some rows are missed!

The dataset has more than 40,000 rows.

Is there any limitation with processing such big dataframes on Jupyter?

Do you recommend using any other platform other than Google Collab?

Appreciate your help in advance!

Thanks

Take a look at Guide to work efficiently with large datasets using Google Colab.

Many thanks for your response.

Is there any other recommendations other than Google Colab?

Not sure if the issue is Jupyter. Pandas limit is based on your memory and that is not dependent of the platform.

But, answering the question, VSCode now supports notebooks.

(Jan-04-2022, 05:44 AM)eyadfr Wrote: [ -> ]Is there any other recommendations other than Google Colab?

You can of course try to do it local on machine that use.
40,000 rows is not that big so should be possible with normal amount of memory,
if if look link under they test different solutions with a dataset with over 100 million rows and 10 columns.
Tutorial on reading large datasets