+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Data Science (https://python-forum.io/forum-44.html)
+--- Thread: Help needed with Jupyter Notebook (/thread-35963.html)
Help needed with Jupyter Notebook - eyadfr - Jan-03-2022
Hello
I'm working on Jupyter Notebook with a dataframe. When manipulating the dataframe using functions some rows are missed!
The dataset has more than 40,000 rows.
Is there any limitation with processing such big dataframes on Jupyter?
Do you recommend using any other platform other than Google Collab?
Appreciate your help in advance!
Thanks
RE: Help needed with Jupyter Notebook - snippsat - Jan-03-2022
Take a look at Guide to work efficiently with large datasets using Google Colab.
RE: Help needed with Jupyter Notebook - eyadfr - Jan-04-2022
Many thanks for your response.
Is there any other recommendations other than Google Colab?
RE: Help needed with Jupyter Notebook - jefsummers - Jan-04-2022
Not sure if the issue is Jupyter. Pandas limit is based on your memory and that is not dependent of the platform.
But, answering the question, VSCode now supports notebooks.
RE: Help needed with Jupyter Notebook - snippsat - Jan-04-2022
(Jan-04-2022, 05:44 AM)eyadfr Wrote: Is there any other recommendations other than Google Colab?You can of course try to do it local on machine that use.
40,000 rows is not that big so should be possible with normal amount of memory,
if if look link under they test different solutions with a dataset with over
100 million rows and 10 columns.Tutorial on reading large datasets