Python Forum
Help needed with Jupyter Notebook
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Help needed with Jupyter Notebook
#1
Hello

I'm working on Jupyter Notebook with a dataframe. When manipulating the dataframe using functions some rows are missed!

The dataset has more than 40,000 rows.

Is there any limitation with processing such big dataframes on Jupyter?

Do you recommend using any other platform other than Google Collab?

Appreciate your help in advance!

Thanks
Reply
#2
Take a look at Guide to work efficiently with large datasets using Google Colab.
Reply
#3
Many thanks for your response.

Is there any other recommendations other than Google Colab?
Reply
#4
Not sure if the issue is Jupyter. Pandas limit is based on your memory and that is not dependent of the platform.

But, answering the question, VSCode now supports notebooks.
Reply
#5
(Jan-04-2022, 05:44 AM)eyadfr Wrote: Is there any other recommendations other than Google Colab?
You can of course try to do it local on machine that use.
40,000 rows is not that big so should be possible with normal amount of memory,
if if look link under they test different solutions with a dataset with over 100 million rows and 10 columns.
Tutorial on reading large datasets
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Jupyter notebooks, jupyter kernels, virtual environment created in Anaconda... bytecrunch 8 2,130 Nov-05-2023, 06:38 PM
Last Post: snippsat
  Jupyter Notebook TypeError Holidays entity loader class bmanning20 0 2,642 Jul-20-2023, 06:52 PM
Last Post: bmanning20
  Why does graph look different in Codecademy and Jupyter Notebook abc123456 1 1,450 Mar-29-2023, 09:22 AM
Last Post: newbieAuggie2019
  Setting up and integrating Jupyter Notebook development environment with VSC Drone4four 3 1,781 Sep-12-2022, 09:18 AM
Last Post: Drone4four
Thumbs Up can't access data from URL in pandas/jupyter notebook aaanoushka 1 1,829 Feb-13-2022, 01:19 PM
Last Post: jefsummers
Exclamation Jupyter Notebook - Help Needed! eyadfr 1 1,807 Jan-26-2022, 06:42 PM
Last Post: jefsummers
  HELP! Importing json file into csv into jupyter notebook vilsef 2 2,531 Jan-22-2021, 11:06 AM
Last Post: snippsat
  How can draw a real-time marker on map using folium/leaflet Jupyter notebook C3PO 0 2,389 Dec-22-2020, 07:04 PM
Last Post: C3PO
  Python Custom Module not working in Jupyter Notebook with Pandas fid 0 2,005 Jul-04-2020, 11:05 AM
Last Post: fid
  How to change working directory in Jupyter Notebook? rajeev1729 2 21,029 Aug-25-2019, 07:25 PM
Last Post: perfringo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020