Python Forum
dealing with big data of timestamp
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
dealing with big data of timestamp
#1
def function(time):
    return df.iloc[df.index.get_loc(time,method='nearest'),9]

series.apply(function)
hello everyone, I have a dataframe indexed by timestamp and I have as well a series of time (not necessarily exist in the dataframe index). I want for every time to extract a value from the dataframe corresponding to the nearest timestamp. I am running the code above, it works fine but when use it with a big dataframe (1 million rows) it takes days to be executed. Can anyone help please !!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Excel from SAP - dealing with formats and VBA MasterOfDestr 7 551 Feb-25-2024, 12:23 PM
Last Post: Pedroski55
  UnicodeEncodeError - Dealing with Japanese Characters fioranosnake 2 2,442 Jul-07-2022, 08:43 PM
Last Post: fioranosnake
  error in timestamp Led_Zeppelin 3 3,203 Jun-15-2022, 08:28 PM
Last Post: deanhystad
  error in timestamp Led_Zeppelin 0 1,006 Jun-10-2022, 07:59 PM
Last Post: Led_Zeppelin
  Dealing with duplicated data in a CSV file bts001 10 11,450 Sep-06-2021, 12:11 AM
Last Post: SamHobbs
  get year information from a timestamp data frame asli 1 1,660 Jan-08-2021, 09:11 PM
Last Post: Larz60+
  Dealing with a .json nightmare... ideas? t4keheart 10 4,377 Jan-28-2020, 10:12 PM
Last Post: t4keheart
  Dealing with Exponential data parthi1705 11 9,760 May-30-2019, 10:16 AM
Last Post: buran
  Timestamp is undefined ErnestTBass 7 7,934 Feb-16-2019, 08:27 PM
Last Post: snippsat
  Dealing with multiple context managers heras 5 4,689 Nov-16-2018, 09:01 AM
Last Post: DeaD_EyE

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020