Python Forum
Extracting rows based on condition on one column
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Extracting rows based on condition on one column
#3
Thanks, it worked!

Out of curiosity, here is a little test I did comparing the execution time. It appears the NumPy method is 75x faster than looping. Do, you know what makes NumPy fast? Does it store the array in some efficient manner or something else?

import time
input = np.arange(4*10**7).reshape((10**7, 4))

# First method: Using NumPy
start_time = time.time()
print(input[(-1000 < input[:, 2]) & (input[:, 2] < 10000)])
print(time.time()-start_time)
start_time = time.time()

# Second method: Without NumPy

diff = []
for row in range(10**7):
    if -1000 < arr[row, 2] < 10000:
          diff.append(arr[row, :])

print(diff)

print(time.time()-start_time)
Reply


Messages In This Thread
RE: Extracting rows based on condition on one column - by Robotguy - Aug-07-2020, 02:27 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Find duplicates in a pandas dataframe list column on other rows Calab 2 2,055 Sep-18-2024, 07:38 PM
Last Post: Calab
  Merging rows and adding columns based on matching index pythonnewbie78 3 1,744 Dec-24-2023, 11:51 AM
Last Post: Pedroski55
  Make unique id in vectorized way based on text data column with similarity scoring ill8 0 1,435 Dec-12-2022, 03:22 AM
Last Post: ill8
  reduce time series based on sum condition amdi40 0 1,569 Apr-06-2022, 09:09 AM
Last Post: amdi40
  Pandas Dataframe Filtering based on rows mvdlm 0 2,053 Apr-02-2022, 06:39 PM
Last Post: mvdlm
  New Dataframe Column Based on Several Conditions nb1214 1 2,495 Nov-16-2021, 10:52 PM
Last Post: jefsummers
  Pandas Data frame column condition check based on length of the value aditi06 1 3,711 Jul-28-2021, 11:08 AM
Last Post: jefsummers
Question [Solved] How to refer to dataframe column name based on a list lorensa74 1 2,984 May-17-2021, 07:02 AM
Last Post: lorensa74
  Add column based on others timste 8 5,581 Apr-03-2021, 07:39 AM
Last Post: devesh_sahu
  Dropping Rows From A Data Frame Based On A Variable JoeDainton123 1 2,858 Aug-03-2020, 02:05 AM
Last Post: scidam

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020