Python Forum
Python/Numpy have I already written the swiftest code for large array?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Python/Numpy have I already written the swiftest code for large array?
#1
**GOAL:**
I would like to get my script total execution time down from 4 minutes to less than 30 secs. I have a large 1d array (3000000+) of distances with many duplicate distances. I am trying to write the swiftest function that returns all distances that appear n times in the array. I have written a function in numpy but there is a bottleneck at one line in the code. Swift performance is an issue because the calculations are done in a for loop for 2400 different large distance arrays. 

Quote: import numpy as np
for t in range(0, 2400):
a=np.random.randint(1000000000, 5000000000, 3000000)
b=np.bincount(a,minlength=np.size(a))
c=np.where(b == 3)[0] #SLOW STATEMENT/BOTTLENECK
return c

**EXPECTED RESULTS:**
Given a 1d array of distances [2000000000,3005670000,2000000000,12345667,4000789000,12345687,12345667,2000000000,12345667]
I would expect back an array of [2000000000,12345667] when queried to return an array of all distances that appear 3 times in the main array.

What should I do?
Reply


Messages In This Thread
Python/Numpy have I already written the swiftest code for large array? - by justforkicks1 - Dec-12-2017, 06:49 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  [Numpy] How to store different data type in one numpy array? water 7 769 Mar-26-2024, 02:18 PM
Last Post: snippsat
  reshaping 2D numpy array paul18fr 3 1,057 Jan-03-2023, 06:45 PM
Last Post: paul18fr
  Numpy returns "TypeError: unsupported operand type(s) for *: 'numpy.ufunc' and 'int'" kalle 2 2,689 Jul-19-2022, 06:31 AM
Last Post: paul18fr
  Numpy array BrianPA 13 5,100 Jan-23-2021, 09:36 AM
Last Post: Serafim
  How to fill datetime64 field in numpy structured array? AlekseyPython 0 2,315 Oct-20-2020, 08:17 AM
Last Post: AlekseyPython
  Adding data in 3D array from 2D numpy array asmasattar 0 2,258 Jul-23-2020, 10:55 AM
Last Post: asmasattar
  converting dataframe to int numpy array glennford49 1 2,349 Apr-04-2020, 06:15 AM
Last Post: snippsat
  Replacing sub array in Numpy array ThemePark 5 4,274 Apr-01-2020, 01:16 PM
Last Post: ThemePark
  How to prepare a NumPy array which include float type array elements subhash 0 1,946 Mar-02-2020, 06:46 AM
Last Post: subhash
  numpy.where array search for string in just one coordinate adetheheat 1 2,317 Jan-09-2020, 07:09 PM
Last Post: paul18fr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020