Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Slow Python Code
#2
You could implement a list comprehension instead of list.extend().

data3 = [data2[index+1],data2[index+2],data2[index+3] for i in data for index, j in enumerate(data2) if i==j]
List comprehensions are faster and you wouldn't be issuing a method call for every match.

You could also sort data and use the bisect module to identify and target specific indices to be checked. Bisecting works by dividing the list in half over and over again until you find a match. Instead of iterating over 2.3 million items in data, you be iterating over log 2 of 2.3 million - only 21 items!

This *should* work but may need some fine tuning. Again, data needs to be sorted first. There are some highly efficient sorting algorithms that should
be able to the do the trick.

import bisect

for i, x in enumerate(data2):
    index = bisect.bisect_left(data, x)
    for y in data[index:]:
        if x != y:
            break
            
        data3.extend([data2[i+1],data2[i+2],data2[i+3]])
Reply


Messages In This Thread
Slow Python Code - by Jay123 - Sep-05-2019, 02:43 PM
RE: Slow Python Code - by stullis - Sep-05-2019, 10:27 PM
RE: Slow Python Code - by scidam - Sep-06-2019, 12:23 AM
RE: Slow Python Code - by Jay123 - Sep-09-2019, 08:46 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  My python code is running very slow on millions of records shantanu97 7 2,650 Dec-28-2021, 11:02 AM
Last Post: Larz60+
  Optmized way to rewrite this very slow code liva28 0 1,513 Jul-18-2021, 12:16 PM
Last Post: liva28
  Python file to slow, how peed up ? Leon 4 3,126 Jan-05-2019, 09:40 AM
Last Post: Gribouillis
  simple code is way too slow JAREDZ 7 8,177 Nov-11-2018, 12:03 PM
Last Post: Larz60+
  Python 2.7 Addition to dict is too slow VolanD 6 4,099 May-04-2018, 09:24 AM
Last Post: Gribouillis

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020