Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Slow Python Code
#3
Another way is to use Pandas:

import pandas as pd
df1 = pd.DataFrame({'x':[1,2.3,3.2,3.3,5,6,7,8,6.2,7]}, dtype=float)
df2 = pd.DataFrame({'y': list(range(100))}, dtype=float)
s = df2.y.isin(df1.x)
pd.np.hstack([df2[s].values, df2[s.shift(1, fill_value=False)].values, df2[s.shift(2, fill_value=False)].values])
Output:
array([[ 1., 2., 3.], [ 5., 6., 7.], [ 6., 7., 8.], [ 7., 8., 9.], [ 8., 9., 10.]])
Note: you need explicitly declare data types for dataframes if you are planning to use .isin method. .isin performs some data type casting (see bug) that can lead to unexpected results, e.g. when comparing arrays of float and integer values. This bug is still actual in Pandas v0.25.0.
Reply


Messages In This Thread
Slow Python Code - by Jay123 - Sep-05-2019, 02:43 PM
RE: Slow Python Code - by stullis - Sep-05-2019, 10:27 PM
RE: Slow Python Code - by scidam - Sep-06-2019, 12:23 AM
RE: Slow Python Code - by Jay123 - Sep-09-2019, 08:46 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  My python code is running very slow on millions of records shantanu97 7 2,653 Dec-28-2021, 11:02 AM
Last Post: Larz60+
  Optmized way to rewrite this very slow code liva28 0 1,514 Jul-18-2021, 12:16 PM
Last Post: liva28
  Python file to slow, how peed up ? Leon 4 3,137 Jan-05-2019, 09:40 AM
Last Post: Gribouillis
  simple code is way too slow JAREDZ 7 8,204 Nov-11-2018, 12:03 PM
Last Post: Larz60+
  Python 2.7 Addition to dict is too slow VolanD 6 4,113 May-04-2018, 09:24 AM
Last Post: Gribouillis

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020