Python Forum
optimization problem for dataframe manipulation
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
optimization problem for dataframe manipulation
#1
Hi, I am kind of new and learning python. Here I have an optimization problem and hope someone can help me through:

Let inventories be a dataframe, alpha, beta, zeta, sigma, gamma, omega, mon are known coefficients and v1 and w6 are functions for data transformation.

What I want to do is the following (original code):
for item in inventories.columns:
        x = inventories[item]                                    # get a single column
        x = x.sort_values(ascending=True)                        # sort its values in ascending order
        x.index= range(len(x))                                   # index should also be in ascending order and used in next line                    
        ind = np.where(x<0, x.index.values, n - x.index.values)  # create a serie depending on if x's value is negative/positive
        fac = np.where(x<0, -omega, zeta)                       # similar to the above
        val = np.where(x<0, -x.values, x.values)                 # similar to the above
        exponent = np.where(x<0, beta, alpha)                    # similar to the above
        v = [ fac * v1(val,exponent)*(w6((ind+1)/mon, sigma, gamma) - w6((ind)/mon, sigma, gamma))] # here is the data transformation to produce 
                                                                                                        # one value for each item
The above code runs but extremely slow because it runs one columns at a time. So Is there any way to optimize the code with python tricks to make it run faster?

Here's what I like to modify (but need to get it work):
	x = inventories
	x = pd.DataFrame(np.sort(x.values, axis=0), index=x.index, columns=x.columns) # This sorting works
	ind = np.where(x<0, x.index.values, n - x.index.values)                       # failed to broadcast due to dimentional difference
	fac = np.where(x<0, -omega, zeta)                                          # failed to broadcast due to dimentional difference
	val = np.where(x<0, -x.values, x.values)                                      # failed to broadcast due to dimentional difference
	exponent = np.where(x<0, beta, alpha)                                       # failed to broadcast due to dimentional difference
	v = fac * v1(val,exponent)*(w6((ind+1)/mon, sigma, gamma) - w6((ind)/mon, sigma, gamma)) # ???
Thanks to anyone who can help!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  2-dataframe, datetime lookup problem Mark17 0 1,230 Jan-27-2022, 01:02 AM
Last Post: Mark17
  Problem in list manipulation CyKlop 6 2,283 Oct-18-2021, 09:03 AM
Last Post: DeaD_EyE
  problem writing dataframe to oracle aliyesami 4 2,651 Sep-25-2021, 11:20 PM
Last Post: SamHobbs
  Problem in saving .xlsm (excel) file using pandas dataframe in python shantanu97 2 4,268 Aug-29-2021, 12:39 PM
Last Post: snippsat
  Dataframe problem stylingpat 3 2,814 Mar-22-2021, 10:36 AM
Last Post: stylingpat
Question Dataframe Manipulation Coping Rows and Removing Dates ashleysnl 1 1,775 Feb-26-2021, 10:00 PM
Last Post: nilamo
  Problem with If statement and dataframe Milfredo 1 1,766 Sep-16-2020, 05:50 AM
Last Post: Milfredo
  Dataframe mean calculation problem: do we have to loop? sparkt 1 2,175 Aug-28-2020, 02:41 PM
Last Post: sparkt
  Problem in solving optimization problem Ruchika 0 1,553 Jul-27-2020, 05:28 AM
Last Post: Ruchika
  Optimization problem ( food menu ) Truman 3 3,561 Apr-01-2019, 09:28 PM
Last Post: Truman

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020