How to use scipy.optimization.brute for multivariable function

How to use scipy.optimization.brute for multivariable function - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: How to use scipy.optimization.brute for multivariable function (/thread-30391.html)

How to use scipy.optimization.brute for multivariable function - Shiladitya - Oct-19-2020

I want to find the global minima of a two-variables function, say V(x,y,T), where (x,y) are the independent variables and `T' is the parameter. My target is to find the global minimum/ minima (minima of the function which has two variables is determined by https://en.wikipedia.org/wiki/Second_partial_derivative_test) of the function V(x,y,T) for a fixed value of the parameter T. I want to use the package 'scipy.optimization.brute'. At first, please confirm whether I am right or wrong.

I am pasting a sample example from (https://scipy-lectures.org/intro/scipy/auto_examples/plot_optimize_example2.html)

[python]
from scipy import optimize

# Global optimization
grid = (-10, 10, 0.1)
xmin_global = optimize.brute(f, (grid, ))
print("Global minima found %s" % xmin_global)

[/python]
This example is for single-variable-function. I have no idea how to put the input for the two-variables case.

RE: How to use scipy.optimization.brute for multivariable function - scidam - Oct-19-2020

You need to define ranges where brute will search for an optimum, e.g.

grid = ((-1, 1, 0.1), (-1, 1, 0.1)) 
# Also, you can use slice objects for this:
# grid = (slice(-1, 1, 0.1), slice(-1, 1, 0.1))

Further, you need to define a function to be optimized:

def f(x, T):
    return (x[0] - T) ** 2 + (x[1] - T) ** 2

And, finally, use brute to find the minimum:

from scipy.optimize import brute
brute(f, grid, (1,))

RE: How to use scipy.optimization.brute for multivariable function - Shiladitya - Oct-20-2020

(Oct-19-2020, 10:08 PM)scidam Wrote: You need to define ranges where brute will search for an optimum, e.g.
grid = ((-1, 1, 0.1), (-1, 1, 0.1)) 
# Also, you can use slice objects for this:
# grid = (slice(-1, 1, 0.1), slice(-1, 1, 0.1))
Further, you need to define a function to be optimized:
def f(x, T):
    return (x[0] - T) ** 2 + (x[1] - T) ** 2
And, finally, use brute to find the minimum:
from scipy.optimize import brute
brute(f, grid, (1,))

Thanks a lot for your kind reply.

And I have a question-

brute(f, grid, (1,))

What does "(1,)" stand for? Is it the value of parameter 'T'?

And when defined the function, you used x[0] and x[1]; why?
Mathematically the function is 'f(x,y,T)=(x - T)^2 + (y - T)^2.
Am I right?

RE: How to use scipy.optimization.brute for multivariable function - scidam - Oct-20-2020

You are right. (1, ) is a value assigned to T (parameter). Note, that brute expects that the first argument of a function responsive for optimization domain. However, this first argument can be a vector. So, if you want to optimize your function in 2D space, you need to pass 2D vector as a first argument. Other arguments are function's parameters.

You unpack x and y, as follows:

def f(X, T):
    x = X[0]
    y = X[1]
    return (x-T)**2 + (y-T)**2

RE: How to use scipy.optimization.brute for multivariable function - Shiladitya - Oct-21-2020

(Oct-20-2020, 10:27 AM)scidam Wrote: You are right. (1, ) is a value assigned to T (parameter). Note, that brute expects that the first argument of a function responsive for optimization domain. However, this first argument can be a vector. So, if you want to optimize your function in 2D space, you need to pass 2D vector as a first argument. Other arguments are function's parameters.

You unpack x and y, as follows:
def f(X, T):
    x = X[0]
    y = X[1]
    return (x-T)**2 + (y-T)**2

Thanks.

I am getting an error for my function f(x,y,T).
At first, I defined the function f(x,y,T). Then I typed

def fT(X,T):
  x=X[0]
  y=X[1]
  return f(x,y,T)


grid = ((1, 400, 10), (1, 400, 10))
xmin_globalT =optimize.brute(fT, grid, (132,) )
print(xmin_globalT)

I am getting an error message in output

[ 9.89129495e+18 -1.31404261e+18]
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:249: RuntimeWarning: overflow encountered in double_scalars
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:247: RuntimeWarning: overflow encountered in double_scalars

Could you kindly help me here, please?

RE: How to use scipy.optimization.brute for multivariable function - scidam - Oct-21-2020

It seems that f(x, y) returns too large values. Could you provide implementation of f(x, y) here? Actually, this is not an error, just a warning.

RE: How to use scipy.optimization.brute for multivariable function - Shiladitya - Oct-25-2020

(Oct-21-2020, 11:44 PM)scidam Wrote: It seems that f(x, y) returns too large values. Could you provide implementation of f(x, y) here? Actually, this is not an error, just a warning.

Sorry for the late reply. I got stuck with some other businesses.
Here is a simplified form of my function. If you will compile, you will see two errors.

import math
import numpy as np
from scipy import optimize

def fT(X,T):
  p=X[0]
  q=X[1]
  r=pow(p,2) + pow(p,4) 
  Vtest = (( -(r**4)*(math.log((r**2)/(T**2))) ))
  return Vtest


grid = ((10, 40, 1), (10, 40, 1))
xmin_globalT =optimize.brute(fT, grid, (567,) )
print(xmin_globalT)

The errors are

/usr/..:RuntimeWarning: overflow encountered in double_scalars
  if __name__ == '__main__':
/usr/../scipy/optimize/optimize.py:597: RuntimeWarning: invalid value encountered in subtract
  numpy.max(numpy.abs(fsim[0] - fsim[1:])) <= fatol):

Thanks in advance.

RE: How to use scipy.optimization.brute for multivariable function - scidam - Oct-25-2020

This is because brute use int32 dtype for X. Just append dots to grid definition, e.g. grid = ((10.0, 40.0, 1.0), (10.0, 40.0, 1.0)). However, there are still large values occur when evaluating fT. Also, q variable is not used at all.

RE: How to use scipy.optimization.brute for multivariable function - Shiladitya - Oct-27-2020

(Oct-25-2020, 11:12 PM)scidam Wrote: This is because brute use int32 dtype for X. Just append dots to grid definition, e.g. grid = ((10.0, 40.0, 1.0), (10.0, 40.0, 1.0)). However, there are still large values occur when evaluating fT. Also, q variable is not used at all.

I am a little bit confused now. Instead of executing 'brute', if I only try to find the value of 'fT', it is not blowing up.

(I changed the fT now)

import math
import numpy as np
from scipy import optimize
 
def fT(X,T):
  p=X[0]
  q=X[1]
  r=pow(p,2) + pow(q,4)
  Vtest = (( -(r**4)*(math.log((r**2)/(T**2))) ))
  return Vtest
 
 
grid = ((5.0, 10.0, 1.0), (5.0, 10.0, 1.0))
#xmin_globalT =optimize.brute(fT, grid, (3,) )
#print(xmin_globalT)


z1=pow(1.0,2) + pow(40.0,4) 
A1=(( -(z1**4)*(math.log((z1**2)/(270**2))) ))
print(A1)

z2=pow(40.0,2) + pow(40.0,4) 
A2=(( -(z2**4)*(math.log((z2**2)/(270**2))) ))
print(A2)

And I. am getting the result without any warning -

-7.865898072821287e+26
-7.886106650018259e+26

Whenever, I run the 'brute', even for a smaller range, I am getting that error.

RE: How to use scipy.optimization.brute for multivariable function - scidam - Oct-28-2020

Your function seems to be quite simple and you can probably try to find out extreme points by differentiating it.
If the function is monotonic within the area of interest, it reaches its extremal value on the edge of the area.
Try to find derivative of the function and equal it to zero. You can also simplify math.log(x ** 2) to 2*math.log(x) etc. In this case you will not need to compute powers (which can lead to very large values).