find hidden python files

[bobsmith76]

I have a built-in operation which is eating up 74% of the run time in my code and I'm having a very difficult time tracking down its location. If look at the pictures you'll see that there is an operation which is simply called __call__ and its located at line 394 of the reference.py file. Then you'll see the number 20 circled, that means 20 seconds and the program takes 33 seconds so that's how much time this operation is taking up. Then below that you'll see the four lines that use this feature and one of them is in _xlmac.py file. I cannot find either the _xlmac or the reference.py file. I think because they are hidden. If you put in those names into the mac spotlight those files do not come up.

[nilamo]

__call__ is a function call. Probably of a class, because a function would have it's name there instead. ie:

class Spam:
    def __call__(self, arg):
        print(f"Spam's instance was called: {arg}")

foo = Spam()
foo("__call__ is called here")

A quick google search shows that xlwings has a _xlmac.py file, but that doesn't mean that's the one you're seeing: https://github.com/ZoomerAnalytics/xlwings

How are you getting these numbers? Are you using an ide? The ide itself, or however your code is being analyzed might be contributing to the numbers.

[bobsmith76]

Yea, I'm using xlwings but that does not solve the larger problem of how to track down hidden python files. I've run into this problem before and need to add this skill to my repertoire. I've solved the problem. openpyxl opens my workbook and processes the data in 45 seconds. That's too slow. So I've moved over to the xlrd module. the xlrd module changes integers to floats so I had to add in the line:

float_x_int = lambda x: int(x) if type(x) in [float, int] and int(x) == x else x

After I put in that line the execution time went from 10 to 27 seconds. I had trouble believing that the above line would cause that but it turns out that it was an xlwings function I was using. I thought I was using the xlwings function before but apparently it wasn't.


Still, in the future I might need to have to locate a hidden python file and place a break point in them and then look in the stack to find out where the code is accessing that python module. So even though I've got the executing time down to a reasonable level, I still need to solve this larger problem.

[nilamo]

I actually just recently started looking at python debuggers, and Thonny might be a good place to start. https://thonny.org/
By stepping into the code, you can see exactly what lines are running, and when, so you can track down what's happening.

[wavic]

Operating system?