function NOT imported from a module

function NOT imported from a module - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: function NOT imported from a module (/thread-12372.html)

Pages: 1 2

function NOT imported from a module - Skaperen - Aug-21-2018

i have a little test module named foo (file foo.py) with 2 functions, aaa() and bbb(). function aaa() calls bbb(). function bbb() prints "woot". in a test script in file bar.py it does from foo import aaa then calls aaa(). when the test script is run the output is "woot". this is fine. but i do have the question, how is it that aaa() can see bbb() to be able to call it?

foo.py:

def aaa():
    bbb()
    return

def bbb():
    print('woot')
    return

bar.py:

from foo import aaa
aaa()

Output:lt1/forums /home/forums 3> cat -n foo.py
     1	def aaa():
     2	    bbb()
     3	    return
     4	 
     5	def bbb():
     6	    print('woot')
     7	    return
lt1/forums /home/forums 4> cat -n bar.py
     1	from foo import aaa
     2	aaa()
lt1/forums /home/forums 5> python3 bar.py
woot
lt1/forums /home/forums 6>

RE: function NOT imported from a module - snippsat - Aug-22-2018

Importing a module,the module is always fully imported (into the sys.modules mapping).
So the function definition aaa has full access to foo.py and call bbb as it would running the file normal.

C:\1_py\mod
λ ptpython
>>> dir()
['__builtins__', '__doc__', '__name__', '__package__']

>>> from foo import aaa

>>> dir()
['_', '_1', '_3', '_5', '_6', '__builtins__', '__doc__', '__name__', '__package__', 'aaa']
>>> aaa
<function aaa at 0x05E6B1E0>

>>> import sys

>>> sys.modules['foo']
<module 'foo' from 'C:\\1_py\\mod\\foo.py'>

>>> aaa()
woot

So doing import foo or from foo import aaa make no difference regarding import as whole foo.py always get imported.
from foo import aaa binds a different name aaa pointing straight at the attribute contained inside of the module.

RE: function NOT imported from a module - Skaperen - Aug-22-2018

what i really want to know is when aaa() calls bbb() does it see it in locals or in globals or in some special namespace view for everything in foo.py? does it copy all these items from the loading of foo.py or does it just make them virtually appear? i know about the local then global search order but i want to know how this fits into that.

the reason this is such a curiosity to me is that from some testing i did, i know that it can see data variables, too. i don't know if it can modify them by name that way (like globals cannot be modified by name unless specified). but referenced data objects can be acted on in a way that changes the object (e.g. appending to a list). a did a test where a function named store(), and another function named fetch(), accessed a dictionary in their module and stored into or fetched from that dictionary by key. it works just fine. so it makes me wonder why we need classes as a different kind of object.

functions in modules can easily keep data between calls and i don't need to code a class to do that.

RE: function NOT imported from a module - Gribouillis - Aug-23-2018

If you disassemble the code of aaa you get

>>> dis.dis(aaa)                                                                            
  2           0 LOAD_GLOBAL              0 (bbb)
              3 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
              6 POP_TOP

  3           7 LOAD_CONST               0 (None)
             10 RETURN_VALUE

Now the LOAD_GLOBAL opcode 'Loads the global named co_names[namei] onto the stack.' You can see that

>>> aaa.__code__.co_names
('bbb',)

and note thas aaa stores a reference to its globals

>>> aaa.__globals__

I think you have most of the mechanism here. The next step is to go see the details of bytecode execution of LOAD_GLOBAL in the C code but it probably won't tell you more.

RE: function NOT imported from a module - Skaperen - Aug-24-2018

if bbb is in globals, would that mean the code that did the import and called aaa() could call bbb() instesd?

what i see is the ability to save state as a single instance. if i need multiple instances then that is when i really use a class.

RE: function NOT imported from a module - Gribouillis - Aug-24-2018

(Aug-24-2018, 03:53 AM)Skaperen Wrote: would that mean the code that did the import and called aaa() could call bbb() instesd?

Here is what you can do with module inspect to explore external variables used in function aaa

>>> from foo import aaa
>>> import inspect
>>> cv = inspect.getclosurevars(aaa)
>>> cv
ClosureVars(nonlocals={}, globals={'bbb': <function bbb at 0x7f2a0b6cb400>}, builtins={}, unbound=set())
>>> for dic in (cv.nonlocals, cv.globals, cv.builtins):
...     if 'bbb' in dic:
...         dic['bbb']()
...         break
... else:
...     if 'bbb' in cv.unbound:
...         print('bbb referenced in aaa but cannot be resolved')
... 
woot

Notice that the cv.globals here is not the same as aaa.__globals__: it contains only the symbols referenced in function aaa. You can check this by adding a variable spam = 'spam' in foo.py. This variable won't appear in cv.globals.

RE: function NOT imported from a module - Skaperen - Aug-24-2018

what mechanism is taking place that defines what symbols go where in this? i ask because i am seeing that non-function symbol names that were not expressed in the import statement are not visible to the caller (which imported the function it calls), but are visible to that function.

1. is it a different global space?

2. is it a mechanism that makes two (or more) namespaces appear as one (like an overlay filesystem does).

3. is python putting the symbols in when the function is being called and taken backout when it returns?

in the unnamed language i was designing, that design does a similar thing, but achieves it by an overlay where all code always sees everything that is where it comes by a means of an overlay with locals on top. if the code is referenced via an instance (a class object), then a 4th layer for the instance is also there (allowing objects to keep data local, or save it with the object instance, or share it among all instances of the same class (more layers here for inherited instances), or globally (the caller can get to here).

i'm wanting to understand how Python does it, in terms of what the language is supposed to give us, as opposed to any artifacts of some implementation. i am planning to make a little storage module that can do this for caller functions. i have dabbled in this and it seems to work. more complex cases like separate imports have yet to be tested. but i hate determining this by test code because it will include the artifacts of the CPython implementation i have.

what is this nonlocals={} ? yet another space? where do the real locals fit in?

RE: function NOT imported from a module - Gribouillis - Aug-25-2018

(Aug-24-2018, 11:51 PM)Skaperen Wrote: what is this nonlocals={} ?

Here is an example with non local external variables.

>>> def spam():
...     eggs = 'eggs'
...     def cheese():
...         print(eggs)
...     return cheese
... 
>>> f = spam()
>>> import inspect
>>> inspect.getclosurevars(f)
ClosureVars(nonlocals={'eggs': 'eggs'}, globals={}, builtins={'print': <built-in function print>}, unbound=set())

RE: function NOT imported from a module - Skaperen - Aug-26-2018

what if i has this (untested) code:

d = {}
o = []

from numbers import Number

def put(v,*a):
    if a:
        for k in a:
            d[k]=v
    else:
        o[:]=[v]
    return v

def get(*a):
    return tuple(d[k] for k in a) if a else o[0]

def clear():
    d.clear()
    o.clear()
    return None

def pop(*a):
    return {k:d.pop(k) for k in d.keys()} if a else o.pop(0)

def append(v,*a):
    if a:
        for k in a:
            if isinstance(d[k],list):
                d[k].append(v)
    else:
        o.append(v)
    return v

def prepend(v,*a):
    if a:
        for k in a:
            if isinstance(d[k],list):
                d[k][0:0]=[v]
    else:
        o[0:0]=[v]
    return v

def update(a,b):
    if isinstance(a,dict):
        d.update(a)
    if isinstance(b,(list,tuple)):
        o[:]=b
    else:
        o[:]=[b]
    return (a,b)

def add(v,*a):
    if a:
        for k in a:
            if k in d:
                if isinstance(d[k],Number):
                    d[k]+=v
            else:
                d[k]=v
    else:
        if o:
            if isinstance(o[0],Number):
                o[0]+=v
        else:
            o[:]=v
    return v

def sub(v,*a):
    return add(-v,*a):

def inc(*a):
    return add(1,*a)

def dec(*a):
    return sub(1,*a)

def zero(*a):
    if a:
        for k in a:
            if isinstance(d[k],Number):
                d[k]=0
    else:
        if o and isinstance(o[0],Number):
            o[0]=0
    return v

def dump():
    return (d,o[0])

def load(a,b):
    clear()
    return update(a,b)

def items():
    return d.items()

def keys():
    return d.keys()

def values():
    return d.values()

def reference():
    return d,o

can some other function (that is not a method of a class) import this and use it to store stuff that will be there the next time it is called? this is a first-off. there are no-doubt some big oopses in it. but hopefully you get the idea.

RE: function NOT imported from a module - Skaperen - Aug-31-2018

i am currently writing a function to output numbers. the function is given one number directly or a list/tuple of them. the way the numbers need to be output is with a space between each and never wrapped from the end of the line to the next one. they must always stay whole on the screen. i envision 2 ways to do this, both of which require some state be retained between calls. method 1 is to simply output a space and the number with no newline unless this number would wrap, in which case reset the column number to this number's length and output a newline and the number. the state between calls is how many characters have been printed (the space counts) which is checked every time against the known or assumed display width. method 2 involves buffering the line in which case the buffer is the state saved between calls, only one instance is needed when there is only one display to print to. so i really don't need a class; i can simply do this as a function.