How to avoid exec(), globals(), locals(), eval()

[paul18fr]

This is the first post (of series?) to find a good way (or practice) to produce a dynamical link code without using exex() and so on.

How can I produce the same results using a list of dictionnary names and not the list of dictionnaries in order to get the same results (without using exec())? (i'm turning around)

Thanks

from itertools import chain

spam = {'foo':1, 'bar':2}
eggs = {'x':0, 'y':3}
 
Dicts = [spam, eggs]                    # list of dictionnaries 
DictsWithNamesOnly = ['spam', 'eggs']   # list of dictionnary names = list of strings
 
#########
def get_keys(*args):
    for some_dict in args:
        print(list(some_dict.keys()))
        print("Dict = {}".format(some_dict))
        print("Type = {}".format(type(some_dict)))
        
get_keys(*Dicts)

[ndc85430]

You don't say why you want to do this in the first place. If you give details about the higher level problem you're actually trying to solve, there might be a more appropriate solution. In any case, as presented, can't you just use a dictionary to map the names to the dictionaries?

[paul18fr]

You don't say why you want to do this in the first place.

I guess you're right; in the case I'm currently working on, I'm facing to different difficulties:

• for a function (i.e. as argument), I need to provide a list of dictionnaries
  
• Dictionnaries names have been previously (dynamically) reccorded in a list, and I cannot use explicit names and numbers accordingly (but use a variable instead)
  
• in the main code, I'm using exec(s) to replace the name by its variable; if a want to rewrite it in a function, well "goto" item 1
  

can't you just use a dictionary to map the names to the dictionaries?

Well it seems I do not know how to proceed

Thanks for your interest

[bowlofred]

Dictionnaries names have been previously (dynamically) reccorded in a list,

If they're in a list, then you should access them by that list, not by an exec. Don't reassign them to independent variables, assign them to elements of a list or of another dict.

# dicts in a list
# dicts in a list
my_listed_dicts = [
    {"foo": 1, "bar": 2},
    {"x": 0, "y": 3},
]

# nested dicts
my_nested_dicts = {
    "spam": {"foo": 1, "bar": 2},
    "eggs": {"x": 0, "y": 3},
}

# access them without exec
print(f"The first dict in the list is {my_listed_dicts[0]}")
print(f"The nested dict stored as 'eggs' is {my_nested_dicts['eggs']}")

Output:
The first dict in the list is {'foo': 1, 'bar': 2}
The nested dict stored as 'eggs' is {'x': 0, 'y': 3}

[paul18fr]

Dictionnary names (even their numbers), as well as for the keys, everything are variables; The only way I've found is to use exec() so far, and I want to proceed correctly

In the code I gave you you can add the following lines to see what I mean

for i in range(len(DictsWithNamesOnly)):
    exec('print("%s dictionary = {}".format(%s))' %(DictsWithNamesOnly[i], DictsWithNamesOnly[i]))

[bowlofred]

You said the dictionaries were in a list. That's not correct. You just have the local variable names in a list.

In your original code you have this:

spam = {'foo':1, 'bar':2}
eggs = {'x':0, 'y':3}

Don't do that. You've assigned them to two different top-level names. Instead, assign them to a single dict and give them keys to reference them.

my_dicts = {
    "spam": {'foo':1, 'bar':2},
    "eggs": {'x':0, 'y':3},
}

for d in my_dicts:
    print(f"{d} dictionary = {my_dicts[d]}")

[paul18fr]

@bowlofred
ok my mistake to not being precise enough; the current post is in the continuity of this one (I thought it was a better idea to open a new one to speak about the use of exec(), locals(), globals() and eval()).
(print is just a way highlight my need)

Currently:

• I'm dealing with a lot of data that are recorded in a first time in dictionaries
  
• the name of a dictionary depends on what I'm getting, so it's implicitly a variable
  
• the number of dictionaries varies
  
• the keys name is also a variable
  
• values are (numpy) arrays (containing millions of rows and up to dozens of columns - in other word the size of the array can be huge)
  
• in parallel to dictionnaries creation, a dictionnary names list is created and updated
  
• because of all these variables, I'm using a mix of exec(), locals(), gobals(), eval() to dynamically work with names (for dictionaries, keys, and so on) and I know it's not a correct way to do under Python (for all reasons ever evocked)
  

In a first stage, and based on the list of dictionary names, I do not know how to pass to a function a variable number of dictionaries except using exec() and so on.

Hope it is clearer now
(one issue after another one )

[deanhystad]

Are you passing a list of dictionary names because you think this has less overhead than passing a list of dictionaries? That is incorrect. A python dictionary variable is going to be 4 or 8 bytes. A python string variable is going to be 4 or 8 bytes. A list of dictionary names will be the same size as a list of dictionaries.

There are several posts in this forum asking how you can programmatically create new variable names. The poster assumes that since the program manipulates multiple variables that each of the variables must have a name. They are nearly always incorrect. Variable names are mnemonics for writing and reading code. Python does not care what your variables are named. "spam", "eggs", dictlist[0] and dictionaries["eggs"] can all reference a dictionary.

If you know there will only be a "spam" dictionary then it is good to use a variable named "spam". If you have two dictionaries it may make sense to think of using a collection. If you have many dictionaries and the number is variable, a collection is definitely the way to go.

[paul18fr]

well I forgot some basics and I'm trying to figure out all you've said (spent last hours to look for informations); I'm doing basic test to understand.

I'm facing to difficulties:

• how to get Mat2 values in MyDict?
  
• as we can see in the picture, using update doe not insert BasicDict: what's the correct method (not found in internet)
  

I'm doing my best to have a good practice

import numpy as np

DictVar = 'NewVar'
Mat = np.random.random((10,1))
BasicDict = {'Mat': Mat}

# New arrays are created
Mat2 = np.random.random((10,1))
Mat3 = np.random.random((10,1))

## A "dictionary of dictionaries" is created (or left empty)
# MyDict = {}
MyDict = { "NewDict": {'Mat1': Mat, 'Mat2': Mat2}, }


if not DictVar in MyDict:
    print("{} is not in MyDictList -> added".format(DictVar))
    MyDict.update({ DictVar: {}, })
    MyDict.update(BasicDict) ## only Mat is inserted!!!
else:
    print("{} is in MyDict".format(DictVar))

del Mat, Mat2, Mat3

# first step=extraction of Mat2 dictionary
# tempo = MyDict(BasicDict[0])
GetMat2 = MyDict(BasicDict["Mat2"])
# second step=add of BasicDict dictionary into DictVar

Error:
KeyError: 'Mat2'

[Yoriz]

Mat2 is inside NewDict which is inside MyDict

get_mat2 = MyDict['NewDict']['Mat2']