importing a config file prefixed with a dot

[Skaperen]

i want to establish a consistent scheme for config files for python program.  for programs made for unix and unix-like platforms i want to use config file names that begin with a dot character so they are hidden like other config files.  many programs have full or nearly-full programming power in the config fill and i want to have the same in the config files for programs in python.   many script written various shell languages use the source command to run the config file which give it full programming power in the shell language.  to do the same for python, the only way i see to do it is importing the config file as a module.  the first problem is that the import command does not like names that begin with a dot.  even the builtin.__import__() call fails at this.  any next ideas to try?


file .tryconfig.py:

# set some variables for a test config
foo = 36
bar = 42

file tryprogram.py:

__import__('.tryconfig',globals())
print(foo,bar)

so i try it:

Output:
lt1/forums /home/forums 4> ls -l .tryconfig.py;cat .tryconfig.py
-rw-r--r-- 1 forums forums 57 Mar 24 23:11 .tryconfig.py
# set some variables for a test config
foo = 36
bar = 42
lt1/forums /home/forums 5> ls -l tryprogram.py;cat tryprogram.py
-rw-r--r-- 1 forums forums 50 Mar 24 23:11 tryprogram.py
__import__('.tryconfig',globals())
print(foo,bar)
lt1/forums /home/forums 6> py3 tryprogram.py
Traceback (most recent call last):
  File "tryprogram.py", line 1, in <module>
    __import__('.tryconfig',globals())
ImportError: No module named '.tryconfig'
lt1/forums /home/forums 7> py2 tryprogram.py
Traceback (most recent call last):
  File "tryprogram.py", line 1, in <module>
    __import__('.tryconfig',globals())
ValueError: Empty module name
lt1/forums /home/forums 8> 

[wavic]

Hm! Why not just load a json file. It's suitable file format for such purposes.

with open('.app_settings', 'r') as conf:
    settings = json.load(conf)

[Skaperen]

that (json) is what i do now.  i want to have a nearly full python coding capability in the config file allow the user that can code in python to use more advanced code to set data values for the config.

here is a silly example:

a = 'f' + 'o'*
b = 'bar'
ab = a + b
time_period_in_seconds = 5 * 60 * 60 # 5 hours

that last line might be a more realistic use case example.  i want to also allow comprehension, loops, i/o, etc.

[wavic]

Then you need some serializing method. Like pickle or dill. Dumping a class object with all its data and methods is perhaps what you need.

I've seen that you may not want to install additional modules. And I found this.

[Skaperen]

serialization would not help as that would compromise the format of the file.  it needs to be directly editable with an editor like emacs or vi.  that basically means the file will generally have python as its content.

[zivoni]

Beside dot meaning relative import, import something creates variable something and variable names cant start with a dot.

You can use/abuse importlib library to load module from a file and execute it, make it importable and import it (for python 3.?+, older ones use imp?).

Example (with big help of importlib docs)

Output:
jacq /tmp> cat .boo.py 
print('boo')

def boo():
    print('boo boo')

>>> import sys
>>> import importlib.util as imput
>>> spec = imput.spec_from_file_location('boo', '.boo.py')
>>> boo_module = imput.module_from_spec(spec)
>>> spec.loader.exec_module(boo_module)
boo
>>> sys.modules['boo'] = boo_module
>>> import boo
>>> boo.boo()
boo boo

But doing such thing just to have hidden config file seems be a little exaggerated. Is not better to put your config file to some hidden directory (like ~/.config/skapeutil), modify sys.path and import it in a standard way?

[ichabod801]

You could use exec:

exec(open('.config').read())

There's more on this at Stack Overflow: http://stackoverflow.com/questions/43619...n-3#437857

[wavic]

I came across this. But it uses some tricky methods like json.dump(list(bytecode)).

See jsonpickle or at least how is done. Just an example:

In [1]: import jsonpickle

In [2]: class Thing(object):
   ...:     def __init__(self, name):
   ...:         self.name = name
   ...: 
   ...: obj = Thing('Awesome')
   ...: 

In [3]: frozen = jsonpickle.encode(obj)

In [4]: frozen
Out[4]: '{"py/object": "__main__.Thing", "name": "Awesome"}'

This is from the webside

[nilamo]

Instead of importing into the global namespace, why not just treat it like a module?

import some_random_config as conf

print(conf.foo)

That way, if someone else is looking at your script, they won't be totally lost about where you're defining variables, and if they make a mistake with naming, they'll get a meaningful error message (conf.foo is not defined).

[Skaperen]

i can get things to work by using conf=__import__(filename) as long as there is no dot in the file name  ('.' not in filename).  avoiding config data going into globals() or locals() is a good idea for run time safety in case the user accidentally makes the config file world writable.  that exec() way was something i was thinking about.  it was what made me think i needed to do the config in a private variable space.  that would let me .pop all the variables i know about and complain about unknown variables that remain.  time to try a module/function to do it.

---

there is configparser.  but it assumes the windows ini format.  i'm not going to use that format.