python subdirectories

[Skaperen]

my laptop (ubuntu 16.04.5 upgraded to be xubuntu) has a bunch of subdirectories with names like "dist-package" and "site-package" which have parents with names like "python" and "python-{version}" for various subversions of 2 and 3. does anyone know what these are, what they are for, and what they are supposed to have?

[snippsat]

dist-packages is a Debian-specific convention that is also present in its derivatives, like Ubuntu.
Debian Python Wiki:

dist-packages instead of site-packages.
Third party Python software installed from Debian packages goes into dist-packages, not site-packages.
This is to reduce conflict between the system Python, and any from-source Python build you might install manually.

Means that if you manually install Python from source,it uses the site-packages directory.
site-packages or dist-packages is the place Python install 3-party modules/package.

Testing on Mint 19.
I use pyenv it will use site-packages folder.

pip -V
pip 19.0.3 from /home/tom/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip (python 3.7)

# Going back to original Python version(3.6.5) on Mint 19
$ pyenv local system
$ pip -V
pip 18.0 from /home/tom/.local/lib/python3.6/site-packages/pip (python 3.6)

So Mint 19 use site-packages.

Test Ubuntu 14 i run on Cloud9,here use dist-packages.

snippsat:~/workspace/bs4_test $ pip -V
pip 9.0.1 from /usr/local/lib/python3.5/dist-packages (python 3.5)

---

As note so has Windows always used site-packages,as folder for 3-party installed modules/package.

[Skaperen]

where to put locally written code so that it does not get mixed up with anything being installed? is there a way to have Python make use of yet-another (3rd) such subdirectory name (for example "host-packages")? environment variable PYTHONPATH?

[snippsat]

where to put locally written code so that it does not get mixed up with anything being installed?

Virtual environment,now also build into Python trough venv.

The main purpose of Python virtual environments is to create an isolated environment for Python projects.
This means that each project can have its own dependencies, regardless of what dependencies every other project has.
These dependencies can be written to a requirements.txt which we look at later.

Is there a way to have Python make use of yet-another (3rd) such subdirectory name (for example "host-packages")? environment variable PYTHONPATH?

That would be confusing,you should never ask user to alter there PYTHONPATH to make your library work,
it can be done automatically in setup.py(when make pip install my_package),so user don't know it,but should be avoided.

For Virtual environment there is pip freeze > requirements.txt,
and restore pip install -r requirements.txt.

For making code available on PyPI or share a wheel pip install my.whl.
There are more stuff to look into,i have a long tutorial about here.
Packaging/Modules--Wheel--pip--setup.py--Freeze.

[Gribouillis]

is there a way to have Python make use of yet-another (3rd) such subdirectory name

Yes there are ways. Look for a file named /usr/lib/python3.5/sitecustomize.py or perhaps /etc/python3.5/sitecustomize.py

In this file add this

import site
site.addsitedir('/path/to/your/favorite/directory')

Then you can use /path/to/your/favorite/directory to add your own modules.

It can also be done on a per-user basis. For this create a file $HOME/.local/lib/python3.5/site-packages/usercustomize.py

In this file, write the same thing as above. You can install there scripts for your user only.

[Skaperen]

i've got 3.6.8, now, so i changed the 3.5 to 3.6. now i have:

/etc/python3.6/sitecustomize.py:

# install the apport exception handler if available
try:
    import apport_python_hook
except ImportError:
    pass
else:
    apport_python_hook.install()
import site
for a in ('/usr/local','/usr/host'):
    for b in ('dist-packages','host-packages','site-packages'):
        site.addsitedir(f'{a}/lib/python3.6/{b}')

python3 still works. at least i didn't break it, yet.

[Gribouillis]

Why are you doing this? You don't need dozens of site-packages directories. Also you may have updated the wrong sitecustomize.py. On my computer I have

Output:
λ python3 -c "import sitecustomize; print(sitecustomize)"
<module 'sitecustomize' from '/usr/lib/python3.5/sitecustomize.py'>

[Skaperen]

you suggested 2 files to look at. i updated them for 3.6. only 1 of them existed so i modified it instead of adding a new one.

---

the reason is because i wanted a directory added that was different than the common ones. i chose to name it "host" because the name is the same length as the other two, and it fits in with other customizing i am doing in /usr/host (same subdirectories as /usr/local has).

[Gribouillis]

I see. I hope it works now.