Packaging/Modules--Wheel--pip--setup.py--Freeze

[snippsat]

This is gone be about Packaging/Modules and sharing your made stuff.

The Dark Age and the Broken past
Python packaging was in messy state between around 2008 to 2013.
It was not clear which libraries or tools to use,which ones were worth investing in or learning.
There was setuptools/Distutils/easy_install/source disruption and pip starting coming into the picture to.
There were critical security flaws in various parts of this toolchain,project was unmaintained for a long time(like setuptools).

From The Ashes,the picture is a lot better now
Donald Stufft stepped in both on Pip and PyPI and improved the availability of the systems it depended upon,
as well as some pretty serious vulnerabilities in the tool itself.
Now there are more agreement and the toolchain pip -- Wheel -- setuptools and PyPI(Share Module/Package with the world).
Python Core started distributing the ensurepip module along with both Python 2.7.9--> and 3.3-->.
This mean that if you have download or have a never version of Python you have pip,
can do pip install module/package eg pip install requests.

---

Wheel ☸
This first part is gone be about making Python Wheels(not about sharing on PyPi yet).
Wheel allows for binary redistribution of libraries,
can give wheel(.whl) file to a friend or share it on a website like Gohlke(Windows Binaries for Python Extension Packages)

I'll be using Win-10 and Linux Mint 18.1,both with Python 3.6.1.
There are basic setup here Windows part-1, part-2 and Linux part-1
Testing that Python is pip is up to date.

Windows: i use cmder but the commands is just the same in cmd.

Output:
C:\
λ python -V
Python 3.6.1
C:\
λ pip -V
pip 9.0.1 from c:\python36\lib\site-packages (python 3.6)

Linux:

Output:
mint@mint ~ $ python -V
Python 3.6.1
mint@mint ~ $ pip -V
pip 9.0.1 from /home/mint/.pyenv/versions/3.6.1/lib/python3.6/site-packages (python 3.6)
mint@mint ~ $ 

Looking at how wheel file is setup.
[Image: GMMBNS.jpg]
Making a wheel
So gone making a simple wheel to see how this work.
Folder setup:

C:\power\
 |-- power.py
 |-- setup.py

# power.py
__author__ = 'snippsat'
__version__ = '0.1'

def power_of(arg):
   '''Calulate Powers of exponent(^) in python'''
   return arg ** arg

if __name__ == '__main__':
   # Test,when "import power" this code will not executed
   print(power_of(4)) #--> 256
   print(power_of(8)) #--> 16777216

# setup.py
from setuptools import setup

__author__ = 'snippsat'

setup(
   name="power",
   version='0.1',
   py_modules=['power'],
   description="A simple power of exponent",
   url='https://python-forum.io/index.php',
   author_email='[email protected]',
)

Now we have all the setup needed.
In setup.py has basic info about building process.
Making the wheel python setup.py bdist_wheel --universal

C:\power
λ python setup.py bdist_wheel --universal
..... installing .....

Now in dist folder is the wheel power-0.1-py2.py3-none-any.whl
Name and version self explanatory,using --universal(mean that file is tested to  work python 2 and 3).
any mean that this wheel file work on (Windows,Linux,OS X).
For Doc look at PyPA and Setuptools .

Testing and installing the wheel that i build on Windows(shall work on Window and Linux).

Output:
# Test Windows
# Install
C:\power\dist
λ pip install power-0.1-py2.py3-none-any.whl

# Use
C:\
λ ptpython (or just python for standard REPL) 
>>> from power import power_of
>>> power_of(20)
104857600000000000000000000

Output:
# Test Linux
# install 
mint@mint ~/Downloads $ sudo pip install power-0.1-py2.py3-none-any.whl

# Use
mint@mint ~/Downloads $ ptpython (or just python for standard REPL) 
>>> from power import power_of
>>> power_of(12)
8916100448256

Success the wheel file work on Windows and Linux.

---

Freeze ❄
The problem of sharing Python made stuff with people that do not have Python(like most Windows user).
Tool i mention here work for newer Python version pyinstaller -- cx_Freeze -- pynsist.

A demo with power.py with Pyinstaller.
Have to rewrite it a little or it's just get executed.

# power.py
# Rewritten to user input for use on "power.exe" run
__author__ = 'snippsat'
__version__ = '0.1'

def power_of(arg):
   '''Calulate Powers of exponent(^) in python'''
   return arg ** arg

if __name__ == '__main__':
   user_input = int(input('Enter number to perform power of exponent: '))
   print(power_of(user_input))
   input('Press <Enter> to exit')

Pyinstaller work for Python 3.6 if use development version.

Output:
# Install
(power_inst) C:\power\power_inst
λ pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip
......
Successfully installed PyInstaller-3.3.dev0+abfc80661 future-0.16.0 pypiwin32-220

Output:
# Use
(power_inst) C:\power\power_inst
λ pyinstaller power.py

In dist folder there is power.exe,now is not Python needed to run power.exe.

For Pyinstaller on Linux is the same way,
has to build power.py on Linux(the Windows build do not work on Linux).
In dist folder there will be power file that is executable on Linux.

---

More Wheel ☸
This one show how to add dependencies in setup.py.
Folder setup:

C:\web
 |-- find_title.py
 |-- setup.py

# find_title.py
import requests
from bs4 import BeautifulSoup

def web_title(url):
   '''find web site title'''
   url_get = requests.get(url)
   soup = BeautifulSoup(url_get.content, 'lxml')
   print(soup.select('head > title')[0].text)

if __name__ == '__main__':
   #url = 'http://CNN.com'
   url = 'https://www.python.org/'
   web_title(url)

# setup.py
from setuptools import setup

__author__ = 'snippsat'

setup(
   name="web_title",
   version='0.1',
   py_modules=['find_title'],
   description="Find web site title",
   url='https://python-forum.io/index.php',
   author_email='[email protected]',
   python_requires='>=3.4',
   classifiers=[
       'Programming Language :: Python :: 3.4',
       'Programming Language :: Python :: 3.5',
       'Programming Language :: Python :: 3.6',
    ],
   # These will be installed when pip wheel run
   install_requires=[
       'beautifulsoup4',
       'requests',
       'lxml',
   ],
)

The important stuff here is install_requires,
these will be installed when run wheel file.
python_requires has set that Python 3.4 or newer is needed.
classifiers meta data info about project(not needed).

Make wheel python setup.py bdist_wheel and use wheel web_title-0.1-py3-none-any.whl:

# Make wheel
C:\web
λ python setup.py bdist_wheel

# Use wheel
C:\web\dist
λ pip install web_title-0.1-py3-none-any.whl
Requirement already satisfied: web-title==0.1 
Requirement already satisfied: beautifulsoup4 in c:\python36\lib\site-packages (from web-title==0.1)
Requirement already satisfied: lxml in c:\python36\lib\site-packages (from web-title==0.1)
Requirement already satisfied: requests in c:\python36\lib\site-packages (from web-title==0.1)

Test:

Output:
C:\
λ ptpython
>>> from find_title import web_title

>>> web_title('http://python-forum.io')
Python Forum

---

Gone continue here with part-2,
with making a Package and upload to PyPi(also demo upload with wheel that has been made here).

---

---

---

Now gone continue with using PyPi ans the new site Warhouse (Will be the new PyPi site).
All of this can be confusing for most.
Here a short description of some word used in topic.

• Module: A single Python file(foo.py),when used as a module import foo it's filename without the .py.
  
• Package: A directory of Python modules/folders containing an additional __init__.py.
  
• Wheel: A binary redistribution of Modules/package pip install some_wheel.whl.
  
• PyPi/Warehouse share Modules/Package with all that have Python an pip.
  
• Freeze: A executable file to distribute to end-users,that contains all of your application code as well as the Python interpreter.
  

---

Setting up for upload to to PyPi/Warehouse
There where some changes in.pypirc since last i used it.
This is tested and work.
For Windows C:\Users\<username>\.pypirc Linux /home/<username>.pypirc
Register at PyPi,email with username and password get back fill in.

[distutils]
index-servers =
 pypi
 pypitest   
 
[pypi]
repository=https://upload.pypi.org/legacy/
username:username
password:password

[pypitest]
repository=https://test.pypi.org/legacy/
username:username
password:password

[server-login]
username:username
password:password

Now pip install twine.
Can now test with web_title that i made before in post.
In C:\web folder twine upload dist/* will find wheel is dist folder and upload it.
So now is web-title(PyPi) and web-title(Warehouse) available for all.
pip install web-title
Usage:

C:\
λ ptpython
>>> from find_title import web_title

>>> web_title('http://python-forum.io')
Python Forum
>>> web_title('https://pypi.org/')
PyPI - the Python Package Index · Warehouse (PyPI)
>>> web_title('http://cnn.com')
CNN - Breaking News, U.S., World, Weather, Entertainment & Video News

---

Making a Package
So now will bring all made before + a Class into one package.
Here is the Packaging_Tutorial
To download:
git clone url_of_repo

So now i can use all from div_package
[Image: fiu4Vn.jpg]
Now see that both find_title and power is available.
Usage:

C:\
λ ptpython
>>> import div_pack
>>> div_pack.find_title.web_title('https://python-forum.io')
Python Forum

>>> div_pack.power.power_of(22)
341427877364219557396646723584

>>> # And my new Class
>>> company = div_pack.CompanyCheck('Volta', 488)
>>> company.name
'Volta'
>>> company._id
488

>>> company.check_name
Company name: <Volta> is correct
>>> company.check_id
Id <488> is correct

It can be smart to change __init__.py in main folder to bring all together.
This can also avoid long imports from main.sub_1.sub_2.an_other_folder import what_i_need

from .company import CompanyCheck
from .power import power
from .web import find_title

[snippsat]

Conclusion and thought.
The tool has gotten a lot better,but software packages and dependencies(Dependency hell) is hard in any language.
Python is used in broad area of fields,this also mean that dependencies 3-party package usage is heavy.

The positive part is that all build tool are now a part of newer version of Python,
pip(wheel,setuptools),virtualenv the only external tool i used for wheel,packing,
was twine(Recommended) for uploading distributions to PyPI(Warehouse).
There are work been done to make tools better Pipfile and Kenneth Reitz try to marriage  together with Pipenv.

Anaconda with 720 packages pre-installed is a interesting choice for more than data science people.
The package list has use case for a wide area.
They have there own pip which is conda.
CONDA-FORGE is really interesting,
they do cross-platform testing(Windows,Linux and OSX) before they distribute the package.