Installing Python and pointing it to required libraries

[hooiberg]

Hello, more experienced people.

I am responsible for installing software on our CentOS cluster. Several users have requested a new Python version be installed, with extra packages. Sadly, I do not have access to the package management system, so I have to do everything by hand. (This does not mean I should not be able to install PIP and use it... I hope)

However, the first big problem is compiling Python itself. I have:

1) Downloaded Python 3.7.2 from https://www.python.org/downloads/source/

2) To get PIP to work I need _ctypes. To get Python to compile with _ctypes I need libffi, which I got from https://sourceware.org/libffi/ which configures and makes smoothly, and generates header file ffi.h, ffi_common.h and library libffi.so. So far so good.

3) I tried to configure Python using:

./configure --with-valgrind --with-libs='-L/trinity/opt/apps/software/libffi/libffi-3.2.1/x86_64-unknown-linux-gnu/.libs -lffi'

as the with-libs argument comes back into the configure log in compile commands such as [snippet from config.log]:

configure:10917: gcc -pthread -o conftest conftest.c
-L/trinity/opt/apps/software/libffi/libffi-3.2.1/x86_64-unknown-linux-gnu/.libs -lffi -lpthread
-ldl >&5

4) Also I hardcoded the path to the header file in the configure script, which I know is ugly, but I saw no way to provide it as an argument.

LIBFFI_INCLUDEDIR="/trinity/opt/apps/software/libffi/libffi-3.2.1/x86_64-unknown-linux-gnu/include"

5) Now I can make, and I am still stuck with the issue that libffi cannot be found, so my efforts to point Python to its location were wrong... (Also a bunch of other misery, but that is another day's worries) :

Error:
running build
running build_ext
INFO: Can't locate Tcl/Tk libs and/or headers
INFO: Could not locate ffi libs and/or headers

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_bz2                  _curses               _curses_panel      
_hashlib              _lzma                 _sqlite3           
_ssl                  _tkinter              _uuid              
readline                                                       
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd                
time                                                           


Failed to build these modules:
_ctypes                                                        


Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

What is the best way to indicate the locations of required libraries and header files?
What probably obvious mistake did I make?

Thank you for your time and expertise!