Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Embedding, windows, and virtual environments
#1
Hi,

I've been writing a C++ app which embeds the python interpreter (tested on 3.6 or 3.7).
I am writing it both for linux and windows.

I would like it to use with virtual environments and have it behave in exactly the same way as python.exe would do in such cases.

In linux everything works out of the box and there is nothing I really need to do to support it. It just works.

In windows I can't get it to work other than manually adjusting the sys.path.
The code is at the bottom of the message.

Virtual environments created with

python -m venv abcd

After a lot of debugging I have found that the issue is sys.executable which is used in site.py to work out the location and settings of the virtual environment.

In linux this is by default the location of the python interpreter, something like /usr/bin/python3 or /tmp/abcd/bin/python3 (if the venv is activated).

In windows it is always the name of the executable.

This is because the 2 files Modules/getpath.c and PC/getpathp.c have completely different logic in
calculate_program_full_path vs get_program_full_path.

The former (linux) searches the ProgramName in the PATH while the second calls GetModuleFileNameW regardless.

This value ends up in Py_GetProgramFullPath() which is the same as sys.executable.
The only solution I see is to locate my app in the same folder where the embedded python.exe is. But this is really a messy and not scalable solution. Or to manually set sys.path, but I would have to replicate site.py or whatever other virtual environment technique is used.

I have a few questions

1) why this different behaviour?
2) how can I make virtual environments works out of the box in windows as well?
3) according to the doc (https://docs.python.org/3/c-api/init.htm...amFullPath) the liunux behaviour is correct and depends on the value set by Py_SetProgramName(), while in windows it ignores it
4) why the doc (https://docs.python.org/3/extending/embedding.html) says that the call to Py_SetProgramName() is recommended? What are the benefits? Actually I would recommend NOT to call it at all. If you set it to argv[0] as in the example, then the virtual environments do not work in linux either and it behaves the same as window.

As I said, if the call below is left out, in linux it just works.

Andrea

Example

#include <Python.h>

int
main(int argc, char *argv[])
{
wchar_t *program = Py_DecodeLocale(argv[0], NULL);
if (program == NULL)
{
fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
exit(1);
}

// try a few alternatives
// Py_SetProgramName(L"python"); /* optional but recommended */
// Py_SetProgramName(program); /* optional but recommended */
// do not call it at all

Py_Initialize();
PyRun_SimpleString(
"import sys\n"
"print('EXECUTABLE', sys.executable)\n"
"print('PATH', sys.path)\n");

if (Py_FinalizeEx() < 0)
{
exit(120);
}

PyMem_RawFree(program);
return 0;
}
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Embedding return in a print statement Tapster 3 294 Oct-07-2019, 03:10 PM
Last Post: Tapster
  More Python Embedding Trouble jibarra 3 541 Jul-11-2019, 09:25 PM
Last Post: Gribouillis
  Python Embedding jibarra 4 628 Jul-11-2019, 03:25 PM
Last Post: jibarra
  Embedding or adding IDE like "repl" inside Flask app sray 1 383 Jul-03-2019, 03:13 PM
Last Post: nilamo
  Embedding Python into a simulator siggi 0 593 Apr-24-2019, 07:42 PM
Last Post: siggi
  Virtual Environments pneveux 1 473 Apr-04-2019, 09:15 PM
Last Post: Larz60+
  Different Virtual environments Pengwain 1 733 Jul-08-2018, 02:48 PM
Last Post: snippsat
  Installing blpapi in two different conda environments snicolaou 0 1,974 Jan-09-2018, 02:12 PM
Last Post: snicolaou
  [CPP] Embedding Python and use GDB Razazel 4 2,103 Nov-12-2017, 05:59 PM
Last Post: Razazel
  Embedding Args in external program call brizflysdrones 5 2,164 May-09-2017, 08:29 PM
Last Post: buran

Forum Jump:


Users browsing this thread: 1 Guest(s)