Python Forum
Embedding, windows, and virtual environments
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Embedding, windows, and virtual environments
#1
Hi,

I've been writing a C++ app which embeds the python interpreter (tested on 3.6 or 3.7).
I am writing it both for linux and windows.

I would like it to use with virtual environments and have it behave in exactly the same way as python.exe would do in such cases.

In linux everything works out of the box and there is nothing I really need to do to support it. It just works.

In windows I can't get it to work other than manually adjusting the sys.path.
The code is at the bottom of the message.

Virtual environments created with

python -m venv abcd

After a lot of debugging I have found that the issue is sys.executable which is used in site.py to work out the location and settings of the virtual environment.

In linux this is by default the location of the python interpreter, something like /usr/bin/python3 or /tmp/abcd/bin/python3 (if the venv is activated).

In windows it is always the name of the executable.

This is because the 2 files Modules/getpath.c and PC/getpathp.c have completely different logic in
calculate_program_full_path vs get_program_full_path.

The former (linux) searches the ProgramName in the PATH while the second calls GetModuleFileNameW regardless.

This value ends up in Py_GetProgramFullPath() which is the same as sys.executable.
The only solution I see is to locate my app in the same folder where the embedded python.exe is. But this is really a messy and not scalable solution. Or to manually set sys.path, but I would have to replicate site.py or whatever other virtual environment technique is used.

I have a few questions

1) why this different behaviour?
2) how can I make virtual environments works out of the box in windows as well?
3) according to the doc (https://docs.python.org/3/c-api/init.htm...amFullPath) the liunux behaviour is correct and depends on the value set by Py_SetProgramName(), while in windows it ignores it
4) why the doc (https://docs.python.org/3/extending/embedding.html) says that the call to Py_SetProgramName() is recommended? What are the benefits? Actually I would recommend NOT to call it at all. If you set it to argv[0] as in the example, then the virtual environments do not work in linux either and it behaves the same as window.

As I said, if the call below is left out, in linux it just works.

Andrea

Example

#include <Python.h>

int
main(int argc, char *argv[])
{
wchar_t *program = Py_DecodeLocale(argv[0], NULL);
if (program == NULL)
{
fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
exit(1);
}

// try a few alternatives
// Py_SetProgramName(L"python"); /* optional but recommended */
// Py_SetProgramName(program); /* optional but recommended */
// do not call it at all

Py_Initialize();
PyRun_SimpleString(
"import sys\n"
"print('EXECUTABLE', sys.executable)\n"
"print('PATH', sys.path)\n");

if (Py_FinalizeEx() < 0)
{
exit(120);
}

PyMem_RawFree(program);
return 0;
}
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Best practice on using virtual environments in Python bytecrunch 6 641 Feb-14-2024, 03:22 PM
Last Post: snippsat
  Embedding python script into html via pyscript pyscript_dude 7 1,452 Apr-16-2023, 11:17 PM
Last Post: pyscript_dude
  Virtual Environments - Organization (VS Code) JaysonWonder 11 1,767 Jan-26-2023, 11:34 PM
Last Post: Larz60+
  Keeping up with IDEs and Virtual Environments... bytecrunch 7 2,363 Sep-05-2022, 08:04 PM
Last Post: snippsat
  C++ python embedding comarius 0 803 Aug-26-2022, 02:01 AM
Last Post: comarius
  I don't understand pip and environments snakes 3 1,200 Jul-31-2022, 08:17 PM
Last Post: snakes
Question Embedding a python file online Dreary35 0 1,476 Jun-10-2021, 05:05 PM
Last Post: Dreary35
  Use different Anaconda environments on Linux Mint and Spyder StaLLoNe_CoBRa 0 1,839 Jan-20-2021, 03:12 AM
Last Post: StaLLoNe_CoBRa
  pip and venv virtual environments soupworks 2 2,230 Dec-30-2020, 11:38 PM
Last Post: soupworks
  Need help merging/embedding duckredbeard 10 3,332 Aug-13-2020, 04:48 AM
Last Post: duckredbeard

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020