embedded python system path configuration

Question

I am trying to embed Python in C++. I am trying to use the documented API for configuring the Python system path so that arbitrary modules can be imported from C++. But the path is not being configured correctly because my module cannot be imported unless I move it into the same directory as the executable. Clearly I am not using the API correctly. Here is my code:

error_t init_path(PyConfig* config, wchar_t** paths, int num_paths) {
    if (num_paths) {
        PyStatus status;
        //status = PyConfig_SetBytesString(config, &config->program_name, "C:/Python311/python.exe");
        //status = PyConfig_SetBytesString(config, &config->program_name, "D:/work/projects/common_hdl/pysim/cosim/x64/Debug/cosim.exe");
        status = PyConfig_Read(config);
        config->module_search_paths_set = 1;
        for (int i = 0; i < num_paths; i++) {
            status = PyWideStringList_Append(&config->module_search_paths, paths[i]);
            if (PyStatus_Exception(status)) {
                fprintf(stderr, "Could not add path %ls\n", paths[i]);
                return PATH_INSERTION_ERR;
            }
            else {
                printf("Appending %ls to system path\n", paths[i]);
            }
        }
        status = Py_InitializeFromConfig(config);
        if (PyStatus_Exception(status)) {
            fprintf(stderr, "Unable to initialize python\n");
            return INIT_ERR;
        }
    }
    return NONE;
}

I think the documented steps to initializing the path are:

Initialize the PyConfig data structure
Set the program_name in the data structure
Read the configuration data
Set the module_search_paths_set field to 1
Use the PyWideStringList_Append method to append the "module_search_paths" field in the data structure.
Run the Py_InitializeFromConfig method with the new configuration.

But...

It is very unclear from the Python documentation what the "program_name" is referring to and why (or if) it needs to be initialized. I have tried some experiments as you can see in my code.

And the code above simply doesn't work. Can anyone spot the problem or fix my interpretation of the configuration process? I know there are other methods that will allow for python code to be run that will manipulate the system path. But I really would like to use the API means of doing this.

More information: When I move the python file I am trying to import into the same directory as the executable and print the contents of "sys.path", I can confirm what I suspected. The call to my "init_path" method listed above is not doing anything to the sys.path variable. The sys.path variable is unchanged regardless of what I do. — sean
– sean, Commented Jul 11, 2023 at 20:46

Aname · Accepted Answer · 2024-04-19 05:07:09Z

0

When I move the python file I am trying to import into the same directory as the executable and print the contents of "sys.path"

Does it work in that case?

That would make me think that something was wrong with your PyWideStringList_Append(&config->module_search_paths...) calls: was the path to that file in the paths list? The value of num_paths correct? Actually, could you share a call to that function? I use something like L"C:\\tests\\pyfiles_to_load" (on windows obviously) in which there's a .py file I then can load.

To my understanding and experiments, and despite the examples in https://docs.python.org/fr/3/c-api/init_config.html the name (&config->program_name) doesn't have to be a path, it can be 'just' a name (e.g. Test_PythonCcall).

To modify sys.path, it seems like you need to use PySys_SetArgvEx https://docs.python.org/3.11/c-api/init.html#c.PySys_SetArgvEx explains that

If updatepath is zero, this is all the function does. If updatepath is non-zero, the function also modifies sys.path according to the following algorithm ...

Good luck!

edited Apr 19, 2024 at 5:07

answered Apr 17, 2024 at 7:29

Aname

6106 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

sean Over a year ago

It's been a long time since I asked this question. I abandoned using the API as documented and used the approach described here: stackoverflow.com/questions/7624529/…

Collectives™ on Stack Overflow

embedded python system path configuration

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related