I want to embed a Python interpreter in my C++ project using the pybind11 embedded interpreter.
During runtime, Python modules from different locations can be loaded and deleted.
To achieve this, I used importlib to refresh my Python cache. However, the cache remains unchanged, and if I try to load a module with the same name but from a different location, the old module is still loaded.
Thanks for your help.
i tried this small example
#include <pybind11/pybind11.h>
#include <pybind11/embed.h>
namespace py = pybind11;
void addModulePath(const std::string &module) {
auto sys = py::module::import("sys");
auto importlib = py::module::import("importlib");
auto path = sys.attr("path");
py::print(path);
path.attr("append")(module);
importlib.attr("invalidate_caches")();
py::print(path);
}
void deleteModulePath(std::string& module) {
auto sys = py::module::import("sys");
auto importlib = py::module::import("importlib");
auto path = sys.attr("path");
py::print(path);
path.attr("remove")(module);
importlib.attr("invalidate_caches")();
py::print(path);
}
PYBIND11_EMBEDDED_MODULE(changeModule, m) {
m.def("add_module", &addModulePath, "add module path");
m.def("delete_module", &deleteModulePath, "delete module path");
}
int main() {
constexpr auto test1Path = "/tmp/test1";
constexpr auto test2Path = "/tmp/test2";
py::scoped_interpreter guard{};
py::module_ module = py::module_::import("changeModule");
module.attr("add_module")(test1Path);
auto test = py::module_::import("test");
test.attr("test")();
module.attr("delete_module")(test1Path);
module.attr("add_module")(test2Path);
test = py::module_::import("test");
test.attr("test")();
return 0;
}
the folders test1 and test2 does both contain a file called "test.py" which prints its folder name.
I don't want to restart my interpreter.
fooholds a reference to version 1 oftestand modulebarholds a reference to version 2 oftest. Just because you've deleted a module fromsys.modulesdoes not mean it is no longer accessible in the interpreter. Have you tried just adding/tmptosys.pathand importingtest1.testandtest2.test?invalidate_caches()isn't enough to load new source code of a module that has already been imported. If a module exists insys.modules, then that module is always used in preference to loading the module from disk. You need to either delete the module fromsys.modulesor useimportlib.reload(mymod).reload()preserves the globals of the module. So names that were present in the old source, but not the new source will be preserved.