I'm using C++/Qt to run my Qt application. Development Version: Python: 3.12 Qt: 6.6.3 C++: std 17 Visual Studio 2022
Inside my code, I use Python/C API to run python script like:
PyObject* pModule = PyImport_ImportModule("DataModule");
if (pModule == nullptr)
{
PyErr_Print();
return QString();
}
PyObject* pFunc = PyObject_GetAttrString(pModule, "RunNeuralNetwork");
PyObject* py_outputDir = QStringToPyObject(outputDir);
PyObject* py_fileName = QStringToPyObject(fileName);
PyObject* pArgs = PyTuple_New(4);
PyTuple_SetItem(pArgs, 0, py_outputDir);
PyTuple_SetItem(pArgs, 1, py_fileName);
PyObject* pReturn = PyObject_CallObject(pFunc, pArgs);
QString qString = PyObjectToQString(pReturn);
In "DataModule.py" file, I import Pytorch and create a Neural NetWork function. the code like:
import torch
from torch import nn
from torch.func import jacrev
from torch.utils.data import Dataset
from torch.utils.data import DataLoader
from multiprocessing import Pool
...
class NeuralNetwork(nn.Module):
def __init__(self,
dim_x: int,
dim_y: int,
num_hidden_layer: int,
num_node: int,
acti_name: str,
kwarg_acti: dict = None,
use_bn: bool = False,
normal=None) -> None:
super().__init__()
activation_func = getattr(nn, acti_name)
if kwarg_acti is None:
kwarg_acti = dict()
dim_list = [dim_x] + [num_node] * num_hidden_layer
layer_list = []
for i in range(num_hidden_layer):
layer_list.append(nn.Linear(dim_list[i], dim_list[i + 1]))
if use_bn:
layer_list.append(nn.BatchNorm1d(dim_list[i + 1]))
layer_list.append(activation_func(**kwarg_acti))
layer_list.append(nn.Linear(dim_list[i + 1], dim_y))
# layer_list.append(nn.BatchNorm1d(dim_y))
self.layers = nn.Sequential(*layer_list)
if normal is None:
self.loc_x = torch.tensor(0.)
self.loc_y = torch.tensor(0.)
self.sca_x = torch.tensor(1.)
self.sca_y = torch.tensor(1.)
else:
self.loc_x, self.loc_y, self.sca_x, self.sca_y = normal
def forward(self, x):
x = self.layers((x - self.loc_x) / self.sca_x)
return x * self.sca_y + self.loc_y
When I run this code in C++ Python/C API, sometimes it runs well, and sometimes it crash.
It can not work neither in main thread or child thread.
The crash info in VS is:
0x00007FFE06A5EE30 (python311.dll) Exception in MyApp.exe
0xC0000005: access violation reading 0xFFFFFFFFFFFFFFFF
I consider it is the problem of Multi Thread, then I use the below code In '.py' script
torch.set_num_threads(1)
It still not work.
Please Help!
use C++ through Python/C API runs '.py' script importing pytorch and no error.
0xFFFFFFFFFFFFFFFFis-1, do you check that all functions succeed ?