Passing dynamic array as an argument in PyObject_CallObject in Python/C API

Question

I want to access some python functions that I wrote from my C code. One of my python functions receives a numpy array as an input. In my C code, this array is allocated in the dynamic memory. Whenever my python function is called I receive a segmentation fault, and I am not sure what am I doing wrong. I tried to pass N, O, O& to the Py_BuildValue function, but I keep on getting the same result.

Here is the python simplified code.

#Python simple function that receives pointer to array
#!/usr/bin/python3

import numpy as np
def getNumpyArrayPointer(pointer):
    data = np.ctypeslib.as_array(pointer, shape=(1000,1)).astype(np.int32) #Doesn't matter if I comment out this line. Fault continues
    print('Python function getNumpyArrayPointer() called')
    return data

Here is the C simplified code

#include <iostream>
#include <Python.h>
#include "pyhelper.h"


int capitest()
{
    Py_Initialize();
    PyObject* sysPath = PySys_GetObject((char*)"path");
    PyObject* programName = PyUnicode_FromString("path-to-python-script");
    PyList_Append(sysPath, programName);
    Py_DECREF(programName);
    PyObject* pName = PyUnicode_FromString("python-script-name");
    PyObject* pModule = PyImport_Import(pName);
    PyObject* pValue;
    CPyObject pFunc;
    PyObject* args;

    if(pModule)
    {
        pFunc = PyObject_GetAttrString(pModule, "getNumpyArrayPointer");
        uint32_t *array = (uint32_t *)malloc (1000);
        args = Py_BuildValue("(N)", array );
        
        if(pFunc && PyCallable_Check(pFunc))
        {
            printf("Calling getNumpyArrayPointer\n");
            pValue = PyObject_CallObject(pFunc, args);
            Py_DECREF(args);
            printf("Called getNumpyArrayPointer\n");
            if (PyErr_Occurred()) 
            { 
                PyErr_Print(); 
                return 0; 
            }   
        }
        else
        {
            printf("ERROR: function getNumpyArrayPointer()\n");
        }
    }
    else
    {
        printf("ERROR: Module not imported\n");
    }
    Py_Finalize();
    return 0;
}

int main()
{
    capitest();
}

Output:

Calling getNumpyArrayPointer
Segmentation fault (core dumped)

DavidW · Accepted Answer · 2020-09-05 11:04:57Z

-2

 uint32_t *array = (uint32_t *)malloc (1000);
 args = Py_BuildValue("(N)", array );

That's nonsense. You're telling it that array is a pointer to a Python object so it'll try to manage the "reference count" of the array.

I'd suggest something along the lines:

PyObject* pointer_as_int = PyLong_FromVoidPtr(array);

You then want to pass this to Python object to ctypes.cast(pointer_as_int, ctypes.POINTER(ctypes.c_uint32)) to get your ctypes pointer. See for example ctypes: construct pointer from arbitrary integer

Alternatively you could try passing the C pointer directly the Numpy C API function PyArray_SimpleNewFromData - you'll find plenty of examples on Stack Overflow and elsewhere.

Note that your C code is responsible for freeing the array once you're done with it.

edited Sep 5, 2020 at 11:04

answered Sep 5, 2020 at 9:13

DavidW

31.2k7 gold badges64 silver badges99 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Dawood Over a year ago

Thanks! On the C side I added the lines PyObject* number = PyLong_FromVoidPtr(array); args = Py_BuildValue("(O)", number );. In the python script I changed it to pyptr = ctypes.POINTER(ctypes.c_uint32).from_address(pointer) data = np.ctypeslib.as_array(pyptr, shape=(3072,3072)) Now I get a NULL pointer access Value error. Do you have any idea why?

DavidW Over a year ago

I think my conversion from int to a ctypes value was wrong. See the edit for something tested and working

Dawood Over a year ago

Thanks @DavidW. Just for references, the python code was the one crashing as I was converting the pointer to a np array without casting it. The way I am handling it right now is as follows

pyPointerObject = ctypes.cast(pointerFromC, ctypes.POINTER(ctypes.c_uint32))     npMatrix = np.ctypeslib.as_array(pyPointerObject, shape=(3072,3072))

. Thanks!

DavidW Over a year ago

I'm still fairly confident that crash was because you were misinterpreting a C array as a Python object (even if the crash only happened when you tried to use the object rather than at the moment you did the cast). Glad it's working now.

Collectives™ on Stack Overflow

Passing dynamic array as an argument in PyObject_CallObject in Python/C API

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related