How to debug a dylib error or compiler bug in a Python-C-API function wrapper?

Question

I am writing a Python wrapper for a C function but I have some very strange behaviour. The C code is:

static PyObject* f12_wrapper(PyObject* self, PyObject* args, PyObject* kwargs)
{
    PyObject* y_obj;
    void* y_data = NULL;
    int64_t y_shape[1];
    int64_t y_strides[1];
    int64_t Out_0001;
    PyObject* Out_0001_obj;
    y_obj = Py_None;
    static char *kwlist[] = {
        "y",
        NULL
    };
    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O", kwlist, &y_obj))
    {
        return NULL;
    }
    // printf("1. %p %p %d\n", y_obj, Py_None, (y_obj == Py_None));
    if (y_obj == Py_None)
    {
        // printf("2. Py_None detected");
        y_data = NULL;
    }
    else if (pyarray_check("y", y_obj, NPY_LONG, INT64_C(1), NO_ORDER_CHECK))
    {
        y_data = PyArray_DATA((PyArrayObject*)(y_obj));
        get_strides_and_shape_from_numpy_array(y_obj, y_shape, y_strides);
    }
    else
    {
        // printf("3. Py_None not detected");
        return NULL;
    }
    Out_0001 = bind_c_f12(y_data, y_shape[INT64_C(0)], y_strides[INT64_C(0)]);
    Out_0001_obj = Int64_to_PyLong(&Out_0001);
    return Out_0001_obj;
}

When I call the function from Python as:

f12()

the code runs into the wrong if block. Instead of entering the first if block (the block containing print 2), it skips over this and tests the second if condition before exiting with an error (in the block containing print 3). Putting the second condition in an else block instead of an else if block does not change the behaviour.

If I uncomment print 1 then I can see that the address stored in y_obj and Py_None are the same. If I uncomment print 3 then I can see that the code is going into the else 3.

The strangest behaviour occurs if I uncomment print 2. In this case I have the error:

Trace/BPT trap: 5

I have no idea why the code is not behaving as described or how to debug the dylib error. Is this a compiler bug? The same code works perfectly on my local linux machine, a GitHub linux runner and a GitHub windows runner

This code is calling Fortran code. If I remove the call to bind_c_f12 and stop linking the associated .o files the problem goes away. But an almost identical version of this code with an identical version of the call to bind_c_f12 has worked in the past.

bourneeoo · Accepted Answer · 2024-08-28 13:20:48Z

I never got to the bottom of the Trace/BPT error but I did manage to fix the code by removing lines until it worked, so I am leaving the fixed version here in case anyone comes across a similar problem. The code that worked was:

static PyObject* f12_wrapper(PyObject* self, PyObject* args, PyObject* kwargs)
{
    PyObject* y_obj;
    void* y_data = NULL;
    int64_t y_shape[1];
    int64_t y_strides[1];
    int64_t Out_0001;
    PyObject* Out_0001_obj;
    y_obj = Py_None;
    static char *kwlist[] = {
        "y",
        NULL
    };
    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O", kwlist, &y_obj))
    {
        return NULL;
    }
    // printf("1. %p %p %d\n", y_obj, Py_None, (y_obj == Py_None));
    if (y_obj == Py_None)
    {
        // printf("2. Py_None detected");
        y_data = NULL;
        y_shape[0] = 0;
        y_strides[0] = 0;
    }
    else if (pyarray_check("y", y_obj, NPY_LONG, INT64_C(1), NO_ORDER_CHECK))
    {
        y_data = PyArray_DATA((PyArrayObject*)(y_obj));
        get_strides_and_shape_from_numpy_array(y_obj, y_shape, y_strides);
    }
    else
    {
        // printf("3. Py_None not detected");
        return NULL;
    }
    Out_0001 = bind_c_f12(y_data, y_shape[INT64_C(0)], y_strides[INT64_C(0)]);
    Out_0001_obj = Int64_to_PyLong(&Out_0001);
    return Out_0001_obj;
}

The only difference is that now the variables y_shape and y_stride (which were unused in bind_c_f12 if y_data == NULL) are initialised. My best guess is that the compiler on macos was over-optimising the code and assuming that we would never go into a block that left variables uninitialised. The compiler was gcc/gfortran-14

Collectives™ on Stack Overflow

How to debug a dylib error or compiler bug in a Python-C-API function wrapper?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related