1,389 questions
0
votes
2
answers
178
views
Building a Python extension with a local C++ library dependency using setuptools and build
I have a library written in C++ for which I would like to make a Python wrapper. The directory structure of my repository is similar to the following:
.
├── include # C++ library headers
│ └── …
├── ...
- 2,139
1
vote
0
answers
89
views
How to let Python runtime load its own Library ZIP file from memory during the initialization
I am using the Python C API and want to embed Python runtime into my app.
The first thing is how to let Python load everything from a memory block, which
stores the Zipped Python standard library ...
- 11
1
vote
0
answers
84
views
pyproject.toml: specify dynamic parameters for C extension
I'm updating a Python package that contains some C extensions.
In a previous version I had all parameters set up in the setuptools setup() function, and now I am porting that setup into a new version ...
- 980
0
votes
0
answers
167
views
Pytsk3 installed with .whl file does not work
I tried to use pytsk3 in my python project to extract files from .img files. Pytsk3 (https://github.com/py4n6/pytsk) is a python binding for the Sleuth Kit. It works fine on my laptop (windows 11 pro ...
0
votes
0
answers
55
views
PyEval_RestoreThread crash when calling matplotlib via Python C API
I have a c++ application that implements its own Python interpreter and console. The user can enter Python code into the console, and that text gets sent to another thread where I make calls to the ...
- 33
0
votes
0
answers
52
views
import Pytorch to create Neural Network. Running Error
I'm using C++/Qt to run my Qt application.
Development Version:
Python: 3.12
Qt: 6.6.3
C++: std 17
Visual Studio 2022
Inside my code, I use Python/C API to run python script like:
PyObject* pModule ...
0
votes
1
answer
68
views
Can a simple mutex pthread_mutex be a replacement for GIL in a multithreaded Python C extension?
Can a simple mutex pthread_mutex be a replacement for GIL in a multithreaded Python C extension? For eg: if I put mutex locks around Py_INCREF or PyMem_Alloc, would that suffice?
Tried using the GIL, ...
- 33
1
vote
1
answer
153
views
Create instance of a imported python class with C API
I want to create an instance of a Python class with the C API.
My specific use case is the PySerial Object. My Python code would be:
import serial
test = serial.Serial()
I couldn't find any clue or ...
- 215
1
vote
1
answer
37
views
How to dispose off a partially initialized list in Python C-API?
The documentation for PyList_New says that these two APIs are safe before the list is fully initialized: PyList_SetItem() and PyList_SET_ITEM(). But if an error happens in the middle of initializing ...
- 2,038
3
votes
1
answer
107
views
How ro run a C++ application with embedded Python in Debug mode
I'm trying to run the script below in Debug mode (Release works fine). I've linked against python310_d.lib and python310_d.dll which have been built from the source code found here https://github.com/...
0
votes
1
answer
262
views
C++ Python/C API - Is exception returned by PyErr_GetRaisedException normalized?
I'm using a new Python/C API PyErr_GetRaisedException added in 3.12 version to retrieve exception and then using other Python/C APIs to get exception type, value, and traceback details.
Then I'm ...
- 13
1
vote
1
answer
40
views
How to debug a dylib error or compiler bug in a Python-C-API function wrapper?
I am writing a Python wrapper for a C function but I have some very strange behaviour.
The C code is:
static PyObject* f12_wrapper(PyObject* self, PyObject* args, PyObject* kwargs)
{
PyObject* ...
- 482
8
votes
1
answer
386
views
How to make CPython report vectorcall as available only when it will actually help performance?
The Vectorcall protocol is a new calling convention for Python's C API defined in PEP 590. The idea is to speed up calls in Python by avoiding the need to build intermediate tuples and dicts, and ...
- 76.3k
0
votes
0
answers
52
views
PySide6 with Python C API: EXC_BAD_ACCESS error when calling a method with inherited class from QRasterWindow
Environement:
Python version: 3.9.6 (default, May 7 2023, 23:32:45)
PySide6 version: 6.7.0
When I create an instance of QRasterWindow, calling the metric method is ok:
Py_Initialize();
PyObject *...
- 1
0
votes
1
answer
136
views
Generate pyi stubfile for not importable C module
I have a package A that uses a special package A.B, both are 3rd and 4th party packages written in C using Boost. Because it there is some special setup A.B is not included as a submodule but only as ...
- 5,166
3
votes
1
answer
98
views
CPython: Usage of `tp_finalize` in C-defined static types with no custom `tp_dealloc`
PEP 442 introduced the tp_finalize callback to Python type definitions (as a one-to-one equivalent of Pythons classes' __del__ function), and recommends using this for any non-trivial destruction.
The ...
- 398
0
votes
0
answers
64
views
How do I clear my Python interpreter cache while executing with pybind11?
I want to embed a Python interpreter in my C++ project using the pybind11 embedded interpreter.
During runtime, Python modules from different locations can be loaded and deleted.
To achieve this, I ...
0
votes
1
answer
90
views
Python's C-extension with numpy randomly crashes after a few calls (5~10) from python code
To speed up custom MFCC calculations, I had to write some code in C, wrap it with Python's C API & numpy's C API, and compile it with disutils.core in order to import it from python code.
I was ...
- 33
1
vote
1
answer
95
views
Why does Python implement cyclic GC on types that reference non-container types
Checking docs:
Supporting Cyclic Garbage Collection
Python’s support for detecting and collecting garbage which involves circular references requires support from object types which are “containers” ...
- 25
0
votes
1
answer
78
views
Create a PyObject with a pointer to a C function
Let's say I have this Python function:
def foo(bar):
bar()
And this C function
void bar() {
}
How can I create a PyObject with Python C API that holds a pointer to bar function so I can call the ...
- 1,099
2
votes
1
answer
58
views
Python C extension using the Limited API: how to get the value of __debug__?
I'm coding a C extension for Python.
I used the non-limited API and recently switched to Python's limited API. And with that new API, I can't see a way to get the value of the global __debug__ ...
- 63
0
votes
0
answers
32
views
Executing a c function in python code space
I have the following module written in C using Python C-api:
#include <Python.h>
static PyObject* init_factorial(PyObject* self, PyObject* args) {
long long n;
if (!PyArg_ParseTuple(...
- 109
1
vote
1
answer
1k
views
Polars complex function via the numba JIT (circumventing the return entity limit of 1)
In a Polars DataFrame, I have 3 columns (A, B, D) storing value of type double.
For n rows, and a starting value of some double number, the calculation is performed as shown below:
value of column A = ...
- 944
1
vote
1
answer
217
views
Python: Heaptypes with metaclasses
In Python c-api: how can I create a heap type that has a metaclass? I'm well aware of PyType_FromSpec and derivatives and they can do this (from documentation):
Changed in version 3.12: The function ...
- 316
0
votes
1
answer
49
views
check if assessing an address will cause a segfault without crashing python
What I've tried:
faulthandler is really useful to get a traceback where segfault occurred but it doesn't allow handling it properly.
import faulthandler
faulthandler.enable()
import ctypes
try:
...
- 469
0
votes
3
answers
108
views
Best/Cleanest way to structure error checking on dynamically allocated variables
I am working on a Python C project that involves a couple of dynamically allocated structures like queues and threading structures. I am mainly looking to improve the readability. It looks something ...
3
votes
1
answer
546
views
Python setuptools multiple extension modules with shared C source code building in parallel
I'm working on a Python project with a setup.py that has something like this1:
setup(
cmdclass={"build_ext": my_build_ext},
ext_modules=[
Extension("A", ["a.c&...
- 1,875
0
votes
1
answer
163
views
Calling a function that execute Python in C++ from Python gives a free() invalid pointer error
I write a function that executes a Python string in C++ using the Python C API.
The function works perfectly. When I run it, it executes the Python string.
The problem occurs when I make a .so file ...
1
vote
1
answer
1k
views
How to set the Python executable name, now that Py_SetProgramName() is deprecated?
The Python 3.12 embedding documentation for embedding gives this example:
#define PY_SSIZE_T_CLEAN
#include <Python.h>
int
main(int argc, char *argv[])
{
wchar_t *program = Py_DecodeLocale(...
- 78.3k
0
votes
1
answer
87
views
Executable under GDB invokes different embedded Python
I have code that embeds Python. That embedded Python uses NumPy and hence, I need to explicitly load libpython, to make NumPy work.
The driving code is in C++ (tests in Google Test). There is a bug ...
- 927
0
votes
2
answers
1k
views
Issue with GIL on Python 3.12.2
I am using this code:
PyThreadState * thread = PyEval_SaveThread();
PyGILState_STATE gil = PyGILState_Ensure();
// ...
PyGILState_Release(gil);
PyEval_RestoreThread(thread);
In Python 3.11.8 it ...
- 21
1
vote
1
answer
255
views
Python3.12 C-API segfaults with openMP
Here is a small c++ program that embeds python.
It works with python 3.11.6, but segfaults with python 3.12.0:
#include <iostream>
#include "omp.h"
#include "Python.h"
int ...
- 697
1
vote
2
answers
177
views
How to implement multi-thread programs using Python C API?
I have the following program which uses Python C API. It creates a number of threads (NUM_THREADS constant). In each thread there is an infinite loop that does a very simple operation: creates a ...
- 12.4k
1
vote
1
answer
107
views
generalized ufunc on two arrays with one non-matching dimension
I want to write a numpy gufunc python extension using the numpy C-api that takes two matrices of arbitrary dimensionality, takes the mean across one dimension, and then subtracts one result from the ...
0
votes
1
answer
173
views
u# format character removed from Python 3.12 C-API, how to account for it?
A bunch of unicode-related functionality was removed from the Python 3.12 C-API. Unfortunately for me, there's a very old piece of code (~2010) in our library that uses these and I need to migrate ...
- 7,498
1
vote
0
answers
63
views
Release of references in PyObject when that object is a dictionary, list or text string
I have a code in Python C API like this:
PyObject* result = PyObject_CallMethod(/* some function with some arguments */);
// Do something with result
Py_XDECREF(result);
I'm almost sure this code is ...
- 12.4k
1
vote
1
answer
118
views
How can I create lambda PyObject from its Python code in a C string?
I have the following code in Python, which uses pyjexl module:
import pyjexl
jexl = pyjexl.JEXL()
jexl.add_transform("lowercase", lambda x: str(x).lower())
I want to do the same using ...
- 12.4k
0
votes
2
answers
88
views
IronPython garbage collection - How does it provides compatibility with C-extensions?
In this part of the talk on GIL by Larry Hastings, there is an explanation about how ironclad provides C-extension compatibility with IronPython. This is the interesting part of the talk:
We ...
- 1,962
0
votes
0
answers
36
views
What is the standard method to numpy extension create modules that support different array datatypes?
Assuming I have a C function I would like to wrap by the Nympy(Python)-C API to create a Python extension module. This module should be capable of accepting Numpy array that can have multiple types ...
- 763
0
votes
1
answer
380
views
PyLong_Check() wrongly detecting PyBool type?
I'm using Python C API in my C++ program. I have a function like this one (simplified) that returns the a string with the type of a given PyObject passed as argument:
#include <Python.h>
static ...
- 12.4k
0
votes
1
answer
86
views
How to properly deepcopy linked objects in C Extension
In Python, I have a pair of classes like
class Internal:
def __init__(self, ref):
self.ref = ref
class External:
def __init__(self):
self.storage = [1,2,3]
self.int = ...
- 1,558
0
votes
1
answer
135
views
How to resolve linking error in C Python API
I am getting a linking error in my Visual Studio C project. Those are related to external functions of C-Python API not being resolved in the linking stage. One of them being: unresolved external ...
- 154
2
votes
1
answer
203
views
Segfault in tp_alloc on Python 3.12
Upgraded to Python 3.12.0 from 3.11.7 and previously working C-API code now segfaults inside tp_alloc somewhere.
I was first scrounging around to see if there was some breaking change between 3.11 and ...
- 139
2
votes
1
answer
70
views
Why do attributes that are being set by a custom type initializer need to be protected?
The CPython Tutorial defines a custom initializer for a custom type which has the following lines :
if (first) {
tmp = self->first;
Py_INCREF(first);
self->first = first;
...
- 9,269
1
vote
0
answers
224
views
How can I modify `sys.path` before python interpreter initializes using Python/C API?
From the official example, I understand that I can modify sys.path before the interpreter initializes using the following code(exception handlings are omitted):
PyConfig config;
...
- 61
1
vote
1
answer
181
views
Python C API - ModuleNotFoundError
I am using the Python C API, and I keep encountering the error ModuleNotFoundError: No module named '/home/user_1/project/ml/multiply. This is my c++ file, its path is /home/user_1/project/ml/testcapi....
- 69
0
votes
0
answers
69
views
How to Import .py file with third-party package(like numpy) from C?
I'm using visual studio to import a python file from a c/c++ file to use a function(named lalala) defined in PythonApplication7.py. I'm trying to use Python-C-API to do this.
The C file is:
#include &...
- 1
1
vote
1
answer
234
views
Correct use of buffer protocol for dynamic array
I have a dynamic array type in C++ that I would like to expose through the buffer protocol. The type is already exposed as a sequence in Python, but I want to build NumPy arrays with the C++ array ...
- 59.9k
3
votes
4
answers
695
views
How to deploy python extensions (shared libraries) with multiple architectures?
Context: one can compile C code such that it can be used as a Python module. The compiled object is a shared library with a specific naming, so Python can find and load it as a module.
Great. I have ...
- 4,590
1
vote
0
answers
77
views
Initialization of a python function from C++ code takes too long
I have a C++ script, where I load a python file to execute a function to evaluate a Graph Neural Network. This function gets executed to evaluate data in an event loop, so the script gets executed for ...
