Python: Heaptypes with metaclasses

Question

In Python c-api: how can I create a heap type that has a metaclass? I'm well aware of PyType_FromSpec and derivatives and they can do this (from documentation):

Changed in version 3.12: The function now finds and uses a metaclass corresponding to the provided base classes. Previously, only type instances were returned.

The tp_new of the metaclass is ignored. which may result in incomplete initialization. Creating classes whose metaclass overrides tp_new is deprecated and in Python 3.14+ it will be no longer allowed.

I have two one issue with this: support for metaclasses was added in Python 3.12 and I want to support Python 3.8+. Second one: if class name is not statically defined, (creating with malloc and eg.: string concatenation), for python < 3.11, this will leak, as that function doesn't copy the buffer in these versions and there is no good way of freeing that buffer (PyTypeObject->tp_name is a public field and can be replaced by anything).

Setting the class name seems fairly easily fixed: why not assign at static name to the type initially, and then use PyObject_SetAttrString(tp, "__name__", realValue) afterwards to let Python manage the memory? — DavidW
– DavidW, Commented May 24, 2024 at 7:05
True, it would work. When I was firstly implementing this, I decided to just leak memory for python versions < 3.11, as my function is limited in terms how many new classes can be created by it, but I had to change my approach when I discovered that metaclasses don't work for python < 3.12 and I think I have a working solution. — oBrstisf8o
– oBrstisf8o, Commented May 24, 2024 at 7:37

oBrstisf8o · Accepted Answer · 2024-06-01 16:04:34Z

0

As in the documentation, there is another way of creating heap types, which existed for longer and inherently is more stable: calling type with three arguments: type(name, mro, dict), or for meta-types: meta(name, mro, dict)/type.__call__(meta, name, mro, dict).

args = Py_BuildValue("sOO", class_name, super_classes, dict);
PyObject *my_type = PyType_Type.tp_call((PyObject *)&meta_type, args);

Its downside is that this api was created for python code in mind, not c extensions, so it is less pleasant to use.

answered Jun 1, 2024 at 16:04

oBrstisf8o

3162 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

DavidW Over a year ago

I think you also can't give your type a custom C struct (which may or may not be a significant limitation depending on what you need)

oBrstisf8o Over a year ago

@DavidW Or it would require a hack-y solution, but it seems that if my class inherits form a c type that has said struct, it works.

Collectives™ on Stack Overflow

Python: Heaptypes with metaclasses

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related