1.. highlight:: c
2
3.. _common-structs:
4
5Common Object Structures
6========================
7
8There are a large number of structures which are used in the definition of
9object types for Python.  This section describes these structures and how they
10are used.
11
12
13Base object types and macros
14----------------------------
15
16All Python objects ultimately share a small number of fields at the beginning
17of the object's representation in memory.  These are represented by the
18:c:type:`PyObject` and :c:type:`PyVarObject` types, which are defined, in turn,
19by the expansions of some macros also used, whether directly or indirectly, in
20the definition of all other Python objects.
21
22
23.. c:type:: PyObject
24
25   All object types are extensions of this type.  This is a type which
26   contains the information Python needs to treat a pointer to an object as an
27   object.  In a normal "release" build, it contains only the object's
28   reference count and a pointer to the corresponding type object.
29   Nothing is actually declared to be a :c:type:`PyObject`, but every pointer
30   to a Python object can be cast to a :c:expr:`PyObject*`.  Access to the
31   members must be done by using the macros :c:macro:`Py_REFCNT` and
32   :c:macro:`Py_TYPE`.
33
34
35.. c:type:: PyVarObject
36
37   This is an extension of :c:type:`PyObject` that adds the :attr:`ob_size`
38   field.  This is only used for objects that have some notion of *length*.
39   This type does not often appear in the Python/C API.
40   Access to the members must be done by using the macros
41   :c:macro:`Py_REFCNT`, :c:macro:`Py_TYPE`, and :c:macro:`Py_SIZE`.
42
43
44.. c:macro:: PyObject_HEAD
45
46   This is a macro used when declaring new types which represent objects
47   without a varying length.  The PyObject_HEAD macro expands to::
48
49      PyObject ob_base;
50
51   See documentation of :c:type:`PyObject` above.
52
53
54.. c:macro:: PyObject_VAR_HEAD
55
56   This is a macro used when declaring new types which represent objects
57   with a length that varies from instance to instance.
58   The PyObject_VAR_HEAD macro expands to::
59
60      PyVarObject ob_base;
61
62   See documentation of :c:type:`PyVarObject` above.
63
64
65.. c:function:: int Py_Is(PyObject *x, PyObject *y)
66
67   Test if the *x* object is the *y* object, the same as ``x is y`` in Python.
68
69   .. versionadded:: 3.10
70
71
72.. c:function:: int Py_IsNone(PyObject *x)
73
74   Test if an object is the ``None`` singleton,
75   the same as ``x is None`` in Python.
76
77   .. versionadded:: 3.10
78
79
80.. c:function:: int Py_IsTrue(PyObject *x)
81
82   Test if an object is the ``True`` singleton,
83   the same as ``x is True`` in Python.
84
85   .. versionadded:: 3.10
86
87
88.. c:function:: int Py_IsFalse(PyObject *x)
89
90   Test if an object is the ``False`` singleton,
91   the same as ``x is False`` in Python.
92
93   .. versionadded:: 3.10
94
95
96.. c:function:: PyTypeObject* Py_TYPE(PyObject *o)
97
98   Get the type of the Python object *o*.
99
100   Return a :term:`borrowed reference`.
101
102   Use the :c:func:`Py_SET_TYPE` function to set an object type.
103
104   .. versionchanged:: 3.11
105      :c:func:`Py_TYPE()` is changed to an inline static function.
106      The parameter type is no longer :c:expr:`const PyObject*`.
107
108
109.. c:function:: int Py_IS_TYPE(PyObject *o, PyTypeObject *type)
110
111   Return non-zero if the object *o* type is *type*. Return zero otherwise.
112   Equivalent to: ``Py_TYPE(o) == type``.
113
114   .. versionadded:: 3.9
115
116
117.. c:function:: void Py_SET_TYPE(PyObject *o, PyTypeObject *type)
118
119   Set the object *o* type to *type*.
120
121   .. versionadded:: 3.9
122
123
124.. c:function:: Py_ssize_t Py_REFCNT(PyObject *o)
125
126   Get the reference count of the Python object *o*.
127
128   Use the :c:func:`Py_SET_REFCNT()` function to set an object reference count.
129
130   .. versionchanged:: 3.11
131      The parameter type is no longer :c:expr:`const PyObject*`.
132
133   .. versionchanged:: 3.10
134      :c:func:`Py_REFCNT()` is changed to the inline static function.
135
136
137.. c:function:: void Py_SET_REFCNT(PyObject *o, Py_ssize_t refcnt)
138
139   Set the object *o* reference counter to *refcnt*.
140
141   .. versionadded:: 3.9
142
143
144.. c:function:: Py_ssize_t Py_SIZE(PyVarObject *o)
145
146   Get the size of the Python object *o*.
147
148   Use the :c:func:`Py_SET_SIZE` function to set an object size.
149
150   .. versionchanged:: 3.11
151      :c:func:`Py_SIZE()` is changed to an inline static function.
152      The parameter type is no longer :c:expr:`const PyVarObject*`.
153
154
155.. c:function:: void Py_SET_SIZE(PyVarObject *o, Py_ssize_t size)
156
157   Set the object *o* size to *size*.
158
159   .. versionadded:: 3.9
160
161
162.. c:macro:: PyObject_HEAD_INIT(type)
163
164   This is a macro which expands to initialization values for a new
165   :c:type:`PyObject` type.  This macro expands to::
166
167      _PyObject_EXTRA_INIT
168      1, type,
169
170
171.. c:macro:: PyVarObject_HEAD_INIT(type, size)
172
173   This is a macro which expands to initialization values for a new
174   :c:type:`PyVarObject` type, including the :attr:`ob_size` field.
175   This macro expands to::
176
177      _PyObject_EXTRA_INIT
178      1, type, size,
179
180
181Implementing functions and methods
182----------------------------------
183
184.. c:type:: PyCFunction
185
186   Type of the functions used to implement most Python callables in C.
187   Functions of this type take two :c:expr:`PyObject*` parameters and return
188   one such value.  If the return value is ``NULL``, an exception shall have
189   been set.  If not ``NULL``, the return value is interpreted as the return
190   value of the function as exposed in Python.  The function must return a new
191   reference.
192
193   The function signature is::
194
195      PyObject *PyCFunction(PyObject *self,
196                            PyObject *args);
197
198.. c:type:: PyCFunctionWithKeywords
199
200   Type of the functions used to implement Python callables in C
201   with signature :const:`METH_VARARGS | METH_KEYWORDS`.
202   The function signature is::
203
204      PyObject *PyCFunctionWithKeywords(PyObject *self,
205                                        PyObject *args,
206                                        PyObject *kwargs);
207
208
209.. c:type:: _PyCFunctionFast
210
211   Type of the functions used to implement Python callables in C
212   with signature :const:`METH_FASTCALL`.
213   The function signature is::
214
215      PyObject *_PyCFunctionFast(PyObject *self,
216                                 PyObject *const *args,
217                                 Py_ssize_t nargs);
218
219.. c:type:: _PyCFunctionFastWithKeywords
220
221   Type of the functions used to implement Python callables in C
222   with signature :const:`METH_FASTCALL | METH_KEYWORDS`.
223   The function signature is::
224
225      PyObject *_PyCFunctionFastWithKeywords(PyObject *self,
226                                             PyObject *const *args,
227                                             Py_ssize_t nargs,
228                                             PyObject *kwnames);
229
230.. c:type:: PyCMethod
231
232   Type of the functions used to implement Python callables in C
233   with signature :const:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS`.
234   The function signature is::
235
236      PyObject *PyCMethod(PyObject *self,
237                          PyTypeObject *defining_class,
238                          PyObject *const *args,
239                          Py_ssize_t nargs,
240                          PyObject *kwnames)
241
242   .. versionadded:: 3.9
243
244
245.. c:type:: PyMethodDef
246
247   Structure used to describe a method of an extension type.  This structure has
248   four fields:
249
250   .. c:member:: const char* ml_name
251
252      name of the method
253
254   .. c:member:: PyCFunction ml_meth
255
256      pointer to the C implementation
257
258   .. c:member:: int ml_flags
259
260      flags bits indicating how the call should be constructed
261
262   .. c:member:: const char* ml_doc
263
264      points to the contents of the docstring
265
266The :c:member:`ml_meth` is a C function pointer.  The functions may be of different
267types, but they always return :c:expr:`PyObject*`.  If the function is not of
268the :c:type:`PyCFunction`, the compiler will require a cast in the method table.
269Even though :c:type:`PyCFunction` defines the first parameter as
270:c:expr:`PyObject*`, it is common that the method implementation uses the
271specific C type of the *self* object.
272
273The :c:member:`ml_flags` field is a bitfield which can include the following flags.
274The individual flags indicate either a calling convention or a binding
275convention.
276
277There are these calling conventions:
278
279.. data:: METH_VARARGS
280
281   This is the typical calling convention, where the methods have the type
282   :c:type:`PyCFunction`. The function expects two :c:expr:`PyObject*` values.
283   The first one is the *self* object for methods; for module functions, it is
284   the module object.  The second parameter (often called *args*) is a tuple
285   object representing all arguments. This parameter is typically processed
286   using :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_UnpackTuple`.
287
288
289.. data:: METH_VARARGS | METH_KEYWORDS
290
291   Methods with these flags must be of type :c:type:`PyCFunctionWithKeywords`.
292   The function expects three parameters: *self*, *args*, *kwargs* where
293   *kwargs* is a dictionary of all the keyword arguments or possibly ``NULL``
294   if there are no keyword arguments.  The parameters are typically processed
295   using :c:func:`PyArg_ParseTupleAndKeywords`.
296
297
298.. data:: METH_FASTCALL
299
300   Fast calling convention supporting only positional arguments.
301   The methods have the type :c:type:`_PyCFunctionFast`.
302   The first parameter is *self*, the second parameter is a C array
303   of :c:expr:`PyObject*` values indicating the arguments and the third
304   parameter is the number of arguments (the length of the array).
305
306   .. versionadded:: 3.7
307
308   .. versionchanged:: 3.10
309
310      ``METH_FASTCALL`` is now part of the stable ABI.
311
312
313.. data:: METH_FASTCALL | METH_KEYWORDS
314
315   Extension of :const:`METH_FASTCALL` supporting also keyword arguments,
316   with methods of type :c:type:`_PyCFunctionFastWithKeywords`.
317   Keyword arguments are passed the same way as in the
318   :ref:`vectorcall protocol <vectorcall>`:
319   there is an additional fourth :c:expr:`PyObject*` parameter
320   which is a tuple representing the names of the keyword arguments
321   (which are guaranteed to be strings)
322   or possibly ``NULL`` if there are no keywords.  The values of the keyword
323   arguments are stored in the *args* array, after the positional arguments.
324
325   .. versionadded:: 3.7
326
327
328.. data:: METH_METHOD | METH_FASTCALL | METH_KEYWORDS
329
330   Extension of :const:`METH_FASTCALL | METH_KEYWORDS` supporting the *defining
331   class*, that is, the class that contains the method in question.
332   The defining class might be a superclass of ``Py_TYPE(self)``.
333
334   The method needs to be of type :c:type:`PyCMethod`, the same as for
335   ``METH_FASTCALL | METH_KEYWORDS`` with ``defining_class`` argument added after
336   ``self``.
337
338   .. versionadded:: 3.9
339
340
341.. data:: METH_NOARGS
342
343   Methods without parameters don't need to check whether arguments are given if
344   they are listed with the :const:`METH_NOARGS` flag.  They need to be of type
345   :c:type:`PyCFunction`.  The first parameter is typically named *self* and will
346   hold a reference to the module or object instance.  In all cases the second
347   parameter will be ``NULL``.
348
349   The function must have 2 parameters. Since the second parameter is unused,
350   :c:macro:`Py_UNUSED` can be used to prevent a compiler warning.
351
352
353.. data:: METH_O
354
355   Methods with a single object argument can be listed with the :const:`METH_O`
356   flag, instead of invoking :c:func:`PyArg_ParseTuple` with a ``"O"`` argument.
357   They have the type :c:type:`PyCFunction`, with the *self* parameter, and a
358   :c:expr:`PyObject*` parameter representing the single argument.
359
360
361These two constants are not used to indicate the calling convention but the
362binding when use with methods of classes.  These may not be used for functions
363defined for modules.  At most one of these flags may be set for any given
364method.
365
366
367.. data:: METH_CLASS
368
369   .. index:: pair: built-in function; classmethod
370
371   The method will be passed the type object as the first parameter rather
372   than an instance of the type.  This is used to create *class methods*,
373   similar to what is created when using the :func:`classmethod` built-in
374   function.
375
376
377.. data:: METH_STATIC
378
379   .. index:: pair: built-in function; staticmethod
380
381   The method will be passed ``NULL`` as the first parameter rather than an
382   instance of the type.  This is used to create *static methods*, similar to
383   what is created when using the :func:`staticmethod` built-in function.
384
385One other constant controls whether a method is loaded in place of another
386definition with the same method name.
387
388
389.. data:: METH_COEXIST
390
391   The method will be loaded in place of existing definitions.  Without
392   *METH_COEXIST*, the default is to skip repeated definitions.  Since slot
393   wrappers are loaded before the method table, the existence of a
394   *sq_contains* slot, for example, would generate a wrapped method named
395   :meth:`__contains__` and preclude the loading of a corresponding
396   PyCFunction with the same name.  With the flag defined, the PyCFunction
397   will be loaded in place of the wrapper object and will co-exist with the
398   slot.  This is helpful because calls to PyCFunctions are optimized more
399   than wrapper object calls.
400
401
402Accessing attributes of extension types
403---------------------------------------
404
405.. c:type:: PyMemberDef
406
407   Structure which describes an attribute of a type which corresponds to a C
408   struct member.  Its fields are:
409
410   +------------------+---------------+-------------------------------+
411   | Field            | C Type        | Meaning                       |
412   +==================+===============+===============================+
413   | :attr:`name`     | const char \* | name of the member            |
414   +------------------+---------------+-------------------------------+
415   | :attr:`!type`    | int           | the type of the member in the |
416   |                  |               | C struct                      |
417   +------------------+---------------+-------------------------------+
418   | :attr:`offset`   | Py_ssize_t    | the offset in bytes that the  |
419   |                  |               | member is located on the      |
420   |                  |               | type's object struct          |
421   +------------------+---------------+-------------------------------+
422   | :attr:`flags`    | int           | flag bits indicating if the   |
423   |                  |               | field should be read-only or  |
424   |                  |               | writable                      |
425   +------------------+---------------+-------------------------------+
426   | :attr:`doc`      | const char \* | points to the contents of the |
427   |                  |               | docstring                     |
428   +------------------+---------------+-------------------------------+
429
430   :attr:`!type` can be one of many ``T_`` macros corresponding to various C
431   types.  When the member is accessed in Python, it will be converted to the
432   equivalent Python type.
433
434   =============== ==================
435   Macro name      C type
436   =============== ==================
437   T_SHORT         short
438   T_INT           int
439   T_LONG          long
440   T_FLOAT         float
441   T_DOUBLE        double
442   T_STRING        const char \*
443   T_OBJECT        PyObject \*
444   T_OBJECT_EX     PyObject \*
445   T_CHAR          char
446   T_BYTE          char
447   T_UBYTE         unsigned char
448   T_UINT          unsigned int
449   T_USHORT        unsigned short
450   T_ULONG         unsigned long
451   T_BOOL          char
452   T_LONGLONG      long long
453   T_ULONGLONG     unsigned long long
454   T_PYSSIZET      Py_ssize_t
455   =============== ==================
456
457   :c:macro:`T_OBJECT` and :c:macro:`T_OBJECT_EX` differ in that
458   :c:macro:`T_OBJECT` returns ``None`` if the member is ``NULL`` and
459   :c:macro:`T_OBJECT_EX` raises an :exc:`AttributeError`.  Try to use
460   :c:macro:`T_OBJECT_EX` over :c:macro:`T_OBJECT` because :c:macro:`T_OBJECT_EX`
461   handles use of the :keyword:`del` statement on that attribute more correctly
462   than :c:macro:`T_OBJECT`.
463
464   :attr:`flags` can be ``0`` for write and read access or :c:macro:`READONLY` for
465   read-only access.  Using :c:macro:`T_STRING` for :attr:`type` implies
466   :c:macro:`READONLY`.  :c:macro:`T_STRING` data is interpreted as UTF-8.
467   Only :c:macro:`T_OBJECT` and :c:macro:`T_OBJECT_EX`
468   members can be deleted.  (They are set to ``NULL``).
469
470   .. _pymemberdef-offsets:
471
472   Heap allocated types (created using :c:func:`PyType_FromSpec` or similar),
473   ``PyMemberDef`` may contain definitions for the special members
474   ``__dictoffset__``, ``__weaklistoffset__`` and ``__vectorcalloffset__``,
475   corresponding to
476   :c:member:`~PyTypeObject.tp_dictoffset`,
477   :c:member:`~PyTypeObject.tp_weaklistoffset` and
478   :c:member:`~PyTypeObject.tp_vectorcall_offset` in type objects.
479   These must be defined with ``T_PYSSIZET`` and ``READONLY``, for example::
480
481      static PyMemberDef spam_type_members[] = {
482          {"__dictoffset__", T_PYSSIZET, offsetof(Spam_object, dict), READONLY},
483          {NULL}  /* Sentinel */
484      };
485
486
487.. c:function:: PyObject* PyMember_GetOne(const char *obj_addr, struct PyMemberDef *m)
488
489   Get an attribute belonging to the object at address *obj_addr*.  The
490   attribute is described by ``PyMemberDef`` *m*.  Returns ``NULL``
491   on error.
492
493
494.. c:function:: int PyMember_SetOne(char *obj_addr, struct PyMemberDef *m, PyObject *o)
495
496   Set an attribute belonging to the object at address *obj_addr* to object *o*.
497   The attribute to set is described by ``PyMemberDef`` *m*.  Returns ``0``
498   if successful and a negative value on failure.
499
500
501.. c:type:: PyGetSetDef
502
503   Structure to define property-like access for a type. See also description of
504   the :c:member:`PyTypeObject.tp_getset` slot.
505
506   +-------------+------------------+-----------------------------------+
507   | Field       | C Type           | Meaning                           |
508   +=============+==================+===================================+
509   | name        | const char \*    | attribute name                    |
510   +-------------+------------------+-----------------------------------+
511   | get         | getter           | C function to get the attribute   |
512   +-------------+------------------+-----------------------------------+
513   | set         | setter           | optional C function to set or     |
514   |             |                  | delete the attribute, if omitted  |
515   |             |                  | the attribute is readonly         |
516   +-------------+------------------+-----------------------------------+
517   | doc         | const char \*    | optional docstring                |
518   +-------------+------------------+-----------------------------------+
519   | closure     | void \*          | optional function pointer,        |
520   |             |                  | providing additional data for     |
521   |             |                  | getter and setter                 |
522   +-------------+------------------+-----------------------------------+
523
524   The ``get`` function takes one :c:expr:`PyObject*` parameter (the
525   instance) and a function pointer (the associated ``closure``)::
526
527      typedef PyObject *(*getter)(PyObject *, void *);
528
529   It should return a new reference on success or ``NULL`` with a set exception
530   on failure.
531
532   ``set`` functions take two :c:expr:`PyObject*` parameters (the instance and
533   the value to be set) and a function pointer (the associated ``closure``)::
534
535      typedef int (*setter)(PyObject *, PyObject *, void *);
536
537   In case the attribute should be deleted the second parameter is ``NULL``.
538   Should return ``0`` on success or ``-1`` with a set exception on failure.
539