1.. highlight:: c 2 3 4.. _initialization: 5 6***************************************** 7Initialization, Finalization, and Threads 8***************************************** 9 10See also :ref:`Python Initialization Configuration <init-config>`. 11 12.. _pre-init-safe: 13 14Before Python Initialization 15============================ 16 17In an application embedding Python, the :c:func:`Py_Initialize` function must 18be called before using any other Python/C API functions; with the exception of 19a few functions and the :ref:`global configuration variables 20<global-conf-vars>`. 21 22The following functions can be safely called before Python is initialized: 23 24* Configuration functions: 25 26 * :c:func:`PyImport_AppendInittab` 27 * :c:func:`PyImport_ExtendInittab` 28 * :c:func:`PyInitFrozenExtensions` 29 * :c:func:`PyMem_SetAllocator` 30 * :c:func:`PyMem_SetupDebugHooks` 31 * :c:func:`PyObject_SetArenaAllocator` 32 * :c:func:`Py_SetPath` 33 * :c:func:`Py_SetProgramName` 34 * :c:func:`Py_SetPythonHome` 35 * :c:func:`Py_SetStandardStreamEncoding` 36 * :c:func:`PySys_AddWarnOption` 37 * :c:func:`PySys_AddXOption` 38 * :c:func:`PySys_ResetWarnOptions` 39 40* Informative functions: 41 42 * :c:func:`Py_IsInitialized` 43 * :c:func:`PyMem_GetAllocator` 44 * :c:func:`PyObject_GetArenaAllocator` 45 * :c:func:`Py_GetBuildInfo` 46 * :c:func:`Py_GetCompiler` 47 * :c:func:`Py_GetCopyright` 48 * :c:func:`Py_GetPlatform` 49 * :c:func:`Py_GetVersion` 50 51* Utilities: 52 53 * :c:func:`Py_DecodeLocale` 54 55* Memory allocators: 56 57 * :c:func:`PyMem_RawMalloc` 58 * :c:func:`PyMem_RawRealloc` 59 * :c:func:`PyMem_RawCalloc` 60 * :c:func:`PyMem_RawFree` 61 62.. note:: 63 64 The following functions **should not be called** before 65 :c:func:`Py_Initialize`: :c:func:`Py_EncodeLocale`, :c:func:`Py_GetPath`, 66 :c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix`, 67 :c:func:`Py_GetProgramFullPath`, :c:func:`Py_GetPythonHome`, 68 :c:func:`Py_GetProgramName` and :c:func:`PyEval_InitThreads`. 69 70 71.. _global-conf-vars: 72 73Global configuration variables 74============================== 75 76Python has variables for the global configuration to control different features 77and options. By default, these flags are controlled by :ref:`command line 78options <using-on-interface-options>`. 79 80When a flag is set by an option, the value of the flag is the number of times 81that the option was set. For example, ``-b`` sets :c:data:`Py_BytesWarningFlag` 82to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. 83 84.. c:var:: int Py_BytesWarningFlag 85 86 Issue a warning when comparing :class:`bytes` or :class:`bytearray` with 87 :class:`str` or :class:`bytes` with :class:`int`. Issue an error if greater 88 or equal to ``2``. 89 90 Set by the :option:`-b` option. 91 92.. c:var:: int Py_DebugFlag 93 94 Turn on parser debugging output (for expert only, depending on compilation 95 options). 96 97 Set by the :option:`-d` option and the :envvar:`PYTHONDEBUG` environment 98 variable. 99 100.. c:var:: int Py_DontWriteBytecodeFlag 101 102 If set to non-zero, Python won't try to write ``.pyc`` files on the 103 import of source modules. 104 105 Set by the :option:`-B` option and the :envvar:`PYTHONDONTWRITEBYTECODE` 106 environment variable. 107 108.. c:var:: int Py_FrozenFlag 109 110 Suppress error messages when calculating the module search path in 111 :c:func:`Py_GetPath`. 112 113 Private flag used by ``_freeze_module`` and ``frozenmain`` programs. 114 115.. c:var:: int Py_HashRandomizationFlag 116 117 Set to ``1`` if the :envvar:`PYTHONHASHSEED` environment variable is set to 118 a non-empty string. 119 120 If the flag is non-zero, read the :envvar:`PYTHONHASHSEED` environment 121 variable to initialize the secret hash seed. 122 123.. c:var:: int Py_IgnoreEnvironmentFlag 124 125 Ignore all :envvar:`PYTHON*` environment variables, e.g. 126 :envvar:`PYTHONPATH` and :envvar:`PYTHONHOME`, that might be set. 127 128 Set by the :option:`-E` and :option:`-I` options. 129 130.. c:var:: int Py_InspectFlag 131 132 When a script is passed as first argument or the :option:`-c` option is used, 133 enter interactive mode after executing the script or the command, even when 134 :data:`sys.stdin` does not appear to be a terminal. 135 136 Set by the :option:`-i` option and the :envvar:`PYTHONINSPECT` environment 137 variable. 138 139.. c:var:: int Py_InteractiveFlag 140 141 Set by the :option:`-i` option. 142 143.. c:var:: int Py_IsolatedFlag 144 145 Run Python in isolated mode. In isolated mode :data:`sys.path` contains 146 neither the script's directory nor the user's site-packages directory. 147 148 Set by the :option:`-I` option. 149 150 .. versionadded:: 3.4 151 152.. c:var:: int Py_LegacyWindowsFSEncodingFlag 153 154 If the flag is non-zero, use the ``mbcs`` encoding with ``replace`` error 155 handler, instead of the UTF-8 encoding with ``surrogatepass`` error handler, 156 for the :term:`filesystem encoding and error handler`. 157 158 Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSFSENCODING` environment 159 variable is set to a non-empty string. 160 161 See :pep:`529` for more details. 162 163 .. availability:: Windows. 164 165.. c:var:: int Py_LegacyWindowsStdioFlag 166 167 If the flag is non-zero, use :class:`io.FileIO` instead of 168 :class:`WindowsConsoleIO` for :mod:`sys` standard streams. 169 170 Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSSTDIO` environment 171 variable is set to a non-empty string. 172 173 See :pep:`528` for more details. 174 175 .. availability:: Windows. 176 177.. c:var:: int Py_NoSiteFlag 178 179 Disable the import of the module :mod:`site` and the site-dependent 180 manipulations of :data:`sys.path` that it entails. Also disable these 181 manipulations if :mod:`site` is explicitly imported later (call 182 :func:`site.main` if you want them to be triggered). 183 184 Set by the :option:`-S` option. 185 186.. c:var:: int Py_NoUserSiteDirectory 187 188 Don't add the :data:`user site-packages directory <site.USER_SITE>` to 189 :data:`sys.path`. 190 191 Set by the :option:`-s` and :option:`-I` options, and the 192 :envvar:`PYTHONNOUSERSITE` environment variable. 193 194.. c:var:: int Py_OptimizeFlag 195 196 Set by the :option:`-O` option and the :envvar:`PYTHONOPTIMIZE` environment 197 variable. 198 199.. c:var:: int Py_QuietFlag 200 201 Don't display the copyright and version messages even in interactive mode. 202 203 Set by the :option:`-q` option. 204 205 .. versionadded:: 3.2 206 207.. c:var:: int Py_UnbufferedStdioFlag 208 209 Force the stdout and stderr streams to be unbuffered. 210 211 Set by the :option:`-u` option and the :envvar:`PYTHONUNBUFFERED` 212 environment variable. 213 214.. c:var:: int Py_VerboseFlag 215 216 Print a message each time a module is initialized, showing the place 217 (filename or built-in module) from which it is loaded. If greater or equal 218 to ``2``, print a message for each file that is checked for when 219 searching for a module. Also provides information on module cleanup at exit. 220 221 Set by the :option:`-v` option and the :envvar:`PYTHONVERBOSE` environment 222 variable. 223 224 225Initializing and finalizing the interpreter 226=========================================== 227 228 229.. c:function:: void Py_Initialize() 230 231 .. index:: 232 single: Py_SetProgramName() 233 single: PyEval_InitThreads() 234 single: modules (in module sys) 235 single: path (in module sys) 236 pair: module; builtins 237 pair: module; __main__ 238 pair: module; sys 239 triple: module; search; path 240 single: PySys_SetArgv() 241 single: PySys_SetArgvEx() 242 single: Py_FinalizeEx() 243 244 Initialize the Python interpreter. In an application embedding Python, 245 this should be called before using any other Python/C API functions; see 246 :ref:`Before Python Initialization <pre-init-safe>` for the few exceptions. 247 248 This initializes 249 the table of loaded modules (``sys.modules``), and creates the fundamental 250 modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`. It also initializes 251 the module search path (``sys.path``). It does not set ``sys.argv``; use 252 :c:func:`PySys_SetArgvEx` for that. This is a no-op when called for a second time 253 (without calling :c:func:`Py_FinalizeEx` first). There is no return value; it is a 254 fatal error if the initialization fails. 255 256 .. note:: 257 On Windows, changes the console mode from ``O_TEXT`` to ``O_BINARY``, which will 258 also affect non-Python uses of the console using the C Runtime. 259 260 261.. c:function:: void Py_InitializeEx(int initsigs) 262 263 This function works like :c:func:`Py_Initialize` if *initsigs* is ``1``. If 264 *initsigs* is ``0``, it skips initialization registration of signal handlers, which 265 might be useful when Python is embedded. 266 267 268.. c:function:: int Py_IsInitialized() 269 270 Return true (nonzero) when the Python interpreter has been initialized, false 271 (zero) if not. After :c:func:`Py_FinalizeEx` is called, this returns false until 272 :c:func:`Py_Initialize` is called again. 273 274 275.. c:function:: int Py_FinalizeEx() 276 277 Undo all initializations made by :c:func:`Py_Initialize` and subsequent use of 278 Python/C API functions, and destroy all sub-interpreters (see 279 :c:func:`Py_NewInterpreter` below) that were created and not yet destroyed since 280 the last call to :c:func:`Py_Initialize`. Ideally, this frees all memory 281 allocated by the Python interpreter. This is a no-op when called for a second 282 time (without calling :c:func:`Py_Initialize` again first). Normally the 283 return value is ``0``. If there were errors during finalization 284 (flushing buffered data), ``-1`` is returned. 285 286 This function is provided for a number of reasons. An embedding application 287 might want to restart Python without having to restart the application itself. 288 An application that has loaded the Python interpreter from a dynamically 289 loadable library (or DLL) might want to free all memory allocated by Python 290 before unloading the DLL. During a hunt for memory leaks in an application a 291 developer might want to free all memory allocated by Python before exiting from 292 the application. 293 294 **Bugs and caveats:** The destruction of modules and objects in modules is done 295 in random order; this may cause destructors (:meth:`__del__` methods) to fail 296 when they depend on other objects (even functions) or modules. Dynamically 297 loaded extension modules loaded by Python are not unloaded. Small amounts of 298 memory allocated by the Python interpreter may not be freed (if you find a leak, 299 please report it). Memory tied up in circular references between objects is not 300 freed. Some memory allocated by extension modules may not be freed. Some 301 extensions may not work properly if their initialization routine is called more 302 than once; this can happen if an application calls :c:func:`Py_Initialize` and 303 :c:func:`Py_FinalizeEx` more than once. 304 305 .. audit-event:: cpython._PySys_ClearAuditHooks "" c.Py_FinalizeEx 306 307 .. versionadded:: 3.6 308 309.. c:function:: void Py_Finalize() 310 311 This is a backwards-compatible version of :c:func:`Py_FinalizeEx` that 312 disregards the return value. 313 314 315Process-wide parameters 316======================= 317 318 319.. c:function:: int Py_SetStandardStreamEncoding(const char *encoding, const char *errors) 320 321 .. index:: 322 single: Py_Initialize() 323 single: main() 324 triple: stdin; stdout; sdterr 325 326 This API is kept for backward compatibility: setting 327 :c:member:`PyConfig.stdio_encoding` and :c:member:`PyConfig.stdio_errors` 328 should be used instead, see :ref:`Python Initialization Configuration 329 <init-config>`. 330 331 This function should be called before :c:func:`Py_Initialize`, if it is 332 called at all. It specifies which encoding and error handling to use 333 with standard IO, with the same meanings as in :func:`str.encode`. 334 335 It overrides :envvar:`PYTHONIOENCODING` values, and allows embedding code 336 to control IO encoding when the environment variable does not work. 337 338 *encoding* and/or *errors* may be ``NULL`` to use 339 :envvar:`PYTHONIOENCODING` and/or default values (depending on other 340 settings). 341 342 Note that :data:`sys.stderr` always uses the "backslashreplace" error 343 handler, regardless of this (or any other) setting. 344 345 If :c:func:`Py_FinalizeEx` is called, this function will need to be called 346 again in order to affect subsequent calls to :c:func:`Py_Initialize`. 347 348 Returns ``0`` if successful, a nonzero value on error (e.g. calling after the 349 interpreter has already been initialized). 350 351 .. versionadded:: 3.4 352 353 .. deprecated:: 3.11 354 355 356.. c:function:: void Py_SetProgramName(const wchar_t *name) 357 358 .. index:: 359 single: Py_Initialize() 360 single: main() 361 single: Py_GetPath() 362 363 This API is kept for backward compatibility: setting 364 :c:member:`PyConfig.program_name` should be used instead, see :ref:`Python 365 Initialization Configuration <init-config>`. 366 367 This function should be called before :c:func:`Py_Initialize` is called for 368 the first time, if it is called at all. It tells the interpreter the value 369 of the ``argv[0]`` argument to the :c:func:`main` function of the program 370 (converted to wide characters). 371 This is used by :c:func:`Py_GetPath` and some other functions below to find 372 the Python run-time libraries relative to the interpreter executable. The 373 default value is ``'python'``. The argument should point to a 374 zero-terminated wide character string in static storage whose contents will not 375 change for the duration of the program's execution. No code in the Python 376 interpreter will change the contents of this storage. 377 378 Use :c:func:`Py_DecodeLocale` to decode a bytes string to get a 379 :c:expr:`wchar_*` string. 380 381 .. deprecated:: 3.11 382 383 384.. c:function:: wchar* Py_GetProgramName() 385 386 .. index:: single: Py_SetProgramName() 387 388 Return the program name set with :c:func:`Py_SetProgramName`, or the default. 389 The returned string points into static storage; the caller should not modify its 390 value. 391 392 This function should not be called before :c:func:`Py_Initialize`, otherwise 393 it returns ``NULL``. 394 395 .. versionchanged:: 3.10 396 It now returns ``NULL`` if called before :c:func:`Py_Initialize`. 397 398 399.. c:function:: wchar_t* Py_GetPrefix() 400 401 Return the *prefix* for installed platform-independent files. This is derived 402 through a number of complicated rules from the program name set with 403 :c:func:`Py_SetProgramName` and some environment variables; for example, if the 404 program name is ``'/usr/local/bin/python'``, the prefix is ``'/usr/local'``. The 405 returned string points into static storage; the caller should not modify its 406 value. This corresponds to the :makevar:`prefix` variable in the top-level 407 :file:`Makefile` and the :option:`--prefix` argument to the :program:`configure` 408 script at build time. The value is available to Python code as ``sys.prefix``. 409 It is only useful on Unix. See also the next function. 410 411 This function should not be called before :c:func:`Py_Initialize`, otherwise 412 it returns ``NULL``. 413 414 .. versionchanged:: 3.10 415 It now returns ``NULL`` if called before :c:func:`Py_Initialize`. 416 417 418.. c:function:: wchar_t* Py_GetExecPrefix() 419 420 Return the *exec-prefix* for installed platform-*dependent* files. This is 421 derived through a number of complicated rules from the program name set with 422 :c:func:`Py_SetProgramName` and some environment variables; for example, if the 423 program name is ``'/usr/local/bin/python'``, the exec-prefix is 424 ``'/usr/local'``. The returned string points into static storage; the caller 425 should not modify its value. This corresponds to the :makevar:`exec_prefix` 426 variable in the top-level :file:`Makefile` and the ``--exec-prefix`` 427 argument to the :program:`configure` script at build time. The value is 428 available to Python code as ``sys.exec_prefix``. It is only useful on Unix. 429 430 Background: The exec-prefix differs from the prefix when platform dependent 431 files (such as executables and shared libraries) are installed in a different 432 directory tree. In a typical installation, platform dependent files may be 433 installed in the :file:`/usr/local/plat` subtree while platform independent may 434 be installed in :file:`/usr/local`. 435 436 Generally speaking, a platform is a combination of hardware and software 437 families, e.g. Sparc machines running the Solaris 2.x operating system are 438 considered the same platform, but Intel machines running Solaris 2.x are another 439 platform, and Intel machines running Linux are yet another platform. Different 440 major revisions of the same operating system generally also form different 441 platforms. Non-Unix operating systems are a different story; the installation 442 strategies on those systems are so different that the prefix and exec-prefix are 443 meaningless, and set to the empty string. Note that compiled Python bytecode 444 files are platform independent (but not independent from the Python version by 445 which they were compiled!). 446 447 System administrators will know how to configure the :program:`mount` or 448 :program:`automount` programs to share :file:`/usr/local` between platforms 449 while having :file:`/usr/local/plat` be a different filesystem for each 450 platform. 451 452 This function should not be called before :c:func:`Py_Initialize`, otherwise 453 it returns ``NULL``. 454 455 .. versionchanged:: 3.10 456 It now returns ``NULL`` if called before :c:func:`Py_Initialize`. 457 458 459.. c:function:: wchar_t* Py_GetProgramFullPath() 460 461 .. index:: 462 single: Py_SetProgramName() 463 single: executable (in module sys) 464 465 Return the full program name of the Python executable; this is computed as a 466 side-effect of deriving the default module search path from the program name 467 (set by :c:func:`Py_SetProgramName` above). The returned string points into 468 static storage; the caller should not modify its value. The value is available 469 to Python code as ``sys.executable``. 470 471 This function should not be called before :c:func:`Py_Initialize`, otherwise 472 it returns ``NULL``. 473 474 .. versionchanged:: 3.10 475 It now returns ``NULL`` if called before :c:func:`Py_Initialize`. 476 477 478.. c:function:: wchar_t* Py_GetPath() 479 480 .. index:: 481 triple: module; search; path 482 single: path (in module sys) 483 single: Py_SetPath() 484 485 Return the default module search path; this is computed from the program name 486 (set by :c:func:`Py_SetProgramName` above) and some environment variables. 487 The returned string consists of a series of directory names separated by a 488 platform dependent delimiter character. The delimiter character is ``':'`` 489 on Unix and macOS, ``';'`` on Windows. The returned string points into 490 static storage; the caller should not modify its value. The list 491 :data:`sys.path` is initialized with this value on interpreter startup; it 492 can be (and usually is) modified later to change the search path for loading 493 modules. 494 495 This function should not be called before :c:func:`Py_Initialize`, otherwise 496 it returns ``NULL``. 497 498 .. XXX should give the exact rules 499 500 .. versionchanged:: 3.10 501 It now returns ``NULL`` if called before :c:func:`Py_Initialize`. 502 503 504.. c:function:: void Py_SetPath(const wchar_t *) 505 506 .. index:: 507 triple: module; search; path 508 single: path (in module sys) 509 single: Py_GetPath() 510 511 This API is kept for backward compatibility: setting 512 :c:member:`PyConfig.module_search_paths` and 513 :c:member:`PyConfig.module_search_paths_set` should be used instead, see 514 :ref:`Python Initialization Configuration <init-config>`. 515 516 Set the default module search path. If this function is called before 517 :c:func:`Py_Initialize`, then :c:func:`Py_GetPath` won't attempt to compute a 518 default search path but uses the one provided instead. This is useful if 519 Python is embedded by an application that has full knowledge of the location 520 of all modules. The path components should be separated by the platform 521 dependent delimiter character, which is ``':'`` on Unix and macOS, ``';'`` 522 on Windows. 523 524 This also causes :data:`sys.executable` to be set to the program 525 full path (see :c:func:`Py_GetProgramFullPath`) and for :data:`sys.prefix` and 526 :data:`sys.exec_prefix` to be empty. It is up to the caller to modify these 527 if required after calling :c:func:`Py_Initialize`. 528 529 Use :c:func:`Py_DecodeLocale` to decode a bytes string to get a 530 :c:expr:`wchar_*` string. 531 532 The path argument is copied internally, so the caller may free it after the 533 call completes. 534 535 .. versionchanged:: 3.8 536 The program full path is now used for :data:`sys.executable`, instead 537 of the program name. 538 539 .. deprecated:: 3.11 540 541 542.. c:function:: const char* Py_GetVersion() 543 544 Return the version of this Python interpreter. This is a string that looks 545 something like :: 546 547 "3.0a5+ (py3k:63103M, May 12 2008, 00:53:55) \n[GCC 4.2.3]" 548 549 .. index:: single: version (in module sys) 550 551 The first word (up to the first space character) is the current Python version; 552 the first characters are the major and minor version separated by a 553 period. The returned string points into static storage; the caller should not 554 modify its value. The value is available to Python code as :data:`sys.version`. 555 556 See also the :c:var:`Py_Version` constant. 557 558 559.. c:function:: const char* Py_GetPlatform() 560 561 .. index:: single: platform (in module sys) 562 563 Return the platform identifier for the current platform. On Unix, this is 564 formed from the "official" name of the operating system, converted to lower 565 case, followed by the major revision number; e.g., for Solaris 2.x, which is 566 also known as SunOS 5.x, the value is ``'sunos5'``. On macOS, it is 567 ``'darwin'``. On Windows, it is ``'win'``. The returned string points into 568 static storage; the caller should not modify its value. The value is available 569 to Python code as ``sys.platform``. 570 571 572.. c:function:: const char* Py_GetCopyright() 573 574 Return the official copyright string for the current Python version, for example 575 576 ``'Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam'`` 577 578 .. index:: single: copyright (in module sys) 579 580 The returned string points into static storage; the caller should not modify its 581 value. The value is available to Python code as ``sys.copyright``. 582 583 584.. c:function:: const char* Py_GetCompiler() 585 586 Return an indication of the compiler used to build the current Python version, 587 in square brackets, for example:: 588 589 "[GCC 2.7.2.2]" 590 591 .. index:: single: version (in module sys) 592 593 The returned string points into static storage; the caller should not modify its 594 value. The value is available to Python code as part of the variable 595 ``sys.version``. 596 597 598.. c:function:: const char* Py_GetBuildInfo() 599 600 Return information about the sequence number and build date and time of the 601 current Python interpreter instance, for example :: 602 603 "#67, Aug 1 1997, 22:34:28" 604 605 .. index:: single: version (in module sys) 606 607 The returned string points into static storage; the caller should not modify its 608 value. The value is available to Python code as part of the variable 609 ``sys.version``. 610 611 612.. c:function:: void PySys_SetArgvEx(int argc, wchar_t **argv, int updatepath) 613 614 .. index:: 615 single: main() 616 single: Py_FatalError() 617 single: argv (in module sys) 618 619 This API is kept for backward compatibility: setting 620 :c:member:`PyConfig.argv`, :c:member:`PyConfig.parse_argv` and 621 :c:member:`PyConfig.safe_path` should be used instead, see :ref:`Python 622 Initialization Configuration <init-config>`. 623 624 Set :data:`sys.argv` based on *argc* and *argv*. These parameters are 625 similar to those passed to the program's :c:func:`main` function with the 626 difference that the first entry should refer to the script file to be 627 executed rather than the executable hosting the Python interpreter. If there 628 isn't a script that will be run, the first entry in *argv* can be an empty 629 string. If this function fails to initialize :data:`sys.argv`, a fatal 630 condition is signalled using :c:func:`Py_FatalError`. 631 632 If *updatepath* is zero, this is all the function does. If *updatepath* 633 is non-zero, the function also modifies :data:`sys.path` according to the 634 following algorithm: 635 636 - If the name of an existing script is passed in ``argv[0]``, the absolute 637 path of the directory where the script is located is prepended to 638 :data:`sys.path`. 639 - Otherwise (that is, if *argc* is ``0`` or ``argv[0]`` doesn't point 640 to an existing file name), an empty string is prepended to 641 :data:`sys.path`, which is the same as prepending the current working 642 directory (``"."``). 643 644 Use :c:func:`Py_DecodeLocale` to decode a bytes string to get a 645 :c:expr:`wchar_*` string. 646 647 See also :c:member:`PyConfig.orig_argv` and :c:member:`PyConfig.argv` 648 members of the :ref:`Python Initialization Configuration <init-config>`. 649 650 .. note:: 651 It is recommended that applications embedding the Python interpreter 652 for purposes other than executing a single script pass ``0`` as *updatepath*, 653 and update :data:`sys.path` themselves if desired. 654 See `CVE-2008-5983 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-5983>`_. 655 656 On versions before 3.1.3, you can achieve the same effect by manually 657 popping the first :data:`sys.path` element after having called 658 :c:func:`PySys_SetArgv`, for example using:: 659 660 PyRun_SimpleString("import sys; sys.path.pop(0)\n"); 661 662 .. versionadded:: 3.1.3 663 664 .. XXX impl. doesn't seem consistent in allowing ``0``/``NULL`` for the params; 665 check w/ Guido. 666 667 .. deprecated:: 3.11 668 669 670.. c:function:: void PySys_SetArgv(int argc, wchar_t **argv) 671 672 This API is kept for backward compatibility: setting 673 :c:member:`PyConfig.argv` and :c:member:`PyConfig.parse_argv` should be used 674 instead, see :ref:`Python Initialization Configuration <init-config>`. 675 676 This function works like :c:func:`PySys_SetArgvEx` with *updatepath* set 677 to ``1`` unless the :program:`python` interpreter was started with the 678 :option:`-I`. 679 680 Use :c:func:`Py_DecodeLocale` to decode a bytes string to get a 681 :c:expr:`wchar_*` string. 682 683 See also :c:member:`PyConfig.orig_argv` and :c:member:`PyConfig.argv` 684 members of the :ref:`Python Initialization Configuration <init-config>`. 685 686 .. versionchanged:: 3.4 The *updatepath* value depends on :option:`-I`. 687 688 .. deprecated:: 3.11 689 690 691.. c:function:: void Py_SetPythonHome(const wchar_t *home) 692 693 This API is kept for backward compatibility: setting 694 :c:member:`PyConfig.home` should be used instead, see :ref:`Python 695 Initialization Configuration <init-config>`. 696 697 Set the default "home" directory, that is, the location of the standard 698 Python libraries. See :envvar:`PYTHONHOME` for the meaning of the 699 argument string. 700 701 The argument should point to a zero-terminated character string in static 702 storage whose contents will not change for the duration of the program's 703 execution. No code in the Python interpreter will change the contents of 704 this storage. 705 706 Use :c:func:`Py_DecodeLocale` to decode a bytes string to get a 707 :c:expr:`wchar_*` string. 708 709 .. deprecated:: 3.11 710 711 712.. c:function:: wchar_t* Py_GetPythonHome() 713 714 Return the default "home", that is, the value set by a previous call to 715 :c:func:`Py_SetPythonHome`, or the value of the :envvar:`PYTHONHOME` 716 environment variable if it is set. 717 718 This function should not be called before :c:func:`Py_Initialize`, otherwise 719 it returns ``NULL``. 720 721 .. versionchanged:: 3.10 722 It now returns ``NULL`` if called before :c:func:`Py_Initialize`. 723 724 725.. _threads: 726 727Thread State and the Global Interpreter Lock 728============================================ 729 730.. index:: 731 single: global interpreter lock 732 single: interpreter lock 733 single: lock, interpreter 734 735The Python interpreter is not fully thread-safe. In order to support 736multi-threaded Python programs, there's a global lock, called the :term:`global 737interpreter lock` or :term:`GIL`, that must be held by the current thread before 738it can safely access Python objects. Without the lock, even the simplest 739operations could cause problems in a multi-threaded program: for example, when 740two threads simultaneously increment the reference count of the same object, the 741reference count could end up being incremented only once instead of twice. 742 743.. index:: single: setswitchinterval() (in module sys) 744 745Therefore, the rule exists that only the thread that has acquired the 746:term:`GIL` may operate on Python objects or call Python/C API functions. 747In order to emulate concurrency of execution, the interpreter regularly 748tries to switch threads (see :func:`sys.setswitchinterval`). The lock is also 749released around potentially blocking I/O operations like reading or writing 750a file, so that other Python threads can run in the meantime. 751 752.. index:: 753 single: PyThreadState 754 single: PyThreadState 755 756The Python interpreter keeps some thread-specific bookkeeping information 757inside a data structure called :c:type:`PyThreadState`. There's also one 758global variable pointing to the current :c:type:`PyThreadState`: it can 759be retrieved using :c:func:`PyThreadState_Get`. 760 761Releasing the GIL from extension code 762------------------------------------- 763 764Most extension code manipulating the :term:`GIL` has the following simple 765structure:: 766 767 Save the thread state in a local variable. 768 Release the global interpreter lock. 769 ... Do some blocking I/O operation ... 770 Reacquire the global interpreter lock. 771 Restore the thread state from the local variable. 772 773This is so common that a pair of macros exists to simplify it:: 774 775 Py_BEGIN_ALLOW_THREADS 776 ... Do some blocking I/O operation ... 777 Py_END_ALLOW_THREADS 778 779.. index:: 780 single: Py_BEGIN_ALLOW_THREADS 781 single: Py_END_ALLOW_THREADS 782 783The :c:macro:`Py_BEGIN_ALLOW_THREADS` macro opens a new block and declares a 784hidden local variable; the :c:macro:`Py_END_ALLOW_THREADS` macro closes the 785block. 786 787The block above expands to the following code:: 788 789 PyThreadState *_save; 790 791 _save = PyEval_SaveThread(); 792 ... Do some blocking I/O operation ... 793 PyEval_RestoreThread(_save); 794 795.. index:: 796 single: PyEval_RestoreThread() 797 single: PyEval_SaveThread() 798 799Here is how these functions work: the global interpreter lock is used to protect the pointer to the 800current thread state. When releasing the lock and saving the thread state, 801the current thread state pointer must be retrieved before the lock is released 802(since another thread could immediately acquire the lock and store its own thread 803state in the global variable). Conversely, when acquiring the lock and restoring 804the thread state, the lock must be acquired before storing the thread state 805pointer. 806 807.. note:: 808 Calling system I/O functions is the most common use case for releasing 809 the GIL, but it can also be useful before calling long-running computations 810 which don't need access to Python objects, such as compression or 811 cryptographic functions operating over memory buffers. For example, the 812 standard :mod:`zlib` and :mod:`hashlib` modules release the GIL when 813 compressing or hashing data. 814 815 816.. _gilstate: 817 818Non-Python created threads 819-------------------------- 820 821When threads are created using the dedicated Python APIs (such as the 822:mod:`threading` module), a thread state is automatically associated to them 823and the code showed above is therefore correct. However, when threads are 824created from C (for example by a third-party library with its own thread 825management), they don't hold the GIL, nor is there a thread state structure 826for them. 827 828If you need to call Python code from these threads (often this will be part 829of a callback API provided by the aforementioned third-party library), 830you must first register these threads with the interpreter by 831creating a thread state data structure, then acquiring the GIL, and finally 832storing their thread state pointer, before you can start using the Python/C 833API. When you are done, you should reset the thread state pointer, release 834the GIL, and finally free the thread state data structure. 835 836The :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` functions do 837all of the above automatically. The typical idiom for calling into Python 838from a C thread is:: 839 840 PyGILState_STATE gstate; 841 gstate = PyGILState_Ensure(); 842 843 /* Perform Python actions here. */ 844 result = CallSomeFunction(); 845 /* evaluate result or handle exception */ 846 847 /* Release the thread. No Python API allowed beyond this point. */ 848 PyGILState_Release(gstate); 849 850Note that the ``PyGILState_*`` functions assume there is only one global 851interpreter (created automatically by :c:func:`Py_Initialize`). Python 852supports the creation of additional interpreters (using 853:c:func:`Py_NewInterpreter`), but mixing multiple interpreters and the 854``PyGILState_*`` API is unsupported. 855 856 857.. _fork-and-threads: 858 859Cautions about fork() 860--------------------- 861 862Another important thing to note about threads is their behaviour in the face 863of the C :c:func:`fork` call. On most systems with :c:func:`fork`, after a 864process forks only the thread that issued the fork will exist. This has a 865concrete impact both on how locks must be handled and on all stored state 866in CPython's runtime. 867 868The fact that only the "current" thread remains 869means any locks held by other threads will never be released. Python solves 870this for :func:`os.fork` by acquiring the locks it uses internally before 871the fork, and releasing them afterwards. In addition, it resets any 872:ref:`lock-objects` in the child. When extending or embedding Python, there 873is no way to inform Python of additional (non-Python) locks that need to be 874acquired before or reset after a fork. OS facilities such as 875:c:func:`pthread_atfork` would need to be used to accomplish the same thing. 876Additionally, when extending or embedding Python, calling :c:func:`fork` 877directly rather than through :func:`os.fork` (and returning to or calling 878into Python) may result in a deadlock by one of Python's internal locks 879being held by a thread that is defunct after the fork. 880:c:func:`PyOS_AfterFork_Child` tries to reset the necessary locks, but is not 881always able to. 882 883The fact that all other threads go away also means that CPython's 884runtime state there must be cleaned up properly, which :func:`os.fork` 885does. This means finalizing all other :c:type:`PyThreadState` objects 886belonging to the current interpreter and all other 887:c:type:`PyInterpreterState` objects. Due to this and the special 888nature of the :ref:`"main" interpreter <sub-interpreter-support>`, 889:c:func:`fork` should only be called in that interpreter's "main" 890thread, where the CPython global runtime was originally initialized. 891The only exception is if :c:func:`exec` will be called immediately 892after. 893 894 895High-level API 896-------------- 897 898These are the most commonly used types and functions when writing C extension 899code, or when embedding the Python interpreter: 900 901.. c:type:: PyInterpreterState 902 903 This data structure represents the state shared by a number of cooperating 904 threads. Threads belonging to the same interpreter share their module 905 administration and a few other internal items. There are no public members in 906 this structure. 907 908 Threads belonging to different interpreters initially share nothing, except 909 process state like available memory, open file descriptors and such. The global 910 interpreter lock is also shared by all threads, regardless of to which 911 interpreter they belong. 912 913 914.. c:type:: PyThreadState 915 916 This data structure represents the state of a single thread. The only public 917 data member is :attr:`interp` (:c:expr:`PyInterpreterState *`), which points to 918 this thread's interpreter state. 919 920 921.. c:function:: void PyEval_InitThreads() 922 923 .. index:: 924 single: PyEval_AcquireThread() 925 single: PyEval_ReleaseThread() 926 single: PyEval_SaveThread() 927 single: PyEval_RestoreThread() 928 929 Deprecated function which does nothing. 930 931 In Python 3.6 and older, this function created the GIL if it didn't exist. 932 933 .. versionchanged:: 3.9 934 The function now does nothing. 935 936 .. versionchanged:: 3.7 937 This function is now called by :c:func:`Py_Initialize()`, so you don't 938 have to call it yourself anymore. 939 940 .. versionchanged:: 3.2 941 This function cannot be called before :c:func:`Py_Initialize()` anymore. 942 943 .. deprecated:: 3.9 944 945 .. index:: pair: module; _thread 946 947 948.. c:function:: int PyEval_ThreadsInitialized() 949 950 Returns a non-zero value if :c:func:`PyEval_InitThreads` has been called. This 951 function can be called without holding the GIL, and therefore can be used to 952 avoid calls to the locking API when running single-threaded. 953 954 .. versionchanged:: 3.7 955 The :term:`GIL` is now initialized by :c:func:`Py_Initialize()`. 956 957 .. deprecated:: 3.9 958 959 960.. c:function:: PyThreadState* PyEval_SaveThread() 961 962 Release the global interpreter lock (if it has been created) and reset the 963 thread state to ``NULL``, returning the previous thread state (which is not 964 ``NULL``). If the lock has been created, the current thread must have 965 acquired it. 966 967 968.. c:function:: void PyEval_RestoreThread(PyThreadState *tstate) 969 970 Acquire the global interpreter lock (if it has been created) and set the 971 thread state to *tstate*, which must not be ``NULL``. If the lock has been 972 created, the current thread must not have acquired it, otherwise deadlock 973 ensues. 974 975 .. note:: 976 Calling this function from a thread when the runtime is finalizing 977 will terminate the thread, even if the thread was not created by Python. 978 You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to 979 check if the interpreter is in process of being finalized before calling 980 this function to avoid unwanted termination. 981 982.. c:function:: PyThreadState* PyThreadState_Get() 983 984 Return the current thread state. The global interpreter lock must be held. 985 When the current thread state is ``NULL``, this issues a fatal error (so that 986 the caller needn't check for ``NULL``). 987 988 989.. c:function:: PyThreadState* PyThreadState_Swap(PyThreadState *tstate) 990 991 Swap the current thread state with the thread state given by the argument 992 *tstate*, which may be ``NULL``. The global interpreter lock must be held 993 and is not released. 994 995 996The following functions use thread-local storage, and are not compatible 997with sub-interpreters: 998 999.. c:function:: PyGILState_STATE PyGILState_Ensure() 1000 1001 Ensure that the current thread is ready to call the Python C API regardless 1002 of the current state of Python, or of the global interpreter lock. This may 1003 be called as many times as desired by a thread as long as each call is 1004 matched with a call to :c:func:`PyGILState_Release`. In general, other 1005 thread-related APIs may be used between :c:func:`PyGILState_Ensure` and 1006 :c:func:`PyGILState_Release` calls as long as the thread state is restored to 1007 its previous state before the Release(). For example, normal usage of the 1008 :c:macro:`Py_BEGIN_ALLOW_THREADS` and :c:macro:`Py_END_ALLOW_THREADS` macros is 1009 acceptable. 1010 1011 The return value is an opaque "handle" to the thread state when 1012 :c:func:`PyGILState_Ensure` was called, and must be passed to 1013 :c:func:`PyGILState_Release` to ensure Python is left in the same state. Even 1014 though recursive calls are allowed, these handles *cannot* be shared - each 1015 unique call to :c:func:`PyGILState_Ensure` must save the handle for its call 1016 to :c:func:`PyGILState_Release`. 1017 1018 When the function returns, the current thread will hold the GIL and be able 1019 to call arbitrary Python code. Failure is a fatal error. 1020 1021 .. note:: 1022 Calling this function from a thread when the runtime is finalizing 1023 will terminate the thread, even if the thread was not created by Python. 1024 You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to 1025 check if the interpreter is in process of being finalized before calling 1026 this function to avoid unwanted termination. 1027 1028.. c:function:: void PyGILState_Release(PyGILState_STATE) 1029 1030 Release any resources previously acquired. After this call, Python's state will 1031 be the same as it was prior to the corresponding :c:func:`PyGILState_Ensure` call 1032 (but generally this state will be unknown to the caller, hence the use of the 1033 GILState API). 1034 1035 Every call to :c:func:`PyGILState_Ensure` must be matched by a call to 1036 :c:func:`PyGILState_Release` on the same thread. 1037 1038 1039.. c:function:: PyThreadState* PyGILState_GetThisThreadState() 1040 1041 Get the current thread state for this thread. May return ``NULL`` if no 1042 GILState API has been used on the current thread. Note that the main thread 1043 always has such a thread-state, even if no auto-thread-state call has been 1044 made on the main thread. This is mainly a helper/diagnostic function. 1045 1046 1047.. c:function:: int PyGILState_Check() 1048 1049 Return ``1`` if the current thread is holding the GIL and ``0`` otherwise. 1050 This function can be called from any thread at any time. 1051 Only if it has had its Python thread state initialized and currently is 1052 holding the GIL will it return ``1``. 1053 This is mainly a helper/diagnostic function. It can be useful 1054 for example in callback contexts or memory allocation functions when 1055 knowing that the GIL is locked can allow the caller to perform sensitive 1056 actions or otherwise behave differently. 1057 1058 .. versionadded:: 3.4 1059 1060 1061The following macros are normally used without a trailing semicolon; look for 1062example usage in the Python source distribution. 1063 1064 1065.. c:macro:: Py_BEGIN_ALLOW_THREADS 1066 1067 This macro expands to ``{ PyThreadState *_save; _save = PyEval_SaveThread();``. 1068 Note that it contains an opening brace; it must be matched with a following 1069 :c:macro:`Py_END_ALLOW_THREADS` macro. See above for further discussion of this 1070 macro. 1071 1072 1073.. c:macro:: Py_END_ALLOW_THREADS 1074 1075 This macro expands to ``PyEval_RestoreThread(_save); }``. Note that it contains 1076 a closing brace; it must be matched with an earlier 1077 :c:macro:`Py_BEGIN_ALLOW_THREADS` macro. See above for further discussion of 1078 this macro. 1079 1080 1081.. c:macro:: Py_BLOCK_THREADS 1082 1083 This macro expands to ``PyEval_RestoreThread(_save);``: it is equivalent to 1084 :c:macro:`Py_END_ALLOW_THREADS` without the closing brace. 1085 1086 1087.. c:macro:: Py_UNBLOCK_THREADS 1088 1089 This macro expands to ``_save = PyEval_SaveThread();``: it is equivalent to 1090 :c:macro:`Py_BEGIN_ALLOW_THREADS` without the opening brace and variable 1091 declaration. 1092 1093 1094Low-level API 1095------------- 1096 1097All of the following functions must be called after :c:func:`Py_Initialize`. 1098 1099.. versionchanged:: 3.7 1100 :c:func:`Py_Initialize()` now initializes the :term:`GIL`. 1101 1102 1103.. c:function:: PyInterpreterState* PyInterpreterState_New() 1104 1105 Create a new interpreter state object. The global interpreter lock need not 1106 be held, but may be held if it is necessary to serialize calls to this 1107 function. 1108 1109 .. audit-event:: cpython.PyInterpreterState_New "" c.PyInterpreterState_New 1110 1111 1112.. c:function:: void PyInterpreterState_Clear(PyInterpreterState *interp) 1113 1114 Reset all information in an interpreter state object. The global interpreter 1115 lock must be held. 1116 1117 .. audit-event:: cpython.PyInterpreterState_Clear "" c.PyInterpreterState_Clear 1118 1119 1120.. c:function:: void PyInterpreterState_Delete(PyInterpreterState *interp) 1121 1122 Destroy an interpreter state object. The global interpreter lock need not be 1123 held. The interpreter state must have been reset with a previous call to 1124 :c:func:`PyInterpreterState_Clear`. 1125 1126 1127.. c:function:: PyThreadState* PyThreadState_New(PyInterpreterState *interp) 1128 1129 Create a new thread state object belonging to the given interpreter object. 1130 The global interpreter lock need not be held, but may be held if it is 1131 necessary to serialize calls to this function. 1132 1133 1134.. c:function:: void PyThreadState_Clear(PyThreadState *tstate) 1135 1136 Reset all information in a thread state object. The global interpreter lock 1137 must be held. 1138 1139 .. versionchanged:: 3.9 1140 This function now calls the :c:member:`PyThreadState.on_delete` callback. 1141 Previously, that happened in :c:func:`PyThreadState_Delete`. 1142 1143 1144.. c:function:: void PyThreadState_Delete(PyThreadState *tstate) 1145 1146 Destroy a thread state object. The global interpreter lock need not be held. 1147 The thread state must have been reset with a previous call to 1148 :c:func:`PyThreadState_Clear`. 1149 1150 1151.. c:function:: void PyThreadState_DeleteCurrent(void) 1152 1153 Destroy the current thread state and release the global interpreter lock. 1154 Like :c:func:`PyThreadState_Delete`, the global interpreter lock need not 1155 be held. The thread state must have been reset with a previous call 1156 to :c:func:`PyThreadState_Clear`. 1157 1158 1159.. c:function:: PyFrameObject* PyThreadState_GetFrame(PyThreadState *tstate) 1160 1161 Get the current frame of the Python thread state *tstate*. 1162 1163 Return a :term:`strong reference`. Return ``NULL`` if no frame is currently 1164 executing. 1165 1166 See also :c:func:`PyEval_GetFrame`. 1167 1168 *tstate* must not be ``NULL``. 1169 1170 .. versionadded:: 3.9 1171 1172 1173.. c:function:: uint64_t PyThreadState_GetID(PyThreadState *tstate) 1174 1175 Get the unique thread state identifier of the Python thread state *tstate*. 1176 1177 *tstate* must not be ``NULL``. 1178 1179 .. versionadded:: 3.9 1180 1181 1182.. c:function:: PyInterpreterState* PyThreadState_GetInterpreter(PyThreadState *tstate) 1183 1184 Get the interpreter of the Python thread state *tstate*. 1185 1186 *tstate* must not be ``NULL``. 1187 1188 .. versionadded:: 3.9 1189 1190 1191.. c:function:: void PyThreadState_EnterTracing(PyThreadState *tstate) 1192 1193 Suspend tracing and profiling in the Python thread state *tstate*. 1194 1195 Resume them using the :c:func:`PyThreadState_LeaveTracing` function. 1196 1197 .. versionadded:: 3.11 1198 1199 1200.. c:function:: void PyThreadState_LeaveTracing(PyThreadState *tstate) 1201 1202 Resume tracing and profiling in the Python thread state *tstate* suspended 1203 by the :c:func:`PyThreadState_EnterTracing` function. 1204 1205 See also :c:func:`PyEval_SetTrace` and :c:func:`PyEval_SetProfile` 1206 functions. 1207 1208 .. versionadded:: 3.11 1209 1210 1211.. c:function:: PyInterpreterState* PyInterpreterState_Get(void) 1212 1213 Get the current interpreter. 1214 1215 Issue a fatal error if there no current Python thread state or no current 1216 interpreter. It cannot return NULL. 1217 1218 The caller must hold the GIL. 1219 1220 .. versionadded:: 3.9 1221 1222 1223.. c:function:: int64_t PyInterpreterState_GetID(PyInterpreterState *interp) 1224 1225 Return the interpreter's unique ID. If there was any error in doing 1226 so then ``-1`` is returned and an error is set. 1227 1228 The caller must hold the GIL. 1229 1230 .. versionadded:: 3.7 1231 1232 1233.. c:function:: PyObject* PyInterpreterState_GetDict(PyInterpreterState *interp) 1234 1235 Return a dictionary in which interpreter-specific data may be stored. 1236 If this function returns ``NULL`` then no exception has been raised and 1237 the caller should assume no interpreter-specific dict is available. 1238 1239 This is not a replacement for :c:func:`PyModule_GetState()`, which 1240 extensions should use to store interpreter-specific state information. 1241 1242 .. versionadded:: 3.8 1243 1244.. c:type:: PyObject* (*_PyFrameEvalFunction)(PyThreadState *tstate, _PyInterpreterFrame *frame, int throwflag) 1245 1246 Type of a frame evaluation function. 1247 1248 The *throwflag* parameter is used by the ``throw()`` method of generators: 1249 if non-zero, handle the current exception. 1250 1251 .. versionchanged:: 3.9 1252 The function now takes a *tstate* parameter. 1253 1254 .. versionchanged:: 3.11 1255 The *frame* parameter changed from ``PyFrameObject*`` to ``_PyInterpreterFrame*``. 1256 1257.. c:function:: _PyFrameEvalFunction _PyInterpreterState_GetEvalFrameFunc(PyInterpreterState *interp) 1258 1259 Get the frame evaluation function. 1260 1261 See the :pep:`523` "Adding a frame evaluation API to CPython". 1262 1263 .. versionadded:: 3.9 1264 1265.. c:function:: void _PyInterpreterState_SetEvalFrameFunc(PyInterpreterState *interp, _PyFrameEvalFunction eval_frame) 1266 1267 Set the frame evaluation function. 1268 1269 See the :pep:`523` "Adding a frame evaluation API to CPython". 1270 1271 .. versionadded:: 3.9 1272 1273 1274.. c:function:: PyObject* PyThreadState_GetDict() 1275 1276 Return a dictionary in which extensions can store thread-specific state 1277 information. Each extension should use a unique key to use to store state in 1278 the dictionary. It is okay to call this function when no current thread state 1279 is available. If this function returns ``NULL``, no exception has been raised and 1280 the caller should assume no current thread state is available. 1281 1282 1283.. c:function:: int PyThreadState_SetAsyncExc(unsigned long id, PyObject *exc) 1284 1285 Asynchronously raise an exception in a thread. The *id* argument is the thread 1286 id of the target thread; *exc* is the exception object to be raised. This 1287 function does not steal any references to *exc*. To prevent naive misuse, you 1288 must write your own C extension to call this. Must be called with the GIL held. 1289 Returns the number of thread states modified; this is normally one, but will be 1290 zero if the thread id isn't found. If *exc* is :const:`NULL`, the pending 1291 exception (if any) for the thread is cleared. This raises no exceptions. 1292 1293 .. versionchanged:: 3.7 1294 The type of the *id* parameter changed from :c:expr:`long` to 1295 :c:expr:`unsigned long`. 1296 1297.. c:function:: void PyEval_AcquireThread(PyThreadState *tstate) 1298 1299 Acquire the global interpreter lock and set the current thread state to 1300 *tstate*, which must not be ``NULL``. The lock must have been created earlier. 1301 If this thread already has the lock, deadlock ensues. 1302 1303 .. note:: 1304 Calling this function from a thread when the runtime is finalizing 1305 will terminate the thread, even if the thread was not created by Python. 1306 You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to 1307 check if the interpreter is in process of being finalized before calling 1308 this function to avoid unwanted termination. 1309 1310 .. versionchanged:: 3.8 1311 Updated to be consistent with :c:func:`PyEval_RestoreThread`, 1312 :c:func:`Py_END_ALLOW_THREADS`, and :c:func:`PyGILState_Ensure`, 1313 and terminate the current thread if called while the interpreter is finalizing. 1314 1315 :c:func:`PyEval_RestoreThread` is a higher-level function which is always 1316 available (even when threads have not been initialized). 1317 1318 1319.. c:function:: void PyEval_ReleaseThread(PyThreadState *tstate) 1320 1321 Reset the current thread state to ``NULL`` and release the global interpreter 1322 lock. The lock must have been created earlier and must be held by the current 1323 thread. The *tstate* argument, which must not be ``NULL``, is only used to check 1324 that it represents the current thread state --- if it isn't, a fatal error is 1325 reported. 1326 1327 :c:func:`PyEval_SaveThread` is a higher-level function which is always 1328 available (even when threads have not been initialized). 1329 1330 1331.. c:function:: void PyEval_AcquireLock() 1332 1333 Acquire the global interpreter lock. The lock must have been created earlier. 1334 If this thread already has the lock, a deadlock ensues. 1335 1336 .. deprecated:: 3.2 1337 This function does not update the current thread state. Please use 1338 :c:func:`PyEval_RestoreThread` or :c:func:`PyEval_AcquireThread` 1339 instead. 1340 1341 .. note:: 1342 Calling this function from a thread when the runtime is finalizing 1343 will terminate the thread, even if the thread was not created by Python. 1344 You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to 1345 check if the interpreter is in process of being finalized before calling 1346 this function to avoid unwanted termination. 1347 1348 .. versionchanged:: 3.8 1349 Updated to be consistent with :c:func:`PyEval_RestoreThread`, 1350 :c:func:`Py_END_ALLOW_THREADS`, and :c:func:`PyGILState_Ensure`, 1351 and terminate the current thread if called while the interpreter is finalizing. 1352 1353 1354.. c:function:: void PyEval_ReleaseLock() 1355 1356 Release the global interpreter lock. The lock must have been created earlier. 1357 1358 .. deprecated:: 3.2 1359 This function does not update the current thread state. Please use 1360 :c:func:`PyEval_SaveThread` or :c:func:`PyEval_ReleaseThread` 1361 instead. 1362 1363 1364.. _sub-interpreter-support: 1365 1366Sub-interpreter support 1367======================= 1368 1369While in most uses, you will only embed a single Python interpreter, there 1370are cases where you need to create several independent interpreters in the 1371same process and perhaps even in the same thread. Sub-interpreters allow 1372you to do that. 1373 1374The "main" interpreter is the first one created when the runtime initializes. 1375It is usually the only Python interpreter in a process. Unlike sub-interpreters, 1376the main interpreter has unique process-global responsibilities like signal 1377handling. It is also responsible for execution during runtime initialization and 1378is usually the active interpreter during runtime finalization. The 1379:c:func:`PyInterpreterState_Main` function returns a pointer to its state. 1380 1381You can switch between sub-interpreters using the :c:func:`PyThreadState_Swap` 1382function. You can create and destroy them using the following functions: 1383 1384 1385.. c:function:: PyThreadState* Py_NewInterpreter() 1386 1387 .. index:: 1388 pair: module; builtins 1389 pair: module; __main__ 1390 pair: module; sys 1391 single: stdout (in module sys) 1392 single: stderr (in module sys) 1393 single: stdin (in module sys) 1394 1395 Create a new sub-interpreter. This is an (almost) totally separate environment 1396 for the execution of Python code. In particular, the new interpreter has 1397 separate, independent versions of all imported modules, including the 1398 fundamental modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`. The 1399 table of loaded modules (``sys.modules``) and the module search path 1400 (``sys.path``) are also separate. The new environment has no ``sys.argv`` 1401 variable. It has new standard I/O stream file objects ``sys.stdin``, 1402 ``sys.stdout`` and ``sys.stderr`` (however these refer to the same underlying 1403 file descriptors). 1404 1405 The return value points to the first thread state created in the new 1406 sub-interpreter. This thread state is made in the current thread state. 1407 Note that no actual thread is created; see the discussion of thread states 1408 below. If creation of the new interpreter is unsuccessful, ``NULL`` is 1409 returned; no exception is set since the exception state is stored in the 1410 current thread state and there may not be a current thread state. (Like all 1411 other Python/C API functions, the global interpreter lock must be held before 1412 calling this function and is still held when it returns; however, unlike most 1413 other Python/C API functions, there needn't be a current thread state on 1414 entry.) 1415 1416 .. index:: 1417 single: Py_FinalizeEx() 1418 single: Py_Initialize() 1419 1420 Extension modules are shared between (sub-)interpreters as follows: 1421 1422 * For modules using multi-phase initialization, 1423 e.g. :c:func:`PyModule_FromDefAndSpec`, a separate module object is 1424 created and initialized for each interpreter. 1425 Only C-level static and global variables are shared between these 1426 module objects. 1427 1428 * For modules using single-phase initialization, 1429 e.g. :c:func:`PyModule_Create`, the first time a particular extension 1430 is imported, it is initialized normally, and a (shallow) copy of its 1431 module's dictionary is squirreled away. 1432 When the same extension is imported by another (sub-)interpreter, a new 1433 module is initialized and filled with the contents of this copy; the 1434 extension's ``init`` function is not called. 1435 Objects in the module's dictionary thus end up shared across 1436 (sub-)interpreters, which might cause unwanted behavior (see 1437 `Bugs and caveats`_ below). 1438 1439 Note that this is different from what happens when an extension is 1440 imported after the interpreter has been completely re-initialized by 1441 calling :c:func:`Py_FinalizeEx` and :c:func:`Py_Initialize`; in that 1442 case, the extension's ``initmodule`` function *is* called again. 1443 As with multi-phase initialization, this means that only C-level static 1444 and global variables are shared between these modules. 1445 1446 .. index:: single: close() (in module os) 1447 1448 1449.. c:function:: void Py_EndInterpreter(PyThreadState *tstate) 1450 1451 .. index:: single: Py_FinalizeEx() 1452 1453 Destroy the (sub-)interpreter represented by the given thread state. The given 1454 thread state must be the current thread state. See the discussion of thread 1455 states below. When the call returns, the current thread state is ``NULL``. All 1456 thread states associated with this interpreter are destroyed. (The global 1457 interpreter lock must be held before calling this function and is still held 1458 when it returns.) :c:func:`Py_FinalizeEx` will destroy all sub-interpreters that 1459 haven't been explicitly destroyed at that point. 1460 1461 1462Bugs and caveats 1463---------------- 1464 1465Because sub-interpreters (and the main interpreter) are part of the same 1466process, the insulation between them isn't perfect --- for example, using 1467low-level file operations like :func:`os.close` they can 1468(accidentally or maliciously) affect each other's open files. Because of the 1469way extensions are shared between (sub-)interpreters, some extensions may not 1470work properly; this is especially likely when using single-phase initialization 1471or (static) global variables. 1472It is possible to insert objects created in one sub-interpreter into 1473a namespace of another (sub-)interpreter; this should be avoided if possible. 1474 1475Special care should be taken to avoid sharing user-defined functions, 1476methods, instances or classes between sub-interpreters, since import 1477operations executed by such objects may affect the wrong (sub-)interpreter's 1478dictionary of loaded modules. It is equally important to avoid sharing 1479objects from which the above are reachable. 1480 1481Also note that combining this functionality with ``PyGILState_*`` APIs 1482is delicate, because these APIs assume a bijection between Python thread states 1483and OS-level threads, an assumption broken by the presence of sub-interpreters. 1484It is highly recommended that you don't switch sub-interpreters between a pair 1485of matching :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` calls. 1486Furthermore, extensions (such as :mod:`ctypes`) using these APIs to allow calling 1487of Python code from non-Python created threads will probably be broken when using 1488sub-interpreters. 1489 1490 1491Asynchronous Notifications 1492========================== 1493 1494A mechanism is provided to make asynchronous notifications to the main 1495interpreter thread. These notifications take the form of a function 1496pointer and a void pointer argument. 1497 1498 1499.. c:function:: int Py_AddPendingCall(int (*func)(void *), void *arg) 1500 1501 .. index:: single: Py_AddPendingCall() 1502 1503 Schedule a function to be called from the main interpreter thread. On 1504 success, ``0`` is returned and *func* is queued for being called in the 1505 main thread. On failure, ``-1`` is returned without setting any exception. 1506 1507 When successfully queued, *func* will be *eventually* called from the 1508 main interpreter thread with the argument *arg*. It will be called 1509 asynchronously with respect to normally running Python code, but with 1510 both these conditions met: 1511 1512 * on a :term:`bytecode` boundary; 1513 * with the main thread holding the :term:`global interpreter lock` 1514 (*func* can therefore use the full C API). 1515 1516 *func* must return ``0`` on success, or ``-1`` on failure with an exception 1517 set. *func* won't be interrupted to perform another asynchronous 1518 notification recursively, but it can still be interrupted to switch 1519 threads if the global interpreter lock is released. 1520 1521 This function doesn't need a current thread state to run, and it doesn't 1522 need the global interpreter lock. 1523 1524 To call this function in a subinterpreter, the caller must hold the GIL. 1525 Otherwise, the function *func* can be scheduled to be called from the wrong 1526 interpreter. 1527 1528 .. warning:: 1529 This is a low-level function, only useful for very special cases. 1530 There is no guarantee that *func* will be called as quick as 1531 possible. If the main thread is busy executing a system call, 1532 *func* won't be called before the system call returns. This 1533 function is generally **not** suitable for calling Python code from 1534 arbitrary C threads. Instead, use the :ref:`PyGILState API<gilstate>`. 1535 1536 .. versionchanged:: 3.9 1537 If this function is called in a subinterpreter, the function *func* is 1538 now scheduled to be called from the subinterpreter, rather than being 1539 called from the main interpreter. Each subinterpreter now has its own 1540 list of scheduled calls. 1541 1542 .. versionadded:: 3.1 1543 1544.. _profiling: 1545 1546Profiling and Tracing 1547===================== 1548 1549.. sectionauthor:: Fred L. Drake, Jr. <[email protected]> 1550 1551 1552The Python interpreter provides some low-level support for attaching profiling 1553and execution tracing facilities. These are used for profiling, debugging, and 1554coverage analysis tools. 1555 1556This C interface allows the profiling or tracing code to avoid the overhead of 1557calling through Python-level callable objects, making a direct C function call 1558instead. The essential attributes of the facility have not changed; the 1559interface allows trace functions to be installed per-thread, and the basic 1560events reported to the trace function are the same as had been reported to the 1561Python-level trace functions in previous versions. 1562 1563 1564.. c:type:: int (*Py_tracefunc)(PyObject *obj, PyFrameObject *frame, int what, PyObject *arg) 1565 1566 The type of the trace function registered using :c:func:`PyEval_SetProfile` and 1567 :c:func:`PyEval_SetTrace`. The first parameter is the object passed to the 1568 registration function as *obj*, *frame* is the frame object to which the event 1569 pertains, *what* is one of the constants :const:`PyTrace_CALL`, 1570 :const:`PyTrace_EXCEPTION`, :const:`PyTrace_LINE`, :const:`PyTrace_RETURN`, 1571 :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION`, :const:`PyTrace_C_RETURN`, 1572 or :const:`PyTrace_OPCODE`, and *arg* depends on the value of *what*: 1573 1574 +------------------------------+----------------------------------------+ 1575 | Value of *what* | Meaning of *arg* | 1576 +==============================+========================================+ 1577 | :const:`PyTrace_CALL` | Always :c:data:`Py_None`. | 1578 +------------------------------+----------------------------------------+ 1579 | :const:`PyTrace_EXCEPTION` | Exception information as returned by | 1580 | | :func:`sys.exc_info`. | 1581 +------------------------------+----------------------------------------+ 1582 | :const:`PyTrace_LINE` | Always :c:data:`Py_None`. | 1583 +------------------------------+----------------------------------------+ 1584 | :const:`PyTrace_RETURN` | Value being returned to the caller, | 1585 | | or ``NULL`` if caused by an exception. | 1586 +------------------------------+----------------------------------------+ 1587 | :const:`PyTrace_C_CALL` | Function object being called. | 1588 +------------------------------+----------------------------------------+ 1589 | :const:`PyTrace_C_EXCEPTION` | Function object being called. | 1590 +------------------------------+----------------------------------------+ 1591 | :const:`PyTrace_C_RETURN` | Function object being called. | 1592 +------------------------------+----------------------------------------+ 1593 | :const:`PyTrace_OPCODE` | Always :c:data:`Py_None`. | 1594 +------------------------------+----------------------------------------+ 1595 1596.. c:var:: int PyTrace_CALL 1597 1598 The value of the *what* parameter to a :c:type:`Py_tracefunc` function when a new 1599 call to a function or method is being reported, or a new entry into a generator. 1600 Note that the creation of the iterator for a generator function is not reported 1601 as there is no control transfer to the Python bytecode in the corresponding 1602 frame. 1603 1604 1605.. c:var:: int PyTrace_EXCEPTION 1606 1607 The value of the *what* parameter to a :c:type:`Py_tracefunc` function when an 1608 exception has been raised. The callback function is called with this value for 1609 *what* when after any bytecode is processed after which the exception becomes 1610 set within the frame being executed. The effect of this is that as exception 1611 propagation causes the Python stack to unwind, the callback is called upon 1612 return to each frame as the exception propagates. Only trace functions receives 1613 these events; they are not needed by the profiler. 1614 1615 1616.. c:var:: int PyTrace_LINE 1617 1618 The value passed as the *what* parameter to a :c:type:`Py_tracefunc` function 1619 (but not a profiling function) when a line-number event is being reported. 1620 It may be disabled for a frame by setting :attr:`f_trace_lines` to *0* on that frame. 1621 1622 1623.. c:var:: int PyTrace_RETURN 1624 1625 The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a 1626 call is about to return. 1627 1628 1629.. c:var:: int PyTrace_C_CALL 1630 1631 The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C 1632 function is about to be called. 1633 1634 1635.. c:var:: int PyTrace_C_EXCEPTION 1636 1637 The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C 1638 function has raised an exception. 1639 1640 1641.. c:var:: int PyTrace_C_RETURN 1642 1643 The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C 1644 function has returned. 1645 1646 1647.. c:var:: int PyTrace_OPCODE 1648 1649 The value for the *what* parameter to :c:type:`Py_tracefunc` functions (but not 1650 profiling functions) when a new opcode is about to be executed. This event is 1651 not emitted by default: it must be explicitly requested by setting 1652 :attr:`f_trace_opcodes` to *1* on the frame. 1653 1654 1655.. c:function:: void PyEval_SetProfile(Py_tracefunc func, PyObject *obj) 1656 1657 Set the profiler function to *func*. The *obj* parameter is passed to the 1658 function as its first parameter, and may be any Python object, or ``NULL``. If 1659 the profile function needs to maintain state, using a different value for *obj* 1660 for each thread provides a convenient and thread-safe place to store it. The 1661 profile function is called for all monitored events except :const:`PyTrace_LINE` 1662 :const:`PyTrace_OPCODE` and :const:`PyTrace_EXCEPTION`. 1663 1664 See also the :func:`sys.setprofile` function. 1665 1666 The caller must hold the :term:`GIL`. 1667 1668 1669.. c:function:: void PyEval_SetTrace(Py_tracefunc func, PyObject *obj) 1670 1671 Set the tracing function to *func*. This is similar to 1672 :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number 1673 events and per-opcode events, but does not receive any event related to C function 1674 objects being called. Any trace function registered using :c:func:`PyEval_SetTrace` 1675 will not receive :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION` or 1676 :const:`PyTrace_C_RETURN` as a value for the *what* parameter. 1677 1678 See also the :func:`sys.settrace` function. 1679 1680 The caller must hold the :term:`GIL`. 1681 1682 1683.. _advanced-debugging: 1684 1685Advanced Debugger Support 1686========================= 1687 1688.. sectionauthor:: Fred L. Drake, Jr. <[email protected]> 1689 1690 1691These functions are only intended to be used by advanced debugging tools. 1692 1693 1694.. c:function:: PyInterpreterState* PyInterpreterState_Head() 1695 1696 Return the interpreter state object at the head of the list of all such objects. 1697 1698 1699.. c:function:: PyInterpreterState* PyInterpreterState_Main() 1700 1701 Return the main interpreter state object. 1702 1703 1704.. c:function:: PyInterpreterState* PyInterpreterState_Next(PyInterpreterState *interp) 1705 1706 Return the next interpreter state object after *interp* from the list of all 1707 such objects. 1708 1709 1710.. c:function:: PyThreadState * PyInterpreterState_ThreadHead(PyInterpreterState *interp) 1711 1712 Return the pointer to the first :c:type:`PyThreadState` object in the list of 1713 threads associated with the interpreter *interp*. 1714 1715 1716.. c:function:: PyThreadState* PyThreadState_Next(PyThreadState *tstate) 1717 1718 Return the next thread state object after *tstate* from the list of all such 1719 objects belonging to the same :c:type:`PyInterpreterState` object. 1720 1721 1722.. _thread-local-storage: 1723 1724Thread Local Storage Support 1725============================ 1726 1727.. sectionauthor:: Masayuki Yamamoto <[email protected]> 1728 1729The Python interpreter provides low-level support for thread-local storage 1730(TLS) which wraps the underlying native TLS implementation to support the 1731Python-level thread local storage API (:class:`threading.local`). The 1732CPython C level APIs are similar to those offered by pthreads and Windows: 1733use a thread key and functions to associate a :c:expr:`void*` value per 1734thread. 1735 1736The GIL does *not* need to be held when calling these functions; they supply 1737their own locking. 1738 1739Note that :file:`Python.h` does not include the declaration of the TLS APIs, 1740you need to include :file:`pythread.h` to use thread-local storage. 1741 1742.. note:: 1743 None of these API functions handle memory management on behalf of the 1744 :c:expr:`void*` values. You need to allocate and deallocate them yourself. 1745 If the :c:expr:`void*` values happen to be :c:expr:`PyObject*`, these 1746 functions don't do refcount operations on them either. 1747 1748.. _thread-specific-storage-api: 1749 1750Thread Specific Storage (TSS) API 1751--------------------------------- 1752 1753TSS API is introduced to supersede the use of the existing TLS API within the 1754CPython interpreter. This API uses a new type :c:type:`Py_tss_t` instead of 1755:c:expr:`int` to represent thread keys. 1756 1757.. versionadded:: 3.7 1758 1759.. seealso:: "A New C-API for Thread-Local Storage in CPython" (:pep:`539`) 1760 1761 1762.. c:type:: Py_tss_t 1763 1764 This data structure represents the state of a thread key, the definition of 1765 which may depend on the underlying TLS implementation, and it has an 1766 internal field representing the key's initialization state. There are no 1767 public members in this structure. 1768 1769 When :ref:`Py_LIMITED_API <stable>` is not defined, static allocation of 1770 this type by :c:macro:`Py_tss_NEEDS_INIT` is allowed. 1771 1772 1773.. c:macro:: Py_tss_NEEDS_INIT 1774 1775 This macro expands to the initializer for :c:type:`Py_tss_t` variables. 1776 Note that this macro won't be defined with :ref:`Py_LIMITED_API <stable>`. 1777 1778 1779Dynamic Allocation 1780~~~~~~~~~~~~~~~~~~ 1781 1782Dynamic allocation of the :c:type:`Py_tss_t`, required in extension modules 1783built with :ref:`Py_LIMITED_API <stable>`, where static allocation of this type 1784is not possible due to its implementation being opaque at build time. 1785 1786 1787.. c:function:: Py_tss_t* PyThread_tss_alloc() 1788 1789 Return a value which is the same state as a value initialized with 1790 :c:macro:`Py_tss_NEEDS_INIT`, or ``NULL`` in the case of dynamic allocation 1791 failure. 1792 1793 1794.. c:function:: void PyThread_tss_free(Py_tss_t *key) 1795 1796 Free the given *key* allocated by :c:func:`PyThread_tss_alloc`, after 1797 first calling :c:func:`PyThread_tss_delete` to ensure any associated 1798 thread locals have been unassigned. This is a no-op if the *key* 1799 argument is ``NULL``. 1800 1801 .. note:: 1802 A freed key becomes a dangling pointer. You should reset the key to 1803 ``NULL``. 1804 1805 1806Methods 1807~~~~~~~ 1808 1809The parameter *key* of these functions must not be ``NULL``. Moreover, the 1810behaviors of :c:func:`PyThread_tss_set` and :c:func:`PyThread_tss_get` are 1811undefined if the given :c:type:`Py_tss_t` has not been initialized by 1812:c:func:`PyThread_tss_create`. 1813 1814 1815.. c:function:: int PyThread_tss_is_created(Py_tss_t *key) 1816 1817 Return a non-zero value if the given :c:type:`Py_tss_t` has been initialized 1818 by :c:func:`PyThread_tss_create`. 1819 1820 1821.. c:function:: int PyThread_tss_create(Py_tss_t *key) 1822 1823 Return a zero value on successful initialization of a TSS key. The behavior 1824 is undefined if the value pointed to by the *key* argument is not 1825 initialized by :c:macro:`Py_tss_NEEDS_INIT`. This function can be called 1826 repeatedly on the same key -- calling it on an already initialized key is a 1827 no-op and immediately returns success. 1828 1829 1830.. c:function:: void PyThread_tss_delete(Py_tss_t *key) 1831 1832 Destroy a TSS key to forget the values associated with the key across all 1833 threads, and change the key's initialization state to uninitialized. A 1834 destroyed key is able to be initialized again by 1835 :c:func:`PyThread_tss_create`. This function can be called repeatedly on 1836 the same key -- calling it on an already destroyed key is a no-op. 1837 1838 1839.. c:function:: int PyThread_tss_set(Py_tss_t *key, void *value) 1840 1841 Return a zero value to indicate successfully associating a :c:expr:`void*` 1842 value with a TSS key in the current thread. Each thread has a distinct 1843 mapping of the key to a :c:expr:`void*` value. 1844 1845 1846.. c:function:: void* PyThread_tss_get(Py_tss_t *key) 1847 1848 Return the :c:expr:`void*` value associated with a TSS key in the current 1849 thread. This returns ``NULL`` if no value is associated with the key in the 1850 current thread. 1851 1852 1853.. _thread-local-storage-api: 1854 1855Thread Local Storage (TLS) API 1856------------------------------ 1857 1858.. deprecated:: 3.7 1859 This API is superseded by 1860 :ref:`Thread Specific Storage (TSS) API <thread-specific-storage-api>`. 1861 1862.. note:: 1863 This version of the API does not support platforms where the native TLS key 1864 is defined in a way that cannot be safely cast to ``int``. On such platforms, 1865 :c:func:`PyThread_create_key` will return immediately with a failure status, 1866 and the other TLS functions will all be no-ops on such platforms. 1867 1868Due to the compatibility problem noted above, this version of the API should not 1869be used in new code. 1870 1871.. c:function:: int PyThread_create_key() 1872.. c:function:: void PyThread_delete_key(int key) 1873.. c:function:: int PyThread_set_key_value(int key, void *value) 1874.. c:function:: void* PyThread_get_key_value(int key) 1875.. c:function:: void PyThread_delete_key_value(int key) 1876.. c:function:: void PyThread_ReInitTLS() 1877 1878