1:mod:`compileall` --- Byte-compile Python libraries 2=================================================== 3 4.. module:: compileall 5 :synopsis: Tools for byte-compiling all Python source files in a directory tree. 6 7**Source code:** :source:`Lib/compileall.py` 8 9-------------- 10 11This module provides some utility functions to support installing Python 12libraries. These functions compile Python source files in a directory tree. 13This module can be used to create the cached byte-code files at library 14installation time, which makes them available for use even by users who don't 15have write permission to the library directories. 16 17.. include:: ../includes/wasm-notavail.rst 18 19Command-line use 20---------------- 21 22This module can work as a script (using :program:`python -m compileall`) to 23compile Python sources. 24 25.. program:: compileall 26 27.. cmdoption:: directory ... 28 file ... 29 30 Positional arguments are files to compile or directories that contain 31 source files, traversed recursively. If no argument is given, behave as if 32 the command line was ``-l <directories from sys.path>``. 33 34.. cmdoption:: -l 35 36 Do not recurse into subdirectories, only compile source code files directly 37 contained in the named or implied directories. 38 39.. cmdoption:: -f 40 41 Force rebuild even if timestamps are up-to-date. 42 43.. cmdoption:: -q 44 45 Do not print the list of files compiled. If passed once, error messages will 46 still be printed. If passed twice (``-qq``), all output is suppressed. 47 48.. cmdoption:: -d destdir 49 50 Directory prepended to the path to each file being compiled. This will 51 appear in compilation time tracebacks, and is also compiled in to the 52 byte-code file, where it will be used in tracebacks and other messages in 53 cases where the source file does not exist at the time the byte-code file is 54 executed. 55 56.. cmdoption:: -s strip_prefix 57.. cmdoption:: -p prepend_prefix 58 59 Remove (``-s``) or append (``-p``) the given prefix of paths 60 recorded in the ``.pyc`` files. 61 Cannot be combined with ``-d``. 62 63.. cmdoption:: -x regex 64 65 regex is used to search the full path to each file considered for 66 compilation, and if the regex produces a match, the file is skipped. 67 68.. cmdoption:: -i list 69 70 Read the file ``list`` and add each line that it contains to the list of 71 files and directories to compile. If ``list`` is ``-``, read lines from 72 ``stdin``. 73 74.. cmdoption:: -b 75 76 Write the byte-code files to their legacy locations and names, which may 77 overwrite byte-code files created by another version of Python. The default 78 is to write files to their :pep:`3147` locations and names, which allows 79 byte-code files from multiple versions of Python to coexist. 80 81.. cmdoption:: -r 82 83 Control the maximum recursion level for subdirectories. 84 If this is given, then ``-l`` option will not be taken into account. 85 :program:`python -m compileall <directory> -r 0` is equivalent to 86 :program:`python -m compileall <directory> -l`. 87 88.. cmdoption:: -j N 89 90 Use *N* workers to compile the files within the given directory. 91 If ``0`` is used, then the result of :func:`os.cpu_count()` 92 will be used. 93 94.. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash] 95 96 Control how the generated byte-code files are invalidated at runtime. 97 The ``timestamp`` value, means that ``.pyc`` files with the source timestamp 98 and size embedded will be generated. The ``checked-hash`` and 99 ``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based 100 pycs embed a hash of the source file contents rather than a timestamp. See 101 :ref:`pyc-invalidation` for more information on how Python validates 102 bytecode cache files at runtime. 103 The default is ``timestamp`` if the :envvar:`SOURCE_DATE_EPOCH` environment 104 variable is not set, and ``checked-hash`` if the ``SOURCE_DATE_EPOCH`` 105 environment variable is set. 106 107.. cmdoption:: -o level 108 109 Compile with the given optimization level. May be used multiple times 110 to compile for multiple levels at a time (for example, 111 ``compileall -o 1 -o 2``). 112 113.. cmdoption:: -e dir 114 115 Ignore symlinks pointing outside the given directory. 116 117.. cmdoption:: --hardlink-dupes 118 119 If two ``.pyc`` files with different optimization level have 120 the same content, use hard links to consolidate duplicate files. 121 122.. versionchanged:: 3.2 123 Added the ``-i``, ``-b`` and ``-h`` options. 124 125.. versionchanged:: 3.5 126 Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option 127 was changed to a multilevel value. ``-b`` will always produce a 128 byte-code file ending in ``.pyc``, never ``.pyo``. 129 130.. versionchanged:: 3.7 131 Added the ``--invalidation-mode`` option. 132 133.. versionchanged:: 3.9 134 Added the ``-s``, ``-p``, ``-e`` and ``--hardlink-dupes`` options. 135 Raised the default recursion limit from 10 to 136 :py:func:`sys.getrecursionlimit()`. 137 Added the possibility to specify the ``-o`` option multiple times. 138 139 140There is no command-line option to control the optimization level used by the 141:func:`compile` function, because the Python interpreter itself already 142provides the option: :program:`python -O -m compileall`. 143 144Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix` 145setting. The generated bytecode cache will only be useful if :func:`compile` is 146run with the same :attr:`sys.pycache_prefix` (if any) that will be used at 147runtime. 148 149Public functions 150---------------- 151 152.. function:: compile_dir(dir, maxlevels=sys.getrecursionlimit(), ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False) 153 154 Recursively descend the directory tree named by *dir*, compiling all :file:`.py` 155 files along the way. Return a true value if all the files compiled successfully, 156 and a false value otherwise. 157 158 The *maxlevels* parameter is used to limit the depth of the recursion; it 159 defaults to ``sys.getrecursionlimit()``. 160 161 If *ddir* is given, it is prepended to the path to each file being compiled 162 for use in compilation time tracebacks, and is also compiled in to the 163 byte-code file, where it will be used in tracebacks and other messages in 164 cases where the source file does not exist at the time the byte-code file is 165 executed. 166 167 If *force* is true, modules are re-compiled even if the timestamps are up to 168 date. 169 170 If *rx* is given, its ``search`` method is called on the complete path to each 171 file considered for compilation, and if it returns a true value, the file 172 is skipped. This can be used to exclude files matching a regular expression, 173 given as a :ref:`re.Pattern <re-objects>` object. 174 175 If *quiet* is ``False`` or ``0`` (the default), the filenames and other 176 information are printed to standard out. Set to ``1``, only errors are 177 printed. Set to ``2``, all output is suppressed. 178 179 If *legacy* is true, byte-code files are written to their legacy locations 180 and names, which may overwrite byte-code files created by another version of 181 Python. The default is to write files to their :pep:`3147` locations and 182 names, which allows byte-code files from multiple versions of Python to 183 coexist. 184 185 *optimize* specifies the optimization level for the compiler. It is passed to 186 the built-in :func:`compile` function. Accepts also a sequence of optimization 187 levels which lead to multiple compilations of one :file:`.py` file in one call. 188 189 The argument *workers* specifies how many workers are used to 190 compile files in parallel. The default is to not use multiple workers. 191 If the platform can't use multiple workers and *workers* argument is given, 192 then sequential compilation will be used as a fallback. If *workers* 193 is 0, the number of cores in the system is used. If *workers* is 194 lower than ``0``, a :exc:`ValueError` will be raised. 195 196 *invalidation_mode* should be a member of the 197 :class:`py_compile.PycInvalidationMode` enum and controls how the generated 198 pycs are invalidated at runtime. 199 200 The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to 201 the ``-s``, ``-p`` and ``-e`` options described above. 202 They may be specified as ``str`` or :py:class:`os.PathLike`. 203 204 If *hardlink_dupes* is true and two ``.pyc`` files with different optimization 205 level have the same content, use hard links to consolidate duplicate files. 206 207 .. versionchanged:: 3.2 208 Added the *legacy* and *optimize* parameter. 209 210 .. versionchanged:: 3.5 211 Added the *workers* parameter. 212 213 .. versionchanged:: 3.5 214 *quiet* parameter was changed to a multilevel value. 215 216 .. versionchanged:: 3.5 217 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 218 no matter what the value of *optimize* is. 219 220 .. versionchanged:: 3.6 221 Accepts a :term:`path-like object`. 222 223 .. versionchanged:: 3.7 224 The *invalidation_mode* parameter was added. 225 226 .. versionchanged:: 3.7.2 227 The *invalidation_mode* parameter's default value is updated to None. 228 229 .. versionchanged:: 3.8 230 Setting *workers* to 0 now chooses the optimal number of cores. 231 232 .. versionchanged:: 3.9 233 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments. 234 Default value of *maxlevels* was changed from ``10`` to ``sys.getrecursionlimit()`` 235 236.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False) 237 238 Compile the file with path *fullname*. Return a true value if the file 239 compiled successfully, and a false value otherwise. 240 241 If *ddir* is given, it is prepended to the path to the file being compiled 242 for use in compilation time tracebacks, and is also compiled in to the 243 byte-code file, where it will be used in tracebacks and other messages in 244 cases where the source file does not exist at the time the byte-code file is 245 executed. 246 247 If *rx* is given, its ``search`` method is passed the full path name to the 248 file being compiled, and if it returns a true value, the file is not 249 compiled and ``True`` is returned. This can be used to exclude files matching 250 a regular expression, given as a :ref:`re.Pattern <re-objects>` object. 251 252 If *quiet* is ``False`` or ``0`` (the default), the filenames and other 253 information are printed to standard out. Set to ``1``, only errors are 254 printed. Set to ``2``, all output is suppressed. 255 256 If *legacy* is true, byte-code files are written to their legacy locations 257 and names, which may overwrite byte-code files created by another version of 258 Python. The default is to write files to their :pep:`3147` locations and 259 names, which allows byte-code files from multiple versions of Python to 260 coexist. 261 262 *optimize* specifies the optimization level for the compiler. It is passed to 263 the built-in :func:`compile` function. Accepts also a sequence of optimization 264 levels which lead to multiple compilations of one :file:`.py` file in one call. 265 266 *invalidation_mode* should be a member of the 267 :class:`py_compile.PycInvalidationMode` enum and controls how the generated 268 pycs are invalidated at runtime. 269 270 The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to 271 the ``-s``, ``-p`` and ``-e`` options described above. 272 They may be specified as ``str`` or :py:class:`os.PathLike`. 273 274 If *hardlink_dupes* is true and two ``.pyc`` files with different optimization 275 level have the same content, use hard links to consolidate duplicate files. 276 277 .. versionadded:: 3.2 278 279 .. versionchanged:: 3.5 280 *quiet* parameter was changed to a multilevel value. 281 282 .. versionchanged:: 3.5 283 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 284 no matter what the value of *optimize* is. 285 286 .. versionchanged:: 3.7 287 The *invalidation_mode* parameter was added. 288 289 .. versionchanged:: 3.7.2 290 The *invalidation_mode* parameter's default value is updated to None. 291 292 .. versionchanged:: 3.9 293 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments. 294 295.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None) 296 297 Byte-compile all the :file:`.py` files found along ``sys.path``. Return a 298 true value if all the files compiled successfully, and a false value otherwise. 299 300 If *skip_curdir* is true (the default), the current directory is not included 301 in the search. All other parameters are passed to the :func:`compile_dir` 302 function. Note that unlike the other compile functions, ``maxlevels`` 303 defaults to ``0``. 304 305 .. versionchanged:: 3.2 306 Added the *legacy* and *optimize* parameter. 307 308 .. versionchanged:: 3.5 309 *quiet* parameter was changed to a multilevel value. 310 311 .. versionchanged:: 3.5 312 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 313 no matter what the value of *optimize* is. 314 315 .. versionchanged:: 3.7 316 The *invalidation_mode* parameter was added. 317 318 .. versionchanged:: 3.7.2 319 The *invalidation_mode* parameter's default value is updated to None. 320 321To force a recompile of all the :file:`.py` files in the :file:`Lib/` 322subdirectory and all its subdirectories:: 323 324 import compileall 325 326 compileall.compile_dir('Lib/', force=True) 327 328 # Perform same compilation, excluding files in .svn directories. 329 import re 330 compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True) 331 332 # pathlib.Path objects can also be used. 333 import pathlib 334 compileall.compile_dir(pathlib.Path('Lib/'), force=True) 335 336.. seealso:: 337 338 Module :mod:`py_compile` 339 Byte-compile a single source file. 340