1.. _home: 2 3Welcome to the ExecuTorch Documentation 4======================================= 5 6.. important:: 7 v0.4.0 is a beta release of ExecuTorch. As of this release, the API will 8 follow the `API Lifecycle and Deprecation Policy <api-life-cycle.html>`__, 9 and the ``.pte`` binary format will comply with the `Runtime Compatibility 10 Policy 11 <https://github.com/pytorch/executorch/tree/main/runtime/COMPATIBILITY.md>`__. 12 This helps ensure that application developers can update to the latest 13 version of ExecuTorch without breaking existing integration code, in 14 accordance with these policies. If any issues arise or compatibility breaks 15 occur, please `report them in GitHub 16 <https://github.com/pytorch/executorch/issues/new/choose>`__. 17 18 We welcome any feedback, suggestions, and bug reports from the community 19 to help us improve the technology. Please use the `PyTorch Forums 20 <https://discuss.pytorch.org/c/executorch>`__ for discussion and feedback 21 about ExecuTorch using the **ExecuTorch** category, and our `GitHub 22 repository <https://github.com/pytorch/executorch/issues>`__ for bug 23 reporting. 24 25.. raw:: html 26 27 <div class="et-page-column-row"> 28 <div class="et-page-column1"><p><strong>ExecuTorch</strong> is a PyTorch platform that provides infrastructure to run PyTorch programs everywhere from AR/VR wearables to standard on-device iOS and Android mobile deployments. One of the main goals for ExecuTorch is to enable wider customization and deployment capabilities of the PyTorch programs.</p> 29 <p>ExecuTorch heavily relies on such PyTorch technologies as <a href="https://pytorch.org/docs/stable/torch.compiler.html">torch.compile</a> and <a href="https://pytorch.org/docs/main/export.html">torch.export</a>. If you are not familiar with these APIs, you might want to read about them in the PyTorch documentation before diving into the ExecuTorch documentation.</p></div> 30 <div class="et-page-column2"><img src="_static/img/ExecuTorch-Logo-cropped.svg" alt="ExecuTorch logo" title="ExecuTorch logo"></div> 31 </div> 32 33The ExecuTorch source is hosted on GitHub at 34https://github.com/pytorch/executorch. 35 36Getting Started 37~~~~~~~~~~~~~~~ 38 39Topics in this section will help you get started with ExecuTorch. 40 41.. grid:: 3 42 43 .. grid-item-card:: :octicon:`file-code;1em` 44 What is ExecuTorch? 45 :img-top: _static/img/card-background.svg 46 :link: intro-overview.html 47 :link-type: url 48 49 A gentle introduction to ExecuTorch. In this section, 50 you will learn about main features of ExecuTorch 51 and how you can use them in your projects. 52 53 .. grid-item-card:: :octicon:`file-code;1em` 54 Getting started with ExecuTorch 55 :img-top: _static/img/card-background.svg 56 :link: getting-started-setup.html 57 :link-type: url 58 59 A step-by-step tutorial on how to get started with 60 ExecuTorch. 61 62 .. grid-item-card:: :octicon:`file-code;1em` 63 ExecuTorch Llama 64 :img-top: _static/img/card-background.svg 65 :link: llm/llama.html 66 :link-type: url 67 68 Learn about running Llama models via ExecuTorch 69 70.. toctree:: 71 :glob: 72 :maxdepth: 1 73 :caption: Introduction 74 :hidden: 75 76 intro-overview 77 intro-how-it-works 78 getting-started-architecture 79 concepts 80 81.. toctree:: 82 :glob: 83 :maxdepth: 1 84 :caption: Getting Started 85 :hidden: 86 87 getting-started-setup 88 export-overview 89 runtime-build-and-cross-compilation 90 91 92.. toctree:: 93 :glob: 94 :maxdepth: 1 95 :caption: Tutorials 96 :hidden: 97 98 tutorials/export-to-executorch-tutorial 99 running-a-model-cpp-tutorial 100 extension-module 101 extension-tensor 102 tutorials/devtools-integration-tutorial 103 apple-runtime 104 demo-apps-ios 105 demo-apps-android 106 examples-end-to-end-to-lower-model-to-delegate 107 tutorial-xnnpack-delegate-lowering 108 build-run-vulkan 109 .. 110 Alphabetical by backend name. Be sure to keep the same order in the 111 customcarditem entries below. 112 executorch-arm-delegate-tutorial 113 build-run-coreml 114 build-run-mediatek-backend 115 build-run-mps 116 build-run-qualcomm-ai-engine-direct-backend 117 build-run-xtensa 118 119.. toctree:: 120 :glob: 121 :maxdepth: 2 122 :caption: Working with LLMs 123 :hidden: 124 125 Llama <llm/llama> 126 Llama on Android <llm/llama-demo-android> 127 Llama on iOS <llm/llama-demo-ios> 128 Llama on Android via Qualcomm backend <llm/build-run-llama3-qualcomm-ai-engine-direct-backend> 129 Intro to LLMs in Executorch <llm/getting-started> 130 131.. toctree:: 132 :glob: 133 :maxdepth: 1 134 :caption: API Reference 135 :hidden: 136 137 export-to-executorch-api-reference 138 executorch-runtime-api-reference 139 runtime-python-api-reference 140 api-life-cycle 141 142.. toctree:: 143 :glob: 144 :maxdepth: 1 145 :caption: IR Specification 146 :hidden: 147 148 ir-exir 149 ir-ops-set-definition 150 151.. toctree:: 152 :glob: 153 :maxdepth: 1 154 :caption: Compiler Entry Points 155 :hidden: 156 157 compiler-delegate-and-partitioner 158 compiler-backend-dialect 159 compiler-custom-compiler-passes 160 compiler-memory-planning 161 162.. toctree:: 163 :glob: 164 :maxdepth: 1 165 :caption: Runtime 166 :hidden: 167 168 runtime-overview 169 runtime-backend-delegate-implementation-and-linking 170 runtime-platform-abstraction-layer 171 portable-cpp-programming 172 pte-file-format 173 174.. toctree:: 175 :glob: 176 :maxdepth: 1 177 :caption: Quantization 178 :hidden: 179 180 quantization-overview 181 182.. toctree:: 183 :glob: 184 :maxdepth: 1 185 :caption: Kernel Library 186 :hidden: 187 188 kernel-library-overview 189 kernel-library-custom-aten-kernel 190 kernel-library-selective-build 191 192.. toctree:: 193 :glob: 194 :maxdepth: 1 195 :caption: Backend Delegates 196 :hidden: 197 198 native-delegates-executorch-xnnpack-delegate 199 native-delegates-executorch-vulkan-delegate 200 backend-delegates-integration 201 backend-delegates-dependencies 202 debug-backend-delegate 203 204.. toctree:: 205 :glob: 206 :maxdepth: 1 207 :caption: Developer Tools 208 :hidden: 209 210 devtools-overview 211 bundled-io 212 etrecord 213 etdump 214 runtime-profiling 215 model-debugging 216 model-inspector 217 memory-planning-inspection 218 delegate-debugging 219 devtools-tutorial 220 221.. toctree:: 222 :glob: 223 :maxdepth: 1 224 :caption: Contributing 225 :hidden: 226 227 contributing 228 229Tutorials and Examples 230~~~~~~~~~~~~~~~~~~~~~~ 231 232Ready to experiment? Check out some of the 233ExecuTorch tutorials. 234 235.. customcardstart:: 236 237.. customcarditem:: 238 :header: Exporting to ExecuTorch Tutorial 239 :card_description: A tutorial for exporting a model and lowering a it to be runnable on the ExecuTorch runtime. 240 :image: _static/img/generic-pytorch-logo.png 241 :link: tutorials/export-to-executorch-tutorial.html 242 :tags: Export,Delegation,Quantization 243 244.. customcarditem:: 245 :header: Running an ExecuTorch Model C++ Tutorial 246 :card_description: A tutorial for setting up memory pools, loading a model, setting inputs, executing the model, and retrieving outputs on device. 247 :image: _static/img/generic-pytorch-logo.png 248 :link: running-a-model-cpp-tutorial.html 249 :tags: 250 251.. customcarditem:: 252 :header: Simplified Runtime APIs Tutorial 253 :card_description: A simplified tutorial for executing the model on device. 254 :image: _static/img/generic-pytorch-logo.png 255 :link: extension-module.html 256 :tags: 257 258.. customcarditem:: 259 :header: Managing Tensor Memory in C++ Tutorial 260 :card_description: A tutorial for managing the dynamic memory when working with tensors. 261 :image: _static/img/generic-pytorch-logo.png 262 :link: extension-tensor.html 263 :tags: 264 265.. customcarditem:: 266 :header: Using the ExecuTorch Developer Tools to Profile a Model 267 :card_description: A tutorial for using the ExecuTorch Developer Tools to profile and analyze a model with linkage back to source code. 268 :image: _static/img/generic-pytorch-logo.png 269 :link: tutorials/devtools-integration-tutorial.html 270 :tags: devtools 271 272.. customcarditem:: 273 :header: Integrating and Running ExecuTorch on Apple Platforms 274 :card_description: A tutorial on integrating, using, and troubleshooting the ExecuTorch runtime on iOS. 275 :image: _static/img/generic-pytorch-logo.png 276 :link: apple-runtime.html 277 :tags: iOS, macOS 278 279.. customcarditem:: 280 :header: Building an ExecuTorch iOS Demo App 281 :card_description: A demo tutorial that explains how to build ExecuTorch into iOS frameworks and run an iOS app. 282 :image: _static/img/demo_ios_app.jpg 283 :link: demo-apps-ios.html 284 :tags: Delegation,iOS 285 286.. customcarditem:: 287 :header: Building an ExecuTorch Android Demo App 288 :card_description: A demo tutorial that explains how to build ExecuTorch into a JNI library and build an Android app. 289 :image: _static/img/android_app.png 290 :link: demo-apps-android.html 291 :tags: Delegation,Android 292 293.. customcarditem:: 294 :header: Lowering a Model as a Delegate 295 :card_description: An end-to-end example showing how to lower a model as a delegate 296 :image: _static/img/generic-pytorch-logo.png 297 :link: examples-end-to-end-to-lower-model-to-delegate.html 298 :tags: Export,Delegation 299 300.. 301 First-party backends that are good intros for readers. 302 303.. customcarditem:: 304 :header: Building and Running ExecuTorch with XNNPACK Backend 305 :card_description: A demo tutorial for lowering and exporting models with the XNNPACK Backend 306 :image: _static/img/generic-pytorch-logo.png 307 :link: tutorial-xnnpack-delegate-lowering.html 308 :tags: Export,Backend,Delegation,Quantization,XNNPACK 309 310.. customcarditem:: 311 :header: Building and Running ExecuTorch with Vulkan Backend 312 :card_description: A tutorial that walks you through the process of building ExecuTorch with Vulkan Backend 313 :image: _static/img/generic-pytorch-logo.png 314 :link: build-run-vulkan.html 315 :tags: Export,Backend,Delegation,Vulkan 316 317.. 318 Alphabetical by backend name. Be sure to keep the same order in the Tutorials 319 toctree entry above. 320 321.. customcarditem:: 322 :header: Building and Running ExecuTorch with ARM Ethos-U Backend 323 :card_description: A tutorial that walks you through the process of building ExecuTorch with ARM Ethos-U Backend 324 :image: _static/img/generic-pytorch-logo.png 325 :link: executorch-arm-delegate-tutorial.html 326 :tags: Export,Backend,Delegation,ARM,Ethos-U 327 328.. customcarditem:: 329 :header: Building and Running ExecuTorch with CoreML Backend 330 :card_description: A tutorial that walks you through the process of building ExecuTorch with CoreML Backend 331 :image: _static/img/generic-pytorch-logo.png 332 :link: build-run-coreml.html 333 :tags: Export,Backend,Delegation,CoreML 334 335.. customcarditem:: 336 :header: Building and Running ExecuTorch with MediaTek Backend 337 :card_description: A tutorial that walks you through the process of building ExecuTorch with MediaTek Backend 338 :image: _static/img/generic-pytorch-logo.png 339 :link: build-run-mediatek-backend.html 340 :tags: Export,Backend,Delegation,MediaTek 341 342.. customcarditem:: 343 :header: Building and Running ExecuTorch with MPS Backend 344 :card_description: A tutorial that walks you through the process of building ExecuTorch with MPSGraph Backend 345 :image: _static/img/generic-pytorch-logo.png 346 :link: build-run-mps.html 347 :tags: Export,Backend,Delegation,MPS,MPSGraph 348 349.. customcarditem:: 350 :header: Building and Running ExecuTorch with Qualcomm AI Engine Direct Backend 351 :card_description: A tutorial that walks you through the process of building ExecuTorch with Qualcomm AI Engine Direct Backend 352 :image: _static/img/generic-pytorch-logo.png 353 :link: build-run-qualcomm-ai-engine-direct-backend.html 354 :tags: Export,Backend,Delegation,QNN 355 356.. customcarditem:: 357 :header: Building and Running ExecuTorch on Xtensa HiFi4 DSP 358 :card_description: A tutorial that walks you through the process of building ExecuTorch for an Xtensa Hifi4 DSP using custom operators 359 :image: _static/img/generic-pytorch-logo.png 360 :link: build-run-xtensa.html 361 :tags: Export,Custom-Operators,DSP,Xtensa 362 363.. customcardend:: 364
