xref: /aosp_15_r20/external/executorch/docs/source/index.rst (revision 523fa7a60841cd1ecfb9cc4201f1ca8b03ed023a)

.. _home:

Welcome to the ExecuTorch Documentation
=======================================

.. important::
   v0.4.0 is a beta release of ExecuTorch. As of this release, the API will
   follow the `API Lifecycle and Deprecation Policy <api-life-cycle.html>`__,
   and the ``.pte`` binary format will comply with the `Runtime Compatibility
   Policy
   <https://github.com/pytorch/executorch/tree/main/runtime/COMPATIBILITY.md>`__.
   This helps ensure that application developers can update to the latest
   version of ExecuTorch without breaking existing integration code, in
   accordance with these policies. If any issues arise or compatibility breaks
   occur, please `report them in GitHub
   <https://github.com/pytorch/executorch/issues/new/choose>`__.

   We welcome any feedback, suggestions, and bug reports from the community
   to help us improve the technology. Please use the `PyTorch Forums
   <https://discuss.pytorch.org/c/executorch>`__ for discussion and feedback
   about ExecuTorch using the **ExecuTorch** category, and our `GitHub
   repository <https://github.com/pytorch/executorch/issues>`__ for bug
   reporting.

.. raw:: html

   <div class="et-page-column-row">
     <div class="et-page-column1"><p><strong>ExecuTorch</strong> is a PyTorch platform that provides infrastructure to run PyTorch programs everywhere from AR/VR wearables to standard on-device iOS and Android mobile deployments. One of the main goals for ExecuTorch is to enable wider customization and deployment capabilities of the PyTorch programs.</p>
     <p>ExecuTorch heavily relies on such PyTorch technologies as <a href="https://pytorch.org/docs/stable/torch.compiler.html">torch.compile</a> and <a href="https://pytorch.org/docs/main/export.html">torch.export</a>. If you are not familiar with these APIs, you might want to read about them in the PyTorch documentation before diving into the ExecuTorch documentation.</p></div>
     <div class="et-page-column2"><img src="_static/img/ExecuTorch-Logo-cropped.svg" alt="ExecuTorch logo" title="ExecuTorch logo"></div>
   </div>

The ExecuTorch source is hosted on GitHub at
https://github.com/pytorch/executorch.

Getting Started
~~~~~~~~~~~~~~~

Topics in this section will help you get started with ExecuTorch.

.. grid:: 3

     .. grid-item-card:: :octicon:`file-code;1em`
        What is ExecuTorch?
        :img-top: _static/img/card-background.svg
        :link: intro-overview.html
        :link-type: url

        A gentle introduction to ExecuTorch. In this section,
        you will learn about main features of ExecuTorch
        and how you can use them in your projects.

     .. grid-item-card:: :octicon:`file-code;1em`
        Getting started with ExecuTorch
        :img-top: _static/img/card-background.svg
        :link: getting-started-setup.html
        :link-type: url

        A step-by-step tutorial on how to get started with
        ExecuTorch.

     .. grid-item-card:: :octicon:`file-code;1em`
        ExecuTorch Llama
        :img-top: _static/img/card-background.svg
        :link: llm/llama.html
        :link-type: url

        Learn about running Llama models via ExecuTorch

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Introduction
   :hidden:

   intro-overview
   intro-how-it-works
   getting-started-architecture
   concepts

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Getting Started
   :hidden:

   getting-started-setup
   export-overview
   runtime-build-and-cross-compilation


.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Tutorials
   :hidden:

   tutorials/export-to-executorch-tutorial
   running-a-model-cpp-tutorial
   extension-module
   extension-tensor
   tutorials/devtools-integration-tutorial
   apple-runtime
   demo-apps-ios
   demo-apps-android
   examples-end-to-end-to-lower-model-to-delegate
   tutorial-xnnpack-delegate-lowering
   build-run-vulkan
   ..
      Alphabetical by backend name. Be sure to keep the same order in the
      customcarditem entries below.
   executorch-arm-delegate-tutorial
   build-run-coreml
   build-run-mediatek-backend
   build-run-mps
   build-run-qualcomm-ai-engine-direct-backend
   build-run-xtensa

.. toctree::
   :glob:
   :maxdepth: 2
   :caption: Working with LLMs
   :hidden:

   Llama <llm/llama>
   Llama on Android <llm/llama-demo-android>
   Llama on iOS <llm/llama-demo-ios>
   Llama on Android via Qualcomm backend <llm/build-run-llama3-qualcomm-ai-engine-direct-backend>
   Intro to LLMs in Executorch <llm/getting-started>

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: API Reference
   :hidden:

   export-to-executorch-api-reference
   executorch-runtime-api-reference
   runtime-python-api-reference
   api-life-cycle

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: IR Specification
   :hidden:

   ir-exir
   ir-ops-set-definition

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Compiler Entry Points
   :hidden:

   compiler-delegate-and-partitioner
   compiler-backend-dialect
   compiler-custom-compiler-passes
   compiler-memory-planning

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Runtime
   :hidden:

   runtime-overview
   runtime-backend-delegate-implementation-and-linking
   runtime-platform-abstraction-layer
   portable-cpp-programming
   pte-file-format

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Quantization
   :hidden:

   quantization-overview

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Kernel Library
   :hidden:

   kernel-library-overview
   kernel-library-custom-aten-kernel
   kernel-library-selective-build

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Backend Delegates
   :hidden:

   native-delegates-executorch-xnnpack-delegate
   native-delegates-executorch-vulkan-delegate
   backend-delegates-integration
   backend-delegates-dependencies
   debug-backend-delegate

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Developer Tools
   :hidden:

   devtools-overview
   bundled-io
   etrecord
   etdump
   runtime-profiling
   model-debugging
   model-inspector
   memory-planning-inspection
   delegate-debugging
   devtools-tutorial

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Contributing
   :hidden:

   contributing

Tutorials and Examples
~~~~~~~~~~~~~~~~~~~~~~

Ready to experiment? Check out some of the
ExecuTorch tutorials.

.. customcardstart::

.. customcarditem::
   :header: Exporting to ExecuTorch Tutorial
   :card_description: A tutorial for exporting a model and lowering a it to be runnable on the ExecuTorch runtime.
   :image: _static/img/generic-pytorch-logo.png
   :link: tutorials/export-to-executorch-tutorial.html
   :tags: Export,Delegation,Quantization

.. customcarditem::
   :header: Running an ExecuTorch Model C++ Tutorial
   :card_description: A tutorial for setting up memory pools, loading a model, setting inputs, executing the model, and retrieving outputs on device.
   :image: _static/img/generic-pytorch-logo.png
   :link: running-a-model-cpp-tutorial.html
   :tags:

.. customcarditem::
   :header: Simplified Runtime APIs Tutorial
   :card_description: A simplified tutorial for executing the model on device.
   :image: _static/img/generic-pytorch-logo.png
   :link: extension-module.html
   :tags:

.. customcarditem::
   :header: Managing Tensor Memory in C++ Tutorial
   :card_description: A tutorial for managing the dynamic memory when working with tensors.
   :image: _static/img/generic-pytorch-logo.png
   :link: extension-tensor.html
   :tags:

.. customcarditem::
   :header: Using the ExecuTorch Developer Tools to Profile a Model
   :card_description: A tutorial for using the ExecuTorch Developer Tools to profile and analyze a model with linkage back to source code.
   :image: _static/img/generic-pytorch-logo.png
   :link: tutorials/devtools-integration-tutorial.html
   :tags: devtools

.. customcarditem::
   :header: Integrating and Running ExecuTorch on Apple Platforms
   :card_description: A tutorial on integrating, using, and troubleshooting the ExecuTorch runtime on iOS.
   :image: _static/img/generic-pytorch-logo.png
   :link: apple-runtime.html
   :tags: iOS, macOS

.. customcarditem::
   :header: Building an ExecuTorch iOS Demo App
   :card_description: A demo tutorial that explains how to build ExecuTorch into iOS frameworks and run an iOS app.
   :image: _static/img/demo_ios_app.jpg
   :link: demo-apps-ios.html
   :tags: Delegation,iOS

.. customcarditem::
   :header: Building an ExecuTorch Android Demo App
   :card_description: A demo tutorial that explains how to build ExecuTorch into a JNI library and build an Android app.
   :image: _static/img/android_app.png
   :link: demo-apps-android.html
   :tags: Delegation,Android

.. customcarditem::
   :header: Lowering a Model as a Delegate
   :card_description: An end-to-end example showing how to lower a model as a delegate
   :image: _static/img/generic-pytorch-logo.png
   :link: examples-end-to-end-to-lower-model-to-delegate.html
   :tags: Export,Delegation

..
   First-party backends that are good intros for readers.

.. customcarditem::
   :header: Building and Running ExecuTorch with XNNPACK Backend
   :card_description: A demo tutorial for lowering and exporting models with the XNNPACK Backend
   :image: _static/img/generic-pytorch-logo.png
   :link: tutorial-xnnpack-delegate-lowering.html
   :tags: Export,Backend,Delegation,Quantization,XNNPACK

.. customcarditem::
   :header: Building and Running ExecuTorch with Vulkan Backend
   :card_description: A tutorial that walks you through the process of building ExecuTorch with Vulkan Backend
   :image: _static/img/generic-pytorch-logo.png
   :link: build-run-vulkan.html
   :tags: Export,Backend,Delegation,Vulkan

..
   Alphabetical by backend name. Be sure to keep the same order in the Tutorials
   toctree entry above.

.. customcarditem::
   :header: Building and Running ExecuTorch with ARM Ethos-U Backend
   :card_description: A tutorial that walks you through the process of building ExecuTorch with ARM Ethos-U Backend
   :image: _static/img/generic-pytorch-logo.png
   :link: executorch-arm-delegate-tutorial.html
   :tags: Export,Backend,Delegation,ARM,Ethos-U

.. customcarditem::
   :header: Building and Running ExecuTorch with CoreML Backend
   :card_description: A tutorial that walks you through the process of building ExecuTorch with CoreML Backend
   :image: _static/img/generic-pytorch-logo.png
   :link: build-run-coreml.html
   :tags: Export,Backend,Delegation,CoreML

.. customcarditem::
   :header: Building and Running ExecuTorch with MediaTek Backend
   :card_description: A tutorial that walks you through the process of building ExecuTorch with MediaTek Backend
   :image: _static/img/generic-pytorch-logo.png
   :link: build-run-mediatek-backend.html
   :tags: Export,Backend,Delegation,MediaTek

.. customcarditem::
   :header: Building and Running ExecuTorch with MPS Backend
   :card_description: A tutorial that walks you through the process of building ExecuTorch with MPSGraph Backend
   :image: _static/img/generic-pytorch-logo.png
   :link: build-run-mps.html
   :tags: Export,Backend,Delegation,MPS,MPSGraph

.. customcarditem::
   :header: Building and Running ExecuTorch with Qualcomm AI Engine Direct Backend
   :card_description: A tutorial that walks you through the process of building ExecuTorch with Qualcomm AI Engine Direct Backend
   :image: _static/img/generic-pytorch-logo.png
   :link: build-run-qualcomm-ai-engine-direct-backend.html
   :tags: Export,Backend,Delegation,QNN

.. customcarditem::
   :header: Building and Running ExecuTorch on Xtensa HiFi4 DSP
   :card_description: A tutorial that walks you through the process of building ExecuTorch for an Xtensa Hifi4 DSP using custom operators
   :image: _static/img/generic-pytorch-logo.png
   :link: build-run-xtensa.html
   :tags: Export,Custom-Operators,DSP,Xtensa

.. customcardend::