1# Kernel Library Selective Build 2 3_Selective build_ is a build mode on ExecuTorch that uses model metadata to guide ExecuTorch build. This build mode contains build tool APIs available on CMake. ExecuTorch users can use selective build APIs to build an ExecuTorch runtime binary with minimal binary size by only including operators required by models. 4 5This document aims to help ExecuTorch users better use selective build, by listing out available APIs, providing an overview of high level architecture and showcasing examples. 6 7Preread: [Overview of the ExecuTorch runtime](./runtime-overview.md), [High-level architecture and components of ExecuTorch](./getting-started-architecture.md) 8 9 10## Design Principles 11 12**Why selective build?** Many ExecuTorch use cases are constrained by binary size. Selective build can reduce the binary size of the ExecuTorch runtime without compromising support for a target model. 13 14**What are we selecting?** Our core ExecuTorch library is around 50kB with no operators/kernels or delegates. If we link in kernel libraries such as the ExecuTorch in-house portable kernel library, the binary size of the whole application surges, due to unused kernels being registered into the ExecuTorch runtime. Selective build is able to apply a filter on the kernel libraries, so that only the kernels actually being used are linked, thus reducing the binary size of the application. 15 16**How do we select?** Selective build provides APIs to allow users to pass in _op info_, operator metadata derived from target models. Selective build tools will gather these op info and build a filter for all kernel libraries being linked in. 17 18 19## High Level Architecture 20 21 22 23 24 25 26Note that all of the selective build tools are running at build-time (to be distinguished from compile-time or runtime). Therefore selective build tools only have access to static data from user input or models. 27 28The basic flow looks like this: 29 30 31 321. For each of the models we plan to run, we extract op info from it, either manually or via a Python tool. Op info will be written into yaml files and generated at build time. 332. An _op info aggregator _will collect these model op info and merge them into a single op info yaml file. 343. A _kernel resolver _takes in the linked kernel libraries as well as the merged op info yaml file, then makes a decision on which kernels to be registered into ExecuTorch runtime. 35 36 37## APIs 38 39We expose a CMake macro `[gen_selected_ops](https://github.com/pytorch/executorch/blob/main/build/Codegen.cmake#L12)`, to allow users specifying op info: 40 41``` 42gen_selected_ops( 43 LIB_NAME # the name of the selective build operator library to be generated 44 OPS_SCHEMA_YAML # path to a yaml file containing operators to be selected 45 ROOT_OPS # comma separated operator names to be selected 46 INCLUDE_ALL_OPS # boolean flag to include all operators 47) 48``` 49 50 51### Select all ops 52 53If this input is set to true, it means we are registering all the kernels from all the kernel libraries linked into the application. If set to true it is effectively turning off selective build mode. 54 55 56### Select ops from schema yaml 57 58Context: each kernel library is designed to have a yaml file associated with it. For more information on this yaml file, see [Kernel Library Overview](./kernel-library-overview.md). This API allows users to pass in the schema yaml for a kernel library directly, effectively allowlisting all kernels in the library to be registered. 59 60 61### Select root ops from operator list 62 63This API lets users pass in a list of operator names. Note that this API can be combined with the API above and we will create a allowlist from the union of both API inputs. 64 65 66## Example Walkthrough 67 68In CMakeLists.txt we have the following logic: 69```cmake 70set(_kernel_lib) 71if(SELECT_ALL_OPS) 72 gen_selected_ops("" "" "${SELECT_ALL_OPS}") 73elseif(SELECT_OPS_LIST) 74 gen_selected_ops("" "${SELECT_OPS_LIST}" "") 75elseif(SELECT_OPS_YAML) 76 set(_custom_ops_yaml ${EXECUTORCH_ROOT}/examples/portable/custom_ops/custom_ops.yaml) 77 gen_selected_ops("${_custom_ops_yaml}" "" "") 78endif() 79``` 80Then when calling CMake, we can do: 81 82``` 83cmake -D… -DSELECT_OPS_LIST="aten::add.out,aten::mm.out” 84``` 85 86Or 87 88``` 89cmake -D… -DSELECT_OPS_YAML=ON 90``` 91 92To select from either an operator name list or a schema yaml from kernel library. 93
