Lines Matching full:backend
1 # Backend developer guide
3 Arm NN allows adding new backends through the 'Pluggable Backend' mechanism.
5 ## How to add a new backend
7 …/backends](./), in separate subfolders. For Linux builds they must have a ```backend.cmake``` file,
8 which is read automatically by [src/backends/backends.cmake](backends.cmake). The ```backend.cmake`…
9 under the backend-specific folder is then included by the main CMakeLists.txt file at the root of t…
12 ### The backend.cmake file argument
14 The ```backend.cmake``` has three main purposes:
17 2. It makes sure that the subdirectory where backend sources reside gets included into the build.
18 3. To include backend-specific unit tests, the object library for the unit tests needs to be added …
20 Example ```backend.cmake``` file taken from [reference/backend.cmake](reference/backend.cmake):
24 # Make sure the reference backend is included in the build.
26 # in the reference (backend) folder.
31 # Add the cmake OBJECT libraries built by the reference backend to the
37 # Backend specific unit tests can be integrated through the
39 # UnitTests executable can run the backend-specific unit
47 As described in the previous section, adding a new backend will require creating a ```CMakeLists.tx…
48 the backend folder. This follows the standard cmake conventions, and is required to build a static …
54 ### The backend.mk file argument
57 ```backend.mk``` file, which has a single variable called ```BACKEND_SOURCES``` listing all cpp
60 Optionally, backend-specific unit tests can be added similarly, by
63 Example taken from [reference/backend.mk](reference/backend.mk):
84 similarly to the backend code. This requires adding three files under a subfolder at the same level
91 They work the same way as the backend files. The only difference between them is that
92 common code is built first, so the backend code can depend on them.
137 For each backend one needs to register a factory function that can
139 The Arm NN framework creates the backend interfaces dynamically when
142 * During optimization Arm NN needs to decide which layers are supported by the backend.
145 * During optimization Arm NN can run backend-specific optimizations. After splitting the graph into
150 * When the LoadedNetwork creates the backend-specific workloads for the layers, it creates a backend
156 about them. Registration requires a unique backend ID string and a lambda function that
159 For registering a backend only this lambda function needs to exist, not the actual backend. This
160 allows dynamically creating the backend objects when they are needed.
163 are able to tell if a given backend is registered or not.
171 supported on a given backend. The backends need a way to communicate this information by implementi…
179 …e [IWorkloadFactory interface](backendsCommon/WorkloadFactory.hpp) is used for creating the backend
194 use it for creating the backend-specific workloads
200 The backends may choose to implement backend-specific optimizations.
202 the backend interface that allows the backends to apply their specific optimizations to a given sub…
207 …representing the part of the original graph that has been optimized by the backend, while the seco…
209 …ed sub-graphs: these are the parts of the original sub-graph that are not supported by the backend,
212 but that can run (unoptimized) on the backend.
221 method for each backend in the Runtime. If the backend returns a valid unique pointer to a backend …
232 intra/inter-layer memory. This particular feature is required if you want a backend to use Protecte…
233 To support this on your own backend you must implement the UseCustomMemoryAllocator interface.
236 the backend. This interface is also used by the lambda function returned by the Backend Registry to…
237 the CustomMemoryAllocator. Within the backend itself there should be a wrapper class to convert the…
238 …emoryAllocator provided by the interface into something that is more suitable for your own backend.
245 This is a list of BackendCapabilities currently supported by the backend. It consists of a constant…
254 To be properly loaded and used, the backend instances must comply to the standard interface for dyn…
259 The dynamic backend shared object must expose the following interface for Arm NN to handle it corre…
274 …of the loading the id already exists in the internal Arm NN's backend registry, the backend will b…
276 * ```GetVersion()```: must return the version of the dynamic backend.
277 The version must indicate the version of the Backend API the dynamic backend has been built with.
278 The current Backend API version can be found by inspecting the IBackendInternal interface.
279 …At the time of loading, the version of the backend will be checked against the version of the Back…
280 …If the backend version is not compatible with the current Backend API, the backend will not be loa…
282 * ```BackendFactory()```: must return a valid instance of the backend.
283 …The backend instance is an object that must inherit from the version of the IBackendInternal inter…
284 …It is the backend developer's responsibility to ensure that the backend implementation correctly r…
288 ## Dynamic backend versioning and ABI compatibility
290 Dynamic backend versioning policy:
292 Updates to Arm NN's Backend API follow these rules: changes to the Backend API (the IBackendInterna…
298 * Dynamic backend version 2.4 (i.e. built with Backend API version 2.4) is compatible with Arm NN's…
299 (same version, backend built against the same Backend API)
300 * Dynamic backend version 2.1 (i.e. built with Backend API version 2.1) is compatible with Arm NN's…
301 (same major version, backend built against earlier compatible API)
302 * Dynamic backend version 2.5 (i.e. built with Backend API version 2.5) is not compatible with Arm …
303 …(same major version, backend built against later incompatible API, backend might require update to…
304 * Dynamic backend version 2.0 (i.e. built with Backend API version 2.0) is not compatible with Arm …
305 (backend requires a completely new API version)
306 * Dynamic backend version 2.0 (i.e. built with Backend API version 2.0) is not compatible with Arm …
307 (backward compatibility in the Backend API is broken)
309 ## Dynamic backend loading paths
311 …time, Arm NN will scan a given set of paths searching for suitable dynamic backend objects to load.
330 dynamic backend loading feature, and no dynamic backends will be loaded into Arm NN's runtime.
332 ## Dynamic backend file naming convention
334 …ct, Arm NN will scan the paths specified for dynamic backend loading searching for suitable backen…
355 However, if those are actually the same dynamic backend, only the first in order of parsing will be…
361 | Arm_GpuAcc_backend.so | valid: basic backend name …
370 | Arm_GpuAcc456_backend.so | valid: digits in backend id are allowe…
372 …_backend.so | not valid: invalid character in backend id |
375 | Arm__backend.so | not valid: missing backend id …
376 | Arm_GpuAcc.so | not valid: missing "backend" at the en…
377 …o | not valid: missing vendor name and backend id |
381 | Arm_CpuAcc_backend.so | valid: basic backend name …
382 | Arm_CpuAcc_backend.so.1 -> Arm_CpuAcc_backend.so | valid: symlink to valid backend file …
386 | pathA/Arm_GpuAcc_backend.so | valid: basic backend name …
391 ## Dynamic backend examples
398 …eful for going through all the use cases that constitute an invalid dynamic backend object, such as
400 that would prevent Arm NN from making use of the dynamic backend.
402 A dynamic implementation of the reference backend is also provided. The source files are:
407 …e simple and straightforward. Since an implementation of this particular backend was already avail…
408 the dynamic version is just a wrapper around the original code that simply returns the backend id, …
409 backend itself via the factory function.
410 For the sake of the example, the source code of the reference backend is used to build the dynamic …
411 dynamic backend), while all the other symbols needed are provided by linking the dynamic backend ag…
413 The makefile used for building the reference dynamic backend is also provided: [CMakeLists.txt](dyn…
415 A unit test that loads the reference backend dynamically and that exercises it is also included in …
417 In the test, a path on the filesystem is scanned for valid dynamic backend files (using the overrid…
418 where only the reference dynamic backend is.
420 …ted in the runtime to represent the newly loaded backend, then the backend is registered in the Ba…
422 …unit test makes sure that the backend is actually registered in Arm NN, before trying to create an…
424 The backend instance is used to verify that everything is in order, testing basic 2D convolution su…
426 At the end of test, the runtime object goes out of scope and the dynamic backend instance is automa…