1Android 2======= 3 4Mesa hardware drivers can be built for Android one of two ways: built 5into the Android OS using the ndk-build build system on older versions 6of Android, or out-of-tree using the Meson build system and the 7Android NDK. 8 9The ndk-build build system has proven to be hard to maintain, as one 10needs a built Android tree to build against, and it has never been 11tested in CI. The Meson build system flow is frequently used by 12Chrome OS developers for building and testing Android drivers. 13 14When building llvmpipe or lavapipe for Android the ndk-build workflow 15is also used, but there are additional steps required to add the driver 16to the Android OS image. 17 18Building using the Android NDK 19------------------------------ 20 21Download and install the NDK using whatever method you normally would. 22Then, create your Meson cross file to use it, something like this 23``~/.local/share/meson/cross/android-aarch64`` file: 24 25.. code-block:: ini 26 27 [binaries] 28 ar = 'NDKDIR/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android-ar' 29 c = ['ccache', 'NDKDIR/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android29-clang'] 30 cpp = ['ccache', 'NDKDIR/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android29-clang++', '-fno-exceptions', '-fno-unwind-tables', '-fno-asynchronous-unwind-tables', '-static-libstdc++'] 31 c_ld = 'lld' 32 cpp_ld = 'lld' 33 strip = 'NDKDIR/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android-strip' 34 # Android doesn't come with a pkg-config, but we need one for Meson to be happy not 35 # finding all the optional deps it looks for. Use system pkg-config pointing at a 36 # directory we get to populate with any .pc files we want to add for Android 37 pkg-config = ['env', 'PKG_CONFIG_LIBDIR=NDKDIR/pkgconfig', '/usr/bin/pkg-config'] 38 39 [host_machine] 40 system = 'android' 41 cpu_family = 'aarch64' 42 cpu = 'armv8' 43 endian = 'little' 44 45Now, use that cross file for your Android build directory (as in this 46one cross-compiling the turnip driver for a stock Pixel phone) 47 48.. code-block:: sh 49 50 meson setup build-android-aarch64 \ 51 --cross-file android-aarch64 \ 52 -Dplatforms=android \ 53 -Dplatform-sdk-version=26 \ 54 -Dandroid-stub=true \ 55 -Dgallium-drivers= \ 56 -Dvulkan-drivers=freedreno \ 57 -Dfreedreno-kmds=kgsl 58 meson compile -C build-android-aarch64 59 60Replacing Android drivers on stock Android 61------------------------------------------ 62 63The vendor partition with the drivers is normally mounted from a 64read-only disk image on ``/vendor``. To be able to replace them for 65driver development, we need to unlock the device and remount 66``/vendor`` read/write. 67 68.. code-block:: sh 69 70 adb disable-verity 71 adb reboot 72 adb remount -R 73 adb remount 74 75Now you can replace drivers as in: 76 77.. code-block:: sh 78 79 adb push build-android-aarch64/src/freedreno/vulkan/libvulkan_freedreno.so /vendor/lib64/hw/vulkan.sdm710.so 80 81Note this command doesn't quite work because libvulkan wants the 82SONAME to match. You can use ``patchelf`` to fix this: 83 84.. code-block:: sh 85 86 cp build-android-aarch64/src/freedreno/vulkan/libvulkan_freedreno.so /tmp/vulkan.sdm710.so 87 patchelf --set-soname vulkan.sdm710.so /tmp/vulkan.sdm710.so 88 adb push /tmp/vulkan.sdm710.so /vendor/lib64/hw/ 89 90Replacing Android drivers on Chrome OS 91-------------------------------------- 92 93Chrome OS's ARC++ is an Android container with hardware drivers inside 94of it. The vendor partition with the drivers is normally mounted from 95a read-only squashfs image on disk. For doing rapid driver 96development, you don't want to regenerate that image. So, we'll take 97the existing squashfs image, copy it out on the host, and then use a 98bind mount instead of a loopback mount so we can update our drivers 99using scp from outside the container. 100 101On your device, you'll want to make ``/`` read-write. ssh in as root 102and run: 103 104.. code-block:: sh 105 106 crossystem dev_boot_signed_only=0 107 /usr/share/vboot/bin/make_dev_ssd.sh --remove_rootfs_verification --partitions 4 108 reboot 109 110Then, we'll switch Android from using an image for ``/vendor`` to using a 111bind-mount from a directory we control. 112 113.. code-block:: sh 114 115 cd /opt/google/containers/android/ 116 mkdir vendor-ro 117 mount -o loop vendor.raw.img vendor-ro 118 cp -a vendor-ro vendor-rw 119 emacs config.json 120 121In the ``config.json``, you want to find the block for ``/vendor`` and 122change it to:: 123 124 { 125 "destination": "/vendor", 126 "type": "bind", 127 "source": "/opt/google/containers/android/vendor-rw", 128 "options": [ 129 "bind", 130 "rw" 131 ] 132 }, 133 134Now, restart the UI to do a full reload: 135 136.. code-block:: sh 137 138 restart ui 139 140At this point, your android container is restarted with your new 141bind-mount ``/vendor``, and if you use ``android-sh`` to shell into it 142then the ``mount`` command should show:: 143 144 /dev/root on /vendor type ext2 (rw,seclabel,relatime) 145 146Now, replacing your DRI driver with a new one built for Android should 147be a matter of: 148 149.. code-block:: sh 150 151 scp msm_dri.so $HOST:/opt/google/containers/android/vendor-rw/lib64/dri/ 152 153You can do your build of your DRI driver using ``emerge-$BOARD 154arc-mesa-freedreno`` (for example) if you have a source tree with 155ARC++, but it should also be possible to build using the NDK as 156described above. There are currently rough edges with this, for 157example the build will require that you have your arc-libdrm build 158available to the NDK, assuming you're building anything but the 159Freedreno Vulkan driver for KGSL. You can mostly put things in place 160with: 161 162.. code-block:: sh 163 164 scp $HOST:/opt/google/containers/android/vendor-rw/lib64/libdrm.so \ 165 NDKDIR/sysroot/usr/lib/aarch64-linux-android/lib/ 166 167 ln -s \ 168 /usr/include/xf86drm.h \ 169 /usr/include/libsync.h \ 170 /usr/include/libdrm \ 171 NDKDIR/sysroot/usr/include/ 172 173It seems that new invocations of an application will often reload the 174DRI driver, but depending on the component you're working on you may 175find you need to reload the whole Android container. To do so without 176having to log in to Chrome again every time, you can just kill the 177container and let it restart: 178 179.. code-block:: sh 180 181 kill $(cat /run/containers/android-run_oci/container.pid ) 182 183Adding drivers to Android OS image 184---------------------------------- 185 186When building your own Android OS images it's possible to add 187drivers built out of tree directly into the OS image. For 188running llvmpipe and lavapipe on Android this step is required 189to ensure Android is able to load the drivers correctly. 190 191The following steps provide and example for building 192the android cuttlefish image following the official Android 193documentation from https://source.android.com/docs/setup 194 195When building llvmpipe or lavapipe for Android, it is required 196to do this so that the permissions for accessing the library 197are set correctly. 198 199Following the Android documentation, we can run the following 200commands 201 202.. code-block:: sh 203 204 repo init -b main -u https://android.googlesource.com/platform/manifest 205 repo sync -c -j8 206 207 source build/envsetup.sh 208 lunch aosp_cf_x86_64_phone-trunk_staging-userdebug 209 210Be aware that the sync command can take a long time to run as 211it will download all of the source code. This will set up 212the ``aosp_cf_x86_64_phone-trunk_staging-userdebug`` build target 213for Android. Please note that the x86_64 cuttlefish target will require 214you to build mesa for 32bit and 64bit. Next we need to copy the build 215driver libraries into the source tree of Android and patch the binary names. 216Note that as of ``9b7bb6cc9fa``, libgallium will include the build tag in the 217name, so the name of that library will need to match the tag used in the build. 218 219.. code-block:: sh 220 221 mkdir prebuilts/mesa 222 mkdir prebuilts/mesa/x86_64 223 mkdir prebuilts/mesa/x86 224 cp ${INSTALL_PREFIX_64}/lib/libEGL.so prebuilts/mesa/x86_64/ 225 cp ${INSTALL_PREFIX_64}/lib/libglapi.so prebuilts/mesa/x86_64/ 226 cp ${INSTALL_PREFIX_64}/lib/libgallium-24.3.0-devel.so prebuilts/mesa/x86_64/ 227 cp ${INSTALL_PREFIX_64}/lib/libGLESv1_CM.so prebuilts/mesa/x86_64/ 228 cp ${INSTALL_PREFIX_64}/lib/libGLESv2.so prebuilts/mesa/x86_64/ 229 cp ${INSTALL_PREFIX_64}/lib/libvulkan_lvp.so prebuilts/mesa/x86_64/ 230 cp ${INSTALL_PREFIX_32}/lib/libEGL.so prebuilts/mesa/x86 231 cp ${INSTALL_PREFIX_32}/lib/libglapi.so prebuilts/mesa/x86 232 cp ${INSTALL_PREFIX_32}/lib/libgallium-24.3.0-devel.so prebuilts/mesa/x86/ 233 cp ${INSTALL_PREFIX_32}/lib/libGLESv1_CM.so prebuilts/mesa/x86 234 cp ${INSTALL_PREFIX_32}/lib/libGLESv2.so prebuilts/mesa/x86 235 cp ${INSTALL_PREFIX_32}/lib/libvulkan_lvp.so prebuilts/mesa/x86 236 237 patchelf --set-soname libEGL_lp.so prebuilts/mesa/x86_64/libEGL.so 238 patchelf --set-soname libGLESv1_CM_lp.so prebuilts/mesa/x86_64/libGLESv1_CM.so 239 patchelf --set-soname libGLESv2_lp.so prebuilts/mesa/x86_64/libGLESv2.so 240 patchelf --set-soname vulkan.lvp.so prebuilts/mesa/x86_64/libvulkan_lvp.so 241 patchelf --set-soname libEGL_lp.so prebuilts/mesa/x86/libEGL.so 242 patchelf --set-soname libGLESv1_CM_lp.so prebuilts/mesa/x86/libGLESv1_CM.so 243 patchelf --set-soname libGLESv2_lp.so prebuilts/mesa/x86/libGLESv2.so 244 patchelf --set-soname vulkan.lvp.so prebuilts/mesa/x86/libvulkan_lvp.so 245 246We then need to create an ``prebuilts/mesa/Android.bp`` build file to include 247the libraries in the build. 248 249.. code-block:: 250 251 cc_prebuilt_library_shared { 252 name: "libglapi", 253 arch: { 254 x86_64: { 255 srcs: ["x86_64/libglapi.so"], 256 }, 257 x86: { 258 srcs: ["x86/libglapi.so"], 259 }, 260 }, 261 strip: { 262 none: true, 263 }, 264 relative_install_path: "egl", 265 shared_libs: ["libc", "libdl", "liblog", "libm"], 266 vendor: true 267 } 268 269 cc_prebuilt_library_shared { 270 name: "libgallium-24.3.0-devel", 271 arch: { 272 x86_64: { 273 srcs: ["x86_64/libgallium-24.3.0-devel.so"], 274 }, 275 x86: { 276 srcs: ["x86/libgallium-24.3.0-devel.so"], 277 }, 278 }, 279 strip: { 280 none: true, 281 }, 282 relative_install_path: "egl", 283 shared_libs: ["libc", "libdl", "liblog", "libm"], 284 check_elf_files: false, 285 vendor: true 286 } 287 288 cc_prebuilt_library_shared { 289 name: "libEGL_lp", 290 arch: { 291 x86_64: { 292 srcs: ["x86_64/libEGL.so"], 293 }, 294 x86: { 295 srcs: ["x86/libEGL.so"], 296 }, 297 }, 298 strip: { 299 none: true, 300 }, 301 relative_install_path: "egl", 302 shared_libs: ["libc", "libdl", "liblog", "libm", "libcutils", "libdrm", "libhardware", "liblog", "libnativewindow", "libsync"], 303 check_elf_files: false, 304 vendor: true 305 } 306 307 cc_prebuilt_library_shared { 308 name: "libGLESv1_CM_lp", 309 arch: { 310 x86_64: { 311 srcs: ["x86_64/libGLESv1_CM.so"], 312 }, 313 x86: { 314 srcs: ["x86/libGLESv1_CM.so"], 315 }, 316 }, 317 strip: { 318 none: true, 319 }, 320 relative_install_path: "egl", 321 shared_libs: ["libc", "libdl", "liblog", "libm"], 322 check_elf_files: false, 323 vendor: true 324 } 325 326 cc_prebuilt_library_shared { 327 name: "libGLESv2_lp", 328 arch: { 329 x86_64: { 330 srcs: ["x86_64/libGLESv2.so"], 331 }, 332 x86: { 333 srcs: ["x86_64/libGLESv2.so"], 334 }, 335 }, 336 strip: { 337 none: true, 338 }, 339 relative_install_path: "egl", 340 shared_libs: ["libc", "libdl", "liblog", "libm"], 341 check_elf_files: false, 342 vendor: true 343 } 344 345 cc_prebuilt_library_shared { 346 name: "vulkan.lvp", 347 arch: { 348 x86_64: { 349 srcs: ["x86_64/libvulkan_lvp.so"], 350 }, 351 x86: { 352 srcs: ["x86/libvulkan_lvp.so"], 353 }, 354 }, 355 strip: { 356 none: true, 357 }, 358 relative_install_path: "hw", 359 shared_libs: ["libc", "libdl", "liblog", "libm", "libcutils", "libdrm", "liblog", "libnativewindow", "libsync", "libz"], 360 vendor: true 361 } 362 363 364Next we need to update the device configuration to include the libraries 365in the build, as well as set the appropriate system properties. We can 366create the file 367``device/google/cuttlefish/shared/mesa/device_vendor.mk`` 368 369 370.. code-block:: makefile 371 372 PRODUCT_SOONG_NAMESPACES += prebuilts/mesa 373 PRODUCT_PACKAGES += libglapi \ 374 libGLESv1_CM_lp \ 375 libGLESv2_lp \ 376 libEGL_lp \ 377 libgallium-24.3.0-devel.so \ 378 vulkan.lvp 379 PRODUCT_VENDOR_PROPERTIES += \ 380 ro.hardware.egl=lp \ 381 ro.hardware.vulkan=lvp \ 382 mesa.libgl.always.software=true \ 383 mesa.android.no.kms.swrast=true \ 384 debug.hwui.renderer=opengl \ 385 ro.gfx.angle.supported=false \ 386 debug.sf.disable_hwc_vds=1 \ 387 ro.vendor.hwcomposer.mode=client 388 389Also the file ``device/google/cuttlefish/shared/mesa/BoardConfig.mk`` 390 391.. code-block:: makefile 392 393 BOARD_VENDOR_SEPOLICY_DIRS += \ 394 device/google/cuttlefish/shared/mesa/sepolicy 395 396Next the file ``device/google/cuttlefish/shared/mesa/sepolicy/file_contexts`` 397 398.. code-block:: sh 399 400 /vendor/lib(64)?/egl/libEGL_lp\.so u:object_r:same_process_hal_file:s0 401 /vendor/lib(64)?/egl/libGLESv1_CM_lp\.so u:object_r:same_process_hal_file:s0 402 /vendor/lib(64)?/egl/libGLESv2_lp\.so u:object_r:same_process_hal_file:s0 403 /vendor/lib(64)?/egl/libglapi\.so u:object_r:same_process_hal_file:s0 404 /vendor/lib(64)?/egl/libgallium\-24.3.0\-devel\.so u:object_r:same_process_hal_file:s0 405 /vendor/lib(64)?/hw/vulkan\.lvp\.so u:object_r:same_process_hal_file:s0 406 407After creating these files we need to modify the existing config files 408to include these build files. First we modify 409``device/google/cuttlefish/shared/phone/device_vendor.mk`` 410to add the below code in the spot where other device_vendor 411files are included. 412 413.. code-block:: sh 414 415 $(call inherit-product, device/google/cuttlefish/shared/mesa/device_vendor.mk) 416 417Lastly we modify 418``device/google/cuttlefish/vsoc_x86_64/BoardConfig.mk`` to include 419the following line where the other BoardConfig files are included 420 421.. code-block:: sh 422 423 -include device/google/cuttlefish/shared/mesa/BoardConfig.mk 424 425Then we are set to continue following the official instructions to 426build the cuttlefish target and run it in the cuttlefish emulator. 427