1# Chromium's Java Toolchain 2 3This doc aims to describe the Chrome build process that takes a set of `.java` 4files and turns them into a `classes.dex` file. 5 6[TOC] 7 8## Core GN Target Types 9 10The following have `supports_android` and `requires_android` set to false by 11default: 12* `java_library()`: Compiles `.java` -> `.jar` 13* `java_prebuilt()`: Imports a prebuilt `.jar` file. 14 15The following have `supports_android` and `requires_android` set to true. They 16also have a default `jar_excluded_patterns` set (more on that later): 17* `android_library()` 18* `android_java_prebuilt()` 19 20All target names must end with "_java" so that the build system can distinguish 21them from non-java targets (or [other variations](https://cs.chromium.org/chromium/src/build/config/android/internal_rules.gni?rcl=ec2c17d7b4e424e060c3c7972842af87343526a1&l=20)). 22 23Most targets produce two separate `.jar` files: 24* Device `.jar`: Used to produce `.dex.jar`, which is used on-device. 25* Host `.jar`: For use on the host machine (`junit_binary` / `java_binary`). 26 * Host `.jar` files live in `lib.java/` so that they are archived in 27 builder/tester bots (which do not archive `obj/`). 28 29## From Source to Final Dex 30 31### Step 1: Create interface .jar with turbine or ijar 32 33What are interface jars?: 34 35* They contain `.class` files with all private symbols and all method bodies 36 removed. 37* Dependant targets use interface `.jar` files to skip having to be rebuilt 38 when only private implementation details change. 39 40For prebuilt `.jar` files: we use [//third_party/ijar] to create interface 41`.jar` files from the prebuilt ones. 42 43For non-prebuilt `.jar` files`: we use [//third_party/turbine] to create 44interface `.jar` files directly from `.java` source files. Turbine is faster 45than javac because it does not compile method bodies. Although Turbine causes 46us to compile files twice, it speeds up builds by allowing `javac` compilation 47of targets to happen concurrently with their dependencies. We also use Turbine 48to run our annotation processors. 49 50[//third_party/ijar]: /third_party/ijar/README.chromium 51[//third_party/turbine]: /third_party/turbine/README.chromium 52 53### Step 2a: Compile with javac 54 55This step is the only step that does not apply to prebuilt targets. 56 57* All `.java` files in a target are compiled by `javac` into `.class` files. 58 * This includes `.java` files that live within `.srcjar` files, referenced 59 through `srcjar_deps`. 60* The `classpath` used when compiling a target is comprised of `.jar` files of 61 its deps. 62 * When deps are library targets, the Step 1 `.jar` file is used. 63 * When deps are prebuilt targets, the original `.jar` file is used. 64 * All `.jar` processing done in subsequent steps does not impact compilation 65 classpath. 66* `.class` files are zipped into an output `.jar` file. 67* There is **no support** for incremental compilation at this level. 68 * If one source file changes within a library, then the entire library is 69 recompiled. 70 * Prefer smaller targets to avoid slow compiles. 71 72### Step 2b: Compile with ErrorProne 73 74This step can be disabled via GN arg: `use_errorprone_java_compiler = false` 75 76* Concurrently with step 1a: [ErrorProne] compiles java files and checks for bug 77 patterns, including some [custom to Chromium][ep_plugins]. 78* ErrorProne used to replace step 1a, but was changed to a concurrent step after 79 being identified as being slower. 80 81[ErrorProne]: https://errorprone.info/ 82[ep_plugins]: /tools/android/errorprone_plugin/ 83 84### Step 3: Desugaring (Device .jar Only) 85 86This step happens only when targets have `supports_android = true`. It is not 87applied to `.jar` files used by `junit_binary`. 88 89* `//third_party/bazel/desugar` converts certain Java 8 constructs, such as 90 lambdas and default interface methods, into constructs that are compatible 91 with Java 7. 92 93### Step 4: Instrumenting (Device .jar Only) 94 95This step happens only when this GN arg is set: `use_jacoco_coverage = true` 96 97* [Jacoco] adds instrumentation hooks to methods. 98 99[Jacoco]: https://www.eclemma.org/jacoco/ 100 101### Step 5: Filtering 102 103This step happens only when targets that have `jar_excluded_patterns` or 104`jar_included_patterns` set (e.g. all `android_` targets). 105 106* Remove `.class` files that match the filters from the `.jar`. These `.class` 107 files are generally those that are re-created with different implementations 108 further on in the build process. 109 * E.g.: `R.class` files - a part of [Android Resources]. 110 * E.g.: `GEN_JNI.class` - a part of our [JNI] glue. 111 112[JNI]: /third_party/jni_zero/README.md 113[Android Resources]: life_of_a_resource.md 114 115### Step 6: Per-Library Dexing 116 117This step happens only when targets have `supports_android = true`. 118 119* [d8] converts `.jar` files containing `.class` files into `.dex.jar` files 120 containing `classes.dex` files. 121* Dexing is incremental - it will reuse dex'ed classes from a previous build if 122 the corresponding `.class` file is unchanged. 123* These per-library `.dex.jar` files are used directly by [incremental install], 124 and are inputs to the Apk step when `enable_proguard = false`. 125 * Even when `is_java_debug = false`, many apk targets do not enable ProGuard 126 (e.g. unit tests). 127 128[d8]: https://developer.android.com/studio/command-line/d8 129[incremental install]: /build/android/incremental_install/README.md 130 131### Step 7: Apk / Bundle Module Compile 132 133* Each `android_apk` and `android_bundle_module` template has a nested 134 `java_library` target. The nested library includes final copies of files 135 stripped out by prior filtering steps. These files include: 136 * Final `R.java` files, created by `compile_resources.py`. 137 * Final `GEN_JNI.java` for [JNI glue]. 138 * `BuildConfig.java` and `NativeLibraries.java` (//base dependencies). 139 140[JNI glue]: /third_party/jni_zero/README.md 141 142### Step 8: Final Dexing 143 144This step is skipped when building using [Incremental Install]. 145 146When `is_java_debug = true`: 147* [d8] merges all library `.dex.jar` files into a final `.mergeddex.jar`. 148 149When `is_java_debug = false`: 150* [R8] performs whole-program optimization on all library `lib.java` `.jar` 151 files and outputs a final `.r8dex.jar`. 152 * For App Bundles, R8 creates a `.r8dex.jar` for each module. 153 154[Incremental Install]: /build/android/incremental_install/README.md 155[R8]: https://r8.googlesource.com/r8 156 157## Test APKs with apk_under_test 158 159Test APKs are normal APKs that contain an `<instrumentation>` tag within their 160`AndroidManifest.xml`. If this tag specifies an `android:targetPackage` 161different from itself, then Android will add that package's `classes.dex` to the 162test APK's Java classpath when run. In GN, you can enable this behavior using 163the `apk_under_test` parameter on `instrumentation_test_apk` targets. Using it 164is discouraged if APKs have `proguard_enabled=true`. 165 166### Difference in Final Dex 167 168When `enable_proguard=false`: 169* Any library depended on by the test APK that is also depended on by the 170 apk-under-test is excluded from the test APK's final dex step. 171 172When `enable_proguard=true`: 173* Test APKs cannot make use of the apk-under-test's dex because only symbols 174 explicitly kept by `-keep` directives are guaranteed to exist after 175 ProGuarding. As a work-around, test APKs include all of the apk-under-test's 176 libraries directly in its own final dex such that the under-test apk's Java 177 code is never used (because it is entirely shadowed by the test apk's dex). 178 * We've found this configuration to be fragile, and are trying to [move away 179 from it](https://bugs.chromium.org/p/chromium/issues/detail?id=890452). 180 181### Difference in GEN_JNI.java 182* Calling native methods using [JNI glue] requires that a `GEN_JNI.java` class 183 be generated that contains all native methods for an APK. There cannot be 184 conflicting `GEN_JNI` classes in both the test apk and the apk-under-test, so 185 only the apk-under-test has one generated for it. As a result this, 186 instrumentation test APKs that use apk-under-test cannot use native methods 187 that aren't already part of the apk-under-test. 188 189## How to Generate Java Source Code 190There are two ways to go about generating source files: Annotation Processors 191and custom build steps. 192 193### Annotation Processors 194* These are run by `javac` as part of the compile step. 195* They **cannot** modify the source files that they apply to. They can only 196 generate new sources. 197* Use these when: 198 * an existing Annotation Processor does what you want 199 (E.g. Dagger, AutoService, etc.), or 200 * you need to understand Java types to do generation. 201 202### Custom Build Steps 203* These use discrete build actions to generate source files. 204 * Some generate `.java` directly, but most generate a zip file of sources 205 (called a `.srcjar`) to simplify the number of inputs / outputs. 206* Examples of existing templates: 207 * `jinja_template`: Generates source files using [Jinja]. 208 * `java_cpp_template`: Generates source files using the C preprocessor. 209 * `java_cpp_enum`: Generates `@IntDef`s based on enums within `.h` files. 210 * `java_cpp_strings`: Generates String constants based on strings defined in 211 `.cc` files. 212* Custom build steps are preferred over Annotation Processors because they are 213 generally easier to understand, and can run in parallel with other steps 214 (rather than being tied to compiles). 215 216[Jinja]: https://palletsprojects.com/p/jinja/ 217 218## Static Analysis & Code Checks 219 220See [static_analysis.md](static_analysis.md) 221