1*03ce13f7SAndroid Build Coastguard WorkerLLVM Dependency 2*03ce13f7SAndroid Build Coastguard Worker=============== 3*03ce13f7SAndroid Build Coastguard Worker 4*03ce13f7SAndroid Build Coastguard WorkerOverview 5*03ce13f7SAndroid Build Coastguard Worker-------- 6*03ce13f7SAndroid Build Coastguard Worker 7*03ce13f7SAndroid Build Coastguard WorkerSwiftShader's [Reactor](Reactor.md) library uses LLVM 8*03ce13f7SAndroid Build Coastguard Workeras one of its JIT-compiler backends. This page contains notes about building and 9*03ce13f7SAndroid Build Coastguard Workerupgrading LLVM. 10*03ce13f7SAndroid Build Coastguard Worker 11*03ce13f7SAndroid Build Coastguard WorkerDirectory structure 12*03ce13f7SAndroid Build Coastguard Worker------------------- 13*03ce13f7SAndroid Build Coastguard Worker 14*03ce13f7SAndroid Build Coastguard WorkerThe current version of LLVM we use is 10, and can be found in 15*03ce13f7SAndroid Build Coastguard Worker`third_party/llvm-10.0`. 16*03ce13f7SAndroid Build Coastguard Worker 17*03ce13f7SAndroid Build Coastguard WorkerIn this folder you will find the following directories: 18*03ce13f7SAndroid Build Coastguard Worker 19*03ce13f7SAndroid Build Coastguard Worker* configs : Contains per-platform headers that LLVM sources include to 20*03ce13f7SAndroid Build Coastguard Worker configure the build. These are generated by running `scripts/update.py` 21*03ce13f7SAndroid Build Coastguard Worker (more on that below). 22*03ce13f7SAndroid Build Coastguard Worker* llvm : Contains a subset of the LLVM source code needed to build the JIT 23*03ce13f7SAndroid Build Coastguard Worker support required by SwiftShader. 24*03ce13f7SAndroid Build Coastguard Worker* scripts : Contains `update.py`, which is used to update the files in the 25*03ce13f7SAndroid Build Coastguard Worker `configs` folder. More on that below. 26*03ce13f7SAndroid Build Coastguard Worker 27*03ce13f7SAndroid Build Coastguard WorkerUpdating the current version of LLVM to latest 28*03ce13f7SAndroid Build Coastguard Worker---------------------------------------------- 29*03ce13f7SAndroid Build Coastguard Worker 30*03ce13f7SAndroid Build Coastguard WorkerUpdating to the latest version of LLVM can be tricky to do manually, especially 31*03ce13f7SAndroid Build Coastguard Workerbecause the [llvm-project repo](https://github.com/llvm/llvm-project) includes 32*03ce13f7SAndroid Build Coastguard Workermuch more than just LLVM (e.g. it includes all the Clang source). Furthermore, 33*03ce13f7SAndroid Build Coastguard Workerwe may have local changes to our copy of LLVM that must be maintained, or at 34*03ce13f7SAndroid Build Coastguard Workerleast considered across updates. 35*03ce13f7SAndroid Build Coastguard Worker 36*03ce13f7SAndroid Build Coastguard WorkerTo ease this pain, run the script `third_party/update-llvm-10.sh` on Linux. This 37*03ce13f7SAndroid Build Coastguard Workerscript works by updating a separate branch of SwiftShader, `llvm10-clean`, on 38*03ce13f7SAndroid Build Coastguard Workerwhich the latest snapshot of LLVM is fetched and committed, and then this branch 39*03ce13f7SAndroid Build Coastguard Workeris merged back into `master`. During the merge, if there are conflicts to 40*03ce13f7SAndroid Build Coastguard Workerresolve because of local changes we've made, these can be resolved in the usual 41*03ce13f7SAndroid Build Coastguard Workermanner, and the merge can be resumed. 42*03ce13f7SAndroid Build Coastguard Worker 43*03ce13f7SAndroid Build Coastguard WorkerThe script is configured to fetch from the branch in `LLVM_REPO_BRANCH`, and 44*03ce13f7SAndroid Build Coastguard Workerwill automatically grab the latest commit on that branch. 45*03ce13f7SAndroid Build Coastguard Worker 46*03ce13f7SAndroid Build Coastguard WorkerAlthough not always necessary, if there were new configuration variables added 47*03ce13f7SAndroid Build Coastguard Workeror modified, you may need to run `update.py` as described below. Otherwise, if 48*03ce13f7SAndroid Build Coastguard Workerall goes well, the update to LLVM can be committed and pushed. 49*03ce13f7SAndroid Build Coastguard Worker 50*03ce13f7SAndroid Build Coastguard WorkerUpdating LLVM configuration files 51*03ce13f7SAndroid Build Coastguard Worker--------------------------------- 52*03ce13f7SAndroid Build Coastguard Worker 53*03ce13f7SAndroid Build Coastguard WorkerThe script `third_party/llvm-10.0/scripts/update.py` is used to update the 54*03ce13f7SAndroid Build Coastguard Workerconfig files in `third_party/llvm-10.0/configs`. 55*03ce13f7SAndroid Build Coastguard Worker 56*03ce13f7SAndroid Build Coastguard WorkerBefore running this script, you must make sure to update two variables in it 57*03ce13f7SAndroid Build Coastguard Worker(and commit this change): 58*03ce13f7SAndroid Build Coastguard Worker 59*03ce13f7SAndroid Build Coastguard Worker``` 60*03ce13f7SAndroid Build Coastguard Worker# LLVM_BRANCH must match the value of the same variable in third_party/update-llvm-10.sh 61*03ce13f7SAndroid Build Coastguard WorkerLLVM_BRANCH = "release/10.x" 62*03ce13f7SAndroid Build Coastguard Worker 63*03ce13f7SAndroid Build Coastguard Worker# LLVM_COMMIT must be set to the commit hash that we last updated to when running third_party/update-llvm-10.sh. 64*03ce13f7SAndroid Build Coastguard Worker# Run 'git show -s origin/llvm10-clean' and look for 'llvm-10-update: <hash>' to retrieve it. 65*03ce13f7SAndroid Build Coastguard WorkerLLVM_COMMIT = "d32170dbd5b0d54436537b6b75beaf44324e0c28" 66*03ce13f7SAndroid Build Coastguard Worker``` 67*03ce13f7SAndroid Build Coastguard Worker 68*03ce13f7SAndroid Build Coastguard WorkerThe script takes a platform as argument, and extra CMake args. For example, to 69*03ce13f7SAndroid Build Coastguard Workerupdate the Linux configs, run: 70*03ce13f7SAndroid Build Coastguard Worker 71*03ce13f7SAndroid Build Coastguard Worker``` 72*03ce13f7SAndroid Build Coastguard Workerpython3 update.py linux -j 200 73*03ce13f7SAndroid Build Coastguard Worker``` 74*03ce13f7SAndroid Build Coastguard Worker 75*03ce13f7SAndroid Build Coastguard WorkerThis script does the following: 76*03ce13f7SAndroid Build Coastguard Worker 77*03ce13f7SAndroid Build Coastguard Worker* Clones the LLVM repo and checks out `LLVM_COMMIT` from `LLVM_BRANCH`. 78*03ce13f7SAndroid Build Coastguard Worker* Builds LLVM specifically for the target architectures specified in 79*03ce13f7SAndroid Build Coastguard Worker `LLVM_TRIPLES` dictionary. 80*03ce13f7SAndroid Build Coastguard Worker* Copies the specified platform config files to 81*03ce13f7SAndroid Build Coastguard Worker `third_party/llvm-10.0/configs`, applying certain transformations to the 82*03ce13f7SAndroid Build Coastguard Worker files, such as undefining macros listed in `LLVM_UNDEF_MACROS` (see the 83*03ce13f7SAndroid Build Coastguard Worker `copy_platform_file` function). 84*03ce13f7SAndroid Build Coastguard Worker 85*03ce13f7SAndroid Build Coastguard WorkerNote that certain configuration options depend on the host OS, you will need to 86*03ce13f7SAndroid Build Coastguard Workerrun the script on the right host OS. See the `LLVM_PLATFORM_TO_HOST_SYSTEM` 87*03ce13f7SAndroid Build Coastguard Workerdictionary for the mapping, which looks like this at the time of this writing: 88*03ce13f7SAndroid Build Coastguard Worker 89*03ce13f7SAndroid Build Coastguard Worker``` 90*03ce13f7SAndroid Build Coastguard Worker# Mapping of target platform to the host it must be built on 91*03ce13f7SAndroid Build Coastguard WorkerLLVM_PLATFORM_TO_HOST_SYSTEM = { 92*03ce13f7SAndroid Build Coastguard Worker 'android': 'Linux', 93*03ce13f7SAndroid Build Coastguard Worker 'darwin': 'Darwin', 94*03ce13f7SAndroid Build Coastguard Worker 'linux': 'Linux', 95*03ce13f7SAndroid Build Coastguard Worker 'windows': 'Windows', 96*03ce13f7SAndroid Build Coastguard Worker 'fuchsia': 'Linux' 97*03ce13f7SAndroid Build Coastguard Worker} 98*03ce13f7SAndroid Build Coastguard Worker``` 99*03ce13f7SAndroid Build Coastguard Worker 100*03ce13f7SAndroid Build Coastguard WorkerGenerally, Windows to build Window, Darwin to build Darwin (MacOS), and Linux 101*03ce13f7SAndroid Build Coastguard Workerfor everything else. Also note that for android and fuchsia, the config is 102*03ce13f7SAndroid Build Coastguard Workerclosest to that of Linux, but you will likely have to manually tweak the configs 103*03ce13f7SAndroid Build Coastguard Worker(in particular, `configs/<platform>/include/llvm/Config/config.h`). 104*03ce13f7SAndroid Build Coastguard Worker 105*03ce13f7SAndroid Build Coastguard WorkerSupported platforms, architectures, and build systems 106*03ce13f7SAndroid Build Coastguard Worker----------------------------------------------------- 107*03ce13f7SAndroid Build Coastguard Worker 108*03ce13f7SAndroid Build Coastguard WorkerSwiftShader is used by many products on many architectures: 109*03ce13f7SAndroid Build Coastguard Worker 110*03ce13f7SAndroid Build Coastguard Worker* OS: Windows, Linux, MacOS, Android, Fuchsia 111*03ce13f7SAndroid Build Coastguard Worker* Architecture: x64, x86, ARM, ARM64, MIPS, MIPS64 112*03ce13f7SAndroid Build Coastguard Worker* Build systems: CMake, GN, Soong, Blaze 113*03ce13f7SAndroid Build Coastguard Worker 114*03ce13f7SAndroid Build Coastguard WorkerUpgrading/updating LLVM usually entails making sure it builds for all of these.