1# Luci Infrastructure 2 3This directory contains the configuration and build recipes run by our luci infrastructure for CI 4and presubmit testing. 5 6Note: Luci applies config and recipes changes asynchronously. Do not submit changes to this 7directory in the same commit as changes to other crosvm source. 8 9## Recipes 10 11### Recipe Documentation 12 13A few links to relevant documentation needed to write recipes: 14 15- [Recipe Engine](https://chromium.googlesource.com/infra/luci/recipes-py.git/+/HEAD/README.recipes.md) 16- [Depot Tools Recipes](https://chromium.googlesource.com/chromium/tools/depot_tools.git/+/HEAD/recipes/README.recipes.md)) 17- [ChromiumOS Recipes](https://chromium.googlesource.com/chromiumos/infra/recipes.git/+/HEAD/README.recipes.md) 18 19Luci also provides a 20[User Guide](https://chromium.googlesource.com/infra/luci/recipes-py/+/master/doc/user_guide.md) and 21[Walkthrough](https://chromium.googlesource.com/infra/luci/recipes-py/+/refs/heads/main/doc/walkthrough.md) 22for getting started with recipes. 23 24For writing tests, documentation can be found in the 25[Recipe test API](https://chromium.googlesource.com/infra/luci/recipes-py/+/HEAD/recipe_engine/recipe_test_api.py) 26and 27[Post Process API](https://chromium.googlesource.com/infra/luci/recipes-py/+/HEAD/recipe_engine/post_process.py) 28 29### Running recipe tests 30 31Recipes must have 100% code coverage to have tests pass. Tests can be run with: 32 33``` 34cd infra && ./recipes.py test run 35``` 36 37Most tests execute a few example invocations, record the commands that would be executed and compare 38them to the json files in `*.expected`. This allows developers to catch unwanted side-effects of 39their changes. 40 41To regenerate the expectation files, run: 42 43``` 44cd infra && ./recipes.py test train 45``` 46 47Then verify the `git diff` to make sure all changes to outcomes are intentional. 48 49### Testing recipes locally 50 51We try to build our recipes to work well locally, so for example build_linux.py can be invoked in 52the recipe engine via: 53 54``` 55cd infra && ./recipes.py run build_linux 56``` 57 58When run locally, recipes that check out crosvm, will run against the current HEAD of the main 59branch. 60 61The recipe will run in the local `infra/.recipe_deps/recipe_engine/workdir` directory and is 62preserved between runs in the same way data is preserved on bots, so incremental builds or the use 63of cached files can be tested. 64 65### Testing recipes on a bot (Googlers only) 66 67Note: See internal [crosvm/infra](http://go/crosvm/infra) documentation on access control. 68 69Some things cannot be tested locally and need to be run on one of our build bots. This can be done 70with the [led](http://go/luci-how-to-led) tool. 71 72Commonly used led commands are: 73 74- `led get-builder $NAME` will download and output the job template for that builder. 75- `led get-build $BBID` will download the job definition of a previous build. 76- `led edit-recipe-bundle` will update the job to use your local version recipes 77- `led edit-cr-cl` will update the job to run on a gerrit change 78- `led launch` launches a new job using the input job definition. 79 80Important: Changes to recipes are applied separately from changes to crosvm code. 81 82#### Testing recipe changes on post-submit builders 83 84To test a local recipe change, you can launch a post-submit build using `led`. First `git commit` 85your recipe changes locally, then combine the led commands to: 86 87``` 88led get-builder luci.crosvm.ci:linux_x86_64 89 | led-edit-recipe-bundle 90 | led launch 91``` 92 93This will run the `linux_x86_64` builder on the current `main` revision of crosvm using the local 94version of recipes. 95 96Important: Changes to crosvm source outside of recipes will not be part of the build. 97 98#### Testing recipe and source changes on pre-submit builders 99 100If we want to test a combination of both recipe and source changes, we can test those on a 101pre-submit builder, which patch in a gerrit change to test. 102 103We can specify that gerrit change via `led edit-cr-cl`. 104 105So to test, first `git commit` and `./tools/cl upload` your local changes. Then build a job 106definition to run: 107 108``` 109led get-builder luci.crosvm.try:linux_x86_64 110 | led-edit-recipe-bundle 111 | led-edit-cr-cl $GERRIT_URL 112 | led launch 113``` 114 115This will launch a presubmit builder using the local version of recipes, and runs it on the gerrit 116change at $GERRIT_URL. 117 118#### Testing a new recipe 119 120This is a little tricker, but you can change the recipe name that is launched by `led`. So you can: 121 122``` 123led get-builder luci.crosvm.ci:linux_x86_64 > job.json 124``` 125 126Then edit the `job.json` file to use the newly added recipe, and launch the job using the local 127version of recipes. 128 129``` 130cat job.json | led-edit-recipe-bundle | led launch 131``` 132