1*c8dee2aaSAndroid Build Coastguard Worker# Prerequisites 2*c8dee2aaSAndroid Build Coastguard Worker 3*c8dee2aaSAndroid Build Coastguard WorkerNode v14 or later is required to run tests. We use npm (the Node Package Manager) to install 4*c8dee2aaSAndroid Build Coastguard Workertest dependencies. Recent installations of Node have npm as well. 5*c8dee2aaSAndroid Build Coastguard WorkerCanvasKit has no other external source dependencies. 6*c8dee2aaSAndroid Build Coastguard Worker 7*c8dee2aaSAndroid Build Coastguard Worker## Compiling with GN 8*c8dee2aaSAndroid Build Coastguard WorkerTo build with GN, you need to have followed the instructions to download Skia and its deps 9*c8dee2aaSAndroid Build Coastguard Worker<https://skia.org/docs/user/download>. 10*c8dee2aaSAndroid Build Coastguard Worker 11*c8dee2aaSAndroid Build Coastguard WorkerTo compile CanvasKit, you will first need to [download and activate `emscripten`][1] using the 12*c8dee2aaSAndroid Build Coastguard Workerscript in `//bin/activate-emsdk` (or `//tools/git-sync-deps` which also calls activate-emsdk). 13*c8dee2aaSAndroid Build Coastguard WorkerThis places the associated files in `//third_party/externals/emsdk` and the GN[2] build scripts 14*c8dee2aaSAndroid Build Coastguard Workerwill use those by default. 15*c8dee2aaSAndroid Build Coastguard WorkerThe compile.sh script automates the default GN settings; users are free to set their own. If users 16*c8dee2aaSAndroid Build Coastguard Workerwant to use their own version of emscripten, they should set the `skia_emsdk_dir` argument 17*c8dee2aaSAndroid Build Coastguard Worker(see `//skia/gn/toolchain/wasm.gni`). For other available arguments, see 18*c8dee2aaSAndroid Build Coastguard Worker`//modules/canvaskit/BUILD.gn`. 19*c8dee2aaSAndroid Build Coastguard Worker 20*c8dee2aaSAndroid Build Coastguard Worker[1]: https://emscripten.org/ 21*c8dee2aaSAndroid Build Coastguard Worker[2]: https://chromium.googlesource.com/chromium/src/tools/gn/+/48062805e19b4697c5fbd926dc649c78b6aaa138/README.md 22*c8dee2aaSAndroid Build Coastguard Worker 23*c8dee2aaSAndroid Build Coastguard Worker### MacOS specific notes 24*c8dee2aaSAndroid Build Coastguard WorkerMake sure you have Python3 installed, otherwise the downloading emscripten toolchain 25*c8dee2aaSAndroid Build Coastguard Workercan fail with errors about SSL certificates. <https://github.com/emscripten-core/emsdk/pull/273> 26*c8dee2aaSAndroid Build Coastguard Worker 27*c8dee2aaSAndroid Build Coastguard WorkerSee also <https://github.com/emscripten-core/emscripten/issues/9036#issuecomment-532092743> 28*c8dee2aaSAndroid Build Coastguard Workerfor a solution to Python3 using the wrong certificates. 29*c8dee2aaSAndroid Build Coastguard Worker 30*c8dee2aaSAndroid Build Coastguard Worker# Compile and Run Local Example 31*c8dee2aaSAndroid Build Coastguard Worker 32*c8dee2aaSAndroid Build Coastguard Worker``` 33*c8dee2aaSAndroid Build Coastguard Worker# The following installs all npm dependencies and only needs to be when setting up 34*c8dee2aaSAndroid Build Coastguard Worker# or if our npm dependencies have changed (rarely). 35*c8dee2aaSAndroid Build Coastguard Workernpm ci 36*c8dee2aaSAndroid Build Coastguard Worker 37*c8dee2aaSAndroid Build Coastguard Workermake release # make debug is much faster and has better error messages 38*c8dee2aaSAndroid Build Coastguard Workermake local-example 39*c8dee2aaSAndroid Build Coastguard Worker``` 40*c8dee2aaSAndroid Build Coastguard Worker 41*c8dee2aaSAndroid Build Coastguard WorkerThis will print a local endpoint for viewing the example. You can experiment 42*c8dee2aaSAndroid Build Coastguard Workerwith the CanvasKit API by modifying `./npm_build/example.html` and refreshing 43*c8dee2aaSAndroid Build Coastguard Workerthe page. For some more experimental APIs, there's also `./npm_build/extra.html`. 44*c8dee2aaSAndroid Build Coastguard Worker 45*c8dee2aaSAndroid Build Coastguard WorkerFor other available build targets, see `Makefile` and `compile.sh`. 46*c8dee2aaSAndroid Build Coastguard WorkerFor example, building a stripped-down version of CanvasKit with no text support or 47*c8dee2aaSAndroid Build Coastguard Workerany of the "extras", one might run: 48*c8dee2aaSAndroid Build Coastguard Worker 49*c8dee2aaSAndroid Build Coastguard Worker ./compile.sh no_skottie no_font 50*c8dee2aaSAndroid Build Coastguard Worker 51*c8dee2aaSAndroid Build Coastguard WorkerSuch a stripped-down version is about half the size of the default release build. 52*c8dee2aaSAndroid Build Coastguard Worker 53*c8dee2aaSAndroid Build Coastguard WorkerIf CanvasKit fails to build and you are getting compile errors that don't look like Skia code, 54*c8dee2aaSAndroid Build Coastguard Workeryou may need to do a fresh install of the npm modules. You can do this by finding the .dts file 55*c8dee2aaSAndroid Build Coastguard Workermentioned in the error message, deleting it, and rerunning `npm ci`. 56*c8dee2aaSAndroid Build Coastguard Worker 57*c8dee2aaSAndroid Build Coastguard WorkerIf you're using the correct modules plus the latest supported typescript and it still fails, 58*c8dee2aaSAndroid Build Coastguard Workerthe module versions listed in package.json may need to be updated as well. 59*c8dee2aaSAndroid Build Coastguard Worker 60*c8dee2aaSAndroid Build Coastguard Worker# Unit tests, performance tests, and coverage. 61*c8dee2aaSAndroid Build Coastguard Worker 62*c8dee2aaSAndroid Build Coastguard WorkerTo run unit tests and compute test coverage on a debug gpu build 63*c8dee2aaSAndroid Build Coastguard Worker 64*c8dee2aaSAndroid Build Coastguard Worker``` 65*c8dee2aaSAndroid Build Coastguard Workermake debug 66*c8dee2aaSAndroid Build Coastguard Workermake test-continuous 67*c8dee2aaSAndroid Build Coastguard Worker``` 68*c8dee2aaSAndroid Build Coastguard Worker 69*c8dee2aaSAndroid Build Coastguard WorkerThis reads karma.conf.js, and opens a Chrome browser and begins running all the test 70*c8dee2aaSAndroid Build Coastguard Workerin `test/` it will detect changes to the tests in that directory and automatically 71*c8dee2aaSAndroid Build Coastguard Workerrun again, however it will automatically rebuild and reload CanvasKit. Closing the 72*c8dee2aaSAndroid Build Coastguard Workerchrome window will just cause it to re-opened. Kill the karma process to stop continuous 73*c8dee2aaSAndroid Build Coastguard Workermonitoring for changes. 74*c8dee2aaSAndroid Build Coastguard Worker 75*c8dee2aaSAndroid Build Coastguard WorkerThe tests are run with whichever build of CanvasKit you last made. be sure to also 76*c8dee2aaSAndroid Build Coastguard Workertest with `release`, `debug_cpu`, and `release_cpu`. testing with release builds will 77*c8dee2aaSAndroid Build Coastguard Workerexpose problems in closure compilation and usually forgotten externs. 78*c8dee2aaSAndroid Build Coastguard Worker 79*c8dee2aaSAndroid Build Coastguard Worker## Coverage 80*c8dee2aaSAndroid Build Coastguard Worker 81*c8dee2aaSAndroid Build Coastguard WorkerCoverage will be automatically computed when running test-continuous locally. Note that 82*c8dee2aaSAndroid Build Coastguard Workerthe results will only be useful when testing a debug build. Open 83*c8dee2aaSAndroid Build Coastguard Worker`coverage/<browser version>/index.html` For a summary and detailed line-by-line result. 84*c8dee2aaSAndroid Build Coastguard Worker 85*c8dee2aaSAndroid Build Coastguard Worker## Measuring Performance 86*c8dee2aaSAndroid Build Coastguard Worker 87*c8dee2aaSAndroid Build Coastguard WorkerWe use puppeteer to run a Chrome browser to gather performance data in a consistent way. 88*c8dee2aaSAndroid Build Coastguard WorkerSee `//tools/perf-canvaskit-puppeteer` for more. 89*c8dee2aaSAndroid Build Coastguard Worker 90*c8dee2aaSAndroid Build Coastguard Worker## Adding tests 91*c8dee2aaSAndroid Build Coastguard Worker 92*c8dee2aaSAndroid Build Coastguard WorkerThe tests in `tests/` are grouped into files by topic. 93*c8dee2aaSAndroid Build Coastguard WorkerWithin each file there are `describe` blocks further organizing the tests, and within those 94*c8dee2aaSAndroid Build Coastguard Worker`it()` functions which test particular behaviors. `describe` and `it` are jasmine methods 95*c8dee2aaSAndroid Build Coastguard Workerwhich can both be temporarily renamed `fdescribe` and `fit`. Which causes jasmine to only those. 96*c8dee2aaSAndroid Build Coastguard Worker 97*c8dee2aaSAndroid Build Coastguard WorkerWe have also defined `gm` which is a method for defining a test which draws something to a canvas 98*c8dee2aaSAndroid Build Coastguard Workerthat is shapshotted and reported to gold.skia.org, where you can compare it with the snapshot at 99*c8dee2aaSAndroid Build Coastguard Workerhead. 100*c8dee2aaSAndroid Build Coastguard Worker 101*c8dee2aaSAndroid Build Coastguard Worker## Testing from Gerrit 102*c8dee2aaSAndroid Build Coastguard Worker 103*c8dee2aaSAndroid Build Coastguard WorkerWhen submitting a CL in gerrit, click "choose tryjobs" and type CanvasKit to filter them. 104*c8dee2aaSAndroid Build Coastguard Workerselect all of them, which at the time of this writing is four jobs, for each combination 105*c8dee2aaSAndroid Build Coastguard Workerof perf/test gpu/cpu. 106*c8dee2aaSAndroid Build Coastguard Worker 107*c8dee2aaSAndroid Build Coastguard WorkerThe performance results are reported to [perf.skia.org] and correctness results are reported to 108*c8dee2aaSAndroid Build Coastguard Worker[gold.skia.org]. 109*c8dee2aaSAndroid Build Coastguard Worker 110*c8dee2aaSAndroid Build Coastguard WorkerCoverage is not measured while running tests this way. 111*c8dee2aaSAndroid Build Coastguard Worker 112*c8dee2aaSAndroid Build Coastguard Worker# Inspecting output WASM 113*c8dee2aaSAndroid Build Coastguard Worker 114*c8dee2aaSAndroid Build Coastguard WorkerThe `wasm2wat` tool from [the WebAssembly Binary Toolkit](https://github.com/WebAssembly/wabt) 115*c8dee2aaSAndroid Build Coastguard Workercan be used to produce a human-readable text version of a `.wasm` file. 116*c8dee2aaSAndroid Build Coastguard Worker 117*c8dee2aaSAndroid Build Coastguard WorkerThe output of `wasm2wat --version` should be `1.0.13 (1.0.17)`. This version has been checked to 118*c8dee2aaSAndroid Build Coastguard Workerwork with the tools in `wasm_tools/SIMD/`. These tools programmatically inspect the `.wasm` output 119*c8dee2aaSAndroid Build Coastguard Workerof a CanvasKit build to detect the presence of [wasm SIMD](https://github.com/WebAssembly/simd) 120*c8dee2aaSAndroid Build Coastguard Workeroperations. 121*c8dee2aaSAndroid Build Coastguard Worker 122*c8dee2aaSAndroid Build Coastguard Worker# Infrastructure Playbook 123*c8dee2aaSAndroid Build Coastguard Worker 124*c8dee2aaSAndroid Build Coastguard WorkerWhen dealing with CanvasKit (or PathKit) on our bots, we use Docker. Check out 125*c8dee2aaSAndroid Build Coastguard Worker$SKIA_ROOT/infra/wasm-common/docker/README.md for more on building/editing the 126*c8dee2aaSAndroid Build Coastguard Workerimages used for building and testing. 127*c8dee2aaSAndroid Build Coastguard Worker 128*c8dee2aaSAndroid Build Coastguard Worker## Updating the version of Emscripten we build/test with 129*c8dee2aaSAndroid Build Coastguard Worker 130*c8dee2aaSAndroid Build Coastguard WorkerThis presumes you have updated emscripten locally to a newer version of the 131*c8dee2aaSAndroid Build Coastguard Workersdk and verified/fixed any build issues that have arisen. 132*c8dee2aaSAndroid Build Coastguard Worker 133*c8dee2aaSAndroid Build Coastguard Worker 1. Edit `//bin/activate-emsdk` to install and activate the desired version of Emscripten. 134*c8dee2aaSAndroid Build Coastguard Worker 2. Upload a CL with all the changes. Run all Test.+CanvasKit, Perf.+Puppeteer, 135*c8dee2aaSAndroid Build Coastguard Worker Test.+PathKit, Perf.+PathKit jobs to make sure the new builds pass all 136*c8dee2aaSAndroid Build Coastguard Worker tests and don't crash the perf harnesses. 137*c8dee2aaSAndroid Build Coastguard Worker 3. Send out CL for review. Feel free to point the reviewer at these steps. 138*c8dee2aaSAndroid Build Coastguard Worker 139*c8dee2aaSAndroid Build Coastguard Worker## Running Skia's GMs and Unit Tests against wasm+WebGL ## 140*c8dee2aaSAndroid Build Coastguard Worker 141*c8dee2aaSAndroid Build Coastguard WorkerGeneral Tips: 142*c8dee2aaSAndroid Build Coastguard Worker - Make use of the skip lists and start indexes in the run-wasm-gm-tests.html to focus in on 143*c8dee2aaSAndroid Build Coastguard Worker problematic tests. 144*c8dee2aaSAndroid Build Coastguard Worker - `Uncaught (in promise) RuntimeError: function signature mismatch` tends to mean null was 145*c8dee2aaSAndroid Build Coastguard Worker dereferenced somewhere. Add SkASSERT to verify. 146*c8dee2aaSAndroid Build Coastguard Worker 147*c8dee2aaSAndroid Build Coastguard Worker### Debugging some GMs / Unit Tests 148*c8dee2aaSAndroid Build Coastguard WorkerFor faster cycle time, it is recommended to focus on specific GMs instead of re-compiling all 149*c8dee2aaSAndroid Build Coastguard Workerof them. This can be done by modifying the `compile_gm.sh` script (but not checking this in) 150*c8dee2aaSAndroid Build Coastguard Workerto set `GMS_TO_BUILD` and/or `TESTS_TO_BUILD` to a minimal set of files. There's an `if false` 151*c8dee2aaSAndroid Build Coastguard Workerthat can be commented out to assist with this. 152*c8dee2aaSAndroid Build Coastguard Worker 153*c8dee2aaSAndroid Build Coastguard WorkerRun `make gm_tests` or `make_gm_tests_debug` from this folder. This will produce a .js and .wasm 154*c8dee2aaSAndroid Build Coastguard Workerin a (not checked in) `build` subfolder. 155*c8dee2aaSAndroid Build Coastguard Worker 156*c8dee2aaSAndroid Build Coastguard WorkerRun `make single-gm` and navigate to <http://localhost:8000/wasm_tools/gms.html>. This will load 157*c8dee2aaSAndroid Build Coastguard Workerthat html file and the freshly built wasm_gm_tests binary and run a single GM and unit test that 158*c8dee2aaSAndroid Build Coastguard Workerwas compiled in. Feel free to modify //modules/canvaskit/wasm_tools/gms.html to run the specific 159*c8dee2aaSAndroid Build Coastguard WorkerGM/unit test or tests that you care about. 160*c8dee2aaSAndroid Build Coastguard Worker 161*c8dee2aaSAndroid Build Coastguard Worker### Testing all GMs / Unit Tests 162*c8dee2aaSAndroid Build Coastguard WorkerWith the current GN build, this can take quite a while to compile and re-compile (the upcoming 163*c8dee2aaSAndroid Build Coastguard WorkerBazel build should alleviate this). 164*c8dee2aaSAndroid Build Coastguard Worker 165*c8dee2aaSAndroid Build Coastguard WorkerRun `make gm_tests` or `make_gm_tests_debug` from this folder. This will produce a .js and .wasm 166*c8dee2aaSAndroid Build Coastguard Workerin a (not checked in) `build` subfolder. 167*c8dee2aaSAndroid Build Coastguard Worker 168*c8dee2aaSAndroid Build Coastguard WorkerChange directory to `//tools/run-wasm-gm-tests`. Run `make run_local`, which will put all PNGs 169*c8dee2aaSAndroid Build Coastguard Workerproduced by GMs into `/tmp/wasm-gmtests` and run all unit tests. 170