1<!-- Generated with Stardoc: http://skydoc.bazel.build --> 2 3Unit testing support. 4 5Unlike most Skylib files, this exports four modules: 6* `unittest` contains functions to declare and define unit tests for ordinary 7 Starlark functions; 8* `analysistest` contains functions to declare and define tests for analysis 9 phase behavior of a rule, such as a given target's providers or registered 10 actions; 11* `loadingtest` contains functions to declare and define tests for loading 12 phase behavior, such as macros and `native.*`; 13* `asserts` contains the assertions used within tests. 14 15See https://bazel.build/extending/concepts for background about macros, rules, 16and the different phases of a build. 17 18<a id="unittest_toolchain"></a> 19 20## unittest_toolchain 21 22<pre> 23unittest_toolchain(<a href="#unittest_toolchain-name">name</a>, <a href="#unittest_toolchain-escape_chars_with">escape_chars_with</a>, <a href="#unittest_toolchain-escape_other_chars_with">escape_other_chars_with</a>, <a href="#unittest_toolchain-failure_templ">failure_templ</a>, <a href="#unittest_toolchain-file_ext">file_ext</a>, 24 <a href="#unittest_toolchain-join_on">join_on</a>, <a href="#unittest_toolchain-success_templ">success_templ</a>) 25</pre> 26 27 28 29**ATTRIBUTES** 30 31 32| Name | Description | Type | Mandatory | Default | 33| :------------- | :------------- | :------------- | :------------- | :------------- | 34| <a id="unittest_toolchain-name"></a>name | A unique name for this target. | <a href="https://bazel.build/concepts/labels#target-names">Name</a> | required | | 35| <a id="unittest_toolchain-escape_chars_with"></a>escape_chars_with | Dictionary of characters that need escaping in test failure message to prefix appended to escape those characters. For example, `{"%": "%", ">": "^"}` would replace `%` with `%%` and `>` with `^>` in the failure message before that is included in `success_templ`. | <a href="https://bazel.build/rules/lib/dict">Dictionary: String -> String</a> | optional | `{}` | 36| <a id="unittest_toolchain-escape_other_chars_with"></a>escape_other_chars_with | String to prefix every character in test failure message which is not a key in `escape_chars_with` before including that in `success_templ`. For example, `""` would prefix every character in the failure message (except those in the keys of `escape_chars_with`) with `\`. | String | optional | `""` | 37| <a id="unittest_toolchain-failure_templ"></a>failure_templ | Test script template with a single `%s`. That placeholder is replaced with the lines in the failure message joined with the string specified in `join_with`. The resulting script should print the failure message and exit with non-zero status. | String | required | | 38| <a id="unittest_toolchain-file_ext"></a>file_ext | File extension for test script, including leading dot. | String | required | | 39| <a id="unittest_toolchain-join_on"></a>join_on | String used to join the lines in the failure message before including the resulting string in the script specified in `failure_templ`. | String | required | | 40| <a id="unittest_toolchain-success_templ"></a>success_templ | Test script generated when the test passes. Should exit with status 0. | String | required | | 41 42 43<a id="analysistest.begin"></a> 44 45## analysistest.begin 46 47<pre> 48analysistest.begin(<a href="#analysistest.begin-ctx">ctx</a>) 49</pre> 50 51Begins an analysis test. 52 53This should be the first function called in an analysis test implementation 54function. It initializes a "test environment" that is used to collect 55assertion failures so that they can be reported and logged at the end of the 56test. 57 58 59**PARAMETERS** 60 61 62| Name | Description | Default Value | 63| :------------- | :------------- | :------------- | 64| <a id="analysistest.begin-ctx"></a>ctx | The Starlark context. Pass the implementation function's `ctx` argument in verbatim. | none | 65 66**RETURNS** 67 68A test environment struct that must be passed to assertions and finally to 69`analysistest.end`. Do not rely on internal details about the fields in this 70struct as it may change. 71 72 73<a id="analysistest.end"></a> 74 75## analysistest.end 76 77<pre> 78analysistest.end(<a href="#analysistest.end-env">env</a>) 79</pre> 80 81Ends an analysis test and logs the results. 82 83This must be called and returned at the end of an analysis test implementation function so 84that the results are reported. 85 86 87**PARAMETERS** 88 89 90| Name | Description | Default Value | 91| :------------- | :------------- | :------------- | 92| <a id="analysistest.end-env"></a>env | The test environment returned by `analysistest.begin`. | none | 93 94**RETURNS** 95 96A list of providers needed to automatically register the analysis test result. 97 98 99<a id="analysistest.fail"></a> 100 101## analysistest.fail 102 103<pre> 104analysistest.fail(<a href="#analysistest.fail-env">env</a>, <a href="#analysistest.fail-msg">msg</a>) 105</pre> 106 107Unconditionally causes the current test to fail. 108 109**PARAMETERS** 110 111 112| Name | Description | Default Value | 113| :------------- | :------------- | :------------- | 114| <a id="analysistest.fail-env"></a>env | The test environment returned by `unittest.begin`. | none | 115| <a id="analysistest.fail-msg"></a>msg | The message to log describing the failure. | none | 116 117 118<a id="analysistest.make"></a> 119 120## analysistest.make 121 122<pre> 123analysistest.make(<a href="#analysistest.make-impl">impl</a>, <a href="#analysistest.make-expect_failure">expect_failure</a>, <a href="#analysistest.make-attrs">attrs</a>, <a href="#analysistest.make-fragments">fragments</a>, <a href="#analysistest.make-config_settings">config_settings</a>, 124 <a href="#analysistest.make-extra_target_under_test_aspects">extra_target_under_test_aspects</a>, <a href="#analysistest.make-doc">doc</a>) 125</pre> 126 127Creates an analysis test rule from its implementation function. 128 129An analysis test verifies the behavior of a "real" rule target by examining 130and asserting on the providers given by the real target. 131 132Each analysis test is defined in an implementation function that must then be 133associated with a rule so that a target can be built. This function handles 134the boilerplate to create and return a test rule and captures the 135implementation function's name so that it can be printed in test feedback. 136 137An example of an analysis test: 138 139``` 140def _your_test(ctx): 141 env = analysistest.begin(ctx) 142 143 # Assert statements go here 144 145 return analysistest.end(env) 146 147your_test = analysistest.make(_your_test) 148``` 149 150Recall that names of test rules must end in `_test`. 151 152 153**PARAMETERS** 154 155 156| Name | Description | Default Value | 157| :------------- | :------------- | :------------- | 158| <a id="analysistest.make-impl"></a>impl | The implementation function of the unit test. | none | 159| <a id="analysistest.make-expect_failure"></a>expect_failure | If true, the analysis test will expect the target_under_test to fail. Assertions can be made on the underlying failure using asserts.expect_failure | `False` | 160| <a id="analysistest.make-attrs"></a>attrs | An optional dictionary to supplement the attrs passed to the unit test's `rule()` constructor. | `{}` | 161| <a id="analysistest.make-fragments"></a>fragments | An optional list of fragment names that can be used to give rules access to language-specific parts of configuration. | `[]` | 162| <a id="analysistest.make-config_settings"></a>config_settings | A dictionary of configuration settings to change for the target under test and its dependencies. This may be used to essentially change 'build flags' for the target under test, and may thus be utilized to test multiple targets with different flags in a single build | `{}` | 163| <a id="analysistest.make-extra_target_under_test_aspects"></a>extra_target_under_test_aspects | An optional list of aspects to apply to the target_under_test in addition to those set up by default for the test harness itself. | `[]` | 164| <a id="analysistest.make-doc"></a>doc | A description of the rule that can be extracted by documentation generating tools. | `""` | 165 166**RETURNS** 167 168A rule definition that should be stored in a global whose name ends in 169`_test`. 170 171 172<a id="analysistest.target_actions"></a> 173 174## analysistest.target_actions 175 176<pre> 177analysistest.target_actions(<a href="#analysistest.target_actions-env">env</a>) 178</pre> 179 180Returns a list of actions registered by the target under test. 181 182**PARAMETERS** 183 184 185| Name | Description | Default Value | 186| :------------- | :------------- | :------------- | 187| <a id="analysistest.target_actions-env"></a>env | The test environment returned by `analysistest.begin`. | none | 188 189**RETURNS** 190 191A list of actions registered by the target under test 192 193 194<a id="analysistest.target_bin_dir_path"></a> 195 196## analysistest.target_bin_dir_path 197 198<pre> 199analysistest.target_bin_dir_path(<a href="#analysistest.target_bin_dir_path-env">env</a>) 200</pre> 201 202Returns ctx.bin_dir.path for the target under test. 203 204**PARAMETERS** 205 206 207| Name | Description | Default Value | 208| :------------- | :------------- | :------------- | 209| <a id="analysistest.target_bin_dir_path-env"></a>env | The test environment returned by `analysistest.begin`. | none | 210 211**RETURNS** 212 213Output bin dir path string. 214 215 216<a id="analysistest.target_under_test"></a> 217 218## analysistest.target_under_test 219 220<pre> 221analysistest.target_under_test(<a href="#analysistest.target_under_test-env">env</a>) 222</pre> 223 224Returns the target under test. 225 226**PARAMETERS** 227 228 229| Name | Description | Default Value | 230| :------------- | :------------- | :------------- | 231| <a id="analysistest.target_under_test-env"></a>env | The test environment returned by `analysistest.begin`. | none | 232 233**RETURNS** 234 235The target under test. 236 237 238<a id="asserts.equals"></a> 239 240## asserts.equals 241 242<pre> 243asserts.equals(<a href="#asserts.equals-env">env</a>, <a href="#asserts.equals-expected">expected</a>, <a href="#asserts.equals-actual">actual</a>, <a href="#asserts.equals-msg">msg</a>) 244</pre> 245 246Asserts that the given `expected` and `actual` values are equal. 247 248**PARAMETERS** 249 250 251| Name | Description | Default Value | 252| :------------- | :------------- | :------------- | 253| <a id="asserts.equals-env"></a>env | The test environment returned by `unittest.begin`. | none | 254| <a id="asserts.equals-expected"></a>expected | The expected value of some computation. | none | 255| <a id="asserts.equals-actual"></a>actual | The actual value returned by some computation. | none | 256| <a id="asserts.equals-msg"></a>msg | An optional message that will be printed that describes the failure. If omitted, a default will be used. | `None` | 257 258 259<a id="asserts.expect_failure"></a> 260 261## asserts.expect_failure 262 263<pre> 264asserts.expect_failure(<a href="#asserts.expect_failure-env">env</a>, <a href="#asserts.expect_failure-expected_failure_msg">expected_failure_msg</a>) 265</pre> 266 267Asserts that the target under test has failed with a given error message. 268 269This requires that the analysis test is created with `analysistest.make()` and 270`expect_failures = True` is specified. 271 272 273**PARAMETERS** 274 275 276| Name | Description | Default Value | 277| :------------- | :------------- | :------------- | 278| <a id="asserts.expect_failure-env"></a>env | The test environment returned by `analysistest.begin`. | none | 279| <a id="asserts.expect_failure-expected_failure_msg"></a>expected_failure_msg | The error message to expect as a result of analysis failures. | `""` | 280 281 282<a id="asserts.false"></a> 283 284## asserts.false 285 286<pre> 287asserts.false(<a href="#asserts.false-env">env</a>, <a href="#asserts.false-condition">condition</a>, <a href="#asserts.false-msg">msg</a>) 288</pre> 289 290Asserts that the given `condition` is false. 291 292**PARAMETERS** 293 294 295| Name | Description | Default Value | 296| :------------- | :------------- | :------------- | 297| <a id="asserts.false-env"></a>env | The test environment returned by `unittest.begin`. | none | 298| <a id="asserts.false-condition"></a>condition | A value that will be evaluated in a Boolean context. | none | 299| <a id="asserts.false-msg"></a>msg | An optional message that will be printed that describes the failure. If omitted, a default will be used. | `"Expected condition to be false, but was true."` | 300 301 302<a id="asserts.new_set_equals"></a> 303 304## asserts.new_set_equals 305 306<pre> 307asserts.new_set_equals(<a href="#asserts.new_set_equals-env">env</a>, <a href="#asserts.new_set_equals-expected">expected</a>, <a href="#asserts.new_set_equals-actual">actual</a>, <a href="#asserts.new_set_equals-msg">msg</a>) 308</pre> 309 310Asserts that the given `expected` and `actual` sets are equal. 311 312**PARAMETERS** 313 314 315| Name | Description | Default Value | 316| :------------- | :------------- | :------------- | 317| <a id="asserts.new_set_equals-env"></a>env | The test environment returned by `unittest.begin`. | none | 318| <a id="asserts.new_set_equals-expected"></a>expected | The expected set resulting from some computation. | none | 319| <a id="asserts.new_set_equals-actual"></a>actual | The actual set returned by some computation. | none | 320| <a id="asserts.new_set_equals-msg"></a>msg | An optional message that will be printed that describes the failure. If omitted, a default will be used. | `None` | 321 322 323<a id="asserts.set_equals"></a> 324 325## asserts.set_equals 326 327<pre> 328asserts.set_equals(<a href="#asserts.set_equals-env">env</a>, <a href="#asserts.set_equals-expected">expected</a>, <a href="#asserts.set_equals-actual">actual</a>, <a href="#asserts.set_equals-msg">msg</a>) 329</pre> 330 331Asserts that the given `expected` and `actual` sets are equal. 332 333**PARAMETERS** 334 335 336| Name | Description | Default Value | 337| :------------- | :------------- | :------------- | 338| <a id="asserts.set_equals-env"></a>env | The test environment returned by `unittest.begin`. | none | 339| <a id="asserts.set_equals-expected"></a>expected | The expected set resulting from some computation. | none | 340| <a id="asserts.set_equals-actual"></a>actual | The actual set returned by some computation. | none | 341| <a id="asserts.set_equals-msg"></a>msg | An optional message that will be printed that describes the failure. If omitted, a default will be used. | `None` | 342 343 344<a id="asserts.true"></a> 345 346## asserts.true 347 348<pre> 349asserts.true(<a href="#asserts.true-env">env</a>, <a href="#asserts.true-condition">condition</a>, <a href="#asserts.true-msg">msg</a>) 350</pre> 351 352Asserts that the given `condition` is true. 353 354**PARAMETERS** 355 356 357| Name | Description | Default Value | 358| :------------- | :------------- | :------------- | 359| <a id="asserts.true-env"></a>env | The test environment returned by `unittest.begin`. | none | 360| <a id="asserts.true-condition"></a>condition | A value that will be evaluated in a Boolean context. | none | 361| <a id="asserts.true-msg"></a>msg | An optional message that will be printed that describes the failure. If omitted, a default will be used. | `"Expected condition to be true, but was false."` | 362 363 364<a id="loadingtest.equals"></a> 365 366## loadingtest.equals 367 368<pre> 369loadingtest.equals(<a href="#loadingtest.equals-env">env</a>, <a href="#loadingtest.equals-test_case">test_case</a>, <a href="#loadingtest.equals-expected">expected</a>, <a href="#loadingtest.equals-actual">actual</a>) 370</pre> 371 372Creates a test case for asserting state at LOADING phase. 373 374**PARAMETERS** 375 376 377| Name | Description | Default Value | 378| :------------- | :------------- | :------------- | 379| <a id="loadingtest.equals-env"></a>env | Loading test env created from loadingtest.make | none | 380| <a id="loadingtest.equals-test_case"></a>test_case | Name of the test case | none | 381| <a id="loadingtest.equals-expected"></a>expected | Expected value to test | none | 382| <a id="loadingtest.equals-actual"></a>actual | Actual value received. | none | 383 384**RETURNS** 385 386None, creates test case 387 388 389<a id="loadingtest.make"></a> 390 391## loadingtest.make 392 393<pre> 394loadingtest.make(<a href="#loadingtest.make-name">name</a>) 395</pre> 396 397Creates a loading phase test environment and test_suite. 398 399**PARAMETERS** 400 401 402| Name | Description | Default Value | 403| :------------- | :------------- | :------------- | 404| <a id="loadingtest.make-name"></a>name | name of the suite of tests to create | none | 405 406**RETURNS** 407 408loading phase environment passed to other loadingtest functions 409 410 411<a id="register_unittest_toolchains"></a> 412 413## register_unittest_toolchains 414 415<pre> 416register_unittest_toolchains() 417</pre> 418 419Registers the toolchains for unittest users. 420 421 422 423<a id="unittest.begin"></a> 424 425## unittest.begin 426 427<pre> 428unittest.begin(<a href="#unittest.begin-ctx">ctx</a>) 429</pre> 430 431Begins a unit test. 432 433This should be the first function called in a unit test implementation 434function. It initializes a "test environment" that is used to collect 435assertion failures so that they can be reported and logged at the end of the 436test. 437 438 439**PARAMETERS** 440 441 442| Name | Description | Default Value | 443| :------------- | :------------- | :------------- | 444| <a id="unittest.begin-ctx"></a>ctx | The Starlark context. Pass the implementation function's `ctx` argument in verbatim. | none | 445 446**RETURNS** 447 448A test environment struct that must be passed to assertions and finally to 449`unittest.end`. Do not rely on internal details about the fields in this 450struct as it may change. 451 452 453<a id="unittest.end"></a> 454 455## unittest.end 456 457<pre> 458unittest.end(<a href="#unittest.end-env">env</a>) 459</pre> 460 461Ends a unit test and logs the results. 462 463This must be called and returned at the end of a unit test implementation function so 464that the results are reported. 465 466 467**PARAMETERS** 468 469 470| Name | Description | Default Value | 471| :------------- | :------------- | :------------- | 472| <a id="unittest.end-env"></a>env | The test environment returned by `unittest.begin`. | none | 473 474**RETURNS** 475 476A list of providers needed to automatically register the test result. 477 478 479<a id="unittest.fail"></a> 480 481## unittest.fail 482 483<pre> 484unittest.fail(<a href="#unittest.fail-env">env</a>, <a href="#unittest.fail-msg">msg</a>) 485</pre> 486 487Unconditionally causes the current test to fail. 488 489**PARAMETERS** 490 491 492| Name | Description | Default Value | 493| :------------- | :------------- | :------------- | 494| <a id="unittest.fail-env"></a>env | The test environment returned by `unittest.begin`. | none | 495| <a id="unittest.fail-msg"></a>msg | The message to log describing the failure. | none | 496 497 498<a id="unittest.make"></a> 499 500## unittest.make 501 502<pre> 503unittest.make(<a href="#unittest.make-impl">impl</a>, <a href="#unittest.make-attrs">attrs</a>, <a href="#unittest.make-doc">doc</a>, <a href="#unittest.make-toolchains">toolchains</a>) 504</pre> 505 506Creates a unit test rule from its implementation function. 507 508Each unit test is defined in an implementation function that must then be 509associated with a rule so that a target can be built. This function handles 510the boilerplate to create and return a test rule and captures the 511implementation function's name so that it can be printed in test feedback. 512 513The optional `attrs` argument can be used to define dependencies for this 514test, in order to form unit tests of rules. 515 516The optional `toolchains` argument can be used to define toolchain 517dependencies for this test. 518 519An example of a unit test: 520 521``` 522def _your_test(ctx): 523 env = unittest.begin(ctx) 524 525 # Assert statements go here 526 527 return unittest.end(env) 528 529your_test = unittest.make(_your_test) 530``` 531 532Recall that names of test rules must end in `_test`. 533 534 535**PARAMETERS** 536 537 538| Name | Description | Default Value | 539| :------------- | :------------- | :------------- | 540| <a id="unittest.make-impl"></a>impl | The implementation function of the unit test. | none | 541| <a id="unittest.make-attrs"></a>attrs | An optional dictionary to supplement the attrs passed to the unit test's `rule()` constructor. | `{}` | 542| <a id="unittest.make-doc"></a>doc | A description of the rule that can be extracted by documentation generating tools. | `""` | 543| <a id="unittest.make-toolchains"></a>toolchains | An optional list to supplement the toolchains passed to the unit test's `rule()` constructor. | `[]` | 544 545**RETURNS** 546 547A rule definition that should be stored in a global whose name ends in 548`_test`. 549 550 551<a id="unittest.suite"></a> 552 553## unittest.suite 554 555<pre> 556unittest.suite(<a href="#unittest.suite-name">name</a>, <a href="#unittest.suite-test_rules">test_rules</a>) 557</pre> 558 559Defines a `test_suite` target that contains multiple tests. 560 561After defining your test rules in a `.bzl` file, you need to create targets 562from those rules so that `blaze test` can execute them. Doing this manually 563in a BUILD file would consist of listing each test in your `load` statement 564and then creating each target one by one. To reduce duplication, we recommend 565writing a macro in your `.bzl` file to instantiate all targets, and calling 566that macro from your BUILD file so you only have to load one symbol. 567 568You can use this function to create the targets and wrap them in a single 569test_suite target. If a test rule requires no arguments, you can simply list 570it as an argument. If you wish to supply attributes explicitly, you can do so 571using `partial.make()`. For instance, in your `.bzl` file, you could write: 572 573``` 574def your_test_suite(): 575 unittest.suite( 576 "your_test_suite", 577 your_test, 578 your_other_test, 579 partial.make(yet_another_test, timeout = "short"), 580 ) 581``` 582 583Then, in your `BUILD` file, simply load the macro and invoke it to have all 584of the targets created: 585 586``` 587load("//path/to/your/package:tests.bzl", "your_test_suite") 588your_test_suite() 589``` 590 591If you pass _N_ unit test rules to `unittest.suite`, _N_ + 1 targets will be 592created: a `test_suite` target named `${name}` (where `${name}` is the name 593argument passed in here) and targets named `${name}_test_${i}`, where `${i}` 594is the index of the test in the `test_rules` list, which is used to uniquely 595name each target. 596 597 598**PARAMETERS** 599 600 601| Name | Description | Default Value | 602| :------------- | :------------- | :------------- | 603| <a id="unittest.suite-name"></a>name | The name of the `test_suite` target, and the prefix of all the test target names. | none | 604| <a id="unittest.suite-test_rules"></a>test_rules | A list of test rules defines by `unittest.test`. | none | 605 606 607