README.rst
1Go rules test suite
2===================
3
4Main test areas
5---------------
6
7.. Child list start
8
9* `Core Go rules tests <core/README.rst>`_
10* `Integration tests <integration/README.rst>`_
11* `Legacy tests <legacy/README.rst>`_
12* `Go rules examples <examples/README.rst>`_
13
14.. Child list end
15
16Adding a new test
17-----------------
18
19All tests in the test suite are expected to obey certain rules.
20
21They must be documented
22 Each test folder must contain a README.rst that documents the area of
23 reponsability for the folder. That README must have a section with
24 the same name as each of the test rules that documents exactly what the
25 test is supposed to be checking for.
26 If the test is in response to a previous issue, the documentation must
27 also link to the issue being addressed.
28
29Test one thing at a time
30 Each test should have a clear and specific responsability, and it should be
31 as tightly targeted as possible.
32 Prefer writing multiple tests in a single folder to a single test that
33 excercises multiple things.
34
35They must be turned on
36 Test that do not run by default on the CI machines are not much use,
37 especially as it's often the only way changes get tested in environments
38 that are not the one they are authored on, and the rules are very sensitive
39 to platform specific variations.
40
41They must not be flakey
42 We will generally just delete tests that flake, and if features cannot be
43 tested without flakes we will probably delete the feature as well.
44
45They must work on all platforms
46 We support mac, linux and windows, and all our code must work across all
47 hosts. There are some tests that need to be platform specific, but it's
48 very rare, and needs a really strong rationale to be accepted.
49
50They must be as fast as possible
51 Some tests need to be large and expensive, but most do not. In particular,
52 downloading large external dependancies to perform a small unit test is not
53 ok, prefer creating a small local replication of the problem instead.
54 Anything that requires external dependancies beyond those of the rules
55 belongs in the integration tests.