1# Patch Review 2 3Anyone can and should review patches. It's the only way to get good at 4patch review and for the project to scale. 5 6## Goals of patch review 7 81. Prevent false positive test results 92. Prevent false negative test results 103. Keep the code as simple as possible, but no simpler 11 12## How to find clear errors 13 14A clear error is one where there is unlikely to be any argument if you 15provide evidence of it. Evidence being an error trace or logical proof 16that an error will occur in a common situation. 17 18The following are examples and may not be appropriate for all tests. 19 20* Merge the patch locally. It should apply cleanly to master. 21* Compile the patch with default and non-default configurations. 22 - Use sanitizers e.g. undefined behaviour, address. 23 - Compile on non-x86 24 - Compile on x86 with -m32 25 - Compile testing patches with GitHub Actions in LTP repo fork can cover 26 various distros/architectures 27* Use `make check` 28* Run effected tests in a VM 29 - Use single vCPU 30 - Use many vCPUs and enable NUMA 31 - Restrict RAM to < 1GB. 32* Run effected tests on an embedded device 33* Run effected tests on non-x86 machine in general 34* Run reproducers on a kernel where the bug is present 35* Run tests with "-i0" and "-i2" 36* Compare usage of system calls with man page descriptions 37* Compare usage of system calls with kernel code 38* Double check commit message 39* Search the LTP library for existing helper functions 40* Check doc formatting, i.e. make doc && chromium docparse/metadata.html 41 42## How to find subtle errors 43 44A subtle error is one where you can expect some argument because you 45do not have clear evidence of an error. It is best to state these as 46questions and not make assertions if possible. 47 48Although if it is a matter of style or "taste" then senior maintainers 49can assert what is correct to avoid bike shedding. 50 51* Ask what happens if there is an error, could it be debugged just 52 with the test output? 53* Are we testing undefined behavior? 54 - Could future kernel behaviour change without "breaking userland"? 55 - Does the kernel behave differently depending on hardware? 56 - Does it behave differently depending on kernel configuration? 57 - Does it behave differently depending on the compiler? 58 - Would it behave differently if the order of checks on syscall parameters 59 changed in the kernel? 60* Will it scale to tiny and huge systems? 61 - What happens if there are 100+ CPUs? 62 - What happens if each CPU core is very slow? 63 - What happens if there are 2TB of RAM? 64* Are we repeating a pattern that can be turned into a library function? 65* Is a single test trying to do too much? 66* Could multiple similar tests be merged? 67* Race conditions 68 - What happens if a process gets preempted? 69 - Could checkpoints or fuzzsync by used instead? 70 - Note, usually you can insert a sleep to prove a race condition 71 exists however finding them is hard 72* Is there a simpler way to achieve the same kernel coverage? 73 74## How to get patches merged 75 76Once you think a patch is good enough you should add your Reviewed-by 77and/or Tested-by tags. This means you will get some credit for getting 78the patch merged. Also some blame if there are problems. 79 80If you ran the test you can add the Tested-by tag. If you read the 81code or used static analysis tools on it, you can add the Reviewed-by 82tag. 83 84In addition you can expect others to review your patches and add their 85tags. This will speed up the process of getting your patches merged. 86 87## Maintainers Checklist 88 89Patchset should be tested locally and ideally also in maintainer's fork in 90GitHub Actions on GitHub. 91 92NOTE: GitHub Actions do only build testing, passing the CI means only that 93 the test compiles fine on variety of different distributions and releases. 94 95The test should be executed at least once locally and should PASS as well. 96 97Commit messages should have 98 99* Author's `Signed-off-by` tag 100* Committer's `Reviewed-by` or `Signed-off-by` tag 101* Check also mailing lists for other reviewers / testers tags, notes and failure reports 102* `Fixes: hash` if it fixes particular LTP commit 103* `Fixes: #N` if it fixes github issue number N, so it's automatically closed 104* LTP WIKI (git: https://github.com/linux-test-project/ltp.wiki.git) should be kept up to date. 105 106After patch is accepted or rejected, set correct state and archive in 107https://patchwork.ozlabs.org/project/ltp/list/[LTP patchwork instance]. 108 109## New tests 110New test should 111 112* Have a record in runtest file 113* Test should work fine with more than one iteration 114 (e.g. run with `-i 100`) 115* Run with `-i 0` to check that setup and cleanup are coded properly (no test is being run) 116* Have a brief description 117* License: the default license for new tests is GPL v2 or later, use 118 GPL-2.0-or-later; the licence for test (e.g. GPL-2.0) should not change 119 unless test is completely rewritten 120* Old copyrights should be kept unless test is completely rewritten 121 122### C tests 123* Use new https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#22-writing-a-test-in-c[C API] 124* Test binaries are added into corresponding `.gitignore` files 125* Check coding style with `make check` 126 (more in https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#21-c-coding-style[C coding style]) 127* Docparse documentation 128* If a test is a regression test it should include tags 129 (more in https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#2238-test-tags[Test tags]) 130 131### Shell tests 132* Use new https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#23-writing-a-testcase-in-shell[shell API] 133* Check coding style with `make check` 134 (more in https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#132-shell-coding-style[Shell coding style]) 135* If a test is a regression test it should include related kernel or glibc commits as a comment 136 137## LTP library 138For patchset touching library please check also 139https://github.com/linux-test-project/ltp/wiki/LTP-Library-API-Writing-Guidelines[LTP Library API Writing Guidelines]. 140