xref: /aosp_15_r20/external/cronet/base/tracing/test/README (revision 6777b5387eb2ff775bb5750e3f5d96f37fb7352b)

# PerfettoSQL Chrome Standard Library tests

This directory contains the [Perfetto Diff Tests](https://perfetto.dev/docs/analysis/trace-processor#diff-tests) to test changes to the Chrome standard library.

The diff tests themselves are in `./trace_processor/diff_tests/chrome`. The `./data` directory contains the Perfetto traces that are used by the diff tests. As well as testing the functionality of your metric, the diff tests help to ensure that the stdlib remains backwards compatible with existing traces recorded from older Chrome versions.

## Running Diff Tests

Currently, the diff tests only run on Linux. You can build and run the diff tests with the following.

```
$ gn gen --args='' out/Linux
$ autoninja -C out/Linux perfetto_diff_tests
$ out/Linux/bin/run_perfetto_diff_tests
```

To run specific diff tests you can specify the `--name-filter` flag on the `run_perfetto_diff_tests` script with regex to filter which tests you want to run.

## Adding a New Diff Test

Your new diff test should go in `base/tracing/test/trace_processor/diff_tests/chrome`. You can either add to an existing TestSuite in one of the files or add a new test in a new file.

If you are adding a **new TestSuite**, be sure to add it to `include_index.py` so the runner knows to run this new TestSuite.

If your test requires modifying or adding new test data i.e. a new trace in `base/tracing/test/data`, you will need to upload this to the GCS bucket. These trace files are too large to be checked-in to the codebase so we check-in only `.sha256` files. You can upload any new traces with this script:

```
$ base/tracing/test/test_data.py upload --verbose
```

This script will upload your file and generate the `.sha256` file in the `base/tracing/test/data/` directory. You can then upload this file with your CL.

## Writing TestTraceProcessor Tests

See [test_trace_processor_example_unittests.cc](../../test/test_trace_processor_example_unittest.cc) for examples you can compile and run.

You can write unit or browser tests with the TestTraceProcessor to record a trace, run a query on it and write expectations against the result.

Instructions:

1. As of 22nd March 2024 all platforms use the Perfetto client library. However, if you do encounter compiler errors you may need to put your test behind the `BUILDFLAG(USE_PERFETTO_CLIENT_LIBRARY)`. [TODO(crbug/42050015): remove after fully launching Perfetto Client Library]

2. You need to add a `base::test::TracingEnvironment` as a member in your test class to handle the setup and teardown between tests. You also need a `base::test::TaskEnvironment` which is needed for starting/stopping tracing.

3. Record a trace:
```
TestTraceProcessor test_trace_processor;
test_trace_processor.StartTrace(/* category_filter_string */);

/* do stuff */

absl::Status status = test_trace_processor.StopAndParseTrace();
ASSERT_TRUE(status.ok()) << status.message();
```

4. Run your query:
```
auto result = test_trace_processor.RunQuery(/* your query */);
ASSERT_TRUE(result.has_value()) << result.message();
```

5. Write expectations against the output:
```
EXPECT_THAT(result.value(), /* your expectations */);
```

The output format is a 2D vector of strings `std::vector<std::vector<std::string>>` where each vector is an SQLite row you would see when querying from the Perfetto UI. The first row will contain the header names for the columns.

#### Best Practices

* Use `ORDER BY` in queries so that the results are deterministic.

* Note that the some data is not stable over long time, in particular ids generated by trace processor, which can change for the same trace is the trace processor under-the-hood parsing logic changes. Slice ids, utids and upids are the most common examples of this.

* In general, it's recommended for tests to focus on the relationships between events, e.g. checking that you find the correct event when filtering by specific id and that its name is as expected, rather than checking specific id values.