1# `__torch_function__` micro-benchmarks 2 3This benchmark suite provides a systemic way to measure the performance of `__torch_function__` overhead. 4 5## Getting started 6### Initial Setup 7Install `py-spy` by doing: 8 9```bash 10pip install py-spy 11``` 12 13Note that more extensive documentation on using `py-spy` is available in `CONTRIBUTING.md`. 14 15### Running the benchmark 16Run one of the following commands in the terminal, with the working directory being `${PYTORCH_CLONE_DIR}/benchmarks/overrides_benchmark`: 17 18```bash 19# Benchmark all the cases 20python bench.py 21 22# Flame graph pertaining to each case. 23py-spy record -o tensor.svg --native -- python pyspybench.py Tensor 24py-spy record -o subtensor.svg --native -- python pyspybench.py SubTensor 25py-spy record -o overridden.svg --native -- python pyspybench.py WithTorchFunction 26py-spy record -o suboverridden.svg --native -- python pyspybench.py SubWithTorchFunction 27``` 28 29Here is a brief overview of what the results should look like, if run correctly: 30 31* Overhead for `torch` functions when run on `torch.Tensor` objects is on the order of 2 μs. 32* `__torch_function__` should add zero overhead for `torch.Tensor` inputs, a small overhead for subclasses of `torch.Tensor`, and a couple of microseconds for `Tensor`-likes with `__torch_function__`. 33* Changing the dispatching mechanism may result in changes that are on the order of 100 ns, which are hard to detect due to noise, but important. 34 35## Reporting benchmark results 36When modifying any of the machinery around `__torch_function__`, run the benchmark for both the feature branch and the point it diverges from `master`. For each of these: 37 38* Run `bench.py`, and include the output in your result. 39* For each case where `bench.py` shows a regression, run the commands described above, prefixing the output SVG filename (the input to the `-o` switch) with `base-` or `branch-` depending on the commit you are running the benchmark on. 40* For each SVG, open it in the browser, take a screenshot and include it in your result. Also include a ZIP file with all SVGs thus produced included. 41
