1*da0073e9SAndroid Build Coastguard Worker# PyTorch CI Stats 2*da0073e9SAndroid Build Coastguard Worker 3*da0073e9SAndroid Build Coastguard WorkerWe track various stats about each CI job. 4*da0073e9SAndroid Build Coastguard Worker 5*da0073e9SAndroid Build Coastguard Worker1. Jobs upload their artifacts to an intermediate data store (either GitHub 6*da0073e9SAndroid Build Coastguard Worker Actions artifacts or S3, depending on what permissions the job has). Example: 7*da0073e9SAndroid Build Coastguard Worker https://github.com/pytorch/pytorch/blob/a9f6a35a33308f3be2413cc5c866baec5cfe3ba1/.github/workflows/_linux-build.yml#L144-L151 8*da0073e9SAndroid Build Coastguard Worker2. When a workflow completes, a `workflow_run` event [triggers 9*da0073e9SAndroid Build Coastguard Worker `upload-test-stats.yml`](https://github.com/pytorch/pytorch/blob/d9fca126fca7d7780ae44170d30bda901f4fe35e/.github/workflows/upload-test-stats.yml#L4). 10*da0073e9SAndroid Build Coastguard Worker3. `upload-test-stats` downloads the raw stats from the intermediate data store 11*da0073e9SAndroid Build Coastguard Worker and uploads them as JSON to Rockset, our metrics backend. 12*da0073e9SAndroid Build Coastguard Worker 13*da0073e9SAndroid Build Coastguard Worker```mermaid 14*da0073e9SAndroid Build Coastguard Workergraph LR 15*da0073e9SAndroid Build Coastguard Worker J1[Job with AWS creds<br>e.g. linux, win] --raw stats--> S3[(AWS S3)] 16*da0073e9SAndroid Build Coastguard Worker J2[Job w/o AWS creds<br>e.g. mac] --raw stats--> GHA[(GH artifacts)] 17*da0073e9SAndroid Build Coastguard Worker 18*da0073e9SAndroid Build Coastguard Worker S3 --> uts[upload-test-stats.yml] 19*da0073e9SAndroid Build Coastguard Worker GHA --> uts 20*da0073e9SAndroid Build Coastguard Worker 21*da0073e9SAndroid Build Coastguard Worker uts --json--> R[(Rockset)] 22*da0073e9SAndroid Build Coastguard Worker``` 23*da0073e9SAndroid Build Coastguard Worker 24*da0073e9SAndroid Build Coastguard WorkerWhy this weird indirection? Because writing to Rockset requires special 25*da0073e9SAndroid Build Coastguard Workerpermissions which, for security reasons, we do not want to give to pull request 26*da0073e9SAndroid Build Coastguard WorkerCI. Instead, we implemented GitHub's [recommended 27*da0073e9SAndroid Build Coastguard Workerpattern](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/) 28*da0073e9SAndroid Build Coastguard Workerfor cases like this. 29*da0073e9SAndroid Build Coastguard Worker 30*da0073e9SAndroid Build Coastguard WorkerFor more details about what stats we export, check out 31*da0073e9SAndroid Build Coastguard Worker[`upload-test-stats.yml`](https://github.com/pytorch/pytorch/blob/d9fca126fca7d7780ae44170d30bda901f4fe35e/.github/workflows/upload-test-stats.yml) 32