1*6dbdd20aSAndroid Build Coastguard Worker# Heap profiler 2*6dbdd20aSAndroid Build Coastguard Worker 3*6dbdd20aSAndroid Build Coastguard WorkerNOTE: **heapprofd requires Android 10 or higher** 4*6dbdd20aSAndroid Build Coastguard Worker 5*6dbdd20aSAndroid Build Coastguard WorkerHeapprofd is a tool that tracks heap allocations & deallocations of an Android 6*6dbdd20aSAndroid Build Coastguard Workerprocess within a given time period. The resulting profile can be used to 7*6dbdd20aSAndroid Build Coastguard Workerattribute memory usage to particular call-stacks, supporting a mix of both 8*6dbdd20aSAndroid Build Coastguard Workernative and java code. The tool can be used by Android platform and app 9*6dbdd20aSAndroid Build Coastguard Workerdevelopers to investigate memory issues. 10*6dbdd20aSAndroid Build Coastguard Worker 11*6dbdd20aSAndroid Build Coastguard WorkerBy default, the tool records native allocations and deallocations done with 12*6dbdd20aSAndroid Build Coastguard Workermalloc/free (or new/delete). It can be configured to record java heap memory 13*6dbdd20aSAndroid Build Coastguard Workerallocations instead: see [Java heap sampling](#java-heap-sampling) below. 14*6dbdd20aSAndroid Build Coastguard Worker 15*6dbdd20aSAndroid Build Coastguard WorkerOn debug Android builds, you can profile all apps and most system services. 16*6dbdd20aSAndroid Build Coastguard WorkerOn "user" builds, you can only use it on apps with the debuggable or 17*6dbdd20aSAndroid Build Coastguard Workerprofileable manifest flag. 18*6dbdd20aSAndroid Build Coastguard Worker 19*6dbdd20aSAndroid Build Coastguard Worker## Quickstart 20*6dbdd20aSAndroid Build Coastguard Worker 21*6dbdd20aSAndroid Build Coastguard WorkerSee the [Memory Guide](/docs/case-studies/memory.md#heapprofd) for getting 22*6dbdd20aSAndroid Build Coastguard Workerstarted with heapprofd. 23*6dbdd20aSAndroid Build Coastguard Worker 24*6dbdd20aSAndroid Build Coastguard Worker## UI 25*6dbdd20aSAndroid Build Coastguard Worker 26*6dbdd20aSAndroid Build Coastguard WorkerDumps from heapprofd are shown as flamegraphs in the UI after clicking on the 27*6dbdd20aSAndroid Build Coastguard Workerdiamond. Each diamond corresponds to a snapshot of the allocations and 28*6dbdd20aSAndroid Build Coastguard Workercallstacks collected at that point in time. 29*6dbdd20aSAndroid Build Coastguard Worker 30*6dbdd20aSAndroid Build Coastguard Worker 31*6dbdd20aSAndroid Build Coastguard Worker 32*6dbdd20aSAndroid Build Coastguard Worker 33*6dbdd20aSAndroid Build Coastguard Worker 34*6dbdd20aSAndroid Build Coastguard Worker## SQL 35*6dbdd20aSAndroid Build Coastguard Worker 36*6dbdd20aSAndroid Build Coastguard WorkerInformation about callstacks is written to the following tables: 37*6dbdd20aSAndroid Build Coastguard Worker 38*6dbdd20aSAndroid Build Coastguard Worker* [`stack_profile_mapping`](/docs/analysis/sql-tables.autogen#stack_profile_mapping) 39*6dbdd20aSAndroid Build Coastguard Worker* [`stack_profile_frame`](/docs/analysis/sql-tables.autogen#stack_profile_frame) 40*6dbdd20aSAndroid Build Coastguard Worker* [`stack_profile_callsite`](/docs/analysis/sql-tables.autogen#stack_profile_callsite) 41*6dbdd20aSAndroid Build Coastguard Worker 42*6dbdd20aSAndroid Build Coastguard WorkerThe allocations themselves are written to 43*6dbdd20aSAndroid Build Coastguard Worker[`heap_profile_allocation`](/docs/analysis/sql-tables.autogen#heap_profile_allocation). 44*6dbdd20aSAndroid Build Coastguard Worker 45*6dbdd20aSAndroid Build Coastguard WorkerOffline symbolization data is stored in 46*6dbdd20aSAndroid Build Coastguard Worker[`stack_profile_symbol`](/docs/analysis/sql-tables.autogen#stack_profile_symbol). 47*6dbdd20aSAndroid Build Coastguard Worker 48*6dbdd20aSAndroid Build Coastguard WorkerSee [Example Queries](#heapprofd-example-queries) for example SQL queries. 49*6dbdd20aSAndroid Build Coastguard Worker 50*6dbdd20aSAndroid Build Coastguard Worker## Recording 51*6dbdd20aSAndroid Build Coastguard Worker 52*6dbdd20aSAndroid Build Coastguard WorkerHeapprofd can be configured and started in three ways. 53*6dbdd20aSAndroid Build Coastguard Worker 54*6dbdd20aSAndroid Build Coastguard Worker#### Manual configuration 55*6dbdd20aSAndroid Build Coastguard Worker 56*6dbdd20aSAndroid Build Coastguard WorkerThis requires manually setting the 57*6dbdd20aSAndroid Build Coastguard Worker[HeapprofdConfig](/docs/reference/trace-config-proto.autogen#HeapprofdConfig) 58*6dbdd20aSAndroid Build Coastguard Workersection of the trace config. The only benefit of doing so is that in this way 59*6dbdd20aSAndroid Build Coastguard Workerheap profiling can be enabled alongside any other tracing data sources. 60*6dbdd20aSAndroid Build Coastguard Worker 61*6dbdd20aSAndroid Build Coastguard Worker#### Using the tools/heap_profile script (recommended) 62*6dbdd20aSAndroid Build Coastguard Worker 63*6dbdd20aSAndroid Build Coastguard WorkerYou can use the `tools/heap_profile` script. If you are having trouble 64*6dbdd20aSAndroid Build Coastguard Workermake sure you are using the 65*6dbdd20aSAndroid Build Coastguard Worker[latest version]( 66*6dbdd20aSAndroid Build Coastguard Workerhttps://raw.githubusercontent.com/google/perfetto/main/tools/heap_profile). 67*6dbdd20aSAndroid Build Coastguard Worker 68*6dbdd20aSAndroid Build Coastguard WorkerYou can target processes either by name (`-n com.example.myapp`) or by PID 69*6dbdd20aSAndroid Build Coastguard Worker(`-p 1234`). In the first case, the heap profile will be initiated on both on 70*6dbdd20aSAndroid Build Coastguard Workeralready-running processes that match the package name and new processes launched 71*6dbdd20aSAndroid Build Coastguard Workerafter the profiling session is started. 72*6dbdd20aSAndroid Build Coastguard WorkerFor the full arguments list see the 73*6dbdd20aSAndroid Build Coastguard Worker[heap_profile cmdline reference page](/docs/reference/heap_profile-cli). 74*6dbdd20aSAndroid Build Coastguard Worker 75*6dbdd20aSAndroid Build Coastguard WorkerYou can use the [Perfetto UI](https://ui.perfetto.dev) to visualize heap dumps. 76*6dbdd20aSAndroid Build Coastguard WorkerUpload the `raw-trace` file in your output directory. You will see all heap 77*6dbdd20aSAndroid Build Coastguard Workerdumps as diamonds on the timeline, click any of them to get a flamegraph. 78*6dbdd20aSAndroid Build Coastguard Worker 79*6dbdd20aSAndroid Build Coastguard WorkerAlternatively [Speedscope](https://speedscope.app) can be used to visualize 80*6dbdd20aSAndroid Build Coastguard Workerthe gzipped protos, but will only show the "Unreleased malloc size" view. 81*6dbdd20aSAndroid Build Coastguard Worker 82*6dbdd20aSAndroid Build Coastguard Worker#### Using the Recording page of Perfetto UI 83*6dbdd20aSAndroid Build Coastguard Worker 84*6dbdd20aSAndroid Build Coastguard WorkerYou can also use the [Perfetto UI](https://ui.perfetto.dev/#!/record/memory) 85*6dbdd20aSAndroid Build Coastguard Workerto record heapprofd profiles. Tick "Heap profiling" in the trace configuration, 86*6dbdd20aSAndroid Build Coastguard Workerenter the processes you want to target, click "Add Device" to pair your phone, 87*6dbdd20aSAndroid Build Coastguard Workerand record profiles straight from your browser. This is also possible on 88*6dbdd20aSAndroid Build Coastguard WorkerWindows. 89*6dbdd20aSAndroid Build Coastguard Worker 90*6dbdd20aSAndroid Build Coastguard Worker## Viewing the data 91*6dbdd20aSAndroid Build Coastguard Worker 92*6dbdd20aSAndroid Build Coastguard Worker 93*6dbdd20aSAndroid Build Coastguard Worker 94*6dbdd20aSAndroid Build Coastguard WorkerThe resulting profile proto contains four views on the data, for each diamond. 95*6dbdd20aSAndroid Build Coastguard Worker 96*6dbdd20aSAndroid Build Coastguard Worker* **Unreleased malloc size**: how many bytes were allocated but not freed at 97*6dbdd20aSAndroid Build Coastguard Worker this callstack, from the moment the recording was started until the timestamp 98*6dbdd20aSAndroid Build Coastguard Worker of the diamond. 99*6dbdd20aSAndroid Build Coastguard Worker* **Total malloc size**: how many bytes were allocated (including ones freed at 100*6dbdd20aSAndroid Build Coastguard Worker the moment of the dump) at this callstack, from the moment the recording was 101*6dbdd20aSAndroid Build Coastguard Worker started until the timestamp of the diamond. 102*6dbdd20aSAndroid Build Coastguard Worker* **Unreleased malloc count**: how many allocations without matching frees were 103*6dbdd20aSAndroid Build Coastguard Worker done at this callstack, from the moment the recording was started until the 104*6dbdd20aSAndroid Build Coastguard Worker timestamp of the diamond. 105*6dbdd20aSAndroid Build Coastguard Worker* **Total malloc count**: how many allocations (including ones with matching 106*6dbdd20aSAndroid Build Coastguard Worker frees) were done at this callstack, from the moment the recording was started 107*6dbdd20aSAndroid Build Coastguard Worker started until the timestamp of the diamond. 108*6dbdd20aSAndroid Build Coastguard Worker 109*6dbdd20aSAndroid Build Coastguard Worker_(Googlers: You can also open the gzipped protos using http://pprof/)_ 110*6dbdd20aSAndroid Build Coastguard Worker 111*6dbdd20aSAndroid Build Coastguard WorkerTIP: you might want to put `libart.so` as a "Hide regex" when profiling apps. 112*6dbdd20aSAndroid Build Coastguard Worker 113*6dbdd20aSAndroid Build Coastguard WorkerTIP: Click Left Heavy on the top left for a good visualization. 114*6dbdd20aSAndroid Build Coastguard Worker 115*6dbdd20aSAndroid Build Coastguard Worker## Continuous dumps 116*6dbdd20aSAndroid Build Coastguard Worker 117*6dbdd20aSAndroid Build Coastguard WorkerBy default, the heap profiler captures all the allocations from the beginning of 118*6dbdd20aSAndroid Build Coastguard Workerthe recording and stores a single snapshot, shown as a single diamond in the UI, 119*6dbdd20aSAndroid Build Coastguard Workerwhich summarizes all allocations/frees. 120*6dbdd20aSAndroid Build Coastguard Worker 121*6dbdd20aSAndroid Build Coastguard WorkerIt is possible to configure the heap profiler to periodically (not just at the 122*6dbdd20aSAndroid Build Coastguard Workerend of the trace) store snapshots (continuous dumps), for example every 5000ms 123*6dbdd20aSAndroid Build Coastguard Worker 124*6dbdd20aSAndroid Build Coastguard Worker* By setting "Continuous dumps interval" in the UI to 5000. 125*6dbdd20aSAndroid Build Coastguard Worker* By adding 126*6dbdd20aSAndroid Build Coastguard Worker ``` 127*6dbdd20aSAndroid Build Coastguard Worker continuous_dump_config { 128*6dbdd20aSAndroid Build Coastguard Worker dump_interval_ms: 5000 129*6dbdd20aSAndroid Build Coastguard Worker } 130*6dbdd20aSAndroid Build Coastguard Worker ``` 131*6dbdd20aSAndroid Build Coastguard Worker in the 132*6dbdd20aSAndroid Build Coastguard Worker [HeapprofdConfig](/docs/reference/trace-config-proto.autogen#HeapprofdConfig). 133*6dbdd20aSAndroid Build Coastguard Worker* By adding `-c 5000` to the invocation of 134*6dbdd20aSAndroid Build Coastguard Worker [`tools/heap_profile`](/docs/reference/heap_profile-cli). 135*6dbdd20aSAndroid Build Coastguard Worker 136*6dbdd20aSAndroid Build Coastguard Worker 137*6dbdd20aSAndroid Build Coastguard Worker 138*6dbdd20aSAndroid Build Coastguard WorkerThe resulting visualization shows multiple diamonds. Clicking on each diamond 139*6dbdd20aSAndroid Build Coastguard Workershows a summary of the allocations/frees from the beginning of the trace until 140*6dbdd20aSAndroid Build Coastguard Workerthat point (i.e. the summary is cumulative). 141*6dbdd20aSAndroid Build Coastguard Worker 142*6dbdd20aSAndroid Build Coastguard Worker## Sampling interval 143*6dbdd20aSAndroid Build Coastguard Worker 144*6dbdd20aSAndroid Build Coastguard WorkerHeapprofd samples heap allocations by hooking calls to malloc/free and C++'s 145*6dbdd20aSAndroid Build Coastguard Workeroperator new/delete. Given a sampling interval of n bytes, one allocation is 146*6dbdd20aSAndroid Build Coastguard Workersampled, on average, every n bytes allocated. This allows to reduce the 147*6dbdd20aSAndroid Build Coastguard Workerperformance impact on the target process. The default sampling rate 148*6dbdd20aSAndroid Build Coastguard Workeris 4096 bytes. 149*6dbdd20aSAndroid Build Coastguard Worker 150*6dbdd20aSAndroid Build Coastguard WorkerThe easiest way to reason about this is to imagine the memory allocations as a 151*6dbdd20aSAndroid Build Coastguard Workerstream of one byte allocations. From this stream, every byte has a 1/n 152*6dbdd20aSAndroid Build Coastguard Workerprobability of being selected as a sample, and the corresponding callstack 153*6dbdd20aSAndroid Build Coastguard Workergets attributed the complete n bytes. For more accuracy, allocations larger than 154*6dbdd20aSAndroid Build Coastguard Workerthe sampling interval bypass the sampling logic and are recorded with their true 155*6dbdd20aSAndroid Build Coastguard Workersize. 156*6dbdd20aSAndroid Build Coastguard WorkerSee the [heapprofd Sampling](/docs/design-docs/heapprofd-sampling) document for 157*6dbdd20aSAndroid Build Coastguard Workerdetails. 158*6dbdd20aSAndroid Build Coastguard Worker 159*6dbdd20aSAndroid Build Coastguard Worker## Startup profiling 160*6dbdd20aSAndroid Build Coastguard Worker 161*6dbdd20aSAndroid Build Coastguard WorkerWhen specifying a target process name (as opposite to the PID), new processes 162*6dbdd20aSAndroid Build Coastguard Workermatching that name are profiled from their startup. The resulting profile will 163*6dbdd20aSAndroid Build Coastguard Workercontain all allocations done between the start of the process and the end 164*6dbdd20aSAndroid Build Coastguard Workerof the profiling session. 165*6dbdd20aSAndroid Build Coastguard Worker 166*6dbdd20aSAndroid Build Coastguard WorkerOn Android, Java apps are usually not exec()-ed from scratch, but fork()-ed from 167*6dbdd20aSAndroid Build Coastguard Workerthe [zygote], which then specializes into the desired app. If the app's name 168*6dbdd20aSAndroid Build Coastguard Workermatches a name specified in the profiling session, profiling will be enabled as 169*6dbdd20aSAndroid Build Coastguard Workerpart of the zygote specialization. The resulting profile contains all 170*6dbdd20aSAndroid Build Coastguard Workerallocations done between that point in zygote specialization and the end of the 171*6dbdd20aSAndroid Build Coastguard Workerprofiling session. Some allocations done early in the specialization process are 172*6dbdd20aSAndroid Build Coastguard Workernot accounted for. 173*6dbdd20aSAndroid Build Coastguard Worker 174*6dbdd20aSAndroid Build Coastguard WorkerAt the trace proto level, the resulting [ProfilePacket] will have the 175*6dbdd20aSAndroid Build Coastguard Worker`from_startup` field set to true in the corresponding `ProcessHeapSamples` 176*6dbdd20aSAndroid Build Coastguard Workermessage. This is not surfaced in the converted pprof compatible proto. 177*6dbdd20aSAndroid Build Coastguard Worker 178*6dbdd20aSAndroid Build Coastguard Worker[ProfilePacket]: /docs/reference/trace-packet-proto.autogen#ProfilePacket 179*6dbdd20aSAndroid Build Coastguard Worker[zygote]: https://developer.android.com/topic/performance/memory-overview#SharingRAM 180*6dbdd20aSAndroid Build Coastguard Worker 181*6dbdd20aSAndroid Build Coastguard Worker## Runtime profiling 182*6dbdd20aSAndroid Build Coastguard Worker 183*6dbdd20aSAndroid Build Coastguard WorkerWhen a profiling session is started, all matching processes (by name or PID) 184*6dbdd20aSAndroid Build Coastguard Workerare enumerated and are signalled to request profiling. Profiling isn't actually 185*6dbdd20aSAndroid Build Coastguard Workerenabled until a few hundred milliseconds after the next allocation that is 186*6dbdd20aSAndroid Build Coastguard Workerdone by the application. If the application is idle when profiling is 187*6dbdd20aSAndroid Build Coastguard Workerrequested, and then does a burst of allocations, these may be missed. 188*6dbdd20aSAndroid Build Coastguard Worker 189*6dbdd20aSAndroid Build Coastguard WorkerThe resulting profile will contain all allocations done between when profiling 190*6dbdd20aSAndroid Build Coastguard Workeris enabled, and the end of the profiling session. 191*6dbdd20aSAndroid Build Coastguard Worker 192*6dbdd20aSAndroid Build Coastguard WorkerThe resulting [ProfilePacket] will have `from_startup` set to false in the 193*6dbdd20aSAndroid Build Coastguard Workercorresponding `ProcessHeapSamples` message. This does not get surfaced in the 194*6dbdd20aSAndroid Build Coastguard Workerconverted pprof compatible proto. 195*6dbdd20aSAndroid Build Coastguard Worker 196*6dbdd20aSAndroid Build Coastguard Worker## Concurrent profiling sessions 197*6dbdd20aSAndroid Build Coastguard Worker 198*6dbdd20aSAndroid Build Coastguard WorkerIf multiple sessions name the same target process (either by name or PID), 199*6dbdd20aSAndroid Build Coastguard Workeronly the first relevant session will profile the process. The other sessions 200*6dbdd20aSAndroid Build Coastguard Workerwill report that the process had already been profiled when converting to 201*6dbdd20aSAndroid Build Coastguard Workerthe pprof compatible proto. 202*6dbdd20aSAndroid Build Coastguard Worker 203*6dbdd20aSAndroid Build Coastguard WorkerIf you see this message but do not expect any other sessions, run 204*6dbdd20aSAndroid Build Coastguard Worker 205*6dbdd20aSAndroid Build Coastguard Worker```shell 206*6dbdd20aSAndroid Build Coastguard Workeradb shell killall perfetto 207*6dbdd20aSAndroid Build Coastguard Worker``` 208*6dbdd20aSAndroid Build Coastguard Worker 209*6dbdd20aSAndroid Build Coastguard Workerto stop any concurrent sessions that may be running. 210*6dbdd20aSAndroid Build Coastguard Worker 211*6dbdd20aSAndroid Build Coastguard WorkerThe resulting [ProfilePacket] will have `rejected_concurrent` set to true in 212*6dbdd20aSAndroid Build Coastguard Workerotherwise empty corresponding `ProcessHeapSamples` message. This does not get 213*6dbdd20aSAndroid Build Coastguard Workersurfaced in the converted pprof compatible proto. 214*6dbdd20aSAndroid Build Coastguard Worker 215*6dbdd20aSAndroid Build Coastguard Worker## {#heapprofd-targets} Target processes 216*6dbdd20aSAndroid Build Coastguard Worker 217*6dbdd20aSAndroid Build Coastguard WorkerDepending on the build of Android that heapprofd is run on, some processes 218*6dbdd20aSAndroid Build Coastguard Workerare not be eligible to be profiled. 219*6dbdd20aSAndroid Build Coastguard Worker 220*6dbdd20aSAndroid Build Coastguard WorkerOn _user_ (i.e. production, non-rootable) builds, only Java applications with 221*6dbdd20aSAndroid Build Coastguard Workereither the profileable or the debuggable manifest flag set can be profiled. 222*6dbdd20aSAndroid Build Coastguard WorkerProfiling requests for non-profileable/debuggable processes will result in an 223*6dbdd20aSAndroid Build Coastguard Workerempty profile. 224*6dbdd20aSAndroid Build Coastguard Worker 225*6dbdd20aSAndroid Build Coastguard WorkerOn userdebug builds, all processes except for a small set of critical 226*6dbdd20aSAndroid Build Coastguard Workerservices can be profiled (to find the set of disallowed targets, look for 227*6dbdd20aSAndroid Build Coastguard Worker`never_profile_heap` in [heapprofd.te]( 228*6dbdd20aSAndroid Build Coastguard Workerhttps://cs.android.com/android/platform/superproject/main/+/main:system/sepolicy/private/heapprofd.te?q=never_profile_heap). 229*6dbdd20aSAndroid Build Coastguard WorkerThis restriction can be lifted by disabling SELinux by running 230*6dbdd20aSAndroid Build Coastguard Worker`adb shell su root setenforce 0` or by passing `--disable-selinux` to the 231*6dbdd20aSAndroid Build Coastguard Worker`heap_profile` script. 232*6dbdd20aSAndroid Build Coastguard Worker 233*6dbdd20aSAndroid Build Coastguard Worker<center> 234*6dbdd20aSAndroid Build Coastguard Worker 235*6dbdd20aSAndroid Build Coastguard Worker| | userdebug setenforce 0 | userdebug | user | 236*6dbdd20aSAndroid Build Coastguard Worker|-------------------------|:----------------------:|:---------:|:----:| 237*6dbdd20aSAndroid Build Coastguard Worker| critical native service | Y | N | N | 238*6dbdd20aSAndroid Build Coastguard Worker| native service | Y | Y | N | 239*6dbdd20aSAndroid Build Coastguard Worker| app | Y | Y | N | 240*6dbdd20aSAndroid Build Coastguard Worker| profileable app | Y | Y | Y | 241*6dbdd20aSAndroid Build Coastguard Worker| debuggable app | Y | Y | Y | 242*6dbdd20aSAndroid Build Coastguard Worker 243*6dbdd20aSAndroid Build Coastguard Worker</center> 244*6dbdd20aSAndroid Build Coastguard Worker 245*6dbdd20aSAndroid Build Coastguard WorkerTo mark an app as profileable, put `<profileable android:shell="true"/>` into 246*6dbdd20aSAndroid Build Coastguard Workerthe `<application>` section of the app manifest. 247*6dbdd20aSAndroid Build Coastguard Worker 248*6dbdd20aSAndroid Build Coastguard Worker```xml 249*6dbdd20aSAndroid Build Coastguard Worker<manifest ...> 250*6dbdd20aSAndroid Build Coastguard Worker <application> 251*6dbdd20aSAndroid Build Coastguard Worker <profileable android:shell="true"/> 252*6dbdd20aSAndroid Build Coastguard Worker ... 253*6dbdd20aSAndroid Build Coastguard Worker </application> 254*6dbdd20aSAndroid Build Coastguard Worker</manifest> 255*6dbdd20aSAndroid Build Coastguard Worker``` 256*6dbdd20aSAndroid Build Coastguard Worker 257*6dbdd20aSAndroid Build Coastguard Worker## {#java-heap-sampling} Java heap sampling 258*6dbdd20aSAndroid Build Coastguard Worker 259*6dbdd20aSAndroid Build Coastguard WorkerNOTE: **Java heap sampling is available on Android 12 or higher** 260*6dbdd20aSAndroid Build Coastguard Worker 261*6dbdd20aSAndroid Build Coastguard WorkerNOTE: **Java heap sampling is not to be confused with [Java heap 262*6dbdd20aSAndroid Build Coastguard Workerdumps](/docs/data-sources/java-heap-profiler.md)** 263*6dbdd20aSAndroid Build Coastguard Worker 264*6dbdd20aSAndroid Build Coastguard WorkerHeapprofd can be configured to track Java allocations instead of native ones. 265*6dbdd20aSAndroid Build Coastguard Worker* By setting adding `heaps: "com.android.art"` in 266*6dbdd20aSAndroid Build Coastguard Worker [HeapprofdConfig](/docs/reference/trace-config-proto.autogen#HeapprofdConfig). 267*6dbdd20aSAndroid Build Coastguard Worker* By adding `--heaps com.android.art` to the invocation of 268*6dbdd20aSAndroid Build Coastguard Worker [`tools/heap_profile`](/docs/reference/heap_profile-cli). 269*6dbdd20aSAndroid Build Coastguard Worker 270*6dbdd20aSAndroid Build Coastguard WorkerUnlike java heap dumps (which show the retention graph of a snapshot of the live 271*6dbdd20aSAndroid Build Coastguard Workerobjects) but like native heap profiles, java heap samples show callstacks of 272*6dbdd20aSAndroid Build Coastguard Workerallocations over time of the entire profile. 273*6dbdd20aSAndroid Build Coastguard Worker 274*6dbdd20aSAndroid Build Coastguard WorkerJava heap samples only show callstacks of when objects are created, not when 275*6dbdd20aSAndroid Build Coastguard Workerthey're deleted or garbage collected. 276*6dbdd20aSAndroid Build Coastguard Worker 277*6dbdd20aSAndroid Build Coastguard Worker 278*6dbdd20aSAndroid Build Coastguard Worker 279*6dbdd20aSAndroid Build Coastguard WorkerThe resulting profile proto contains two views on the data: 280*6dbdd20aSAndroid Build Coastguard Worker 281*6dbdd20aSAndroid Build Coastguard Worker* **Total allocation size**: how many bytes were allocated at this callstack 282*6dbdd20aSAndroid Build Coastguard Worker over time of the profile until this point. The bytes might have been freed or 283*6dbdd20aSAndroid Build Coastguard Worker not, the tool does not keep track of that. 284*6dbdd20aSAndroid Build Coastguard Worker* **Total allocation count**: how many object were allocated at this callstack 285*6dbdd20aSAndroid Build Coastguard Worker over time of the profile until this point. The objects might have been freed 286*6dbdd20aSAndroid Build Coastguard Worker or not, the tool does not keep track of that. 287*6dbdd20aSAndroid Build Coastguard Worker 288*6dbdd20aSAndroid Build Coastguard WorkerJava heap samples are useful to understand memory churn showing the call stack 289*6dbdd20aSAndroid Build Coastguard Workerof which parts of the code large allocations are attributed to as well as the 290*6dbdd20aSAndroid Build Coastguard Workerallocation type from the ART runtime. 291*6dbdd20aSAndroid Build Coastguard Worker 292*6dbdd20aSAndroid Build Coastguard Worker## DEDUPED frames 293*6dbdd20aSAndroid Build Coastguard Worker 294*6dbdd20aSAndroid Build Coastguard WorkerIf the name of a Java method includes `[DEDUPED]`, this means that multiple 295*6dbdd20aSAndroid Build Coastguard Workermethods share the same code. ART only stores the name of a single one in its 296*6dbdd20aSAndroid Build Coastguard Workermetadata, which is displayed here. This is not necessarily the one that was 297*6dbdd20aSAndroid Build Coastguard Workercalled. 298*6dbdd20aSAndroid Build Coastguard Worker 299*6dbdd20aSAndroid Build Coastguard Worker## Triggering heap snapshots on demand 300*6dbdd20aSAndroid Build Coastguard Worker 301*6dbdd20aSAndroid Build Coastguard WorkerHeap snapshot are recorded into the trace either at regular time intervals, if 302*6dbdd20aSAndroid Build Coastguard Workerusing the `continuous_dump_config` field, or at the end of the session. 303*6dbdd20aSAndroid Build Coastguard Worker 304*6dbdd20aSAndroid Build Coastguard WorkerYou can also trigger a snapshot of all currently profiled processes by running 305*6dbdd20aSAndroid Build Coastguard Worker`adb shell killall -USR1 heapprofd`. This can be useful in lab tests for 306*6dbdd20aSAndroid Build Coastguard Workerrecording the current memory usage of the target in a specific state. 307*6dbdd20aSAndroid Build Coastguard Worker 308*6dbdd20aSAndroid Build Coastguard WorkerThis dump will show up in addition to the dump at the end of the profile that is 309*6dbdd20aSAndroid Build Coastguard Workeralways produced. You can create multiple of these dumps, and they will be 310*6dbdd20aSAndroid Build Coastguard Workerenumerated in the output directory. 311*6dbdd20aSAndroid Build Coastguard Worker 312*6dbdd20aSAndroid Build Coastguard Worker## Symbolization 313*6dbdd20aSAndroid Build Coastguard Worker 314*6dbdd20aSAndroid Build Coastguard Worker### Set up llvm-symbolizer 315*6dbdd20aSAndroid Build Coastguard Worker 316*6dbdd20aSAndroid Build Coastguard WorkerYou only need to do this once. 317*6dbdd20aSAndroid Build Coastguard Worker 318*6dbdd20aSAndroid Build Coastguard WorkerTo use symbolization, your system must have llvm-symbolizer installed and 319*6dbdd20aSAndroid Build Coastguard Workeraccessible from `$PATH` as `llvm-symbolizer`. On Debian, you can install it 320*6dbdd20aSAndroid Build Coastguard Workerusing `sudo apt install llvm`. 321*6dbdd20aSAndroid Build Coastguard Worker 322*6dbdd20aSAndroid Build Coastguard Worker### Symbolize your profile 323*6dbdd20aSAndroid Build Coastguard Worker 324*6dbdd20aSAndroid Build Coastguard WorkerIf the profiled binary or libraries do not have symbol names, you can 325*6dbdd20aSAndroid Build Coastguard Workersymbolize profiles offline. Even if they do, you might want to symbolize in 326*6dbdd20aSAndroid Build Coastguard Workerorder to get inlined function and line number information. All tools 327*6dbdd20aSAndroid Build Coastguard Worker(traceconv, trace_processor_shell, the heap_profile script) support specifying 328*6dbdd20aSAndroid Build Coastguard Workerthe `PERFETTO_BINARY_PATH` as an environment variable. 329*6dbdd20aSAndroid Build Coastguard Worker 330*6dbdd20aSAndroid Build Coastguard Worker``` 331*6dbdd20aSAndroid Build Coastguard WorkerPERFETTO_BINARY_PATH=somedir tools/heap_profile --name ${NAME} 332*6dbdd20aSAndroid Build Coastguard Worker``` 333*6dbdd20aSAndroid Build Coastguard Worker 334*6dbdd20aSAndroid Build Coastguard WorkerYou can persist symbols for a trace by running 335*6dbdd20aSAndroid Build Coastguard Worker`PERFETTO_BINARY_PATH=somedir tools/traceconv symbolize raw-trace > symbols`. 336*6dbdd20aSAndroid Build Coastguard WorkerYou can then concatenate the symbols to the trace ( 337*6dbdd20aSAndroid Build Coastguard Worker`cat raw-trace symbols > symbolized-trace`) and the symbols will part of 338*6dbdd20aSAndroid Build Coastguard Worker`symbolized-trace`. The `tools/heap_profile` script will also generate this 339*6dbdd20aSAndroid Build Coastguard Workerfile in your output directory, if `PERFETTO_BINARY_PATH` is used. 340*6dbdd20aSAndroid Build Coastguard Worker 341*6dbdd20aSAndroid Build Coastguard WorkerThe symbol file is the first with matching Build ID in the following order: 342*6dbdd20aSAndroid Build Coastguard Worker 343*6dbdd20aSAndroid Build Coastguard Worker1. absolute path of library file relative to binary path. 344*6dbdd20aSAndroid Build Coastguard Worker2. absolute path of library file relative to binary path, but with base.apk! 345*6dbdd20aSAndroid Build Coastguard Worker removed from filename. 346*6dbdd20aSAndroid Build Coastguard Worker3. basename of library file relative to binary path. 347*6dbdd20aSAndroid Build Coastguard Worker4. basename of library file relative to binary path, but with base.apk! 348*6dbdd20aSAndroid Build Coastguard Worker removed from filename. 349*6dbdd20aSAndroid Build Coastguard Worker5. in the subdirectory .build-id: the first two hex digits of the build-id 350*6dbdd20aSAndroid Build Coastguard Worker as subdirectory, then the rest of the hex digits, with ".debug" appended. 351*6dbdd20aSAndroid Build Coastguard Worker See 352*6dbdd20aSAndroid Build Coastguard Worker https://fedoraproject.org/wiki/RolandMcGrath/BuildID#Find_files_by_build_ID 353*6dbdd20aSAndroid Build Coastguard Worker 354*6dbdd20aSAndroid Build Coastguard WorkerFor example, "/system/lib/base.apk!foo.so" with build id abcd1234, 355*6dbdd20aSAndroid Build Coastguard Workeris looked for at: 356*6dbdd20aSAndroid Build Coastguard Worker 357*6dbdd20aSAndroid Build Coastguard Worker1. $PERFETTO_BINARY_PATH/system/lib/base.apk!foo.so 358*6dbdd20aSAndroid Build Coastguard Worker2. $PERFETTO_BINARY_PATH/system/lib/foo.so 359*6dbdd20aSAndroid Build Coastguard Worker3. $PERFETTO_BINARY_PATH/base.apk!foo.so 360*6dbdd20aSAndroid Build Coastguard Worker4. $PERFETTO_BINARY_PATH/foo.so 361*6dbdd20aSAndroid Build Coastguard Worker5. $PERFETTO_BINARY_PATH/.build-id/ab/cd1234.debug 362*6dbdd20aSAndroid Build Coastguard Worker 363*6dbdd20aSAndroid Build Coastguard WorkerAlternatively, you can set the `PERFETTO_SYMBOLIZER_MODE` environment variable 364*6dbdd20aSAndroid Build Coastguard Workerto `index`, and the symbolizer will recursively search the given directory for 365*6dbdd20aSAndroid Build Coastguard Workeran ELF file with the given build id. This way, you will not have to worry 366*6dbdd20aSAndroid Build Coastguard Workerabout correct filenames. 367*6dbdd20aSAndroid Build Coastguard Worker 368*6dbdd20aSAndroid Build Coastguard Worker## Deobfuscation 369*6dbdd20aSAndroid Build Coastguard Worker 370*6dbdd20aSAndroid Build Coastguard WorkerIf your profile contains obfuscated Java methods (like `fsd.a`), you can 371*6dbdd20aSAndroid Build Coastguard Workerprovide a deobfuscation map to turn them back into human readable. 372*6dbdd20aSAndroid Build Coastguard WorkerTo do so, use the `PERFETTO_PROGUARD_MAP` environment variable, using the 373*6dbdd20aSAndroid Build Coastguard Workerformat `packagename=map_filename[:packagename=map_filename...]`, e.g. 374*6dbdd20aSAndroid Build Coastguard Worker`PERFETTO_PROGUARD_MAP=com.example.pkg1=foo.txt:com.example.pkg2=bar.txt`. 375*6dbdd20aSAndroid Build Coastguard WorkerAll tools (traceconv, trace_processor_shell, the heap_profile script) support 376*6dbdd20aSAndroid Build Coastguard Workerspecifying the `PERFETTO_PROGUARD_MAP` as an environment variable. 377*6dbdd20aSAndroid Build Coastguard Worker 378*6dbdd20aSAndroid Build Coastguard Worker``` 379*6dbdd20aSAndroid Build Coastguard WorkerPERFETTO_PROGUARD_MAP=com.example.pkg1=proguard_map1.txt:com.example.pkg2=proguard_map2.txt ./tools/heap_profile -n com.example.app 380*6dbdd20aSAndroid Build Coastguard Worker``` 381*6dbdd20aSAndroid Build Coastguard Worker 382*6dbdd20aSAndroid Build Coastguard WorkerYou can get a deobfuscation map for the trace you already collected using 383*6dbdd20aSAndroid Build Coastguard Worker`tools/traceconv deobfuscate`. Then concatenate the resulting file to your 384*6dbdd20aSAndroid Build Coastguard Workertrace to get a deobfuscated version of it (the input trace should be in the 385*6dbdd20aSAndroid Build Coastguard Workerperfetto format, otherwise concatenation will not produce a reasonable output). 386*6dbdd20aSAndroid Build Coastguard Worker 387*6dbdd20aSAndroid Build Coastguard Worker``` 388*6dbdd20aSAndroid Build Coastguard WorkerPERFETTO_PROGUARD_MAP=com.example.pkg=proguard_map.txt tools/traceconv deobfuscate ${TRACE} > deobfuscation_map 389*6dbdd20aSAndroid Build Coastguard Workercat ${TRACE} deobfuscation_map > deobfuscated_trace 390*6dbdd20aSAndroid Build Coastguard Worker``` 391*6dbdd20aSAndroid Build Coastguard Worker 392*6dbdd20aSAndroid Build Coastguard Worker`deobfuscated_trace` can be viewed in the 393*6dbdd20aSAndroid Build Coastguard Worker[Perfetto UI](https://ui.perfetto.dev). 394*6dbdd20aSAndroid Build Coastguard Worker 395*6dbdd20aSAndroid Build Coastguard Worker## Troubleshooting 396*6dbdd20aSAndroid Build Coastguard Worker 397*6dbdd20aSAndroid Build Coastguard Worker### Buffer overrun 398*6dbdd20aSAndroid Build Coastguard Worker 399*6dbdd20aSAndroid Build Coastguard WorkerIf the rate of allocations is too high for heapprofd to keep up, the profiling 400*6dbdd20aSAndroid Build Coastguard Workersession will end early due to a buffer overrun. If the buffer overrun is 401*6dbdd20aSAndroid Build Coastguard Workercaused by a transient spike in allocations, increasing the shared memory buffer 402*6dbdd20aSAndroid Build Coastguard Workersize (passing `--shmem-size` to `tools/heap_profile`) can resolve the issue. 403*6dbdd20aSAndroid Build Coastguard WorkerOtherwise the sampling interval can be increased (at the expense of lower 404*6dbdd20aSAndroid Build Coastguard Workeraccuracy in the resulting profile) by passing `--interval=16000` or higher. 405*6dbdd20aSAndroid Build Coastguard Worker 406*6dbdd20aSAndroid Build Coastguard Worker### Profile is empty 407*6dbdd20aSAndroid Build Coastguard Worker 408*6dbdd20aSAndroid Build Coastguard WorkerCheck whether your target process is eligible to be profiled by consulting 409*6dbdd20aSAndroid Build Coastguard Worker[Target processes](#heapprofd-targets) above. 410*6dbdd20aSAndroid Build Coastguard Worker 411*6dbdd20aSAndroid Build Coastguard WorkerAlso check the [Known Issues](#known-issues). 412*6dbdd20aSAndroid Build Coastguard Worker 413*6dbdd20aSAndroid Build Coastguard Worker### Implausible callstacks 414*6dbdd20aSAndroid Build Coastguard Worker 415*6dbdd20aSAndroid Build Coastguard WorkerIf you see a callstack that seems to impossible from looking at the code, make 416*6dbdd20aSAndroid Build Coastguard Workersure no [DEDUPED frames](#deduped-frames) are involved. 417*6dbdd20aSAndroid Build Coastguard Worker 418*6dbdd20aSAndroid Build Coastguard WorkerAlso, if your code is linked using _Identical Code Folding_ 419*6dbdd20aSAndroid Build Coastguard Worker(ICF), i.e. passing `-Wl,--icf=...` to the linker, most trivial functions, often 420*6dbdd20aSAndroid Build Coastguard Workerconstructors and destructors, can be aliased to binary-equivalent operators 421*6dbdd20aSAndroid Build Coastguard Workerof completely unrelated classes. 422*6dbdd20aSAndroid Build Coastguard Worker 423*6dbdd20aSAndroid Build Coastguard Worker### Symbolization: Could not find library 424*6dbdd20aSAndroid Build Coastguard Worker 425*6dbdd20aSAndroid Build Coastguard WorkerWhen symbolizing a profile, you might come across messages like this: 426*6dbdd20aSAndroid Build Coastguard Worker 427*6dbdd20aSAndroid Build Coastguard Worker```bash 428*6dbdd20aSAndroid Build Coastguard WorkerCould not find /data/app/invalid.app-wFgo3GRaod02wSvPZQ==/lib/arm64/somelib.so 429*6dbdd20aSAndroid Build Coastguard Worker(Build ID: 44b7138abd5957b8d0a56ce86216d478). 430*6dbdd20aSAndroid Build Coastguard Worker``` 431*6dbdd20aSAndroid Build Coastguard Worker 432*6dbdd20aSAndroid Build Coastguard WorkerCheck whether your library (in this example somelib.so) exists in 433*6dbdd20aSAndroid Build Coastguard Worker`PERFETTO_BINARY_PATH`. Then compare the Build ID to the one in your 434*6dbdd20aSAndroid Build Coastguard Workersymbol file, which you can get by running 435*6dbdd20aSAndroid Build Coastguard Worker`readelf -n /path/in/binary/path/somelib.so`. If it does not match, the 436*6dbdd20aSAndroid Build Coastguard Workersymbolized file has a different version than the one on device, and cannot 437*6dbdd20aSAndroid Build Coastguard Workerbe used for symbolization. 438*6dbdd20aSAndroid Build Coastguard WorkerIf it does, try moving somelib.so to the root of `PERFETTO_BINARY_PATH` and 439*6dbdd20aSAndroid Build Coastguard Workertry again. 440*6dbdd20aSAndroid Build Coastguard Worker 441*6dbdd20aSAndroid Build Coastguard Worker### Only one frame shown 442*6dbdd20aSAndroid Build Coastguard WorkerIf you only see a single frame for functions in a specific library, make sure 443*6dbdd20aSAndroid Build Coastguard Workerthat the library has unwind information. We need one of 444*6dbdd20aSAndroid Build Coastguard Worker 445*6dbdd20aSAndroid Build Coastguard Worker* `.gnu_debugdata` 446*6dbdd20aSAndroid Build Coastguard Worker* `.eh_frame` (+ preferably `.eh_frame_hdr`) 447*6dbdd20aSAndroid Build Coastguard Worker* `.debug_frame`. 448*6dbdd20aSAndroid Build Coastguard Worker 449*6dbdd20aSAndroid Build Coastguard WorkerFrame-pointer unwinding is *not supported*. 450*6dbdd20aSAndroid Build Coastguard Worker 451*6dbdd20aSAndroid Build Coastguard WorkerTo check if an ELF file has any of those, run 452*6dbdd20aSAndroid Build Coastguard Worker 453*6dbdd20aSAndroid Build Coastguard Worker```console 454*6dbdd20aSAndroid Build Coastguard Worker$ readelf -S file.so | grep "gnu_debugdata\|eh_frame\|debug_frame" 455*6dbdd20aSAndroid Build Coastguard Worker [12] .eh_frame_hdr PROGBITS 000000000000c2b0 0000c2b0 456*6dbdd20aSAndroid Build Coastguard Worker [13] .eh_frame PROGBITS 0000000000011000 00011000 457*6dbdd20aSAndroid Build Coastguard Worker [24] .gnu_debugdata PROGBITS 0000000000000000 000f7292 458*6dbdd20aSAndroid Build Coastguard Worker``` 459*6dbdd20aSAndroid Build Coastguard Worker 460*6dbdd20aSAndroid Build Coastguard WorkerIf this does not show one or more of the sections, change your build system 461*6dbdd20aSAndroid Build Coastguard Workerto not strip them. 462*6dbdd20aSAndroid Build Coastguard Worker 463*6dbdd20aSAndroid Build Coastguard Worker## (non-Android) Linux support 464*6dbdd20aSAndroid Build Coastguard Worker 465*6dbdd20aSAndroid Build Coastguard WorkerNOTE: Do not use this for production purposes. 466*6dbdd20aSAndroid Build Coastguard Worker 467*6dbdd20aSAndroid Build Coastguard WorkerYou can use a standalone library to profile memory allocations on Linux. 468*6dbdd20aSAndroid Build Coastguard WorkerFirst [build Perfetto](/docs/contributing/build-instructions.md). You only need 469*6dbdd20aSAndroid Build Coastguard Workerto do this once. 470*6dbdd20aSAndroid Build Coastguard Worker 471*6dbdd20aSAndroid Build Coastguard Worker``` 472*6dbdd20aSAndroid Build Coastguard Workertools/setup_all_configs.py 473*6dbdd20aSAndroid Build Coastguard Workerninja -C out/linux_clang_release 474*6dbdd20aSAndroid Build Coastguard Worker``` 475*6dbdd20aSAndroid Build Coastguard Worker 476*6dbdd20aSAndroid Build Coastguard WorkerThen, run traced 477*6dbdd20aSAndroid Build Coastguard Worker 478*6dbdd20aSAndroid Build Coastguard Worker``` 479*6dbdd20aSAndroid Build Coastguard Workerout/linux_clang_release/traced 480*6dbdd20aSAndroid Build Coastguard Worker``` 481*6dbdd20aSAndroid Build Coastguard Worker 482*6dbdd20aSAndroid Build Coastguard WorkerStart the profile (e.g. targeting trace_processor_shell) 483*6dbdd20aSAndroid Build Coastguard Worker 484*6dbdd20aSAndroid Build Coastguard Worker``` 485*6dbdd20aSAndroid Build Coastguard Workertools/heap_profile -n trace_processor_shell --print-config | \ 486*6dbdd20aSAndroid Build Coastguard Workerout/linux_clang_release/perfetto \ 487*6dbdd20aSAndroid Build Coastguard Worker -c - --txt \ 488*6dbdd20aSAndroid Build Coastguard Worker -o ~/heapprofd-trace 489*6dbdd20aSAndroid Build Coastguard Worker``` 490*6dbdd20aSAndroid Build Coastguard Worker 491*6dbdd20aSAndroid Build Coastguard WorkerFinally, run your target (e.g. trace_processor_shell) with LD_PRELOAD 492*6dbdd20aSAndroid Build Coastguard Worker 493*6dbdd20aSAndroid Build Coastguard Worker``` 494*6dbdd20aSAndroid Build Coastguard WorkerLD_PRELOAD=out/linux_clang_release/libheapprofd_glibc_preload.so out/linux_clang_release/trace_processor_shell <trace> 495*6dbdd20aSAndroid Build Coastguard Worker``` 496*6dbdd20aSAndroid Build Coastguard Worker 497*6dbdd20aSAndroid Build Coastguard WorkerThen, Ctrl-C the Perfetto invocation and upload ~/heapprofd-trace to the 498*6dbdd20aSAndroid Build Coastguard Worker[Perfetto UI](https://ui.perfetto.dev). 499*6dbdd20aSAndroid Build Coastguard Worker 500*6dbdd20aSAndroid Build Coastguard WorkerNOTE: by default, heapprofd lazily initalizes to avoid blocking your program's 501*6dbdd20aSAndroid Build Coastguard Workermain thread. However, if your program makes memory allocations on startup, 502*6dbdd20aSAndroid Build Coastguard Workerthese can be missed. To avoid this from happening, set the enironment variable 503*6dbdd20aSAndroid Build Coastguard Worker`PERFETTO_HEAPPROFD_BLOCKING_INIT=1`; on the first malloc, your program will 504*6dbdd20aSAndroid Build Coastguard Workerbe blocked until heapprofd initializes fully but means every allocation will 505*6dbdd20aSAndroid Build Coastguard Workerbe correctly tracked. 506*6dbdd20aSAndroid Build Coastguard Worker 507*6dbdd20aSAndroid Build Coastguard Worker## Known Issues 508*6dbdd20aSAndroid Build Coastguard Worker 509*6dbdd20aSAndroid Build Coastguard Worker### {#known-issues-android13} Android 13 510*6dbdd20aSAndroid Build Coastguard Worker 511*6dbdd20aSAndroid Build Coastguard Worker* Unwinding java frames might not work properly, depending on the ART module 512*6dbdd20aSAndroid Build Coastguard Worker version in use. The UI reports a single "unknown" frame at the top of the 513*6dbdd20aSAndroid Build Coastguard Worker stack in this case. The problem is fixed in Android 13 QPR1. 514*6dbdd20aSAndroid Build Coastguard Worker 515*6dbdd20aSAndroid Build Coastguard Worker### {#known-issues-android12} Android 12 516*6dbdd20aSAndroid Build Coastguard Worker 517*6dbdd20aSAndroid Build Coastguard Worker* Unwinding java frames might not work properly, depending on the ART module 518*6dbdd20aSAndroid Build Coastguard Worker version in use. The UI reports a single "unknown" frame at the top of the 519*6dbdd20aSAndroid Build Coastguard Worker stack in this case. 520*6dbdd20aSAndroid Build Coastguard Worker 521*6dbdd20aSAndroid Build Coastguard Worker### {#known-issues-android11} Android 11 522*6dbdd20aSAndroid Build Coastguard Worker 523*6dbdd20aSAndroid Build Coastguard Worker* 32-bit programs cannot be targeted on 64-bit devices. 524*6dbdd20aSAndroid Build Coastguard Worker* Setting `sampling_interval_bytes` to 0 crashes the target process. 525*6dbdd20aSAndroid Build Coastguard Worker This is an invalid config that should be rejected instead. 526*6dbdd20aSAndroid Build Coastguard Worker* For startup profiles, some frame names might be missing. This will be 527*6dbdd20aSAndroid Build Coastguard Worker resolved in Android 12. 528*6dbdd20aSAndroid Build Coastguard Worker* `Failed to send control socket byte.` is displayed in logcat at the end of 529*6dbdd20aSAndroid Build Coastguard Worker every profile. This is benign. 530*6dbdd20aSAndroid Build Coastguard Worker* The object count may be incorrect in `dump_at_max` profiles. 531*6dbdd20aSAndroid Build Coastguard Worker* Choosing a low shared memory buffer size and `block_client` mode might 532*6dbdd20aSAndroid Build Coastguard Worker lock up the target process. 533*6dbdd20aSAndroid Build Coastguard Worker 534*6dbdd20aSAndroid Build Coastguard Worker### {#known-issues-android10} Android 10 535*6dbdd20aSAndroid Build Coastguard Worker* Function names in libraries with load bias might be incorrect. Use 536*6dbdd20aSAndroid Build Coastguard Worker [offline symbolization](#symbolization) to resolve this issue. 537*6dbdd20aSAndroid Build Coastguard Worker* For startup profiles, some frame names might be missing. This will be 538*6dbdd20aSAndroid Build Coastguard Worker resolved in Android 12. 539*6dbdd20aSAndroid Build Coastguard Worker* 32-bit programs cannot be targeted on 64-bit devices. 540*6dbdd20aSAndroid Build Coastguard Worker* x86 / x86_64 platforms are not supported. This includes the Android 541*6dbdd20aSAndroid Build Coastguard Worker_Cuttlefish_. 542*6dbdd20aSAndroid Build Coastguard Worker emulator. 543*6dbdd20aSAndroid Build Coastguard Worker* On ARM32, the bottom-most frame is always `ERROR 2`. This is harmless and 544*6dbdd20aSAndroid Build Coastguard Worker the callstacks are still complete. 545*6dbdd20aSAndroid Build Coastguard Worker* If heapprofd is run standalone (by running `heapprofd` in a root shell, rather 546*6dbdd20aSAndroid Build Coastguard Worker than through init), `/dev/socket/heapprofd` get assigned an incorrect SELinux 547*6dbdd20aSAndroid Build Coastguard Worker domain. You will not be able to profile any processes unless you disable 548*6dbdd20aSAndroid Build Coastguard Worker SELinux enforcement. 549*6dbdd20aSAndroid Build Coastguard Worker Run `restorecon /dev/socket/heapprofd` in a root shell to resolve. 550*6dbdd20aSAndroid Build Coastguard Worker* Using `vfork(2)` or `clone(2)` with `CLONE_VM` and allocating / freeing 551*6dbdd20aSAndroid Build Coastguard Worker memory in the child process will prematurely end the profile. 552*6dbdd20aSAndroid Build Coastguard Worker `java.lang.Runtime.exec` does this, calling it will prematurely end 553*6dbdd20aSAndroid Build Coastguard Worker the profile. Note that this is in violation of the POSIX standard. 554*6dbdd20aSAndroid Build Coastguard Worker* Setting `sampling_interval_bytes` to 0 crashes the target process. 555*6dbdd20aSAndroid Build Coastguard Worker This is an invalid config that should be rejected instead. 556*6dbdd20aSAndroid Build Coastguard Worker* `Failed to send control socket byte.` is displayed in logcat at the end of 557*6dbdd20aSAndroid Build Coastguard Worker every profile. This is benign. 558*6dbdd20aSAndroid Build Coastguard Worker* The object count may be incorrect in `dump_at_max` profiles. 559*6dbdd20aSAndroid Build Coastguard Worker* Choosing a low shared memory buffer size and `block_client` mode might 560*6dbdd20aSAndroid Build Coastguard Worker lock up the target process. 561*6dbdd20aSAndroid Build Coastguard Worker 562*6dbdd20aSAndroid Build Coastguard Worker## Heapprofd vs malloc_info() vs RSS 563*6dbdd20aSAndroid Build Coastguard Worker 564*6dbdd20aSAndroid Build Coastguard WorkerWhen using heapprofd and interpreting results, it is important to know the 565*6dbdd20aSAndroid Build Coastguard Workerprecise meaning of the different memory metrics that can be obtained from the 566*6dbdd20aSAndroid Build Coastguard Workeroperating system. 567*6dbdd20aSAndroid Build Coastguard Worker 568*6dbdd20aSAndroid Build Coastguard Worker**heapprofd** gives you the number of bytes the target program 569*6dbdd20aSAndroid Build Coastguard Workerrequested from the default C/C++ allocator. If you are profiling a Java app from 570*6dbdd20aSAndroid Build Coastguard Workerstartup, allocations that happen early in the application's initialization will 571*6dbdd20aSAndroid Build Coastguard Workernot be visible to heapprofd. Native services that do not fork from the Zygote 572*6dbdd20aSAndroid Build Coastguard Workerare not affected by this. 573*6dbdd20aSAndroid Build Coastguard Worker 574*6dbdd20aSAndroid Build Coastguard Worker**malloc\_info** is a libc function that gives you information about the 575*6dbdd20aSAndroid Build Coastguard Workerallocator. This can be triggered on userdebug builds by using 576*6dbdd20aSAndroid Build Coastguard Worker`am dumpheap -m <PID> /data/local/tmp/heap.txt`. This will in general be more 577*6dbdd20aSAndroid Build Coastguard Workerthan the memory seen by heapprofd, depending on the allocator not all memory 578*6dbdd20aSAndroid Build Coastguard Workeris immediately freed. In particular, jemalloc retains some freed memory in 579*6dbdd20aSAndroid Build Coastguard Workerthread caches. 580*6dbdd20aSAndroid Build Coastguard Worker 581*6dbdd20aSAndroid Build Coastguard Worker**Heap RSS** is the amount of memory requested from the operating system by the 582*6dbdd20aSAndroid Build Coastguard Workerallocator. This is larger than the previous two numbers because memory can only 583*6dbdd20aSAndroid Build Coastguard Workerbe obtained in page size chunks, and fragmentation causes some of that memory to 584*6dbdd20aSAndroid Build Coastguard Workerbe wasted. This can be obtained by running `adb shell dumpsys meminfo <PID>` and 585*6dbdd20aSAndroid Build Coastguard Workerlooking at the "Private Dirty" column. 586*6dbdd20aSAndroid Build Coastguard WorkerRSS can also end up being smaller than the other two if the device kernel uses 587*6dbdd20aSAndroid Build Coastguard Workermemory compression (ZRAM, enabled by default on recent versions of android) and 588*6dbdd20aSAndroid Build Coastguard Workerthe memory of the process get swapped out onto ZRAM. 589*6dbdd20aSAndroid Build Coastguard Worker 590*6dbdd20aSAndroid Build Coastguard Worker| | heapprofd | malloc\_info | RSS | 591*6dbdd20aSAndroid Build Coastguard Worker|---------------------|:-----------------:|:------------:|:---:| 592*6dbdd20aSAndroid Build Coastguard Worker| from native startup | x | x | x | 593*6dbdd20aSAndroid Build Coastguard Worker| after zygote init | x | x | x | 594*6dbdd20aSAndroid Build Coastguard Worker| before zygote init | | x | x | 595*6dbdd20aSAndroid Build Coastguard Worker| thread caches | | x | x | 596*6dbdd20aSAndroid Build Coastguard Worker| fragmentation | | | x | 597*6dbdd20aSAndroid Build Coastguard Worker 598*6dbdd20aSAndroid Build Coastguard WorkerIf you observe high RSS or malloc\_info metrics but heapprofd does not match, 599*6dbdd20aSAndroid Build Coastguard Workeryou might be hitting some pathological fragmentation problem in the allocator. 600*6dbdd20aSAndroid Build Coastguard Worker 601*6dbdd20aSAndroid Build Coastguard Worker## Convert to pprof 602*6dbdd20aSAndroid Build Coastguard Worker 603*6dbdd20aSAndroid Build Coastguard WorkerYou can use [traceconv](/docs/quickstart/traceconv.md) to convert the heap dumps 604*6dbdd20aSAndroid Build Coastguard Workerin a trace into the [pprof](https://github.com/google/pprof) format. These can 605*6dbdd20aSAndroid Build Coastguard Workerthen be viewed using the pprof CLI or a UI (e.g. Speedscope, or Google-internal 606*6dbdd20aSAndroid Build Coastguard Workerpprof/). 607*6dbdd20aSAndroid Build Coastguard Worker 608*6dbdd20aSAndroid Build Coastguard Worker```bash 609*6dbdd20aSAndroid Build Coastguard Workertools/traceconv profile /tmp/profile 610*6dbdd20aSAndroid Build Coastguard Worker``` 611*6dbdd20aSAndroid Build Coastguard Worker 612*6dbdd20aSAndroid Build Coastguard WorkerThis will create a directory in `/tmp/` containing the heap dumps. Run: 613*6dbdd20aSAndroid Build Coastguard Worker 614*6dbdd20aSAndroid Build Coastguard Worker```bash 615*6dbdd20aSAndroid Build Coastguard Workergzip /tmp/heap_profile-XXXXXX/*.pb 616*6dbdd20aSAndroid Build Coastguard Worker``` 617*6dbdd20aSAndroid Build Coastguard Worker 618*6dbdd20aSAndroid Build Coastguard Workerto get gzipped protos, which tools handling pprof profile protos expect. 619*6dbdd20aSAndroid Build Coastguard Worker 620*6dbdd20aSAndroid Build Coastguard Worker## {#heapprofd-example-queries} Example SQL Queries 621*6dbdd20aSAndroid Build Coastguard Worker 622*6dbdd20aSAndroid Build Coastguard WorkerWe can get the callstacks that allocated using an SQL Query in the 623*6dbdd20aSAndroid Build Coastguard WorkerTrace Processor. For each frame, we get one row for the number of allocated 624*6dbdd20aSAndroid Build Coastguard Workerbytes, where `count` and `size` is positive, and, if any of them were already 625*6dbdd20aSAndroid Build Coastguard Workerfreed, another line with negative `count` and `size`. The sum of those gets us 626*6dbdd20aSAndroid Build Coastguard Workerthe `Unreleased malloc size` view. 627*6dbdd20aSAndroid Build Coastguard Worker 628*6dbdd20aSAndroid Build Coastguard Worker```sql 629*6dbdd20aSAndroid Build Coastguard Workerselect a.callsite_id, a.ts, a.upid, f.name, f.rel_pc, m.build_id, m.name as mapping_name, 630*6dbdd20aSAndroid Build Coastguard Worker sum(a.size) as space_size, sum(a.count) as space_count 631*6dbdd20aSAndroid Build Coastguard Worker from heap_profile_allocation a join 632*6dbdd20aSAndroid Build Coastguard Worker stack_profile_callsite c ON (a.callsite_id = c.id) join 633*6dbdd20aSAndroid Build Coastguard Worker stack_profile_frame f ON (c.frame_id = f.id) join 634*6dbdd20aSAndroid Build Coastguard Worker stack_profile_mapping m ON (f.mapping = m.id) 635*6dbdd20aSAndroid Build Coastguard Worker group by 1, 2, 3, 4, 5, 6, 7 order by space_size desc; 636*6dbdd20aSAndroid Build Coastguard Worker``` 637*6dbdd20aSAndroid Build Coastguard Worker 638*6dbdd20aSAndroid Build Coastguard Worker| callsite_id | ts | upid | name | rel_pc | build_id | mapping_name | space_size | space_count | 639*6dbdd20aSAndroid Build Coastguard Worker|-------------|----|------|-------|-----------|------|--------|----------|------| 640*6dbdd20aSAndroid Build Coastguard Worker|6660|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |106496|4| 641*6dbdd20aSAndroid Build Coastguard Worker|192 |5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26624 |1| 642*6dbdd20aSAndroid Build Coastguard Worker|1421|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26624 |1| 643*6dbdd20aSAndroid Build Coastguard Worker|1537|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26624 |1| 644*6dbdd20aSAndroid Build Coastguard Worker|8843|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26424 |1| 645*6dbdd20aSAndroid Build Coastguard Worker|8618|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |24576 |4| 646*6dbdd20aSAndroid Build Coastguard Worker|3750|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |12288 |1| 647*6dbdd20aSAndroid Build Coastguard Worker|2820|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |8192 |2| 648*6dbdd20aSAndroid Build Coastguard Worker|3788|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |8192 |2| 649*6dbdd20aSAndroid Build Coastguard Worker 650*6dbdd20aSAndroid Build Coastguard WorkerWe can see all the functions are "malloc" and "realloc", which is not terribly 651*6dbdd20aSAndroid Build Coastguard Workerinformative. Usually we are interested in the _cumulative_ bytes allocated in 652*6dbdd20aSAndroid Build Coastguard Workera function (otherwise, we will always only see malloc / realloc). Chasing the 653*6dbdd20aSAndroid Build Coastguard Workerparent_id of a callsite (not shown in this table) recursively is very hard in 654*6dbdd20aSAndroid Build Coastguard WorkerSQL. 655*6dbdd20aSAndroid Build Coastguard Worker 656*6dbdd20aSAndroid Build Coastguard WorkerThere is an **experimental** table that surfaces this information. The **API is 657*6dbdd20aSAndroid Build Coastguard Workersubject to change**. 658*6dbdd20aSAndroid Build Coastguard Worker 659*6dbdd20aSAndroid Build Coastguard Worker```sql 660*6dbdd20aSAndroid Build Coastguard Workerselect 661*6dbdd20aSAndroid Build Coastguard Worker name, 662*6dbdd20aSAndroid Build Coastguard Worker map_name, 663*6dbdd20aSAndroid Build Coastguard Worker cumulative_size 664*6dbdd20aSAndroid Build Coastguard Workerfrom experimental_flamegraph( 665*6dbdd20aSAndroid Build Coastguard Worker -- The type of the profile from which the flamegraph is being generated. 666*6dbdd20aSAndroid Build Coastguard Worker -- Always 'native' for native heap profiles. 667*6dbdd20aSAndroid Build Coastguard Worker 'native', 668*6dbdd20aSAndroid Build Coastguard Worker -- The timestamp of the heap profile. 669*6dbdd20aSAndroid Build Coastguard Worker 8300973884377, 670*6dbdd20aSAndroid Build Coastguard Worker -- Timestamp constraints: not relevant and always null for native heap 671*6dbdd20aSAndroid Build Coastguard Worker -- profiles. 672*6dbdd20aSAndroid Build Coastguard Worker NULL, 673*6dbdd20aSAndroid Build Coastguard Worker -- The upid of the heap profile. 674*6dbdd20aSAndroid Build Coastguard Worker 1, 675*6dbdd20aSAndroid Build Coastguard Worker -- The upid group: not relevant and always null for native heap profiles. 676*6dbdd20aSAndroid Build Coastguard Worker NULL, 677*6dbdd20aSAndroid Build Coastguard Worker -- A regex for focusing on a particular node in the heapgraph: for advanced 678*6dbdd20aSAndroid Build Coastguard Worker -- use only. 679*6dbdd20aSAndroid Build Coastguard Worker NULL 680*6dbdd20aSAndroid Build Coastguard Worker) 681*6dbdd20aSAndroid Build Coastguard Workerorder by abs(cumulative_size) desc; 682*6dbdd20aSAndroid Build Coastguard Worker``` 683*6dbdd20aSAndroid Build Coastguard Worker 684*6dbdd20aSAndroid Build Coastguard Worker| name | map_name | cumulative_size | 685*6dbdd20aSAndroid Build Coastguard Worker|------|----------|----------------| 686*6dbdd20aSAndroid Build Coastguard Worker|__start_thread|/apex/com.android.runtime/lib64/bionic/libc.so|392608| 687*6dbdd20aSAndroid Build Coastguard Worker|_ZL15__pthread_startPv|/apex/com.android.runtime/lib64/bionic/libc.so|392608| 688*6dbdd20aSAndroid Build Coastguard Worker|_ZN13thread_data_t10trampolineEPKS|/system/lib64/libutils.so|199496| 689*6dbdd20aSAndroid Build Coastguard Worker|_ZN7android14AndroidRuntime15javaThreadShellEPv|/system/lib64/libandroid_runtime.so|199496| 690*6dbdd20aSAndroid Build Coastguard Worker|_ZN7android6Thread11_threadLoopEPv|/system/lib64/libutils.so|199496| 691*6dbdd20aSAndroid Build Coastguard Worker|_ZN3art6Thread14CreateCallbackEPv|/apex/com.android.art/lib64/libart.so|193112| 692*6dbdd20aSAndroid Build Coastguard Worker|_ZN3art35InvokeVirtualOrInterface...|/apex/com.android.art/lib64/libart.so|193112| 693*6dbdd20aSAndroid Build Coastguard Worker|_ZN3art9ArtMethod6InvokeEPNS_6ThreadEPjjPNS_6JValueEPKc|/apex/com.android.art/lib64/libart.so|193112| 694*6dbdd20aSAndroid Build Coastguard Worker|art_quick_invoke_stub|/apex/com.android.art/lib64/libart.so|193112| 695