xref: /aosp_15_r20/external/perfetto/docs/data-sources/native-heap-profiler.md (revision 6dbdd20afdafa5e3ca9b8809fa73465d530080dc)
1*6dbdd20aSAndroid Build Coastguard Worker# Heap profiler
2*6dbdd20aSAndroid Build Coastguard Worker
3*6dbdd20aSAndroid Build Coastguard WorkerNOTE: **heapprofd requires Android 10 or higher**
4*6dbdd20aSAndroid Build Coastguard Worker
5*6dbdd20aSAndroid Build Coastguard WorkerHeapprofd is a tool that tracks heap allocations & deallocations of an Android
6*6dbdd20aSAndroid Build Coastguard Workerprocess within a given time period. The resulting profile can be used to
7*6dbdd20aSAndroid Build Coastguard Workerattribute memory usage to particular call-stacks, supporting a mix of both
8*6dbdd20aSAndroid Build Coastguard Workernative and java code. The tool can be used by Android platform and app
9*6dbdd20aSAndroid Build Coastguard Workerdevelopers to investigate memory issues.
10*6dbdd20aSAndroid Build Coastguard Worker
11*6dbdd20aSAndroid Build Coastguard WorkerBy default, the tool records native allocations and deallocations done with
12*6dbdd20aSAndroid Build Coastguard Workermalloc/free (or new/delete). It can be configured to record java heap memory
13*6dbdd20aSAndroid Build Coastguard Workerallocations instead: see [Java heap sampling](#java-heap-sampling) below.
14*6dbdd20aSAndroid Build Coastguard Worker
15*6dbdd20aSAndroid Build Coastguard WorkerOn debug Android builds, you can profile all apps and most system services.
16*6dbdd20aSAndroid Build Coastguard WorkerOn "user" builds, you can only use it on apps with the debuggable or
17*6dbdd20aSAndroid Build Coastguard Workerprofileable manifest flag.
18*6dbdd20aSAndroid Build Coastguard Worker
19*6dbdd20aSAndroid Build Coastguard Worker## Quickstart
20*6dbdd20aSAndroid Build Coastguard Worker
21*6dbdd20aSAndroid Build Coastguard WorkerSee the [Memory Guide](/docs/case-studies/memory.md#heapprofd) for getting
22*6dbdd20aSAndroid Build Coastguard Workerstarted with heapprofd.
23*6dbdd20aSAndroid Build Coastguard Worker
24*6dbdd20aSAndroid Build Coastguard Worker## UI
25*6dbdd20aSAndroid Build Coastguard Worker
26*6dbdd20aSAndroid Build Coastguard WorkerDumps from heapprofd are shown as flamegraphs in the UI after clicking on the
27*6dbdd20aSAndroid Build Coastguard Workerdiamond. Each diamond corresponds to a snapshot of the allocations and
28*6dbdd20aSAndroid Build Coastguard Workercallstacks collected at that point in time.
29*6dbdd20aSAndroid Build Coastguard Worker
30*6dbdd20aSAndroid Build Coastguard Worker![heapprofd snapshots in the UI tracks](/docs/images/profile-diamond.png)
31*6dbdd20aSAndroid Build Coastguard Worker
32*6dbdd20aSAndroid Build Coastguard Worker![heapprofd flamegraph](/docs/images/native-heap-prof.png)
33*6dbdd20aSAndroid Build Coastguard Worker
34*6dbdd20aSAndroid Build Coastguard Worker## SQL
35*6dbdd20aSAndroid Build Coastguard Worker
36*6dbdd20aSAndroid Build Coastguard WorkerInformation about callstacks is written to the following tables:
37*6dbdd20aSAndroid Build Coastguard Worker
38*6dbdd20aSAndroid Build Coastguard Worker* [`stack_profile_mapping`](/docs/analysis/sql-tables.autogen#stack_profile_mapping)
39*6dbdd20aSAndroid Build Coastguard Worker* [`stack_profile_frame`](/docs/analysis/sql-tables.autogen#stack_profile_frame)
40*6dbdd20aSAndroid Build Coastguard Worker* [`stack_profile_callsite`](/docs/analysis/sql-tables.autogen#stack_profile_callsite)
41*6dbdd20aSAndroid Build Coastguard Worker
42*6dbdd20aSAndroid Build Coastguard WorkerThe allocations themselves are written to
43*6dbdd20aSAndroid Build Coastguard Worker[`heap_profile_allocation`](/docs/analysis/sql-tables.autogen#heap_profile_allocation).
44*6dbdd20aSAndroid Build Coastguard Worker
45*6dbdd20aSAndroid Build Coastguard WorkerOffline symbolization data is stored in
46*6dbdd20aSAndroid Build Coastguard Worker[`stack_profile_symbol`](/docs/analysis/sql-tables.autogen#stack_profile_symbol).
47*6dbdd20aSAndroid Build Coastguard Worker
48*6dbdd20aSAndroid Build Coastguard WorkerSee [Example Queries](#heapprofd-example-queries) for example SQL queries.
49*6dbdd20aSAndroid Build Coastguard Worker
50*6dbdd20aSAndroid Build Coastguard Worker## Recording
51*6dbdd20aSAndroid Build Coastguard Worker
52*6dbdd20aSAndroid Build Coastguard WorkerHeapprofd can be configured and started in three ways.
53*6dbdd20aSAndroid Build Coastguard Worker
54*6dbdd20aSAndroid Build Coastguard Worker#### Manual configuration
55*6dbdd20aSAndroid Build Coastguard Worker
56*6dbdd20aSAndroid Build Coastguard WorkerThis requires manually setting the
57*6dbdd20aSAndroid Build Coastguard Worker[HeapprofdConfig](/docs/reference/trace-config-proto.autogen#HeapprofdConfig)
58*6dbdd20aSAndroid Build Coastguard Workersection of the trace config. The only benefit of doing so is that in this way
59*6dbdd20aSAndroid Build Coastguard Workerheap profiling can be enabled alongside any other tracing data sources.
60*6dbdd20aSAndroid Build Coastguard Worker
61*6dbdd20aSAndroid Build Coastguard Worker#### Using the tools/heap_profile script (recommended)
62*6dbdd20aSAndroid Build Coastguard Worker
63*6dbdd20aSAndroid Build Coastguard WorkerYou can use the `tools/heap_profile` script. If you are having trouble
64*6dbdd20aSAndroid Build Coastguard Workermake sure you are using the
65*6dbdd20aSAndroid Build Coastguard Worker[latest version](
66*6dbdd20aSAndroid Build Coastguard Workerhttps://raw.githubusercontent.com/google/perfetto/main/tools/heap_profile).
67*6dbdd20aSAndroid Build Coastguard Worker
68*6dbdd20aSAndroid Build Coastguard WorkerYou can target processes either by name (`-n com.example.myapp`) or by PID
69*6dbdd20aSAndroid Build Coastguard Worker(`-p 1234`). In the first case, the heap profile will be initiated on both on
70*6dbdd20aSAndroid Build Coastguard Workeralready-running processes that match the package name and new processes launched
71*6dbdd20aSAndroid Build Coastguard Workerafter the profiling session is started.
72*6dbdd20aSAndroid Build Coastguard WorkerFor the full arguments list see the
73*6dbdd20aSAndroid Build Coastguard Worker[heap_profile cmdline reference page](/docs/reference/heap_profile-cli).
74*6dbdd20aSAndroid Build Coastguard Worker
75*6dbdd20aSAndroid Build Coastguard WorkerYou can use the [Perfetto UI](https://ui.perfetto.dev) to visualize heap dumps.
76*6dbdd20aSAndroid Build Coastguard WorkerUpload the `raw-trace` file in your output directory. You will see all heap
77*6dbdd20aSAndroid Build Coastguard Workerdumps as diamonds on the timeline, click any of them to get a flamegraph.
78*6dbdd20aSAndroid Build Coastguard Worker
79*6dbdd20aSAndroid Build Coastguard WorkerAlternatively [Speedscope](https://speedscope.app) can be used to visualize
80*6dbdd20aSAndroid Build Coastguard Workerthe gzipped protos, but will only show the "Unreleased malloc size" view.
81*6dbdd20aSAndroid Build Coastguard Worker
82*6dbdd20aSAndroid Build Coastguard Worker#### Using the Recording page of Perfetto UI
83*6dbdd20aSAndroid Build Coastguard Worker
84*6dbdd20aSAndroid Build Coastguard WorkerYou can also use the [Perfetto UI](https://ui.perfetto.dev/#!/record/memory)
85*6dbdd20aSAndroid Build Coastguard Workerto record heapprofd profiles. Tick "Heap profiling" in the trace configuration,
86*6dbdd20aSAndroid Build Coastguard Workerenter the processes you want to target, click "Add Device" to pair your phone,
87*6dbdd20aSAndroid Build Coastguard Workerand record profiles straight from your browser. This is also possible on
88*6dbdd20aSAndroid Build Coastguard WorkerWindows.
89*6dbdd20aSAndroid Build Coastguard Worker
90*6dbdd20aSAndroid Build Coastguard Worker## Viewing the data
91*6dbdd20aSAndroid Build Coastguard Worker
92*6dbdd20aSAndroid Build Coastguard Worker![Profile Diamond](/docs/images/profile-diamond.png)
93*6dbdd20aSAndroid Build Coastguard Worker
94*6dbdd20aSAndroid Build Coastguard WorkerThe resulting profile proto contains four views on the data, for each diamond.
95*6dbdd20aSAndroid Build Coastguard Worker
96*6dbdd20aSAndroid Build Coastguard Worker* **Unreleased malloc size**: how many bytes were allocated but not freed at
97*6dbdd20aSAndroid Build Coastguard Worker  this callstack, from the moment the recording was started until the timestamp
98*6dbdd20aSAndroid Build Coastguard Worker  of the diamond.
99*6dbdd20aSAndroid Build Coastguard Worker* **Total malloc size**: how many bytes were allocated (including ones freed at
100*6dbdd20aSAndroid Build Coastguard Worker  the moment of the dump) at this callstack, from the moment the recording was
101*6dbdd20aSAndroid Build Coastguard Worker  started until the timestamp of the diamond.
102*6dbdd20aSAndroid Build Coastguard Worker* **Unreleased malloc count**: how many allocations without matching frees were
103*6dbdd20aSAndroid Build Coastguard Worker  done at this callstack, from the moment the recording was started until the
104*6dbdd20aSAndroid Build Coastguard Worker  timestamp of the diamond.
105*6dbdd20aSAndroid Build Coastguard Worker* **Total malloc count**: how many allocations (including ones with matching
106*6dbdd20aSAndroid Build Coastguard Worker  frees) were done at this callstack, from the moment the recording was started
107*6dbdd20aSAndroid Build Coastguard Worker  started until the timestamp of the diamond.
108*6dbdd20aSAndroid Build Coastguard Worker
109*6dbdd20aSAndroid Build Coastguard Worker_(Googlers: You can also open the gzipped protos using http://pprof/)_
110*6dbdd20aSAndroid Build Coastguard Worker
111*6dbdd20aSAndroid Build Coastguard WorkerTIP: you might want to put `libart.so` as a "Hide regex" when profiling apps.
112*6dbdd20aSAndroid Build Coastguard Worker
113*6dbdd20aSAndroid Build Coastguard WorkerTIP: Click Left Heavy on the top left for a good visualization.
114*6dbdd20aSAndroid Build Coastguard Worker
115*6dbdd20aSAndroid Build Coastguard Worker## Continuous dumps
116*6dbdd20aSAndroid Build Coastguard Worker
117*6dbdd20aSAndroid Build Coastguard WorkerBy default, the heap profiler captures all the allocations from the beginning of
118*6dbdd20aSAndroid Build Coastguard Workerthe recording and stores a single snapshot, shown as a single diamond in the UI,
119*6dbdd20aSAndroid Build Coastguard Workerwhich summarizes all allocations/frees.
120*6dbdd20aSAndroid Build Coastguard Worker
121*6dbdd20aSAndroid Build Coastguard WorkerIt is possible to configure the heap profiler to periodically (not just at the
122*6dbdd20aSAndroid Build Coastguard Workerend of the trace) store snapshots (continuous dumps), for example every 5000ms
123*6dbdd20aSAndroid Build Coastguard Worker
124*6dbdd20aSAndroid Build Coastguard Worker* By setting "Continuous dumps interval" in the UI to 5000.
125*6dbdd20aSAndroid Build Coastguard Worker* By adding
126*6dbdd20aSAndroid Build Coastguard Worker  ```
127*6dbdd20aSAndroid Build Coastguard Worker  continuous_dump_config {
128*6dbdd20aSAndroid Build Coastguard Worker    dump_interval_ms: 5000
129*6dbdd20aSAndroid Build Coastguard Worker  }
130*6dbdd20aSAndroid Build Coastguard Worker  ```
131*6dbdd20aSAndroid Build Coastguard Worker  in the
132*6dbdd20aSAndroid Build Coastguard Worker  [HeapprofdConfig](/docs/reference/trace-config-proto.autogen#HeapprofdConfig).
133*6dbdd20aSAndroid Build Coastguard Worker* By adding `-c 5000` to the invocation of
134*6dbdd20aSAndroid Build Coastguard Worker  [`tools/heap_profile`](/docs/reference/heap_profile-cli).
135*6dbdd20aSAndroid Build Coastguard Worker
136*6dbdd20aSAndroid Build Coastguard Worker![Continuous dump flamegraph](/docs/images/heap_prof_continuous.png)
137*6dbdd20aSAndroid Build Coastguard Worker
138*6dbdd20aSAndroid Build Coastguard WorkerThe resulting visualization shows multiple diamonds. Clicking on each diamond
139*6dbdd20aSAndroid Build Coastguard Workershows a summary of the allocations/frees from the beginning of the trace until
140*6dbdd20aSAndroid Build Coastguard Workerthat point (i.e. the summary is cumulative).
141*6dbdd20aSAndroid Build Coastguard Worker
142*6dbdd20aSAndroid Build Coastguard Worker## Sampling interval
143*6dbdd20aSAndroid Build Coastguard Worker
144*6dbdd20aSAndroid Build Coastguard WorkerHeapprofd samples heap allocations by hooking calls to malloc/free and C++'s
145*6dbdd20aSAndroid Build Coastguard Workeroperator new/delete. Given a sampling interval of n bytes, one allocation is
146*6dbdd20aSAndroid Build Coastguard Workersampled, on average, every n bytes allocated. This allows to reduce the
147*6dbdd20aSAndroid Build Coastguard Workerperformance impact on the target process. The default sampling rate
148*6dbdd20aSAndroid Build Coastguard Workeris 4096 bytes.
149*6dbdd20aSAndroid Build Coastguard Worker
150*6dbdd20aSAndroid Build Coastguard WorkerThe easiest way to reason about this is to imagine the memory allocations as a
151*6dbdd20aSAndroid Build Coastguard Workerstream of one byte allocations. From this stream, every byte has a 1/n
152*6dbdd20aSAndroid Build Coastguard Workerprobability of being selected as a sample, and the corresponding callstack
153*6dbdd20aSAndroid Build Coastguard Workergets attributed the complete n bytes. For more accuracy, allocations larger than
154*6dbdd20aSAndroid Build Coastguard Workerthe sampling interval bypass the sampling logic and are recorded with their true
155*6dbdd20aSAndroid Build Coastguard Workersize.
156*6dbdd20aSAndroid Build Coastguard WorkerSee the [heapprofd Sampling](/docs/design-docs/heapprofd-sampling) document for
157*6dbdd20aSAndroid Build Coastguard Workerdetails.
158*6dbdd20aSAndroid Build Coastguard Worker
159*6dbdd20aSAndroid Build Coastguard Worker## Startup profiling
160*6dbdd20aSAndroid Build Coastguard Worker
161*6dbdd20aSAndroid Build Coastguard WorkerWhen specifying a target process name (as opposite to the PID), new processes
162*6dbdd20aSAndroid Build Coastguard Workermatching that name are profiled from their startup. The resulting profile will
163*6dbdd20aSAndroid Build Coastguard Workercontain all allocations done between the start of the process and the end
164*6dbdd20aSAndroid Build Coastguard Workerof the profiling session.
165*6dbdd20aSAndroid Build Coastguard Worker
166*6dbdd20aSAndroid Build Coastguard WorkerOn Android, Java apps are usually not exec()-ed from scratch, but fork()-ed from
167*6dbdd20aSAndroid Build Coastguard Workerthe [zygote], which then specializes into the desired app. If the app's name
168*6dbdd20aSAndroid Build Coastguard Workermatches a name specified in the profiling session, profiling will be enabled as
169*6dbdd20aSAndroid Build Coastguard Workerpart of the zygote specialization. The resulting profile contains all
170*6dbdd20aSAndroid Build Coastguard Workerallocations done between that point in zygote specialization and the end of the
171*6dbdd20aSAndroid Build Coastguard Workerprofiling session. Some allocations done early in the specialization process are
172*6dbdd20aSAndroid Build Coastguard Workernot accounted for.
173*6dbdd20aSAndroid Build Coastguard Worker
174*6dbdd20aSAndroid Build Coastguard WorkerAt the trace proto level, the resulting [ProfilePacket] will have the
175*6dbdd20aSAndroid Build Coastguard Worker`from_startup` field set to true in the corresponding `ProcessHeapSamples`
176*6dbdd20aSAndroid Build Coastguard Workermessage. This is not surfaced in the converted pprof compatible proto.
177*6dbdd20aSAndroid Build Coastguard Worker
178*6dbdd20aSAndroid Build Coastguard Worker[ProfilePacket]: /docs/reference/trace-packet-proto.autogen#ProfilePacket
179*6dbdd20aSAndroid Build Coastguard Worker[zygote]: https://developer.android.com/topic/performance/memory-overview#SharingRAM
180*6dbdd20aSAndroid Build Coastguard Worker
181*6dbdd20aSAndroid Build Coastguard Worker## Runtime profiling
182*6dbdd20aSAndroid Build Coastguard Worker
183*6dbdd20aSAndroid Build Coastguard WorkerWhen a profiling session is started, all matching processes (by name or PID)
184*6dbdd20aSAndroid Build Coastguard Workerare enumerated and are signalled to request profiling. Profiling isn't actually
185*6dbdd20aSAndroid Build Coastguard Workerenabled until a few hundred milliseconds after the next allocation that is
186*6dbdd20aSAndroid Build Coastguard Workerdone by the application. If the application is idle when profiling is
187*6dbdd20aSAndroid Build Coastguard Workerrequested, and then does a burst of allocations, these may be missed.
188*6dbdd20aSAndroid Build Coastguard Worker
189*6dbdd20aSAndroid Build Coastguard WorkerThe resulting profile will contain all allocations done between when profiling
190*6dbdd20aSAndroid Build Coastguard Workeris enabled, and the end of the profiling session.
191*6dbdd20aSAndroid Build Coastguard Worker
192*6dbdd20aSAndroid Build Coastguard WorkerThe resulting [ProfilePacket] will have `from_startup` set to false in the
193*6dbdd20aSAndroid Build Coastguard Workercorresponding `ProcessHeapSamples` message. This does not get surfaced in the
194*6dbdd20aSAndroid Build Coastguard Workerconverted pprof compatible proto.
195*6dbdd20aSAndroid Build Coastguard Worker
196*6dbdd20aSAndroid Build Coastguard Worker## Concurrent profiling sessions
197*6dbdd20aSAndroid Build Coastguard Worker
198*6dbdd20aSAndroid Build Coastguard WorkerIf multiple sessions name the same target process (either by name or PID),
199*6dbdd20aSAndroid Build Coastguard Workeronly the first relevant session will profile the process. The other sessions
200*6dbdd20aSAndroid Build Coastguard Workerwill report that the process had already been profiled when converting to
201*6dbdd20aSAndroid Build Coastguard Workerthe pprof compatible proto.
202*6dbdd20aSAndroid Build Coastguard Worker
203*6dbdd20aSAndroid Build Coastguard WorkerIf you see this message but do not expect any other sessions, run
204*6dbdd20aSAndroid Build Coastguard Worker
205*6dbdd20aSAndroid Build Coastguard Worker```shell
206*6dbdd20aSAndroid Build Coastguard Workeradb shell killall perfetto
207*6dbdd20aSAndroid Build Coastguard Worker```
208*6dbdd20aSAndroid Build Coastguard Worker
209*6dbdd20aSAndroid Build Coastguard Workerto stop any concurrent sessions that may be running.
210*6dbdd20aSAndroid Build Coastguard Worker
211*6dbdd20aSAndroid Build Coastguard WorkerThe resulting [ProfilePacket] will have `rejected_concurrent` set  to true in
212*6dbdd20aSAndroid Build Coastguard Workerotherwise empty corresponding `ProcessHeapSamples` message. This does not get
213*6dbdd20aSAndroid Build Coastguard Workersurfaced in the converted pprof compatible proto.
214*6dbdd20aSAndroid Build Coastguard Worker
215*6dbdd20aSAndroid Build Coastguard Worker## {#heapprofd-targets} Target processes
216*6dbdd20aSAndroid Build Coastguard Worker
217*6dbdd20aSAndroid Build Coastguard WorkerDepending on the build of Android that heapprofd is run on, some processes
218*6dbdd20aSAndroid Build Coastguard Workerare not be eligible to be profiled.
219*6dbdd20aSAndroid Build Coastguard Worker
220*6dbdd20aSAndroid Build Coastguard WorkerOn _user_ (i.e. production, non-rootable) builds, only Java applications with
221*6dbdd20aSAndroid Build Coastguard Workereither the profileable or the debuggable manifest flag set can be profiled.
222*6dbdd20aSAndroid Build Coastguard WorkerProfiling requests for non-profileable/debuggable processes will result in an
223*6dbdd20aSAndroid Build Coastguard Workerempty profile.
224*6dbdd20aSAndroid Build Coastguard Worker
225*6dbdd20aSAndroid Build Coastguard WorkerOn userdebug builds, all processes except for a small set of critical
226*6dbdd20aSAndroid Build Coastguard Workerservices can be profiled (to find the set of disallowed targets, look for
227*6dbdd20aSAndroid Build Coastguard Worker`never_profile_heap` in [heapprofd.te](
228*6dbdd20aSAndroid Build Coastguard Workerhttps://cs.android.com/android/platform/superproject/main/+/main:system/sepolicy/private/heapprofd.te?q=never_profile_heap).
229*6dbdd20aSAndroid Build Coastguard WorkerThis restriction can be lifted by disabling SELinux by running
230*6dbdd20aSAndroid Build Coastguard Worker`adb shell su root setenforce 0` or by passing `--disable-selinux` to the
231*6dbdd20aSAndroid Build Coastguard Worker`heap_profile` script.
232*6dbdd20aSAndroid Build Coastguard Worker
233*6dbdd20aSAndroid Build Coastguard Worker<center>
234*6dbdd20aSAndroid Build Coastguard Worker
235*6dbdd20aSAndroid Build Coastguard Worker|                         | userdebug setenforce 0 | userdebug | user |
236*6dbdd20aSAndroid Build Coastguard Worker|-------------------------|:----------------------:|:---------:|:----:|
237*6dbdd20aSAndroid Build Coastguard Worker| critical native service |            Y           |     N     |  N   |
238*6dbdd20aSAndroid Build Coastguard Worker| native service          |            Y           |     Y     |  N   |
239*6dbdd20aSAndroid Build Coastguard Worker| app                     |            Y           |     Y     |  N   |
240*6dbdd20aSAndroid Build Coastguard Worker| profileable app         |            Y           |     Y     |  Y   |
241*6dbdd20aSAndroid Build Coastguard Worker| debuggable app          |            Y           |     Y     |  Y   |
242*6dbdd20aSAndroid Build Coastguard Worker
243*6dbdd20aSAndroid Build Coastguard Worker</center>
244*6dbdd20aSAndroid Build Coastguard Worker
245*6dbdd20aSAndroid Build Coastguard WorkerTo mark an app as profileable, put `<profileable android:shell="true"/>` into
246*6dbdd20aSAndroid Build Coastguard Workerthe `<application>` section of the app manifest.
247*6dbdd20aSAndroid Build Coastguard Worker
248*6dbdd20aSAndroid Build Coastguard Worker```xml
249*6dbdd20aSAndroid Build Coastguard Worker<manifest ...>
250*6dbdd20aSAndroid Build Coastguard Worker    <application>
251*6dbdd20aSAndroid Build Coastguard Worker        <profileable android:shell="true"/>
252*6dbdd20aSAndroid Build Coastguard Worker        ...
253*6dbdd20aSAndroid Build Coastguard Worker    </application>
254*6dbdd20aSAndroid Build Coastguard Worker</manifest>
255*6dbdd20aSAndroid Build Coastguard Worker```
256*6dbdd20aSAndroid Build Coastguard Worker
257*6dbdd20aSAndroid Build Coastguard Worker## {#java-heap-sampling} Java heap sampling
258*6dbdd20aSAndroid Build Coastguard Worker
259*6dbdd20aSAndroid Build Coastguard WorkerNOTE: **Java heap sampling is available on Android 12 or higher**
260*6dbdd20aSAndroid Build Coastguard Worker
261*6dbdd20aSAndroid Build Coastguard WorkerNOTE: **Java heap sampling is not to be confused with [Java heap
262*6dbdd20aSAndroid Build Coastguard Workerdumps](/docs/data-sources/java-heap-profiler.md)**
263*6dbdd20aSAndroid Build Coastguard Worker
264*6dbdd20aSAndroid Build Coastguard WorkerHeapprofd can be configured to track Java allocations instead of native ones.
265*6dbdd20aSAndroid Build Coastguard Worker* By setting adding `heaps: "com.android.art"` in
266*6dbdd20aSAndroid Build Coastguard Worker  [HeapprofdConfig](/docs/reference/trace-config-proto.autogen#HeapprofdConfig).
267*6dbdd20aSAndroid Build Coastguard Worker* By adding `--heaps com.android.art` to the invocation of
268*6dbdd20aSAndroid Build Coastguard Worker  [`tools/heap_profile`](/docs/reference/heap_profile-cli).
269*6dbdd20aSAndroid Build Coastguard Worker
270*6dbdd20aSAndroid Build Coastguard WorkerUnlike java heap dumps (which show the retention graph of a snapshot of the live
271*6dbdd20aSAndroid Build Coastguard Workerobjects) but like native heap profiles, java heap samples show callstacks of
272*6dbdd20aSAndroid Build Coastguard Workerallocations over time of the entire profile.
273*6dbdd20aSAndroid Build Coastguard Worker
274*6dbdd20aSAndroid Build Coastguard WorkerJava heap samples only show callstacks of when objects are created, not when
275*6dbdd20aSAndroid Build Coastguard Workerthey're deleted or garbage collected.
276*6dbdd20aSAndroid Build Coastguard Worker
277*6dbdd20aSAndroid Build Coastguard Worker![javaheapsamples](/docs/images/java-heap-samples.png)
278*6dbdd20aSAndroid Build Coastguard Worker
279*6dbdd20aSAndroid Build Coastguard WorkerThe resulting profile proto contains two views on the data:
280*6dbdd20aSAndroid Build Coastguard Worker
281*6dbdd20aSAndroid Build Coastguard Worker* **Total allocation size**: how many bytes were allocated at this callstack
282*6dbdd20aSAndroid Build Coastguard Worker  over time of the profile until this point. The bytes might have been freed or
283*6dbdd20aSAndroid Build Coastguard Worker  not, the tool does not keep track of that.
284*6dbdd20aSAndroid Build Coastguard Worker* **Total allocation count**: how many object were allocated at this callstack
285*6dbdd20aSAndroid Build Coastguard Worker  over time of the profile until this point. The objects might have been freed
286*6dbdd20aSAndroid Build Coastguard Worker  or not, the tool does not keep track of that.
287*6dbdd20aSAndroid Build Coastguard Worker
288*6dbdd20aSAndroid Build Coastguard WorkerJava heap samples are useful to understand memory churn showing the call stack
289*6dbdd20aSAndroid Build Coastguard Workerof which parts of the code large allocations are attributed to as well as the
290*6dbdd20aSAndroid Build Coastguard Workerallocation type from the ART runtime.
291*6dbdd20aSAndroid Build Coastguard Worker
292*6dbdd20aSAndroid Build Coastguard Worker## DEDUPED frames
293*6dbdd20aSAndroid Build Coastguard Worker
294*6dbdd20aSAndroid Build Coastguard WorkerIf the name of a Java method includes `[DEDUPED]`, this means that multiple
295*6dbdd20aSAndroid Build Coastguard Workermethods share the same code. ART only stores the name of a single one in its
296*6dbdd20aSAndroid Build Coastguard Workermetadata, which is displayed here. This is not necessarily the one that was
297*6dbdd20aSAndroid Build Coastguard Workercalled.
298*6dbdd20aSAndroid Build Coastguard Worker
299*6dbdd20aSAndroid Build Coastguard Worker## Triggering heap snapshots on demand
300*6dbdd20aSAndroid Build Coastguard Worker
301*6dbdd20aSAndroid Build Coastguard WorkerHeap snapshot are recorded into the trace either at regular time intervals, if
302*6dbdd20aSAndroid Build Coastguard Workerusing the `continuous_dump_config` field, or at the end of the session.
303*6dbdd20aSAndroid Build Coastguard Worker
304*6dbdd20aSAndroid Build Coastguard WorkerYou can also trigger a snapshot of all currently profiled processes by running
305*6dbdd20aSAndroid Build Coastguard Worker`adb shell killall -USR1 heapprofd`. This can be useful in lab tests for
306*6dbdd20aSAndroid Build Coastguard Workerrecording the current memory usage of the target in a specific state.
307*6dbdd20aSAndroid Build Coastguard Worker
308*6dbdd20aSAndroid Build Coastguard WorkerThis dump will show up in addition to the dump at the end of the profile that is
309*6dbdd20aSAndroid Build Coastguard Workeralways produced. You can create multiple of these dumps, and they will be
310*6dbdd20aSAndroid Build Coastguard Workerenumerated in the output directory.
311*6dbdd20aSAndroid Build Coastguard Worker
312*6dbdd20aSAndroid Build Coastguard Worker## Symbolization
313*6dbdd20aSAndroid Build Coastguard Worker
314*6dbdd20aSAndroid Build Coastguard Worker### Set up llvm-symbolizer
315*6dbdd20aSAndroid Build Coastguard Worker
316*6dbdd20aSAndroid Build Coastguard WorkerYou only need to do this once.
317*6dbdd20aSAndroid Build Coastguard Worker
318*6dbdd20aSAndroid Build Coastguard WorkerTo use symbolization, your system must have llvm-symbolizer installed and
319*6dbdd20aSAndroid Build Coastguard Workeraccessible from `$PATH` as `llvm-symbolizer`. On Debian, you can install it
320*6dbdd20aSAndroid Build Coastguard Workerusing `sudo apt install llvm`.
321*6dbdd20aSAndroid Build Coastguard Worker
322*6dbdd20aSAndroid Build Coastguard Worker### Symbolize your profile
323*6dbdd20aSAndroid Build Coastguard Worker
324*6dbdd20aSAndroid Build Coastguard WorkerIf the profiled binary or libraries do not have symbol names, you can
325*6dbdd20aSAndroid Build Coastguard Workersymbolize profiles offline. Even if they do, you might want to symbolize in
326*6dbdd20aSAndroid Build Coastguard Workerorder to get inlined function and line number information. All tools
327*6dbdd20aSAndroid Build Coastguard Worker(traceconv, trace_processor_shell, the heap_profile script) support specifying
328*6dbdd20aSAndroid Build Coastguard Workerthe `PERFETTO_BINARY_PATH` as an environment variable.
329*6dbdd20aSAndroid Build Coastguard Worker
330*6dbdd20aSAndroid Build Coastguard Worker```
331*6dbdd20aSAndroid Build Coastguard WorkerPERFETTO_BINARY_PATH=somedir tools/heap_profile --name ${NAME}
332*6dbdd20aSAndroid Build Coastguard Worker```
333*6dbdd20aSAndroid Build Coastguard Worker
334*6dbdd20aSAndroid Build Coastguard WorkerYou can persist symbols for a trace by running
335*6dbdd20aSAndroid Build Coastguard Worker`PERFETTO_BINARY_PATH=somedir tools/traceconv symbolize raw-trace > symbols`.
336*6dbdd20aSAndroid Build Coastguard WorkerYou can then concatenate the symbols to the trace (
337*6dbdd20aSAndroid Build Coastguard Worker`cat raw-trace symbols > symbolized-trace`) and the symbols will part of
338*6dbdd20aSAndroid Build Coastguard Worker`symbolized-trace`. The `tools/heap_profile` script will also generate this
339*6dbdd20aSAndroid Build Coastguard Workerfile in your output directory, if `PERFETTO_BINARY_PATH` is used.
340*6dbdd20aSAndroid Build Coastguard Worker
341*6dbdd20aSAndroid Build Coastguard WorkerThe symbol file is the first with matching Build ID in the following order:
342*6dbdd20aSAndroid Build Coastguard Worker
343*6dbdd20aSAndroid Build Coastguard Worker1. absolute path of library file relative to binary path.
344*6dbdd20aSAndroid Build Coastguard Worker2. absolute path of library file relative to binary path, but with base.apk!
345*6dbdd20aSAndroid Build Coastguard Worker    removed from filename.
346*6dbdd20aSAndroid Build Coastguard Worker3. basename of library file relative to binary path.
347*6dbdd20aSAndroid Build Coastguard Worker4. basename of library file relative to binary path, but with base.apk!
348*6dbdd20aSAndroid Build Coastguard Worker    removed from filename.
349*6dbdd20aSAndroid Build Coastguard Worker5. in the subdirectory .build-id: the first two hex digits of the build-id
350*6dbdd20aSAndroid Build Coastguard Worker    as subdirectory, then the rest of the hex digits, with ".debug" appended.
351*6dbdd20aSAndroid Build Coastguard Worker    See
352*6dbdd20aSAndroid Build Coastguard Worker    https://fedoraproject.org/wiki/RolandMcGrath/BuildID#Find_files_by_build_ID
353*6dbdd20aSAndroid Build Coastguard Worker
354*6dbdd20aSAndroid Build Coastguard WorkerFor example, "/system/lib/base.apk!foo.so" with build id abcd1234,
355*6dbdd20aSAndroid Build Coastguard Workeris looked for at:
356*6dbdd20aSAndroid Build Coastguard Worker
357*6dbdd20aSAndroid Build Coastguard Worker1. $PERFETTO_BINARY_PATH/system/lib/base.apk!foo.so
358*6dbdd20aSAndroid Build Coastguard Worker2. $PERFETTO_BINARY_PATH/system/lib/foo.so
359*6dbdd20aSAndroid Build Coastguard Worker3. $PERFETTO_BINARY_PATH/base.apk!foo.so
360*6dbdd20aSAndroid Build Coastguard Worker4. $PERFETTO_BINARY_PATH/foo.so
361*6dbdd20aSAndroid Build Coastguard Worker5. $PERFETTO_BINARY_PATH/.build-id/ab/cd1234.debug
362*6dbdd20aSAndroid Build Coastguard Worker
363*6dbdd20aSAndroid Build Coastguard WorkerAlternatively, you can set the `PERFETTO_SYMBOLIZER_MODE` environment variable
364*6dbdd20aSAndroid Build Coastguard Workerto `index`, and the symbolizer will recursively search the given directory for
365*6dbdd20aSAndroid Build Coastguard Workeran ELF file with the given build id. This way, you will not have to worry
366*6dbdd20aSAndroid Build Coastguard Workerabout correct filenames.
367*6dbdd20aSAndroid Build Coastguard Worker
368*6dbdd20aSAndroid Build Coastguard Worker## Deobfuscation
369*6dbdd20aSAndroid Build Coastguard Worker
370*6dbdd20aSAndroid Build Coastguard WorkerIf your profile contains obfuscated Java methods (like `fsd.a`), you can
371*6dbdd20aSAndroid Build Coastguard Workerprovide a deobfuscation map to turn them back into human readable.
372*6dbdd20aSAndroid Build Coastguard WorkerTo do so, use the `PERFETTO_PROGUARD_MAP` environment variable, using the
373*6dbdd20aSAndroid Build Coastguard Workerformat `packagename=map_filename[:packagename=map_filename...]`, e.g.
374*6dbdd20aSAndroid Build Coastguard Worker`PERFETTO_PROGUARD_MAP=com.example.pkg1=foo.txt:com.example.pkg2=bar.txt`.
375*6dbdd20aSAndroid Build Coastguard WorkerAll tools (traceconv, trace_processor_shell, the heap_profile script) support
376*6dbdd20aSAndroid Build Coastguard Workerspecifying the `PERFETTO_PROGUARD_MAP` as an environment variable.
377*6dbdd20aSAndroid Build Coastguard Worker
378*6dbdd20aSAndroid Build Coastguard Worker```
379*6dbdd20aSAndroid Build Coastguard WorkerPERFETTO_PROGUARD_MAP=com.example.pkg1=proguard_map1.txt:com.example.pkg2=proguard_map2.txt ./tools/heap_profile -n com.example.app
380*6dbdd20aSAndroid Build Coastguard Worker```
381*6dbdd20aSAndroid Build Coastguard Worker
382*6dbdd20aSAndroid Build Coastguard WorkerYou can get a deobfuscation map for the trace you already collected using
383*6dbdd20aSAndroid Build Coastguard Worker`tools/traceconv deobfuscate`. Then concatenate the resulting file to your
384*6dbdd20aSAndroid Build Coastguard Workertrace to get a deobfuscated version of it (the input trace should be in the
385*6dbdd20aSAndroid Build Coastguard Workerperfetto format, otherwise concatenation will not produce a reasonable output).
386*6dbdd20aSAndroid Build Coastguard Worker
387*6dbdd20aSAndroid Build Coastguard Worker```
388*6dbdd20aSAndroid Build Coastguard WorkerPERFETTO_PROGUARD_MAP=com.example.pkg=proguard_map.txt tools/traceconv deobfuscate ${TRACE} > deobfuscation_map
389*6dbdd20aSAndroid Build Coastguard Workercat ${TRACE} deobfuscation_map > deobfuscated_trace
390*6dbdd20aSAndroid Build Coastguard Worker```
391*6dbdd20aSAndroid Build Coastguard Worker
392*6dbdd20aSAndroid Build Coastguard Worker`deobfuscated_trace` can be viewed in the
393*6dbdd20aSAndroid Build Coastguard Worker[Perfetto UI](https://ui.perfetto.dev).
394*6dbdd20aSAndroid Build Coastguard Worker
395*6dbdd20aSAndroid Build Coastguard Worker## Troubleshooting
396*6dbdd20aSAndroid Build Coastguard Worker
397*6dbdd20aSAndroid Build Coastguard Worker### Buffer overrun
398*6dbdd20aSAndroid Build Coastguard Worker
399*6dbdd20aSAndroid Build Coastguard WorkerIf the rate of allocations is too high for heapprofd to keep up, the profiling
400*6dbdd20aSAndroid Build Coastguard Workersession will end early due to a buffer overrun. If the buffer overrun is
401*6dbdd20aSAndroid Build Coastguard Workercaused by a transient spike in allocations, increasing the shared memory buffer
402*6dbdd20aSAndroid Build Coastguard Workersize (passing `--shmem-size` to `tools/heap_profile`) can resolve the issue.
403*6dbdd20aSAndroid Build Coastguard WorkerOtherwise the sampling interval can be increased (at the expense of lower
404*6dbdd20aSAndroid Build Coastguard Workeraccuracy in the resulting profile) by passing `--interval=16000` or higher.
405*6dbdd20aSAndroid Build Coastguard Worker
406*6dbdd20aSAndroid Build Coastguard Worker### Profile is empty
407*6dbdd20aSAndroid Build Coastguard Worker
408*6dbdd20aSAndroid Build Coastguard WorkerCheck whether your target process is eligible to be profiled by consulting
409*6dbdd20aSAndroid Build Coastguard Worker[Target processes](#heapprofd-targets) above.
410*6dbdd20aSAndroid Build Coastguard Worker
411*6dbdd20aSAndroid Build Coastguard WorkerAlso check the [Known Issues](#known-issues).
412*6dbdd20aSAndroid Build Coastguard Worker
413*6dbdd20aSAndroid Build Coastguard Worker### Implausible callstacks
414*6dbdd20aSAndroid Build Coastguard Worker
415*6dbdd20aSAndroid Build Coastguard WorkerIf you see a callstack that seems to impossible from looking at the code, make
416*6dbdd20aSAndroid Build Coastguard Workersure no [DEDUPED frames](#deduped-frames) are involved.
417*6dbdd20aSAndroid Build Coastguard Worker
418*6dbdd20aSAndroid Build Coastguard WorkerAlso, if your code is linked using _Identical Code Folding_
419*6dbdd20aSAndroid Build Coastguard Worker(ICF), i.e. passing `-Wl,--icf=...` to the linker, most trivial functions, often
420*6dbdd20aSAndroid Build Coastguard Workerconstructors and destructors, can be aliased to binary-equivalent operators
421*6dbdd20aSAndroid Build Coastguard Workerof completely unrelated classes.
422*6dbdd20aSAndroid Build Coastguard Worker
423*6dbdd20aSAndroid Build Coastguard Worker### Symbolization: Could not find library
424*6dbdd20aSAndroid Build Coastguard Worker
425*6dbdd20aSAndroid Build Coastguard WorkerWhen symbolizing a profile, you might come across messages like this:
426*6dbdd20aSAndroid Build Coastguard Worker
427*6dbdd20aSAndroid Build Coastguard Worker```bash
428*6dbdd20aSAndroid Build Coastguard WorkerCould not find /data/app/invalid.app-wFgo3GRaod02wSvPZQ==/lib/arm64/somelib.so
429*6dbdd20aSAndroid Build Coastguard Worker(Build ID: 44b7138abd5957b8d0a56ce86216d478).
430*6dbdd20aSAndroid Build Coastguard Worker```
431*6dbdd20aSAndroid Build Coastguard Worker
432*6dbdd20aSAndroid Build Coastguard WorkerCheck whether your library (in this example somelib.so) exists in
433*6dbdd20aSAndroid Build Coastguard Worker`PERFETTO_BINARY_PATH`. Then compare the Build ID to the one in your
434*6dbdd20aSAndroid Build Coastguard Workersymbol file, which you can get by running
435*6dbdd20aSAndroid Build Coastguard Worker`readelf -n /path/in/binary/path/somelib.so`. If it does not match, the
436*6dbdd20aSAndroid Build Coastguard Workersymbolized file has a different version than the one on device, and cannot
437*6dbdd20aSAndroid Build Coastguard Workerbe used for symbolization.
438*6dbdd20aSAndroid Build Coastguard WorkerIf it does, try moving somelib.so to the root of `PERFETTO_BINARY_PATH` and
439*6dbdd20aSAndroid Build Coastguard Workertry again.
440*6dbdd20aSAndroid Build Coastguard Worker
441*6dbdd20aSAndroid Build Coastguard Worker### Only one frame shown
442*6dbdd20aSAndroid Build Coastguard WorkerIf you only see a single frame for functions in a specific library, make sure
443*6dbdd20aSAndroid Build Coastguard Workerthat the library has unwind information. We need one of
444*6dbdd20aSAndroid Build Coastguard Worker
445*6dbdd20aSAndroid Build Coastguard Worker* `.gnu_debugdata`
446*6dbdd20aSAndroid Build Coastguard Worker* `.eh_frame` (+ preferably `.eh_frame_hdr`)
447*6dbdd20aSAndroid Build Coastguard Worker* `.debug_frame`.
448*6dbdd20aSAndroid Build Coastguard Worker
449*6dbdd20aSAndroid Build Coastguard WorkerFrame-pointer unwinding is *not supported*.
450*6dbdd20aSAndroid Build Coastguard Worker
451*6dbdd20aSAndroid Build Coastguard WorkerTo check if an ELF file has any of those, run
452*6dbdd20aSAndroid Build Coastguard Worker
453*6dbdd20aSAndroid Build Coastguard Worker```console
454*6dbdd20aSAndroid Build Coastguard Worker$ readelf -S file.so | grep "gnu_debugdata\|eh_frame\|debug_frame"
455*6dbdd20aSAndroid Build Coastguard Worker  [12] .eh_frame_hdr     PROGBITS         000000000000c2b0  0000c2b0
456*6dbdd20aSAndroid Build Coastguard Worker  [13] .eh_frame         PROGBITS         0000000000011000  00011000
457*6dbdd20aSAndroid Build Coastguard Worker  [24] .gnu_debugdata    PROGBITS         0000000000000000  000f7292
458*6dbdd20aSAndroid Build Coastguard Worker```
459*6dbdd20aSAndroid Build Coastguard Worker
460*6dbdd20aSAndroid Build Coastguard WorkerIf this does not show one or more of the sections, change your build system
461*6dbdd20aSAndroid Build Coastguard Workerto not strip them.
462*6dbdd20aSAndroid Build Coastguard Worker
463*6dbdd20aSAndroid Build Coastguard Worker## (non-Android) Linux support
464*6dbdd20aSAndroid Build Coastguard Worker
465*6dbdd20aSAndroid Build Coastguard WorkerNOTE: Do not use this for production purposes.
466*6dbdd20aSAndroid Build Coastguard Worker
467*6dbdd20aSAndroid Build Coastguard WorkerYou can use a standalone library to profile memory allocations on Linux.
468*6dbdd20aSAndroid Build Coastguard WorkerFirst [build Perfetto](/docs/contributing/build-instructions.md). You only need
469*6dbdd20aSAndroid Build Coastguard Workerto do this once.
470*6dbdd20aSAndroid Build Coastguard Worker
471*6dbdd20aSAndroid Build Coastguard Worker```
472*6dbdd20aSAndroid Build Coastguard Workertools/setup_all_configs.py
473*6dbdd20aSAndroid Build Coastguard Workerninja -C out/linux_clang_release
474*6dbdd20aSAndroid Build Coastguard Worker```
475*6dbdd20aSAndroid Build Coastguard Worker
476*6dbdd20aSAndroid Build Coastguard WorkerThen, run traced
477*6dbdd20aSAndroid Build Coastguard Worker
478*6dbdd20aSAndroid Build Coastguard Worker```
479*6dbdd20aSAndroid Build Coastguard Workerout/linux_clang_release/traced
480*6dbdd20aSAndroid Build Coastguard Worker```
481*6dbdd20aSAndroid Build Coastguard Worker
482*6dbdd20aSAndroid Build Coastguard WorkerStart the profile (e.g. targeting trace_processor_shell)
483*6dbdd20aSAndroid Build Coastguard Worker
484*6dbdd20aSAndroid Build Coastguard Worker```
485*6dbdd20aSAndroid Build Coastguard Workertools/heap_profile -n trace_processor_shell --print-config  | \
486*6dbdd20aSAndroid Build Coastguard Workerout/linux_clang_release/perfetto \
487*6dbdd20aSAndroid Build Coastguard Worker  -c - --txt \
488*6dbdd20aSAndroid Build Coastguard Worker  -o ~/heapprofd-trace
489*6dbdd20aSAndroid Build Coastguard Worker```
490*6dbdd20aSAndroid Build Coastguard Worker
491*6dbdd20aSAndroid Build Coastguard WorkerFinally, run your target (e.g. trace_processor_shell) with LD_PRELOAD
492*6dbdd20aSAndroid Build Coastguard Worker
493*6dbdd20aSAndroid Build Coastguard Worker```
494*6dbdd20aSAndroid Build Coastguard WorkerLD_PRELOAD=out/linux_clang_release/libheapprofd_glibc_preload.so out/linux_clang_release/trace_processor_shell <trace>
495*6dbdd20aSAndroid Build Coastguard Worker```
496*6dbdd20aSAndroid Build Coastguard Worker
497*6dbdd20aSAndroid Build Coastguard WorkerThen, Ctrl-C the Perfetto invocation and upload ~/heapprofd-trace to the
498*6dbdd20aSAndroid Build Coastguard Worker[Perfetto UI](https://ui.perfetto.dev).
499*6dbdd20aSAndroid Build Coastguard Worker
500*6dbdd20aSAndroid Build Coastguard WorkerNOTE: by default, heapprofd lazily initalizes to avoid blocking your program's
501*6dbdd20aSAndroid Build Coastguard Workermain thread. However, if your program makes memory allocations on startup,
502*6dbdd20aSAndroid Build Coastguard Workerthese can be missed. To avoid this from happening, set the enironment variable
503*6dbdd20aSAndroid Build Coastguard Worker`PERFETTO_HEAPPROFD_BLOCKING_INIT=1`; on the first malloc, your program will
504*6dbdd20aSAndroid Build Coastguard Workerbe blocked until heapprofd initializes fully but means every allocation will
505*6dbdd20aSAndroid Build Coastguard Workerbe correctly tracked.
506*6dbdd20aSAndroid Build Coastguard Worker
507*6dbdd20aSAndroid Build Coastguard Worker## Known Issues
508*6dbdd20aSAndroid Build Coastguard Worker
509*6dbdd20aSAndroid Build Coastguard Worker### {#known-issues-android13} Android 13
510*6dbdd20aSAndroid Build Coastguard Worker
511*6dbdd20aSAndroid Build Coastguard Worker* Unwinding java frames might not work properly, depending on the ART module
512*6dbdd20aSAndroid Build Coastguard Worker  version in use. The UI reports a single "unknown" frame at the top of the
513*6dbdd20aSAndroid Build Coastguard Worker  stack in this case. The problem is fixed in Android 13 QPR1.
514*6dbdd20aSAndroid Build Coastguard Worker
515*6dbdd20aSAndroid Build Coastguard Worker### {#known-issues-android12} Android 12
516*6dbdd20aSAndroid Build Coastguard Worker
517*6dbdd20aSAndroid Build Coastguard Worker* Unwinding java frames might not work properly, depending on the ART module
518*6dbdd20aSAndroid Build Coastguard Worker  version in use. The UI reports a single "unknown" frame at the top of the
519*6dbdd20aSAndroid Build Coastguard Worker  stack in this case.
520*6dbdd20aSAndroid Build Coastguard Worker
521*6dbdd20aSAndroid Build Coastguard Worker### {#known-issues-android11} Android 11
522*6dbdd20aSAndroid Build Coastguard Worker
523*6dbdd20aSAndroid Build Coastguard Worker* 32-bit programs cannot be targeted on 64-bit devices.
524*6dbdd20aSAndroid Build Coastguard Worker* Setting `sampling_interval_bytes` to 0 crashes the target process.
525*6dbdd20aSAndroid Build Coastguard Worker  This is an invalid config that should be rejected instead.
526*6dbdd20aSAndroid Build Coastguard Worker* For startup profiles, some frame names might be missing. This will be
527*6dbdd20aSAndroid Build Coastguard Worker  resolved in Android 12.
528*6dbdd20aSAndroid Build Coastguard Worker* `Failed to send control socket byte.` is displayed in logcat at the end of
529*6dbdd20aSAndroid Build Coastguard Worker  every profile. This is benign.
530*6dbdd20aSAndroid Build Coastguard Worker* The object count may be incorrect in `dump_at_max` profiles.
531*6dbdd20aSAndroid Build Coastguard Worker* Choosing a low shared memory buffer size and `block_client` mode might
532*6dbdd20aSAndroid Build Coastguard Worker  lock up the target process.
533*6dbdd20aSAndroid Build Coastguard Worker
534*6dbdd20aSAndroid Build Coastguard Worker### {#known-issues-android10} Android 10
535*6dbdd20aSAndroid Build Coastguard Worker* Function names in libraries with load bias might be incorrect. Use
536*6dbdd20aSAndroid Build Coastguard Worker  [offline symbolization](#symbolization) to resolve this issue.
537*6dbdd20aSAndroid Build Coastguard Worker* For startup profiles, some frame names might be missing. This will be
538*6dbdd20aSAndroid Build Coastguard Worker  resolved in Android 12.
539*6dbdd20aSAndroid Build Coastguard Worker* 32-bit programs cannot be targeted on 64-bit devices.
540*6dbdd20aSAndroid Build Coastguard Worker* x86 / x86_64 platforms are not supported. This includes the Android
541*6dbdd20aSAndroid Build Coastguard Worker_Cuttlefish_.
542*6dbdd20aSAndroid Build Coastguard Worker  emulator.
543*6dbdd20aSAndroid Build Coastguard Worker* On ARM32, the bottom-most frame is always `ERROR 2`. This is harmless and
544*6dbdd20aSAndroid Build Coastguard Worker  the callstacks are still complete.
545*6dbdd20aSAndroid Build Coastguard Worker* If heapprofd is run standalone (by running `heapprofd` in a root shell, rather
546*6dbdd20aSAndroid Build Coastguard Worker  than through init), `/dev/socket/heapprofd` get assigned an incorrect SELinux
547*6dbdd20aSAndroid Build Coastguard Worker  domain. You will not be able to profile any processes unless you disable
548*6dbdd20aSAndroid Build Coastguard Worker  SELinux enforcement.
549*6dbdd20aSAndroid Build Coastguard Worker  Run `restorecon /dev/socket/heapprofd` in a root shell to resolve.
550*6dbdd20aSAndroid Build Coastguard Worker* Using `vfork(2)` or `clone(2)` with `CLONE_VM` and allocating / freeing
551*6dbdd20aSAndroid Build Coastguard Worker  memory in the child process will prematurely end the profile.
552*6dbdd20aSAndroid Build Coastguard Worker  `java.lang.Runtime.exec` does this, calling it will prematurely end
553*6dbdd20aSAndroid Build Coastguard Worker  the profile. Note that this is in violation of the POSIX standard.
554*6dbdd20aSAndroid Build Coastguard Worker* Setting `sampling_interval_bytes` to 0 crashes the target process.
555*6dbdd20aSAndroid Build Coastguard Worker  This is an invalid config that should be rejected instead.
556*6dbdd20aSAndroid Build Coastguard Worker* `Failed to send control socket byte.` is displayed in logcat at the end of
557*6dbdd20aSAndroid Build Coastguard Worker  every profile. This is benign.
558*6dbdd20aSAndroid Build Coastguard Worker* The object count may be incorrect in `dump_at_max` profiles.
559*6dbdd20aSAndroid Build Coastguard Worker* Choosing a low shared memory buffer size and `block_client` mode might
560*6dbdd20aSAndroid Build Coastguard Worker  lock up the target process.
561*6dbdd20aSAndroid Build Coastguard Worker
562*6dbdd20aSAndroid Build Coastguard Worker## Heapprofd vs malloc_info() vs RSS
563*6dbdd20aSAndroid Build Coastguard Worker
564*6dbdd20aSAndroid Build Coastguard WorkerWhen using heapprofd and interpreting results, it is important to know the
565*6dbdd20aSAndroid Build Coastguard Workerprecise meaning of the different memory metrics that can be obtained from the
566*6dbdd20aSAndroid Build Coastguard Workeroperating system.
567*6dbdd20aSAndroid Build Coastguard Worker
568*6dbdd20aSAndroid Build Coastguard Worker**heapprofd** gives you the number of bytes the target program
569*6dbdd20aSAndroid Build Coastguard Workerrequested from the default C/C++ allocator. If you are profiling a Java app from
570*6dbdd20aSAndroid Build Coastguard Workerstartup, allocations that happen early in the application's initialization will
571*6dbdd20aSAndroid Build Coastguard Workernot be visible to heapprofd. Native services that do not fork from the Zygote
572*6dbdd20aSAndroid Build Coastguard Workerare not affected by this.
573*6dbdd20aSAndroid Build Coastguard Worker
574*6dbdd20aSAndroid Build Coastguard Worker**malloc\_info** is a libc function that gives you information about the
575*6dbdd20aSAndroid Build Coastguard Workerallocator. This can be triggered on userdebug builds by using
576*6dbdd20aSAndroid Build Coastguard Worker`am dumpheap -m <PID> /data/local/tmp/heap.txt`. This will in general be more
577*6dbdd20aSAndroid Build Coastguard Workerthan the memory seen by heapprofd, depending on the allocator not all memory
578*6dbdd20aSAndroid Build Coastguard Workeris immediately freed. In particular, jemalloc retains some freed memory in
579*6dbdd20aSAndroid Build Coastguard Workerthread caches.
580*6dbdd20aSAndroid Build Coastguard Worker
581*6dbdd20aSAndroid Build Coastguard Worker**Heap RSS** is the amount of memory requested from the operating system by the
582*6dbdd20aSAndroid Build Coastguard Workerallocator. This is larger than the previous two numbers because memory can only
583*6dbdd20aSAndroid Build Coastguard Workerbe obtained in page size chunks, and fragmentation causes some of that memory to
584*6dbdd20aSAndroid Build Coastguard Workerbe wasted. This can be obtained by running `adb shell dumpsys meminfo <PID>` and
585*6dbdd20aSAndroid Build Coastguard Workerlooking at the "Private Dirty" column.
586*6dbdd20aSAndroid Build Coastguard WorkerRSS can also end up being smaller than the other two if the device kernel uses
587*6dbdd20aSAndroid Build Coastguard Workermemory compression (ZRAM, enabled by default on recent versions of android) and
588*6dbdd20aSAndroid Build Coastguard Workerthe memory of the process get swapped out onto ZRAM.
589*6dbdd20aSAndroid Build Coastguard Worker
590*6dbdd20aSAndroid Build Coastguard Worker|                     | heapprofd         | malloc\_info | RSS |
591*6dbdd20aSAndroid Build Coastguard Worker|---------------------|:-----------------:|:------------:|:---:|
592*6dbdd20aSAndroid Build Coastguard Worker| from native startup |          x        |      x       |  x  |
593*6dbdd20aSAndroid Build Coastguard Worker| after zygote init   |          x        |      x       |  x  |
594*6dbdd20aSAndroid Build Coastguard Worker| before zygote init  |                   |      x       |  x  |
595*6dbdd20aSAndroid Build Coastguard Worker| thread caches       |                   |      x       |  x  |
596*6dbdd20aSAndroid Build Coastguard Worker| fragmentation       |                   |              |  x  |
597*6dbdd20aSAndroid Build Coastguard Worker
598*6dbdd20aSAndroid Build Coastguard WorkerIf you observe high RSS or malloc\_info metrics but heapprofd does not match,
599*6dbdd20aSAndroid Build Coastguard Workeryou might be hitting some pathological fragmentation problem in the allocator.
600*6dbdd20aSAndroid Build Coastguard Worker
601*6dbdd20aSAndroid Build Coastguard Worker## Convert to pprof
602*6dbdd20aSAndroid Build Coastguard Worker
603*6dbdd20aSAndroid Build Coastguard WorkerYou can use [traceconv](/docs/quickstart/traceconv.md) to convert the heap dumps
604*6dbdd20aSAndroid Build Coastguard Workerin a trace into the [pprof](https://github.com/google/pprof) format. These can
605*6dbdd20aSAndroid Build Coastguard Workerthen be viewed using the pprof CLI or a UI (e.g. Speedscope, or Google-internal
606*6dbdd20aSAndroid Build Coastguard Workerpprof/).
607*6dbdd20aSAndroid Build Coastguard Worker
608*6dbdd20aSAndroid Build Coastguard Worker```bash
609*6dbdd20aSAndroid Build Coastguard Workertools/traceconv profile /tmp/profile
610*6dbdd20aSAndroid Build Coastguard Worker```
611*6dbdd20aSAndroid Build Coastguard Worker
612*6dbdd20aSAndroid Build Coastguard WorkerThis will create a directory in `/tmp/` containing the heap dumps. Run:
613*6dbdd20aSAndroid Build Coastguard Worker
614*6dbdd20aSAndroid Build Coastguard Worker```bash
615*6dbdd20aSAndroid Build Coastguard Workergzip /tmp/heap_profile-XXXXXX/*.pb
616*6dbdd20aSAndroid Build Coastguard Worker```
617*6dbdd20aSAndroid Build Coastguard Worker
618*6dbdd20aSAndroid Build Coastguard Workerto get gzipped protos, which tools handling pprof profile protos expect.
619*6dbdd20aSAndroid Build Coastguard Worker
620*6dbdd20aSAndroid Build Coastguard Worker## {#heapprofd-example-queries} Example SQL Queries
621*6dbdd20aSAndroid Build Coastguard Worker
622*6dbdd20aSAndroid Build Coastguard WorkerWe can get the callstacks that allocated using an SQL Query in the
623*6dbdd20aSAndroid Build Coastguard WorkerTrace Processor. For each frame, we get one row for the number of allocated
624*6dbdd20aSAndroid Build Coastguard Workerbytes, where `count` and `size` is positive, and, if any of them were already
625*6dbdd20aSAndroid Build Coastguard Workerfreed, another line with negative `count` and `size`. The sum of those gets us
626*6dbdd20aSAndroid Build Coastguard Workerthe `Unreleased malloc size` view.
627*6dbdd20aSAndroid Build Coastguard Worker
628*6dbdd20aSAndroid Build Coastguard Worker```sql
629*6dbdd20aSAndroid Build Coastguard Workerselect a.callsite_id, a.ts, a.upid, f.name, f.rel_pc, m.build_id, m.name as mapping_name,
630*6dbdd20aSAndroid Build Coastguard Worker        sum(a.size) as space_size, sum(a.count) as space_count
631*6dbdd20aSAndroid Build Coastguard Worker      from heap_profile_allocation a join
632*6dbdd20aSAndroid Build Coastguard Worker           stack_profile_callsite c ON (a.callsite_id = c.id) join
633*6dbdd20aSAndroid Build Coastguard Worker           stack_profile_frame f ON (c.frame_id = f.id) join
634*6dbdd20aSAndroid Build Coastguard Worker           stack_profile_mapping m ON (f.mapping = m.id)
635*6dbdd20aSAndroid Build Coastguard Worker      group by 1, 2, 3, 4, 5, 6, 7 order by space_size desc;
636*6dbdd20aSAndroid Build Coastguard Worker```
637*6dbdd20aSAndroid Build Coastguard Worker
638*6dbdd20aSAndroid Build Coastguard Worker| callsite_id | ts | upid | name | rel_pc | build_id | mapping_name | space_size | space_count |
639*6dbdd20aSAndroid Build Coastguard Worker|-------------|----|------|-------|-----------|------|--------|----------|------|
640*6dbdd20aSAndroid Build Coastguard Worker|6660|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |106496|4|
641*6dbdd20aSAndroid Build Coastguard Worker|192 |5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26624 |1|
642*6dbdd20aSAndroid Build Coastguard Worker|1421|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26624 |1|
643*6dbdd20aSAndroid Build Coastguard Worker|1537|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26624 |1|
644*6dbdd20aSAndroid Build Coastguard Worker|8843|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26424 |1|
645*6dbdd20aSAndroid Build Coastguard Worker|8618|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |24576 |4|
646*6dbdd20aSAndroid Build Coastguard Worker|3750|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |12288 |1|
647*6dbdd20aSAndroid Build Coastguard Worker|2820|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |8192  |2|
648*6dbdd20aSAndroid Build Coastguard Worker|3788|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |8192  |2|
649*6dbdd20aSAndroid Build Coastguard Worker
650*6dbdd20aSAndroid Build Coastguard WorkerWe can see all the functions are "malloc" and "realloc", which is not terribly
651*6dbdd20aSAndroid Build Coastguard Workerinformative. Usually we are interested in the _cumulative_ bytes allocated in
652*6dbdd20aSAndroid Build Coastguard Workera function (otherwise, we will always only see malloc / realloc). Chasing the
653*6dbdd20aSAndroid Build Coastguard Workerparent_id of a callsite (not shown in this table) recursively is very hard in
654*6dbdd20aSAndroid Build Coastguard WorkerSQL.
655*6dbdd20aSAndroid Build Coastguard Worker
656*6dbdd20aSAndroid Build Coastguard WorkerThere is an **experimental** table that surfaces this information. The **API is
657*6dbdd20aSAndroid Build Coastguard Workersubject to change**.
658*6dbdd20aSAndroid Build Coastguard Worker
659*6dbdd20aSAndroid Build Coastguard Worker```sql
660*6dbdd20aSAndroid Build Coastguard Workerselect
661*6dbdd20aSAndroid Build Coastguard Worker  name,
662*6dbdd20aSAndroid Build Coastguard Worker  map_name,
663*6dbdd20aSAndroid Build Coastguard Worker  cumulative_size
664*6dbdd20aSAndroid Build Coastguard Workerfrom experimental_flamegraph(
665*6dbdd20aSAndroid Build Coastguard Worker  -- The type of the profile from which the flamegraph is being generated.
666*6dbdd20aSAndroid Build Coastguard Worker  -- Always 'native' for native heap profiles.
667*6dbdd20aSAndroid Build Coastguard Worker  'native',
668*6dbdd20aSAndroid Build Coastguard Worker  -- The timestamp of the heap profile.
669*6dbdd20aSAndroid Build Coastguard Worker  8300973884377,
670*6dbdd20aSAndroid Build Coastguard Worker  -- Timestamp constraints: not relevant and always null for native heap
671*6dbdd20aSAndroid Build Coastguard Worker  -- profiles.
672*6dbdd20aSAndroid Build Coastguard Worker  NULL,
673*6dbdd20aSAndroid Build Coastguard Worker  -- The upid of the heap profile.
674*6dbdd20aSAndroid Build Coastguard Worker  1,
675*6dbdd20aSAndroid Build Coastguard Worker  -- The upid group: not relevant and always null for native heap profiles.
676*6dbdd20aSAndroid Build Coastguard Worker  NULL,
677*6dbdd20aSAndroid Build Coastguard Worker  -- A regex for focusing on a particular node in the heapgraph: for advanced
678*6dbdd20aSAndroid Build Coastguard Worker  -- use only.
679*6dbdd20aSAndroid Build Coastguard Worker  NULL
680*6dbdd20aSAndroid Build Coastguard Worker)
681*6dbdd20aSAndroid Build Coastguard Workerorder by abs(cumulative_size) desc;
682*6dbdd20aSAndroid Build Coastguard Worker```
683*6dbdd20aSAndroid Build Coastguard Worker
684*6dbdd20aSAndroid Build Coastguard Worker| name | map_name | cumulative_size |
685*6dbdd20aSAndroid Build Coastguard Worker|------|----------|----------------|
686*6dbdd20aSAndroid Build Coastguard Worker|__start_thread|/apex/com.android.runtime/lib64/bionic/libc.so|392608|
687*6dbdd20aSAndroid Build Coastguard Worker|_ZL15__pthread_startPv|/apex/com.android.runtime/lib64/bionic/libc.so|392608|
688*6dbdd20aSAndroid Build Coastguard Worker|_ZN13thread_data_t10trampolineEPKS|/system/lib64/libutils.so|199496|
689*6dbdd20aSAndroid Build Coastguard Worker|_ZN7android14AndroidRuntime15javaThreadShellEPv|/system/lib64/libandroid_runtime.so|199496|
690*6dbdd20aSAndroid Build Coastguard Worker|_ZN7android6Thread11_threadLoopEPv|/system/lib64/libutils.so|199496|
691*6dbdd20aSAndroid Build Coastguard Worker|_ZN3art6Thread14CreateCallbackEPv|/apex/com.android.art/lib64/libart.so|193112|
692*6dbdd20aSAndroid Build Coastguard Worker|_ZN3art35InvokeVirtualOrInterface...|/apex/com.android.art/lib64/libart.so|193112|
693*6dbdd20aSAndroid Build Coastguard Worker|_ZN3art9ArtMethod6InvokeEPNS_6ThreadEPjjPNS_6JValueEPKc|/apex/com.android.art/lib64/libart.so|193112|
694*6dbdd20aSAndroid Build Coastguard Worker|art_quick_invoke_stub|/apex/com.android.art/lib64/libart.so|193112|
695