xref: /aosp_15_r20/external/perfetto/docs/data-sources/native-heap-profiler.md (revision 6dbdd20afdafa5e3ca9b8809fa73465d530080dc)
1# Heap profiler
2
3NOTE: **heapprofd requires Android 10 or higher**
4
5Heapprofd is a tool that tracks heap allocations & deallocations of an Android
6process within a given time period. The resulting profile can be used to
7attribute memory usage to particular call-stacks, supporting a mix of both
8native and java code. The tool can be used by Android platform and app
9developers to investigate memory issues.
10
11By default, the tool records native allocations and deallocations done with
12malloc/free (or new/delete). It can be configured to record java heap memory
13allocations instead: see [Java heap sampling](#java-heap-sampling) below.
14
15On debug Android builds, you can profile all apps and most system services.
16On "user" builds, you can only use it on apps with the debuggable or
17profileable manifest flag.
18
19## Quickstart
20
21See the [Memory Guide](/docs/case-studies/memory.md#heapprofd) for getting
22started with heapprofd.
23
24## UI
25
26Dumps from heapprofd are shown as flamegraphs in the UI after clicking on the
27diamond. Each diamond corresponds to a snapshot of the allocations and
28callstacks collected at that point in time.
29
30![heapprofd snapshots in the UI tracks](/docs/images/profile-diamond.png)
31
32![heapprofd flamegraph](/docs/images/native-heap-prof.png)
33
34## SQL
35
36Information about callstacks is written to the following tables:
37
38* [`stack_profile_mapping`](/docs/analysis/sql-tables.autogen#stack_profile_mapping)
39* [`stack_profile_frame`](/docs/analysis/sql-tables.autogen#stack_profile_frame)
40* [`stack_profile_callsite`](/docs/analysis/sql-tables.autogen#stack_profile_callsite)
41
42The allocations themselves are written to
43[`heap_profile_allocation`](/docs/analysis/sql-tables.autogen#heap_profile_allocation).
44
45Offline symbolization data is stored in
46[`stack_profile_symbol`](/docs/analysis/sql-tables.autogen#stack_profile_symbol).
47
48See [Example Queries](#heapprofd-example-queries) for example SQL queries.
49
50## Recording
51
52Heapprofd can be configured and started in three ways.
53
54#### Manual configuration
55
56This requires manually setting the
57[HeapprofdConfig](/docs/reference/trace-config-proto.autogen#HeapprofdConfig)
58section of the trace config. The only benefit of doing so is that in this way
59heap profiling can be enabled alongside any other tracing data sources.
60
61#### Using the tools/heap_profile script (recommended)
62
63You can use the `tools/heap_profile` script. If you are having trouble
64make sure you are using the
65[latest version](
66https://raw.githubusercontent.com/google/perfetto/main/tools/heap_profile).
67
68You can target processes either by name (`-n com.example.myapp`) or by PID
69(`-p 1234`). In the first case, the heap profile will be initiated on both on
70already-running processes that match the package name and new processes launched
71after the profiling session is started.
72For the full arguments list see the
73[heap_profile cmdline reference page](/docs/reference/heap_profile-cli).
74
75You can use the [Perfetto UI](https://ui.perfetto.dev) to visualize heap dumps.
76Upload the `raw-trace` file in your output directory. You will see all heap
77dumps as diamonds on the timeline, click any of them to get a flamegraph.
78
79Alternatively [Speedscope](https://speedscope.app) can be used to visualize
80the gzipped protos, but will only show the "Unreleased malloc size" view.
81
82#### Using the Recording page of Perfetto UI
83
84You can also use the [Perfetto UI](https://ui.perfetto.dev/#!/record/memory)
85to record heapprofd profiles. Tick "Heap profiling" in the trace configuration,
86enter the processes you want to target, click "Add Device" to pair your phone,
87and record profiles straight from your browser. This is also possible on
88Windows.
89
90## Viewing the data
91
92![Profile Diamond](/docs/images/profile-diamond.png)
93
94The resulting profile proto contains four views on the data, for each diamond.
95
96* **Unreleased malloc size**: how many bytes were allocated but not freed at
97  this callstack, from the moment the recording was started until the timestamp
98  of the diamond.
99* **Total malloc size**: how many bytes were allocated (including ones freed at
100  the moment of the dump) at this callstack, from the moment the recording was
101  started until the timestamp of the diamond.
102* **Unreleased malloc count**: how many allocations without matching frees were
103  done at this callstack, from the moment the recording was started until the
104  timestamp of the diamond.
105* **Total malloc count**: how many allocations (including ones with matching
106  frees) were done at this callstack, from the moment the recording was started
107  started until the timestamp of the diamond.
108
109_(Googlers: You can also open the gzipped protos using http://pprof/)_
110
111TIP: you might want to put `libart.so` as a "Hide regex" when profiling apps.
112
113TIP: Click Left Heavy on the top left for a good visualization.
114
115## Continuous dumps
116
117By default, the heap profiler captures all the allocations from the beginning of
118the recording and stores a single snapshot, shown as a single diamond in the UI,
119which summarizes all allocations/frees.
120
121It is possible to configure the heap profiler to periodically (not just at the
122end of the trace) store snapshots (continuous dumps), for example every 5000ms
123
124* By setting "Continuous dumps interval" in the UI to 5000.
125* By adding
126  ```
127  continuous_dump_config {
128    dump_interval_ms: 5000
129  }
130  ```
131  in the
132  [HeapprofdConfig](/docs/reference/trace-config-proto.autogen#HeapprofdConfig).
133* By adding `-c 5000` to the invocation of
134  [`tools/heap_profile`](/docs/reference/heap_profile-cli).
135
136![Continuous dump flamegraph](/docs/images/heap_prof_continuous.png)
137
138The resulting visualization shows multiple diamonds. Clicking on each diamond
139shows a summary of the allocations/frees from the beginning of the trace until
140that point (i.e. the summary is cumulative).
141
142## Sampling interval
143
144Heapprofd samples heap allocations by hooking calls to malloc/free and C++'s
145operator new/delete. Given a sampling interval of n bytes, one allocation is
146sampled, on average, every n bytes allocated. This allows to reduce the
147performance impact on the target process. The default sampling rate
148is 4096 bytes.
149
150The easiest way to reason about this is to imagine the memory allocations as a
151stream of one byte allocations. From this stream, every byte has a 1/n
152probability of being selected as a sample, and the corresponding callstack
153gets attributed the complete n bytes. For more accuracy, allocations larger than
154the sampling interval bypass the sampling logic and are recorded with their true
155size.
156See the [heapprofd Sampling](/docs/design-docs/heapprofd-sampling) document for
157details.
158
159## Startup profiling
160
161When specifying a target process name (as opposite to the PID), new processes
162matching that name are profiled from their startup. The resulting profile will
163contain all allocations done between the start of the process and the end
164of the profiling session.
165
166On Android, Java apps are usually not exec()-ed from scratch, but fork()-ed from
167the [zygote], which then specializes into the desired app. If the app's name
168matches a name specified in the profiling session, profiling will be enabled as
169part of the zygote specialization. The resulting profile contains all
170allocations done between that point in zygote specialization and the end of the
171profiling session. Some allocations done early in the specialization process are
172not accounted for.
173
174At the trace proto level, the resulting [ProfilePacket] will have the
175`from_startup` field set to true in the corresponding `ProcessHeapSamples`
176message. This is not surfaced in the converted pprof compatible proto.
177
178[ProfilePacket]: /docs/reference/trace-packet-proto.autogen#ProfilePacket
179[zygote]: https://developer.android.com/topic/performance/memory-overview#SharingRAM
180
181## Runtime profiling
182
183When a profiling session is started, all matching processes (by name or PID)
184are enumerated and are signalled to request profiling. Profiling isn't actually
185enabled until a few hundred milliseconds after the next allocation that is
186done by the application. If the application is idle when profiling is
187requested, and then does a burst of allocations, these may be missed.
188
189The resulting profile will contain all allocations done between when profiling
190is enabled, and the end of the profiling session.
191
192The resulting [ProfilePacket] will have `from_startup` set to false in the
193corresponding `ProcessHeapSamples` message. This does not get surfaced in the
194converted pprof compatible proto.
195
196## Concurrent profiling sessions
197
198If multiple sessions name the same target process (either by name or PID),
199only the first relevant session will profile the process. The other sessions
200will report that the process had already been profiled when converting to
201the pprof compatible proto.
202
203If you see this message but do not expect any other sessions, run
204
205```shell
206adb shell killall perfetto
207```
208
209to stop any concurrent sessions that may be running.
210
211The resulting [ProfilePacket] will have `rejected_concurrent` set  to true in
212otherwise empty corresponding `ProcessHeapSamples` message. This does not get
213surfaced in the converted pprof compatible proto.
214
215## {#heapprofd-targets} Target processes
216
217Depending on the build of Android that heapprofd is run on, some processes
218are not be eligible to be profiled.
219
220On _user_ (i.e. production, non-rootable) builds, only Java applications with
221either the profileable or the debuggable manifest flag set can be profiled.
222Profiling requests for non-profileable/debuggable processes will result in an
223empty profile.
224
225On userdebug builds, all processes except for a small set of critical
226services can be profiled (to find the set of disallowed targets, look for
227`never_profile_heap` in [heapprofd.te](
228https://cs.android.com/android/platform/superproject/main/+/main:system/sepolicy/private/heapprofd.te?q=never_profile_heap).
229This restriction can be lifted by disabling SELinux by running
230`adb shell su root setenforce 0` or by passing `--disable-selinux` to the
231`heap_profile` script.
232
233<center>
234
235|                         | userdebug setenforce 0 | userdebug | user |
236|-------------------------|:----------------------:|:---------:|:----:|
237| critical native service |            Y           |     N     |  N   |
238| native service          |            Y           |     Y     |  N   |
239| app                     |            Y           |     Y     |  N   |
240| profileable app         |            Y           |     Y     |  Y   |
241| debuggable app          |            Y           |     Y     |  Y   |
242
243</center>
244
245To mark an app as profileable, put `<profileable android:shell="true"/>` into
246the `<application>` section of the app manifest.
247
248```xml
249<manifest ...>
250    <application>
251        <profileable android:shell="true"/>
252        ...
253    </application>
254</manifest>
255```
256
257## {#java-heap-sampling} Java heap sampling
258
259NOTE: **Java heap sampling is available on Android 12 or higher**
260
261NOTE: **Java heap sampling is not to be confused with [Java heap
262dumps](/docs/data-sources/java-heap-profiler.md)**
263
264Heapprofd can be configured to track Java allocations instead of native ones.
265* By setting adding `heaps: "com.android.art"` in
266  [HeapprofdConfig](/docs/reference/trace-config-proto.autogen#HeapprofdConfig).
267* By adding `--heaps com.android.art` to the invocation of
268  [`tools/heap_profile`](/docs/reference/heap_profile-cli).
269
270Unlike java heap dumps (which show the retention graph of a snapshot of the live
271objects) but like native heap profiles, java heap samples show callstacks of
272allocations over time of the entire profile.
273
274Java heap samples only show callstacks of when objects are created, not when
275they're deleted or garbage collected.
276
277![javaheapsamples](/docs/images/java-heap-samples.png)
278
279The resulting profile proto contains two views on the data:
280
281* **Total allocation size**: how many bytes were allocated at this callstack
282  over time of the profile until this point. The bytes might have been freed or
283  not, the tool does not keep track of that.
284* **Total allocation count**: how many object were allocated at this callstack
285  over time of the profile until this point. The objects might have been freed
286  or not, the tool does not keep track of that.
287
288Java heap samples are useful to understand memory churn showing the call stack
289of which parts of the code large allocations are attributed to as well as the
290allocation type from the ART runtime.
291
292## DEDUPED frames
293
294If the name of a Java method includes `[DEDUPED]`, this means that multiple
295methods share the same code. ART only stores the name of a single one in its
296metadata, which is displayed here. This is not necessarily the one that was
297called.
298
299## Triggering heap snapshots on demand
300
301Heap snapshot are recorded into the trace either at regular time intervals, if
302using the `continuous_dump_config` field, or at the end of the session.
303
304You can also trigger a snapshot of all currently profiled processes by running
305`adb shell killall -USR1 heapprofd`. This can be useful in lab tests for
306recording the current memory usage of the target in a specific state.
307
308This dump will show up in addition to the dump at the end of the profile that is
309always produced. You can create multiple of these dumps, and they will be
310enumerated in the output directory.
311
312## Symbolization
313
314### Set up llvm-symbolizer
315
316You only need to do this once.
317
318To use symbolization, your system must have llvm-symbolizer installed and
319accessible from `$PATH` as `llvm-symbolizer`. On Debian, you can install it
320using `sudo apt install llvm`.
321
322### Symbolize your profile
323
324If the profiled binary or libraries do not have symbol names, you can
325symbolize profiles offline. Even if they do, you might want to symbolize in
326order to get inlined function and line number information. All tools
327(traceconv, trace_processor_shell, the heap_profile script) support specifying
328the `PERFETTO_BINARY_PATH` as an environment variable.
329
330```
331PERFETTO_BINARY_PATH=somedir tools/heap_profile --name ${NAME}
332```
333
334You can persist symbols for a trace by running
335`PERFETTO_BINARY_PATH=somedir tools/traceconv symbolize raw-trace > symbols`.
336You can then concatenate the symbols to the trace (
337`cat raw-trace symbols > symbolized-trace`) and the symbols will part of
338`symbolized-trace`. The `tools/heap_profile` script will also generate this
339file in your output directory, if `PERFETTO_BINARY_PATH` is used.
340
341The symbol file is the first with matching Build ID in the following order:
342
3431. absolute path of library file relative to binary path.
3442. absolute path of library file relative to binary path, but with base.apk!
345    removed from filename.
3463. basename of library file relative to binary path.
3474. basename of library file relative to binary path, but with base.apk!
348    removed from filename.
3495. in the subdirectory .build-id: the first two hex digits of the build-id
350    as subdirectory, then the rest of the hex digits, with ".debug" appended.
351    See
352    https://fedoraproject.org/wiki/RolandMcGrath/BuildID#Find_files_by_build_ID
353
354For example, "/system/lib/base.apk!foo.so" with build id abcd1234,
355is looked for at:
356
3571. $PERFETTO_BINARY_PATH/system/lib/base.apk!foo.so
3582. $PERFETTO_BINARY_PATH/system/lib/foo.so
3593. $PERFETTO_BINARY_PATH/base.apk!foo.so
3604. $PERFETTO_BINARY_PATH/foo.so
3615. $PERFETTO_BINARY_PATH/.build-id/ab/cd1234.debug
362
363Alternatively, you can set the `PERFETTO_SYMBOLIZER_MODE` environment variable
364to `index`, and the symbolizer will recursively search the given directory for
365an ELF file with the given build id. This way, you will not have to worry
366about correct filenames.
367
368## Deobfuscation
369
370If your profile contains obfuscated Java methods (like `fsd.a`), you can
371provide a deobfuscation map to turn them back into human readable.
372To do so, use the `PERFETTO_PROGUARD_MAP` environment variable, using the
373format `packagename=map_filename[:packagename=map_filename...]`, e.g.
374`PERFETTO_PROGUARD_MAP=com.example.pkg1=foo.txt:com.example.pkg2=bar.txt`.
375All tools (traceconv, trace_processor_shell, the heap_profile script) support
376specifying the `PERFETTO_PROGUARD_MAP` as an environment variable.
377
378```
379PERFETTO_PROGUARD_MAP=com.example.pkg1=proguard_map1.txt:com.example.pkg2=proguard_map2.txt ./tools/heap_profile -n com.example.app
380```
381
382You can get a deobfuscation map for the trace you already collected using
383`tools/traceconv deobfuscate`. Then concatenate the resulting file to your
384trace to get a deobfuscated version of it (the input trace should be in the
385perfetto format, otherwise concatenation will not produce a reasonable output).
386
387```
388PERFETTO_PROGUARD_MAP=com.example.pkg=proguard_map.txt tools/traceconv deobfuscate ${TRACE} > deobfuscation_map
389cat ${TRACE} deobfuscation_map > deobfuscated_trace
390```
391
392`deobfuscated_trace` can be viewed in the
393[Perfetto UI](https://ui.perfetto.dev).
394
395## Troubleshooting
396
397### Buffer overrun
398
399If the rate of allocations is too high for heapprofd to keep up, the profiling
400session will end early due to a buffer overrun. If the buffer overrun is
401caused by a transient spike in allocations, increasing the shared memory buffer
402size (passing `--shmem-size` to `tools/heap_profile`) can resolve the issue.
403Otherwise the sampling interval can be increased (at the expense of lower
404accuracy in the resulting profile) by passing `--interval=16000` or higher.
405
406### Profile is empty
407
408Check whether your target process is eligible to be profiled by consulting
409[Target processes](#heapprofd-targets) above.
410
411Also check the [Known Issues](#known-issues).
412
413### Implausible callstacks
414
415If you see a callstack that seems to impossible from looking at the code, make
416sure no [DEDUPED frames](#deduped-frames) are involved.
417
418Also, if your code is linked using _Identical Code Folding_
419(ICF), i.e. passing `-Wl,--icf=...` to the linker, most trivial functions, often
420constructors and destructors, can be aliased to binary-equivalent operators
421of completely unrelated classes.
422
423### Symbolization: Could not find library
424
425When symbolizing a profile, you might come across messages like this:
426
427```bash
428Could not find /data/app/invalid.app-wFgo3GRaod02wSvPZQ==/lib/arm64/somelib.so
429(Build ID: 44b7138abd5957b8d0a56ce86216d478).
430```
431
432Check whether your library (in this example somelib.so) exists in
433`PERFETTO_BINARY_PATH`. Then compare the Build ID to the one in your
434symbol file, which you can get by running
435`readelf -n /path/in/binary/path/somelib.so`. If it does not match, the
436symbolized file has a different version than the one on device, and cannot
437be used for symbolization.
438If it does, try moving somelib.so to the root of `PERFETTO_BINARY_PATH` and
439try again.
440
441### Only one frame shown
442If you only see a single frame for functions in a specific library, make sure
443that the library has unwind information. We need one of
444
445* `.gnu_debugdata`
446* `.eh_frame` (+ preferably `.eh_frame_hdr`)
447* `.debug_frame`.
448
449Frame-pointer unwinding is *not supported*.
450
451To check if an ELF file has any of those, run
452
453```console
454$ readelf -S file.so | grep "gnu_debugdata\|eh_frame\|debug_frame"
455  [12] .eh_frame_hdr     PROGBITS         000000000000c2b0  0000c2b0
456  [13] .eh_frame         PROGBITS         0000000000011000  00011000
457  [24] .gnu_debugdata    PROGBITS         0000000000000000  000f7292
458```
459
460If this does not show one or more of the sections, change your build system
461to not strip them.
462
463## (non-Android) Linux support
464
465NOTE: Do not use this for production purposes.
466
467You can use a standalone library to profile memory allocations on Linux.
468First [build Perfetto](/docs/contributing/build-instructions.md). You only need
469to do this once.
470
471```
472tools/setup_all_configs.py
473ninja -C out/linux_clang_release
474```
475
476Then, run traced
477
478```
479out/linux_clang_release/traced
480```
481
482Start the profile (e.g. targeting trace_processor_shell)
483
484```
485tools/heap_profile -n trace_processor_shell --print-config  | \
486out/linux_clang_release/perfetto \
487  -c - --txt \
488  -o ~/heapprofd-trace
489```
490
491Finally, run your target (e.g. trace_processor_shell) with LD_PRELOAD
492
493```
494LD_PRELOAD=out/linux_clang_release/libheapprofd_glibc_preload.so out/linux_clang_release/trace_processor_shell <trace>
495```
496
497Then, Ctrl-C the Perfetto invocation and upload ~/heapprofd-trace to the
498[Perfetto UI](https://ui.perfetto.dev).
499
500NOTE: by default, heapprofd lazily initalizes to avoid blocking your program's
501main thread. However, if your program makes memory allocations on startup,
502these can be missed. To avoid this from happening, set the enironment variable
503`PERFETTO_HEAPPROFD_BLOCKING_INIT=1`; on the first malloc, your program will
504be blocked until heapprofd initializes fully but means every allocation will
505be correctly tracked.
506
507## Known Issues
508
509### {#known-issues-android13} Android 13
510
511* Unwinding java frames might not work properly, depending on the ART module
512  version in use. The UI reports a single "unknown" frame at the top of the
513  stack in this case. The problem is fixed in Android 13 QPR1.
514
515### {#known-issues-android12} Android 12
516
517* Unwinding java frames might not work properly, depending on the ART module
518  version in use. The UI reports a single "unknown" frame at the top of the
519  stack in this case.
520
521### {#known-issues-android11} Android 11
522
523* 32-bit programs cannot be targeted on 64-bit devices.
524* Setting `sampling_interval_bytes` to 0 crashes the target process.
525  This is an invalid config that should be rejected instead.
526* For startup profiles, some frame names might be missing. This will be
527  resolved in Android 12.
528* `Failed to send control socket byte.` is displayed in logcat at the end of
529  every profile. This is benign.
530* The object count may be incorrect in `dump_at_max` profiles.
531* Choosing a low shared memory buffer size and `block_client` mode might
532  lock up the target process.
533
534### {#known-issues-android10} Android 10
535* Function names in libraries with load bias might be incorrect. Use
536  [offline symbolization](#symbolization) to resolve this issue.
537* For startup profiles, some frame names might be missing. This will be
538  resolved in Android 12.
539* 32-bit programs cannot be targeted on 64-bit devices.
540* x86 / x86_64 platforms are not supported. This includes the Android
541_Cuttlefish_.
542  emulator.
543* On ARM32, the bottom-most frame is always `ERROR 2`. This is harmless and
544  the callstacks are still complete.
545* If heapprofd is run standalone (by running `heapprofd` in a root shell, rather
546  than through init), `/dev/socket/heapprofd` get assigned an incorrect SELinux
547  domain. You will not be able to profile any processes unless you disable
548  SELinux enforcement.
549  Run `restorecon /dev/socket/heapprofd` in a root shell to resolve.
550* Using `vfork(2)` or `clone(2)` with `CLONE_VM` and allocating / freeing
551  memory in the child process will prematurely end the profile.
552  `java.lang.Runtime.exec` does this, calling it will prematurely end
553  the profile. Note that this is in violation of the POSIX standard.
554* Setting `sampling_interval_bytes` to 0 crashes the target process.
555  This is an invalid config that should be rejected instead.
556* `Failed to send control socket byte.` is displayed in logcat at the end of
557  every profile. This is benign.
558* The object count may be incorrect in `dump_at_max` profiles.
559* Choosing a low shared memory buffer size and `block_client` mode might
560  lock up the target process.
561
562## Heapprofd vs malloc_info() vs RSS
563
564When using heapprofd and interpreting results, it is important to know the
565precise meaning of the different memory metrics that can be obtained from the
566operating system.
567
568**heapprofd** gives you the number of bytes the target program
569requested from the default C/C++ allocator. If you are profiling a Java app from
570startup, allocations that happen early in the application's initialization will
571not be visible to heapprofd. Native services that do not fork from the Zygote
572are not affected by this.
573
574**malloc\_info** is a libc function that gives you information about the
575allocator. This can be triggered on userdebug builds by using
576`am dumpheap -m <PID> /data/local/tmp/heap.txt`. This will in general be more
577than the memory seen by heapprofd, depending on the allocator not all memory
578is immediately freed. In particular, jemalloc retains some freed memory in
579thread caches.
580
581**Heap RSS** is the amount of memory requested from the operating system by the
582allocator. This is larger than the previous two numbers because memory can only
583be obtained in page size chunks, and fragmentation causes some of that memory to
584be wasted. This can be obtained by running `adb shell dumpsys meminfo <PID>` and
585looking at the "Private Dirty" column.
586RSS can also end up being smaller than the other two if the device kernel uses
587memory compression (ZRAM, enabled by default on recent versions of android) and
588the memory of the process get swapped out onto ZRAM.
589
590|                     | heapprofd         | malloc\_info | RSS |
591|---------------------|:-----------------:|:------------:|:---:|
592| from native startup |          x        |      x       |  x  |
593| after zygote init   |          x        |      x       |  x  |
594| before zygote init  |                   |      x       |  x  |
595| thread caches       |                   |      x       |  x  |
596| fragmentation       |                   |              |  x  |
597
598If you observe high RSS or malloc\_info metrics but heapprofd does not match,
599you might be hitting some pathological fragmentation problem in the allocator.
600
601## Convert to pprof
602
603You can use [traceconv](/docs/quickstart/traceconv.md) to convert the heap dumps
604in a trace into the [pprof](https://github.com/google/pprof) format. These can
605then be viewed using the pprof CLI or a UI (e.g. Speedscope, or Google-internal
606pprof/).
607
608```bash
609tools/traceconv profile /tmp/profile
610```
611
612This will create a directory in `/tmp/` containing the heap dumps. Run:
613
614```bash
615gzip /tmp/heap_profile-XXXXXX/*.pb
616```
617
618to get gzipped protos, which tools handling pprof profile protos expect.
619
620## {#heapprofd-example-queries} Example SQL Queries
621
622We can get the callstacks that allocated using an SQL Query in the
623Trace Processor. For each frame, we get one row for the number of allocated
624bytes, where `count` and `size` is positive, and, if any of them were already
625freed, another line with negative `count` and `size`. The sum of those gets us
626the `Unreleased malloc size` view.
627
628```sql
629select a.callsite_id, a.ts, a.upid, f.name, f.rel_pc, m.build_id, m.name as mapping_name,
630        sum(a.size) as space_size, sum(a.count) as space_count
631      from heap_profile_allocation a join
632           stack_profile_callsite c ON (a.callsite_id = c.id) join
633           stack_profile_frame f ON (c.frame_id = f.id) join
634           stack_profile_mapping m ON (f.mapping = m.id)
635      group by 1, 2, 3, 4, 5, 6, 7 order by space_size desc;
636```
637
638| callsite_id | ts | upid | name | rel_pc | build_id | mapping_name | space_size | space_count |
639|-------------|----|------|-------|-----------|------|--------|----------|------|
640|6660|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |106496|4|
641|192 |5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26624 |1|
642|1421|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26624 |1|
643|1537|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26624 |1|
644|8843|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |26424 |1|
645|8618|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |24576 |4|
646|3750|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |12288 |1|
647|2820|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |8192  |2|
648|3788|5|1| malloc |244716| 8126fd.. | /apex/com.android.runtime/lib64/bionic/libc.so |8192  |2|
649
650We can see all the functions are "malloc" and "realloc", which is not terribly
651informative. Usually we are interested in the _cumulative_ bytes allocated in
652a function (otherwise, we will always only see malloc / realloc). Chasing the
653parent_id of a callsite (not shown in this table) recursively is very hard in
654SQL.
655
656There is an **experimental** table that surfaces this information. The **API is
657subject to change**.
658
659```sql
660select
661  name,
662  map_name,
663  cumulative_size
664from experimental_flamegraph(
665  -- The type of the profile from which the flamegraph is being generated.
666  -- Always 'native' for native heap profiles.
667  'native',
668  -- The timestamp of the heap profile.
669  8300973884377,
670  -- Timestamp constraints: not relevant and always null for native heap
671  -- profiles.
672  NULL,
673  -- The upid of the heap profile.
674  1,
675  -- The upid group: not relevant and always null for native heap profiles.
676  NULL,
677  -- A regex for focusing on a particular node in the heapgraph: for advanced
678  -- use only.
679  NULL
680)
681order by abs(cumulative_size) desc;
682```
683
684| name | map_name | cumulative_size |
685|------|----------|----------------|
686|__start_thread|/apex/com.android.runtime/lib64/bionic/libc.so|392608|
687|_ZL15__pthread_startPv|/apex/com.android.runtime/lib64/bionic/libc.so|392608|
688|_ZN13thread_data_t10trampolineEPKS|/system/lib64/libutils.so|199496|
689|_ZN7android14AndroidRuntime15javaThreadShellEPv|/system/lib64/libandroid_runtime.so|199496|
690|_ZN7android6Thread11_threadLoopEPv|/system/lib64/libutils.so|199496|
691|_ZN3art6Thread14CreateCallbackEPv|/apex/com.android.art/lib64/libart.so|193112|
692|_ZN3art35InvokeVirtualOrInterface...|/apex/com.android.art/lib64/libart.so|193112|
693|_ZN3art9ArtMethod6InvokeEPNS_6ThreadEPjjPNS_6JValueEPKc|/apex/com.android.art/lib64/libart.so|193112|
694|art_quick_invoke_stub|/apex/com.android.art/lib64/libart.so|193112|
695