xref: /aosp_15_r20/cts/hostsidetests/theme/README (revision b7c941bb3fa97aba169d73cee0bed2de8ac964bf)
1*b7c941bbSAndroid Build Coastguard Worker* Copyright (C) 2015 The Android Open Source Project
2*b7c941bbSAndroid Build Coastguard Worker*
3*b7c941bbSAndroid Build Coastguard Worker* Licensed under the Apache License, Version 2.0 (the "License");
4*b7c941bbSAndroid Build Coastguard Worker* you may not use this file except in compliance with the License.
5*b7c941bbSAndroid Build Coastguard Worker* You may obtain a copy of the License at
6*b7c941bbSAndroid Build Coastguard Worker*
7*b7c941bbSAndroid Build Coastguard Worker*      http://www.apache.org/licenses/LICENSE-2.0
8*b7c941bbSAndroid Build Coastguard Worker*
9*b7c941bbSAndroid Build Coastguard Worker* Unless required by applicable law or agreed to in writing, software
10*b7c941bbSAndroid Build Coastguard Worker* distributed under the License is distributed on an "AS IS" BASIS,
11*b7c941bbSAndroid Build Coastguard Worker* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12*b7c941bbSAndroid Build Coastguard Worker* See the License for the specific language governing permissions and
13*b7c941bbSAndroid Build Coastguard Worker* limitations under the License.
14*b7c941bbSAndroid Build Coastguard Worker
15*b7c941bbSAndroid Build Coastguard Worker
16*b7c941bbSAndroid Build Coastguard WorkerINTRODUCTION
17*b7c941bbSAndroid Build Coastguard Worker
18*b7c941bbSAndroid Build Coastguard WorkerThe Android theme tests ensure that the Holo and Material themes have not been
19*b7c941bbSAndroid Build Coastguard Workermodified. They consist of API-specific sets of reference images representing
20*b7c941bbSAndroid Build Coastguard Workerspecific themes and widgets that must be identical across devices. To pass the
21*b7c941bbSAndroid Build Coastguard Workertheme tests, a device must be able to generate images that are identical to the
22*b7c941bbSAndroid Build Coastguard Workerreference images.
23*b7c941bbSAndroid Build Coastguard Worker
24*b7c941bbSAndroid Build Coastguard WorkerNOTE: Reference images should only be updated by the CTS test maintainers. Any
25*b7c941bbSAndroid Build Coastguard Worker      modifications to the reference images will invalidate the test results.
26*b7c941bbSAndroid Build Coastguard Worker
27*b7c941bbSAndroid Build Coastguard Worker
28*b7c941bbSAndroid Build Coastguard WorkerINSTRUCTIONS
29*b7c941bbSAndroid Build Coastguard Worker
30*b7c941bbSAndroid Build Coastguard WorkerI. Generating reference images (CTS maintainers only)
31*b7c941bbSAndroid Build Coastguard Worker
32*b7c941bbSAndroid Build Coastguard WorkerReference images are typically only generated for new API revisions. To
33*b7c941bbSAndroid Build Coastguard Workergenerate a new set of reference images from an emulator, do the following:
34*b7c941bbSAndroid Build Coastguard Worker
35*b7c941bbSAndroid Build Coastguard Worker  1. Ensure the Android SDK is installed locally. Either a public or internal
36*b7c941bbSAndroid Build Coastguard Worker     distribution is fine. From the console, set the ANDROID_SDK_ROOT env var:
37*b7c941bbSAndroid Build Coastguard Worker
38*b7c941bbSAndroid Build Coastguard Worker     export ANDROID_SDK_ROOT = /path/to/sdk
39*b7c941bbSAndroid Build Coastguard Worker
40*b7c941bbSAndroid Build Coastguard Worker  2. Obtain an x86_64 emulator image from the build server by using the script
41*b7c941bbSAndroid Build Coastguard Worker     available internally at go/emu-dev. This script will install the image in
42*b7c941bbSAndroid Build Coastguard Worker     your SDK.
43*b7c941bbSAndroid Build Coastguard Worker
44*b7c941bbSAndroid Build Coastguard Worker  3. Use the SDK's AVD Manager tool to create a single virtual device using the
45*b7c941bbSAndroid Build Coastguard Worker     emulator image from step 2. The exact configuration doesn't really matter;
46*b7c941bbSAndroid Build Coastguard Worker     you can use Nexus 5 as a template. Name the emulator "theme_emulator".
47*b7c941bbSAndroid Build Coastguard Worker
48*b7c941bbSAndroid Build Coastguard Worker  4. From the console, set up your build environment for x86_64 and build CTS:
49*b7c941bbSAndroid Build Coastguard Worker
50*b7c941bbSAndroid Build Coastguard Worker     source build/envsetup.sh && lunch sdk_x86_64-eng && make cts -j32
51*b7c941bbSAndroid Build Coastguard Worker
52*b7c941bbSAndroid Build Coastguard Worker  5. Use the reference image script to generate the reference images. The script
53*b7c941bbSAndroid Build Coastguard Worker     will automatically start the emulator in the required configurations and
54*b7c941bbSAndroid Build Coastguard Worker     install the resulting reference images in assets/<platform>/<dpi>.zip,
55*b7c941bbSAndroid Build Coastguard Worker     overwriting any existing images.
56*b7c941bbSAndroid Build Coastguard Worker
57*b7c941bbSAndroid Build Coastguard Worker     ./cts/hostsidetests/theme/generate_images.py theme_emulator
58*b7c941bbSAndroid Build Coastguard Worker
59*b7c941bbSAndroid Build Coastguard WorkerYou can also generate reference images using a real device. To generate a new set
60*b7c941bbSAndroid Build Coastguard Workerof reference images from a real device, do the following:
61*b7c941bbSAndroid Build Coastguard Worker
62*b7c941bbSAndroid Build Coastguard Worker  1. Connect the device. Verify the device is connected:
63*b7c941bbSAndroid Build Coastguard Worker
64*b7c941bbSAndroid Build Coastguard Worker     adb devices
65*b7c941bbSAndroid Build Coastguard Worker
66*b7c941bbSAndroid Build Coastguard Worker  2. Use the reference image script to generate the reference images:
67*b7c941bbSAndroid Build Coastguard Worker
68*b7c941bbSAndroid Build Coastguard Worker     ./cts/hostsidetests/theme/generate_images.py
69*b7c941bbSAndroid Build Coastguard Worker
70*b7c941bbSAndroid Build Coastguard WorkerThere is an option to build locally an Android system image and use an emulator that is stored in
71*b7c941bbSAndroid Build Coastguard WorkerAndroid source tree under "prebuilts/android-emulator/linux-x86_64/emulator". This option does not
72*b7c941bbSAndroid Build Coastguard Workerrequire a SDK and can be used to generate images with locally modified source code: for example
73*b7c941bbSAndroid Build Coastguard Workerright before making a test breaking change.
74*b7c941bbSAndroid Build Coastguard Worker
75*b7c941bbSAndroid Build Coastguard Worker  1. From the console, set up your build environment for sdk_phone_x86_64 and build Android and CTS:
76*b7c941bbSAndroid Build Coastguard Worker
77*b7c941bbSAndroid Build Coastguard Worker     lunch sdk_phone_x86_64-userdebug && make -j32 && make cts -j32
78*b7c941bbSAndroid Build Coastguard Worker
79*b7c941bbSAndroid Build Coastguard Worker  2. Use the reference image script to generate the reference images. The script
80*b7c941bbSAndroid Build Coastguard Worker     will automatically start the emulator in the required configurations and
81*b7c941bbSAndroid Build Coastguard Worker     install the resulting reference images in assets/<platform>/<dpi>.zip,
82*b7c941bbSAndroid Build Coastguard Worker     overwriting any existing images.
83*b7c941bbSAndroid Build Coastguard Worker
84*b7c941bbSAndroid Build Coastguard Worker     ./cts/hostsidetests/theme/generate_images.py local
85*b7c941bbSAndroid Build Coastguard Worker
86*b7c941bbSAndroid Build Coastguard WorkerA complete collection of reference images for a given API revision must include
87*b7c941bbSAndroid Build Coastguard Workera set for each possible DPI bucket (tvdpi, xxhdpi, etc.) that may be tested.
88*b7c941bbSAndroid Build Coastguard Worker
89*b7c941bbSAndroid Build Coastguard WorkerFor a list of devices and their DPI buckets, see Device Metrics:
90*b7c941bbSAndroid Build Coastguard Worker
91*b7c941bbSAndroid Build Coastguard Worker    https://design.google.com/devices/
92*b7c941bbSAndroid Build Coastguard Worker
93*b7c941bbSAndroid Build Coastguard Worker
94*b7c941bbSAndroid Build Coastguard WorkerII. Building theme tests
95*b7c941bbSAndroid Build Coastguard Worker
96*b7c941bbSAndroid Build Coastguard Worker1. If you have not already built the CTS tests, run an initial make:
97*b7c941bbSAndroid Build Coastguard Worker
98*b7c941bbSAndroid Build Coastguard Worker   make cts -j32
99*b7c941bbSAndroid Build Coastguard Worker
100*b7c941bbSAndroid Build Coastguard Worker2. Subsequent changes to the theme tests, including changes to the reference
101*b7c941bbSAndroid Build Coastguard Worker   images, may be built using mmm:
102*b7c941bbSAndroid Build Coastguard Worker
103*b7c941bbSAndroid Build Coastguard Worker   mmm cts/hostsidetests/theme -j32
104*b7c941bbSAndroid Build Coastguard Worker
105*b7c941bbSAndroid Build Coastguard Worker
106*b7c941bbSAndroid Build Coastguard WorkerIII. Running theme tests
107*b7c941bbSAndroid Build Coastguard Worker
108*b7c941bbSAndroid Build Coastguard WorkerTo obtain reliable results, theme tests should be run against the device's
109*b7c941bbSAndroid Build Coastguard Workernative density; however, many of the tests will also work in a scaled density.
110*b7c941bbSAndroid Build Coastguard WorkerIf you don't possess a device for a given density and would still like to run
111*b7c941bbSAndroid Build Coastguard Workertests for that density, you can manually force scaling via:
112*b7c941bbSAndroid Build Coastguard Worker
113*b7c941bbSAndroid Build Coastguard Worker  adb shell wm density <dpi>
114*b7c941bbSAndroid Build Coastguard Worker
115*b7c941bbSAndroid Build Coastguard WorkerAs of API 24, the results will NOT be 100% reliable due to scaling artifacts.
116*b7c941bbSAndroid Build Coastguard WorkerTo reset the device to its native density, run:
117*b7c941bbSAndroid Build Coastguard Worker
118*b7c941bbSAndroid Build Coastguard Worker  adb shell wm density reset
119*b7c941bbSAndroid Build Coastguard Worker
120*b7c941bbSAndroid Build Coastguard WorkerOnce the device is in the desired state, do the following:
121*b7c941bbSAndroid Build Coastguard Worker
122*b7c941bbSAndroid Build Coastguard Worker  1. Connect the device that you wish to test. Confirm that is is connected with:
123*b7c941bbSAndroid Build Coastguard Worker
124*b7c941bbSAndroid Build Coastguard Worker     adb devices
125*b7c941bbSAndroid Build Coastguard Worker
126*b7c941bbSAndroid Build Coastguard Worker  2. Run the theme tests using cts-tradefed:
127*b7c941bbSAndroid Build Coastguard Worker
128*b7c941bbSAndroid Build Coastguard Worker     cts-tradefed run singleCommand cts-dev --module CtsThemeHostTestCases \
129*b7c941bbSAndroid Build Coastguard Worker         --test android.theme.cts.ThemeHostTest
130*b7c941bbSAndroid Build Coastguard Worker
131*b7c941bbSAndroid Build Coastguard Worker  3. Wait for the tests to complete. This should take less than five minutes.
132*b7c941bbSAndroid Build Coastguard Worker
133*b7c941bbSAndroid Build Coastguard WorkerIf any test failures are encountered, diff PNGs will be generated and collected
134*b7c941bbSAndroid Build Coastguard Workerin the "diffs" directory of the CTS results indicated by "Test Result:" in the
135*b7c941bbSAndroid Build Coastguard Workertest output.
136