1*b7c941bbSAndroid Build Coastguard Worker* Copyright (C) 2015 The Android Open Source Project 2*b7c941bbSAndroid Build Coastguard Worker* 3*b7c941bbSAndroid Build Coastguard Worker* Licensed under the Apache License, Version 2.0 (the "License"); 4*b7c941bbSAndroid Build Coastguard Worker* you may not use this file except in compliance with the License. 5*b7c941bbSAndroid Build Coastguard Worker* You may obtain a copy of the License at 6*b7c941bbSAndroid Build Coastguard Worker* 7*b7c941bbSAndroid Build Coastguard Worker* http://www.apache.org/licenses/LICENSE-2.0 8*b7c941bbSAndroid Build Coastguard Worker* 9*b7c941bbSAndroid Build Coastguard Worker* Unless required by applicable law or agreed to in writing, software 10*b7c941bbSAndroid Build Coastguard Worker* distributed under the License is distributed on an "AS IS" BASIS, 11*b7c941bbSAndroid Build Coastguard Worker* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12*b7c941bbSAndroid Build Coastguard Worker* See the License for the specific language governing permissions and 13*b7c941bbSAndroid Build Coastguard Worker* limitations under the License. 14*b7c941bbSAndroid Build Coastguard Worker 15*b7c941bbSAndroid Build Coastguard Worker 16*b7c941bbSAndroid Build Coastguard WorkerINTRODUCTION 17*b7c941bbSAndroid Build Coastguard Worker 18*b7c941bbSAndroid Build Coastguard WorkerThe Android theme tests ensure that the Holo and Material themes have not been 19*b7c941bbSAndroid Build Coastguard Workermodified. They consist of API-specific sets of reference images representing 20*b7c941bbSAndroid Build Coastguard Workerspecific themes and widgets that must be identical across devices. To pass the 21*b7c941bbSAndroid Build Coastguard Workertheme tests, a device must be able to generate images that are identical to the 22*b7c941bbSAndroid Build Coastguard Workerreference images. 23*b7c941bbSAndroid Build Coastguard Worker 24*b7c941bbSAndroid Build Coastguard WorkerNOTE: Reference images should only be updated by the CTS test maintainers. Any 25*b7c941bbSAndroid Build Coastguard Worker modifications to the reference images will invalidate the test results. 26*b7c941bbSAndroid Build Coastguard Worker 27*b7c941bbSAndroid Build Coastguard Worker 28*b7c941bbSAndroid Build Coastguard WorkerINSTRUCTIONS 29*b7c941bbSAndroid Build Coastguard Worker 30*b7c941bbSAndroid Build Coastguard WorkerI. Generating reference images (CTS maintainers only) 31*b7c941bbSAndroid Build Coastguard Worker 32*b7c941bbSAndroid Build Coastguard WorkerReference images are typically only generated for new API revisions. To 33*b7c941bbSAndroid Build Coastguard Workergenerate a new set of reference images from an emulator, do the following: 34*b7c941bbSAndroid Build Coastguard Worker 35*b7c941bbSAndroid Build Coastguard Worker 1. Ensure the Android SDK is installed locally. Either a public or internal 36*b7c941bbSAndroid Build Coastguard Worker distribution is fine. From the console, set the ANDROID_SDK_ROOT env var: 37*b7c941bbSAndroid Build Coastguard Worker 38*b7c941bbSAndroid Build Coastguard Worker export ANDROID_SDK_ROOT = /path/to/sdk 39*b7c941bbSAndroid Build Coastguard Worker 40*b7c941bbSAndroid Build Coastguard Worker 2. Obtain an x86_64 emulator image from the build server by using the script 41*b7c941bbSAndroid Build Coastguard Worker available internally at go/emu-dev. This script will install the image in 42*b7c941bbSAndroid Build Coastguard Worker your SDK. 43*b7c941bbSAndroid Build Coastguard Worker 44*b7c941bbSAndroid Build Coastguard Worker 3. Use the SDK's AVD Manager tool to create a single virtual device using the 45*b7c941bbSAndroid Build Coastguard Worker emulator image from step 2. The exact configuration doesn't really matter; 46*b7c941bbSAndroid Build Coastguard Worker you can use Nexus 5 as a template. Name the emulator "theme_emulator". 47*b7c941bbSAndroid Build Coastguard Worker 48*b7c941bbSAndroid Build Coastguard Worker 4. From the console, set up your build environment for x86_64 and build CTS: 49*b7c941bbSAndroid Build Coastguard Worker 50*b7c941bbSAndroid Build Coastguard Worker source build/envsetup.sh && lunch sdk_x86_64-eng && make cts -j32 51*b7c941bbSAndroid Build Coastguard Worker 52*b7c941bbSAndroid Build Coastguard Worker 5. Use the reference image script to generate the reference images. The script 53*b7c941bbSAndroid Build Coastguard Worker will automatically start the emulator in the required configurations and 54*b7c941bbSAndroid Build Coastguard Worker install the resulting reference images in assets/<platform>/<dpi>.zip, 55*b7c941bbSAndroid Build Coastguard Worker overwriting any existing images. 56*b7c941bbSAndroid Build Coastguard Worker 57*b7c941bbSAndroid Build Coastguard Worker ./cts/hostsidetests/theme/generate_images.py theme_emulator 58*b7c941bbSAndroid Build Coastguard Worker 59*b7c941bbSAndroid Build Coastguard WorkerYou can also generate reference images using a real device. To generate a new set 60*b7c941bbSAndroid Build Coastguard Workerof reference images from a real device, do the following: 61*b7c941bbSAndroid Build Coastguard Worker 62*b7c941bbSAndroid Build Coastguard Worker 1. Connect the device. Verify the device is connected: 63*b7c941bbSAndroid Build Coastguard Worker 64*b7c941bbSAndroid Build Coastguard Worker adb devices 65*b7c941bbSAndroid Build Coastguard Worker 66*b7c941bbSAndroid Build Coastguard Worker 2. Use the reference image script to generate the reference images: 67*b7c941bbSAndroid Build Coastguard Worker 68*b7c941bbSAndroid Build Coastguard Worker ./cts/hostsidetests/theme/generate_images.py 69*b7c941bbSAndroid Build Coastguard Worker 70*b7c941bbSAndroid Build Coastguard WorkerThere is an option to build locally an Android system image and use an emulator that is stored in 71*b7c941bbSAndroid Build Coastguard WorkerAndroid source tree under "prebuilts/android-emulator/linux-x86_64/emulator". This option does not 72*b7c941bbSAndroid Build Coastguard Workerrequire a SDK and can be used to generate images with locally modified source code: for example 73*b7c941bbSAndroid Build Coastguard Workerright before making a test breaking change. 74*b7c941bbSAndroid Build Coastguard Worker 75*b7c941bbSAndroid Build Coastguard Worker 1. From the console, set up your build environment for sdk_phone_x86_64 and build Android and CTS: 76*b7c941bbSAndroid Build Coastguard Worker 77*b7c941bbSAndroid Build Coastguard Worker lunch sdk_phone_x86_64-userdebug && make -j32 && make cts -j32 78*b7c941bbSAndroid Build Coastguard Worker 79*b7c941bbSAndroid Build Coastguard Worker 2. Use the reference image script to generate the reference images. The script 80*b7c941bbSAndroid Build Coastguard Worker will automatically start the emulator in the required configurations and 81*b7c941bbSAndroid Build Coastguard Worker install the resulting reference images in assets/<platform>/<dpi>.zip, 82*b7c941bbSAndroid Build Coastguard Worker overwriting any existing images. 83*b7c941bbSAndroid Build Coastguard Worker 84*b7c941bbSAndroid Build Coastguard Worker ./cts/hostsidetests/theme/generate_images.py local 85*b7c941bbSAndroid Build Coastguard Worker 86*b7c941bbSAndroid Build Coastguard WorkerA complete collection of reference images for a given API revision must include 87*b7c941bbSAndroid Build Coastguard Workera set for each possible DPI bucket (tvdpi, xxhdpi, etc.) that may be tested. 88*b7c941bbSAndroid Build Coastguard Worker 89*b7c941bbSAndroid Build Coastguard WorkerFor a list of devices and their DPI buckets, see Device Metrics: 90*b7c941bbSAndroid Build Coastguard Worker 91*b7c941bbSAndroid Build Coastguard Worker https://design.google.com/devices/ 92*b7c941bbSAndroid Build Coastguard Worker 93*b7c941bbSAndroid Build Coastguard Worker 94*b7c941bbSAndroid Build Coastguard WorkerII. Building theme tests 95*b7c941bbSAndroid Build Coastguard Worker 96*b7c941bbSAndroid Build Coastguard Worker1. If you have not already built the CTS tests, run an initial make: 97*b7c941bbSAndroid Build Coastguard Worker 98*b7c941bbSAndroid Build Coastguard Worker make cts -j32 99*b7c941bbSAndroid Build Coastguard Worker 100*b7c941bbSAndroid Build Coastguard Worker2. Subsequent changes to the theme tests, including changes to the reference 101*b7c941bbSAndroid Build Coastguard Worker images, may be built using mmm: 102*b7c941bbSAndroid Build Coastguard Worker 103*b7c941bbSAndroid Build Coastguard Worker mmm cts/hostsidetests/theme -j32 104*b7c941bbSAndroid Build Coastguard Worker 105*b7c941bbSAndroid Build Coastguard Worker 106*b7c941bbSAndroid Build Coastguard WorkerIII. Running theme tests 107*b7c941bbSAndroid Build Coastguard Worker 108*b7c941bbSAndroid Build Coastguard WorkerTo obtain reliable results, theme tests should be run against the device's 109*b7c941bbSAndroid Build Coastguard Workernative density; however, many of the tests will also work in a scaled density. 110*b7c941bbSAndroid Build Coastguard WorkerIf you don't possess a device for a given density and would still like to run 111*b7c941bbSAndroid Build Coastguard Workertests for that density, you can manually force scaling via: 112*b7c941bbSAndroid Build Coastguard Worker 113*b7c941bbSAndroid Build Coastguard Worker adb shell wm density <dpi> 114*b7c941bbSAndroid Build Coastguard Worker 115*b7c941bbSAndroid Build Coastguard WorkerAs of API 24, the results will NOT be 100% reliable due to scaling artifacts. 116*b7c941bbSAndroid Build Coastguard WorkerTo reset the device to its native density, run: 117*b7c941bbSAndroid Build Coastguard Worker 118*b7c941bbSAndroid Build Coastguard Worker adb shell wm density reset 119*b7c941bbSAndroid Build Coastguard Worker 120*b7c941bbSAndroid Build Coastguard WorkerOnce the device is in the desired state, do the following: 121*b7c941bbSAndroid Build Coastguard Worker 122*b7c941bbSAndroid Build Coastguard Worker 1. Connect the device that you wish to test. Confirm that is is connected with: 123*b7c941bbSAndroid Build Coastguard Worker 124*b7c941bbSAndroid Build Coastguard Worker adb devices 125*b7c941bbSAndroid Build Coastguard Worker 126*b7c941bbSAndroid Build Coastguard Worker 2. Run the theme tests using cts-tradefed: 127*b7c941bbSAndroid Build Coastguard Worker 128*b7c941bbSAndroid Build Coastguard Worker cts-tradefed run singleCommand cts-dev --module CtsThemeHostTestCases \ 129*b7c941bbSAndroid Build Coastguard Worker --test android.theme.cts.ThemeHostTest 130*b7c941bbSAndroid Build Coastguard Worker 131*b7c941bbSAndroid Build Coastguard Worker 3. Wait for the tests to complete. This should take less than five minutes. 132*b7c941bbSAndroid Build Coastguard Worker 133*b7c941bbSAndroid Build Coastguard WorkerIf any test failures are encountered, diff PNGs will be generated and collected 134*b7c941bbSAndroid Build Coastguard Workerin the "diffs" directory of the CTS results indicated by "Test Result:" in the 135*b7c941bbSAndroid Build Coastguard Workertest output. 136