xref: /aosp_15_r20/external/pigweed/pw_env_setup/docs.rst (revision 61c4878ac05f98d0ceed94b57d316916de578985)
1*61c4878aSAndroid Build Coastguard Worker.. _module-pw_env_setup:
2*61c4878aSAndroid Build Coastguard Worker
3*61c4878aSAndroid Build Coastguard Worker------------
4*61c4878aSAndroid Build Coastguard Workerpw_env_setup
5*61c4878aSAndroid Build Coastguard Worker------------
6*61c4878aSAndroid Build Coastguard WorkerA classic problem in the embedded space is reducing the time from git clone
7*61c4878aSAndroid Build Coastguard Workerto having a binary executing on a device. The issue is that an entire suite
8*61c4878aSAndroid Build Coastguard Workerof tools is needed for non-trivial production embedded projects. For example:
9*61c4878aSAndroid Build Coastguard Worker
10*61c4878aSAndroid Build Coastguard Worker- A C++ compiler for your target device, and also for your host
11*61c4878aSAndroid Build Coastguard Worker- A build system or three; for example, GN, Ninja, CMake, Bazel
12*61c4878aSAndroid Build Coastguard Worker- A code formatting program like clang-format
13*61c4878aSAndroid Build Coastguard Worker- A debugger like OpenOCD to flash and debug your embedded device (OpenOCD
14*61c4878aSAndroid Build Coastguard Worker  support removed for Windows)
15*61c4878aSAndroid Build Coastguard Worker- A known Python version with known modules installed for scripting
16*61c4878aSAndroid Build Coastguard Worker- A Go compiler for the Go-based command line tools
17*61c4878aSAndroid Build Coastguard Worker
18*61c4878aSAndroid Build Coastguard Worker...and so on
19*61c4878aSAndroid Build Coastguard Worker
20*61c4878aSAndroid Build Coastguard WorkerIn the server space, container solutions like Docker or Podman solve this;
21*61c4878aSAndroid Build Coastguard Workerhowever, in our experience container solutions are a mixed bag for embedded
22*61c4878aSAndroid Build Coastguard Workersystems development where one frequently needs access to native system
23*61c4878aSAndroid Build Coastguard Workerresources like USB devices, or must operate on Windows.
24*61c4878aSAndroid Build Coastguard Worker
25*61c4878aSAndroid Build Coastguard Worker``pw_env_setup`` is our compromise solution for this problem that works on Mac,
26*61c4878aSAndroid Build Coastguard WorkerWindows, and Linux. It leverages the Chrome packaging system `CIPD`_ to
27*61c4878aSAndroid Build Coastguard Workerbootstrap a Python installation, which in turn inflates a virtual
28*61c4878aSAndroid Build Coastguard Workerenvironment. The tooling is installed into your workspace, and makes no
29*61c4878aSAndroid Build Coastguard Workerchanges to your system. This tooling is designed to be reused by any
30*61c4878aSAndroid Build Coastguard Workerproject.
31*61c4878aSAndroid Build Coastguard Worker
32*61c4878aSAndroid Build Coastguard Worker.. _CIPD: https://github.com/luci/luci-go/tree/HEAD/cipd
33*61c4878aSAndroid Build Coastguard Worker
34*61c4878aSAndroid Build Coastguard WorkerUsers interact with  ``pw_env_setup`` with two commands: ``. bootstrap.sh`` and
35*61c4878aSAndroid Build Coastguard Worker``. activate.sh``. The bootstrap command always pulls down the current versions
36*61c4878aSAndroid Build Coastguard Workerof CIPD packages and sets up the Python virtual environment. The activate
37*61c4878aSAndroid Build Coastguard Workercommand reinitializes a previously configured environment, and if none is found,
38*61c4878aSAndroid Build Coastguard Workerruns bootstrap.
39*61c4878aSAndroid Build Coastguard Worker
40*61c4878aSAndroid Build Coastguard Worker.. note::
41*61c4878aSAndroid Build Coastguard Worker
42*61c4878aSAndroid Build Coastguard Worker   On Windows the scripts used to set up the environment are ``bootstrap.bat``
43*61c4878aSAndroid Build Coastguard Worker   and ``activate.bat``.
44*61c4878aSAndroid Build Coastguard Worker
45*61c4878aSAndroid Build Coastguard Worker   ``bootstrap.fish`` and ``activate.fish`` are also available for `Fish shell
46*61c4878aSAndroid Build Coastguard Worker   <https://fishshell.com/>`_ users.
47*61c4878aSAndroid Build Coastguard Worker
48*61c4878aSAndroid Build Coastguard Worker   For simplicity they will be referred to with the ``.sh`` endings unless the
49*61c4878aSAndroid Build Coastguard Worker   distinction is relevant.
50*61c4878aSAndroid Build Coastguard Worker
51*61c4878aSAndroid Build Coastguard WorkerOn POSIX systems, the environment can be deactivated by running ``deactivate``.
52*61c4878aSAndroid Build Coastguard Worker
53*61c4878aSAndroid Build Coastguard Worker==================================
54*61c4878aSAndroid Build Coastguard WorkerUsing pw_env_setup in your project
55*61c4878aSAndroid Build Coastguard Worker==================================
56*61c4878aSAndroid Build Coastguard Worker
57*61c4878aSAndroid Build Coastguard WorkerDownstream Projects Using Pigweed's Packages
58*61c4878aSAndroid Build Coastguard Worker********************************************
59*61c4878aSAndroid Build Coastguard Worker
60*61c4878aSAndroid Build Coastguard WorkerProjects using Pigweed can leverage ``pw_env_setup`` to install Pigweed's
61*61c4878aSAndroid Build Coastguard Workerdependencies or their own dependencies. Projects that only want to use Pigweed's
62*61c4878aSAndroid Build Coastguard Workerdependencies without modifying them can just source Pigweed's ``bootstrap.sh``
63*61c4878aSAndroid Build Coastguard Workerand ``activate.sh`` scripts.
64*61c4878aSAndroid Build Coastguard Worker
65*61c4878aSAndroid Build Coastguard WorkerAn example of what your project's `bootstrap.sh` could look like is below. This
66*61c4878aSAndroid Build Coastguard Workerassumes `bootstrap.sh` is at the top level of your repository.
67*61c4878aSAndroid Build Coastguard Worker
68*61c4878aSAndroid Build Coastguard Worker.. code-block:: bash
69*61c4878aSAndroid Build Coastguard Worker
70*61c4878aSAndroid Build Coastguard Worker   # Do not include a "#!" line, this must be sourced and not executed.
71*61c4878aSAndroid Build Coastguard Worker
72*61c4878aSAndroid Build Coastguard Worker   # This assumes the user is sourcing this file from it's parent directory. See
73*61c4878aSAndroid Build Coastguard Worker   # below for a more flexible way to handle this.
74*61c4878aSAndroid Build Coastguard Worker   PROJ_SETUP_SCRIPT_PATH="$(pwd)/bootstrap.sh"
75*61c4878aSAndroid Build Coastguard Worker
76*61c4878aSAndroid Build Coastguard Worker   export PW_PROJECT_ROOT="$(_python_abspath "$(dirname "$PROJ_SETUP_SCRIPT_PATH")")"
77*61c4878aSAndroid Build Coastguard Worker
78*61c4878aSAndroid Build Coastguard Worker   # You may wish to check if the user is attempting to execute this script
79*61c4878aSAndroid Build Coastguard Worker   # instead of sourcing it. See below for an example of how to handle that
80*61c4878aSAndroid Build Coastguard Worker   # situation.
81*61c4878aSAndroid Build Coastguard Worker
82*61c4878aSAndroid Build Coastguard Worker   # Source Pigweed's bootstrap utility script.
83*61c4878aSAndroid Build Coastguard Worker   # Using '.' instead of 'source' for POSIX compatibility. Since users don't use
84*61c4878aSAndroid Build Coastguard Worker   # dash directly, using 'source' in most documentation so users don't get
85*61c4878aSAndroid Build Coastguard Worker   # confused and try to `./bootstrap.sh`.
86*61c4878aSAndroid Build Coastguard Worker   . "$PW_PROJECT_ROOT/third_party/pigweed/pw_env_setup/util.sh"
87*61c4878aSAndroid Build Coastguard Worker
88*61c4878aSAndroid Build Coastguard Worker   pw_check_root "$PW_ROOT"
89*61c4878aSAndroid Build Coastguard Worker   _PW_ACTUAL_ENVIRONMENT_ROOT="$(pw_get_env_root)"
90*61c4878aSAndroid Build Coastguard Worker   export _PW_ACTUAL_ENVIRONMENT_ROOT
91*61c4878aSAndroid Build Coastguard Worker   SETUP_SH="$_PW_ACTUAL_ENVIRONMENT_ROOT/activate.sh"
92*61c4878aSAndroid Build Coastguard Worker   pw_bootstrap --args...  # See below for details about args.
93*61c4878aSAndroid Build Coastguard Worker   pw_finalize bootstrap "$SETUP_SH"
94*61c4878aSAndroid Build Coastguard Worker
95*61c4878aSAndroid Build Coastguard Worker
96*61c4878aSAndroid Build Coastguard WorkerBazel Usage
97*61c4878aSAndroid Build Coastguard Worker-----------
98*61c4878aSAndroid Build Coastguard WorkerIt is possible to pull in a CIPD dependency into Bazel using WORKSPACE rules
99*61c4878aSAndroid Build Coastguard Workerrather than using `bootstrap.sh`. e.g.
100*61c4878aSAndroid Build Coastguard Worker
101*61c4878aSAndroid Build Coastguard Worker.. code-block:: python
102*61c4878aSAndroid Build Coastguard Worker
103*61c4878aSAndroid Build Coastguard Worker   # WORKSPACE
104*61c4878aSAndroid Build Coastguard Worker
105*61c4878aSAndroid Build Coastguard Worker   load("//pw_env_setup/bazel/cipd_setup:cipd_rules.bzl", "pigweed_deps")
106*61c4878aSAndroid Build Coastguard Worker
107*61c4878aSAndroid Build Coastguard Worker   # Setup CIPD client and packages.
108*61c4878aSAndroid Build Coastguard Worker   # Required by: pigweed.
109*61c4878aSAndroid Build Coastguard Worker   # Used by modules: all.
110*61c4878aSAndroid Build Coastguard Worker   pigweed_deps()
111*61c4878aSAndroid Build Coastguard Worker
112*61c4878aSAndroid Build Coastguard Worker   load("@cipd_deps//:cipd_init.bzl", "cipd_init")
113*61c4878aSAndroid Build Coastguard Worker
114*61c4878aSAndroid Build Coastguard Worker   cipd_init()
115*61c4878aSAndroid Build Coastguard Worker
116*61c4878aSAndroid Build Coastguard Worker
117*61c4878aSAndroid Build Coastguard WorkerThis will make the entire set of Pigweeds remote repositories available to your
118*61c4878aSAndroid Build Coastguard Workerproject. Though these repositories will only be donwloaded if you use them. To
119*61c4878aSAndroid Build Coastguard Workerget a full list of the remote repositories that this configures, run:
120*61c4878aSAndroid Build Coastguard Worker
121*61c4878aSAndroid Build Coastguard Worker.. code-block:: sh
122*61c4878aSAndroid Build Coastguard Worker
123*61c4878aSAndroid Build Coastguard Worker   bazel query //external:all | grep cipd_
124*61c4878aSAndroid Build Coastguard Worker
125*61c4878aSAndroid Build Coastguard WorkerAll files and executables in each CIPD remote repository is exported and visible
126*61c4878aSAndroid Build Coastguard Workereither directely (`@cipd_<dep>//:<file>`) or from 'all' filegroup
127*61c4878aSAndroid Build Coastguard Worker(`@cipd_<dep>//:all`).
128*61c4878aSAndroid Build Coastguard Worker
129*61c4878aSAndroid Build Coastguard WorkerFrom here it is possible to get access to the Bloaty binaries using the
130*61c4878aSAndroid Build Coastguard Workerfollowing command. For example;
131*61c4878aSAndroid Build Coastguard Worker
132*61c4878aSAndroid Build Coastguard Worker.. code-block:: sh
133*61c4878aSAndroid Build Coastguard Worker
134*61c4878aSAndroid Build Coastguard Worker   bazel run @cipd_pigweed_third_party_bloaty_embedded_linux_amd64//:bloaty \
135*61c4878aSAndroid Build Coastguard Worker    -- --help
136*61c4878aSAndroid Build Coastguard Worker
137*61c4878aSAndroid Build Coastguard WorkerUser-Friendliness
138*61c4878aSAndroid Build Coastguard Worker-----------------
139*61c4878aSAndroid Build Coastguard Worker
140*61c4878aSAndroid Build Coastguard WorkerYou may wish to allow sourcing `bootstrap.sh` from a different directory. In
141*61c4878aSAndroid Build Coastguard Workerthat case you'll need the following at the top of `bootstrap.sh`.
142*61c4878aSAndroid Build Coastguard Worker
143*61c4878aSAndroid Build Coastguard Worker.. code-block:: bash
144*61c4878aSAndroid Build Coastguard Worker
145*61c4878aSAndroid Build Coastguard Worker   _python_abspath () {
146*61c4878aSAndroid Build Coastguard Worker     python -c "import os.path; print(os.path.abspath('$@'))"
147*61c4878aSAndroid Build Coastguard Worker   }
148*61c4878aSAndroid Build Coastguard Worker
149*61c4878aSAndroid Build Coastguard Worker   # Use this code from Pigweed's bootstrap to find the path to this script when
150*61c4878aSAndroid Build Coastguard Worker   # sourced. This should work with common shells. PW_CHECKOUT_ROOT is only used in
151*61c4878aSAndroid Build Coastguard Worker   # presubmit tests with strange setups, and can be omitted if you're not using
152*61c4878aSAndroid Build Coastguard Worker   # Pigweed's automated testing infrastructure.
153*61c4878aSAndroid Build Coastguard Worker   if test -n "$PW_CHECKOUT_ROOT"; then
154*61c4878aSAndroid Build Coastguard Worker     PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$PW_CHECKOUT_ROOT/bootstrap.sh")"
155*61c4878aSAndroid Build Coastguard Worker     unset PW_CHECKOUT_ROOT
156*61c4878aSAndroid Build Coastguard Worker   # Shell: bash.
157*61c4878aSAndroid Build Coastguard Worker   elif test -n "$BASH"; then
158*61c4878aSAndroid Build Coastguard Worker     PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$BASH_SOURCE")"
159*61c4878aSAndroid Build Coastguard Worker   # Shell: zsh.
160*61c4878aSAndroid Build Coastguard Worker   elif test -n "$ZSH_NAME"; then
161*61c4878aSAndroid Build Coastguard Worker     PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "${(%):-%N}")"
162*61c4878aSAndroid Build Coastguard Worker   # Shell: dash.
163*61c4878aSAndroid Build Coastguard Worker   elif test ${0##*/} = dash; then
164*61c4878aSAndroid Build Coastguard Worker     PROJ_SETUP_SCRIPT_PATH="$(_python_abspath \
165*61c4878aSAndroid Build Coastguard Worker       "$(lsof -p $$ -Fn0 | tail -1 | sed 's#^[^/]*##;')")"
166*61c4878aSAndroid Build Coastguard Worker   # If everything else fails, try $0. It could work.
167*61c4878aSAndroid Build Coastguard Worker   else
168*61c4878aSAndroid Build Coastguard Worker     PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$0")"
169*61c4878aSAndroid Build Coastguard Worker   fi
170*61c4878aSAndroid Build Coastguard Worker
171*61c4878aSAndroid Build Coastguard WorkerYou may also wish to check if the user is attempting to execute `bootstrap.sh`
172*61c4878aSAndroid Build Coastguard Workerinstead of sourcing it. Executing `bootstrap.sh` would download everything
173*61c4878aSAndroid Build Coastguard Workerrequired for the environment, but cannot modify the environment of the parent
174*61c4878aSAndroid Build Coastguard Workerprocess. To check for this add the following.
175*61c4878aSAndroid Build Coastguard Worker
176*61c4878aSAndroid Build Coastguard Worker.. code-block:: bash
177*61c4878aSAndroid Build Coastguard Worker
178*61c4878aSAndroid Build Coastguard Worker   # Check if this file is being executed or sourced.
179*61c4878aSAndroid Build Coastguard Worker   _pw_sourced=0
180*61c4878aSAndroid Build Coastguard Worker   # If not running in Pigweed's automated testing infrastructure the
181*61c4878aSAndroid Build Coastguard Worker   # SWARMING_BOT_ID check is unnecessary.
182*61c4878aSAndroid Build Coastguard Worker   if [ -n "$SWARMING_BOT_ID" ]; then
183*61c4878aSAndroid Build Coastguard Worker     # If set we're running on swarming and don't need this check.
184*61c4878aSAndroid Build Coastguard Worker     _pw_sourced=1
185*61c4878aSAndroid Build Coastguard Worker   elif [ -n "$ZSH_EVAL_CONTEXT" ]; then
186*61c4878aSAndroid Build Coastguard Worker     case $ZSH_EVAL_CONTEXT in *:file) _pw_sourced=1;; esac
187*61c4878aSAndroid Build Coastguard Worker   elif [ -n "$KSH_VERSION" ]; then
188*61c4878aSAndroid Build Coastguard Worker     [ "$(cd $(dirname -- $0) && pwd -P)/$(basename -- $0)" != \
189*61c4878aSAndroid Build Coastguard Worker       "$(cd $(dirname -- ${.sh.file}) && pwd -P)/$(basename -- ${.sh.file})" ] \
190*61c4878aSAndroid Build Coastguard Worker       && _pw_sourced=1
191*61c4878aSAndroid Build Coastguard Worker   elif [ -n "$BASH_VERSION" ]; then
192*61c4878aSAndroid Build Coastguard Worker     (return 0 2>/dev/null) && _pw_sourced=1
193*61c4878aSAndroid Build Coastguard Worker   else  # All other shells: examine $0 for known shell binary filenames
194*61c4878aSAndroid Build Coastguard Worker     # Detects `sh` and `dash`; add additional shell filenames as needed.
195*61c4878aSAndroid Build Coastguard Worker     case ${0##*/} in sh|dash) _pw_sourced=1;; esac
196*61c4878aSAndroid Build Coastguard Worker   fi
197*61c4878aSAndroid Build Coastguard Worker
198*61c4878aSAndroid Build Coastguard Worker   _pw_eval_sourced "$_pw_sourced"
199*61c4878aSAndroid Build Coastguard Worker
200*61c4878aSAndroid Build Coastguard WorkerDownstream Projects Using Different Packages
201*61c4878aSAndroid Build Coastguard Worker********************************************
202*61c4878aSAndroid Build Coastguard WorkerProjects depending on Pigweed but using additional or different packages should
203*61c4878aSAndroid Build Coastguard Workercopy the Pigweed `sample project`'s ``bootstrap.sh`` and ``pigweed.json`` and
204*61c4878aSAndroid Build Coastguard Workerupdate the call to ``pw_bootstrap``. Search for "downstream" for other places
205*61c4878aSAndroid Build Coastguard Workerthat may require changes, like setting the ``PW_ROOT`` and ``PW_PROJECT_ROOT``
206*61c4878aSAndroid Build Coastguard Workerenvironment variables. Explanations of parts of ``pigweed.json`` are described
207*61c4878aSAndroid Build Coastguard Workerhere.
208*61c4878aSAndroid Build Coastguard Worker
209*61c4878aSAndroid Build Coastguard Worker.. _sample project: https://pigweed.googlesource.com/pigweed/sample_project/+/HEAD
210*61c4878aSAndroid Build Coastguard Worker
211*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.root_variable``
212*61c4878aSAndroid Build Coastguard Worker  Variable used to point to the root of the source tree. Optional, can always
213*61c4878aSAndroid Build Coastguard Worker  use ``PW_PROJECT_ROOT`` instead. (That variable will be set regardless of
214*61c4878aSAndroid Build Coastguard Worker  whether this is provided.)
215*61c4878aSAndroid Build Coastguard Worker
216*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.relative_pigweed_root``
217*61c4878aSAndroid Build Coastguard Worker  Location of the Pigweed submodule within the source tree. Optional—environment
218*61c4878aSAndroid Build Coastguard Worker  setup will work correctly without this. If present, will confirm that it's
219*61c4878aSAndroid Build Coastguard Worker  correct. May be used by other tooling.
220*61c4878aSAndroid Build Coastguard Worker
221*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.cipd_package_files``
222*61c4878aSAndroid Build Coastguard Worker  CIPD package file. JSON file consisting of a list of additional CIPD package
223*61c4878aSAndroid Build Coastguard Worker  files to import and a list of dictionaries with "path", "platforms", "subdir",
224*61c4878aSAndroid Build Coastguard Worker  "tags", and "version_file" keys. Both top-level lists are optional. An
225*61c4878aSAndroid Build Coastguard Worker  example is below. Only "path", "platforms", and "tags" are required. If
226*61c4878aSAndroid Build Coastguard Worker  "version_file" is specified then ``pw doctor`` will fail if that version file
227*61c4878aSAndroid Build Coastguard Worker  is not present. If "subdir" is specified then this packages will be installed
228*61c4878aSAndroid Build Coastguard Worker  in a subdirectory of the directory created for packages in this file.
229*61c4878aSAndroid Build Coastguard Worker
230*61c4878aSAndroid Build Coastguard Worker.. code-block:: json
231*61c4878aSAndroid Build Coastguard Worker
232*61c4878aSAndroid Build Coastguard Worker   {
233*61c4878aSAndroid Build Coastguard Worker     "included_files": [
234*61c4878aSAndroid Build Coastguard Worker       "foo.json"
235*61c4878aSAndroid Build Coastguard Worker     ],
236*61c4878aSAndroid Build Coastguard Worker     "packages": [
237*61c4878aSAndroid Build Coastguard Worker       {
238*61c4878aSAndroid Build Coastguard Worker         "path": "infra/3pp/tools/go/${platform}",
239*61c4878aSAndroid Build Coastguard Worker         "platforms": [
240*61c4878aSAndroid Build Coastguard Worker             "linux-amd64",
241*61c4878aSAndroid Build Coastguard Worker             "linux-arm64",
242*61c4878aSAndroid Build Coastguard Worker             "mac-amd64",
243*61c4878aSAndroid Build Coastguard Worker             "windows-amd64"
244*61c4878aSAndroid Build Coastguard Worker         ],
245*61c4878aSAndroid Build Coastguard Worker         "subdir": "pa/th",
246*61c4878aSAndroid Build Coastguard Worker         "tags": [
247*61c4878aSAndroid Build Coastguard Worker           "version:[email protected]"
248*61c4878aSAndroid Build Coastguard Worker         ],
249*61c4878aSAndroid Build Coastguard Worker         "version_file": ".versions/go.cipd_version"
250*61c4878aSAndroid Build Coastguard Worker       }
251*61c4878aSAndroid Build Coastguard Worker     ]
252*61c4878aSAndroid Build Coastguard Worker   }
253*61c4878aSAndroid Build Coastguard Worker
254*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.project_actions``
255*61c4878aSAndroid Build Coastguard Worker  A list of plugins to load and run after CIPD setup, but prior to virtualenv
256*61c4878aSAndroid Build Coastguard Worker  setup, for e.g. downloading project-specific tools or artifacts needed by
257*61c4878aSAndroid Build Coastguard Worker  later steps. Particularly useful for downstream projects with limited CIPD
258*61c4878aSAndroid Build Coastguard Worker  access.
259*61c4878aSAndroid Build Coastguard Worker
260*61c4878aSAndroid Build Coastguard Worker  A plugin is specified as a dictionary with two keys: "import_path" and
261*61c4878aSAndroid Build Coastguard Worker  "module_name". The "import_path" is relative to the root of the checkout.
262*61c4878aSAndroid Build Coastguard Worker
263*61c4878aSAndroid Build Coastguard Worker  The specified module must provide a "run_actions" method which takes a single
264*61c4878aSAndroid Build Coastguard Worker  argument, "env_vars", which is a pw_env_setup.Environment instance.
265*61c4878aSAndroid Build Coastguard Worker
266*61c4878aSAndroid Build Coastguard Worker  Sample plugin and pigweed.json blob:
267*61c4878aSAndroid Build Coastguard Worker
268*61c4878aSAndroid Build Coastguard Worker.. code-block:: python
269*61c4878aSAndroid Build Coastguard Worker
270*61c4878aSAndroid Build Coastguard Worker   """Sample pw_env_setup project action plugin.
271*61c4878aSAndroid Build Coastguard Worker
272*61c4878aSAndroid Build Coastguard Worker   A sample/starter project action plugin template for pw_env_setup.
273*61c4878aSAndroid Build Coastguard Worker   """
274*61c4878aSAndroid Build Coastguard Worker   def run_action(**kwargs):
275*61c4878aSAndroid Build Coastguard Worker       """Sample project action."""
276*61c4878aSAndroid Build Coastguard Worker       if "env" not in kwargs:
277*61c4878aSAndroid Build Coastguard Worker           raise ValueError(f"Missing required kwarg 'env', got %{kwargs}")
278*61c4878aSAndroid Build Coastguard Worker
279*61c4878aSAndroid Build Coastguard Worker       kwargs["env"].prepend("PATH", "PATH_TO_NEW_TOOLS")
280*61c4878aSAndroid Build Coastguard Worker       raise NotImplementedError("Sample project action running!")
281*61c4878aSAndroid Build Coastguard Worker
282*61c4878aSAndroid Build Coastguard Worker.. code-block:: json
283*61c4878aSAndroid Build Coastguard Worker
284*61c4878aSAndroid Build Coastguard Worker   "project_actions" : [
285*61c4878aSAndroid Build Coastguard Worker      {
286*61c4878aSAndroid Build Coastguard Worker       "import_path": "pw_env_setup",
287*61c4878aSAndroid Build Coastguard Worker       "module_name": "sample_project_action"
288*61c4878aSAndroid Build Coastguard Worker      }
289*61c4878aSAndroid Build Coastguard Worker   ],
290*61c4878aSAndroid Build Coastguard Worker
291*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.virtualenv.gn_args``
292*61c4878aSAndroid Build Coastguard Worker  Any necessary GN args to be used when installing Python packages.
293*61c4878aSAndroid Build Coastguard Worker
294*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.virtualenv.gn_targets``
295*61c4878aSAndroid Build Coastguard Worker  Target for installing Python packages. Downstream projects will need to
296*61c4878aSAndroid Build Coastguard Worker  create targets to install their packages or only use Pigweed Python packages.
297*61c4878aSAndroid Build Coastguard Worker
298*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.virtualenv.gn_root``
299*61c4878aSAndroid Build Coastguard Worker  The root directory of your GN build tree, relative to ``PW_PROJECT_ROOT``.
300*61c4878aSAndroid Build Coastguard Worker  This is the directory your project's ``.gn`` file is located in. If you're
301*61c4878aSAndroid Build Coastguard Worker  only installing Pigweed Python packages, use the location of the Pigweed
302*61c4878aSAndroid Build Coastguard Worker  submodule.
303*61c4878aSAndroid Build Coastguard Worker
304*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.virtualenv.requirements``
305*61c4878aSAndroid Build Coastguard Worker  A list of Python Pip requirements files for installing into the Pigweed
306*61c4878aSAndroid Build Coastguard Worker  virtualenv. Each file will be passed as additional ``--requirement`` argument
307*61c4878aSAndroid Build Coastguard Worker  to a single ```pip install`` at the beginning of bootstrap's ``Python
308*61c4878aSAndroid Build Coastguard Worker  environment`` setup stage. See the `Requirements Files documentation`_ for
309*61c4878aSAndroid Build Coastguard Worker  details on what can be specified using requirements files.
310*61c4878aSAndroid Build Coastguard Worker
311*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.virtualenv.constraints``
312*61c4878aSAndroid Build Coastguard Worker  A list of Python Pip constraints files. These constraints will be passed to
313*61c4878aSAndroid Build Coastguard Worker  every ``pip`` invocation as an additional ``--constraint`` argument during
314*61c4878aSAndroid Build Coastguard Worker  bootstrap.  virtualenv. See the `Constraints Files documentation`_ for details
315*61c4878aSAndroid Build Coastguard Worker  on formatting.
316*61c4878aSAndroid Build Coastguard Worker
317*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.virtualenv.system_packages``
318*61c4878aSAndroid Build Coastguard Worker  A boolean value that can be used the give the Python virtual environment
319*61c4878aSAndroid Build Coastguard Worker  access to the system site packages. Defaults to ``false``.
320*61c4878aSAndroid Build Coastguard Worker
321*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.virtualenv.pip_install_offline``
322*61c4878aSAndroid Build Coastguard Worker  A boolean value that adds ``--no-index`` to all ``pip install`` commands that
323*61c4878aSAndroid Build Coastguard Worker  are part of bootstrap. This forces pip to not reach out to the internet
324*61c4878aSAndroid Build Coastguard Worker  (usually `pypi.org <https://pypi.org/>`_) to download packages. Using this
325*61c4878aSAndroid Build Coastguard Worker  option requires setting
326*61c4878aSAndroid Build Coastguard Worker  ``pw.pw_env_setup.virtualenv.pip_install_find_links``. Defaults to
327*61c4878aSAndroid Build Coastguard Worker  ``false``.
328*61c4878aSAndroid Build Coastguard Worker
329*61c4878aSAndroid Build Coastguard Worker  .. seealso::
330*61c4878aSAndroid Build Coastguard Worker     The Python GN guide for offline pip installation:
331*61c4878aSAndroid Build Coastguard Worker     :ref:`docs-python-build-installing-offline`
332*61c4878aSAndroid Build Coastguard Worker
333*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.virtualenv.pip_install_find_links``
334*61c4878aSAndroid Build Coastguard Worker  List of paths to folders containing Python wheels (``*.whl``) or source tar
335*61c4878aSAndroid Build Coastguard Worker  files (``*.tar.gz``). Pip will check each of these directories when looking
336*61c4878aSAndroid Build Coastguard Worker  for potential install candidates. Each path will be passed to all ``pip
337*61c4878aSAndroid Build Coastguard Worker  install`` commands as ``--find-links PATH``.
338*61c4878aSAndroid Build Coastguard Worker
339*61c4878aSAndroid Build Coastguard Worker  .. tip::
340*61c4878aSAndroid Build Coastguard Worker     Environment variables may be used in these paths. For example:
341*61c4878aSAndroid Build Coastguard Worker
342*61c4878aSAndroid Build Coastguard Worker     .. code-block:: json
343*61c4878aSAndroid Build Coastguard Worker
344*61c4878aSAndroid Build Coastguard Worker        "virtualenv": {
345*61c4878aSAndroid Build Coastguard Worker           "pip_install_find_links": [
346*61c4878aSAndroid Build Coastguard Worker             "${PW_PROJECT_ROOT}/pip_cache"
347*61c4878aSAndroid Build Coastguard Worker           ]
348*61c4878aSAndroid Build Coastguard Worker         }
349*61c4878aSAndroid Build Coastguard Worker
350*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.virtualenv.pip_install_require_hashes``
351*61c4878aSAndroid Build Coastguard Worker  Adds ``--require-hashes`` This option enforces hash checking on Python
352*61c4878aSAndroid Build Coastguard Worker  package files. Defaults to ``false``.
353*61c4878aSAndroid Build Coastguard Worker
354*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.virtualenv.pip_install_disable_cache``
355*61c4878aSAndroid Build Coastguard Worker  A boolean value that adds ``--no-cache-dir`` to all ``pip install`` commands
356*61c4878aSAndroid Build Coastguard Worker  that are part of bootstrap. This forces pip to ignore any previously cached
357*61c4878aSAndroid Build Coastguard Worker  Python packages. On most systems this is located in
358*61c4878aSAndroid Build Coastguard Worker  ``~/.cache/pip/``. Defaults to ``false``.
359*61c4878aSAndroid Build Coastguard Worker
360*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.optional_submodules``
361*61c4878aSAndroid Build Coastguard Worker  By default environment setup will check that all submodules are present in
362*61c4878aSAndroid Build Coastguard Worker  the checkout. Any submodules in this list are excluded from that check.
363*61c4878aSAndroid Build Coastguard Worker
364*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.required_submodules``
365*61c4878aSAndroid Build Coastguard Worker  If this is specified instead of ``optional_submodules`` bootstrap will only
366*61c4878aSAndroid Build Coastguard Worker  complain if one of the required submodules is not present. Combining this
367*61c4878aSAndroid Build Coastguard Worker  with ``optional_submodules`` is not supported.
368*61c4878aSAndroid Build Coastguard Worker
369*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.pw_packages``
370*61c4878aSAndroid Build Coastguard Worker  A list of packages to install using :ref:`pw_package <module-pw_package>`
371*61c4878aSAndroid Build Coastguard Worker  after the rest of bootstrap completes.
372*61c4878aSAndroid Build Coastguard Worker
373*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.gni_file``
374*61c4878aSAndroid Build Coastguard Worker  Location to write a ``.gni`` file containing paths to many things within the
375*61c4878aSAndroid Build Coastguard Worker  environment directory. Defaults to
376*61c4878aSAndroid Build Coastguard Worker  ``build_overrides/pigweed_environment.gni``.
377*61c4878aSAndroid Build Coastguard Worker
378*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.json_file``
379*61c4878aSAndroid Build Coastguard Worker  Location to write a ``.json`` file containing step-by-step modifications to
380*61c4878aSAndroid Build Coastguard Worker  the environment, for reading by tools that don't inherit an environment from
381*61c4878aSAndroid Build Coastguard Worker  a sourced ``bootstrap.sh``.
382*61c4878aSAndroid Build Coastguard Worker
383*61c4878aSAndroid Build Coastguard Worker``pw.pw_env_setup.rosetta``
384*61c4878aSAndroid Build Coastguard Worker  Whether to use Rosetta to use amd64 packages on arm64 Macs. Accepted values
385*61c4878aSAndroid Build Coastguard Worker  are  ``never``, ``allow``, and ``force``. For now, ``allow`` means ``force``.
386*61c4878aSAndroid Build Coastguard Worker  At some point in the future ``allow`` will be changed to mean ``never``.
387*61c4878aSAndroid Build Coastguard Worker
388*61c4878aSAndroid Build Coastguard WorkerAn example of a config file is below.
389*61c4878aSAndroid Build Coastguard Worker
390*61c4878aSAndroid Build Coastguard Worker.. code-block:: json
391*61c4878aSAndroid Build Coastguard Worker
392*61c4878aSAndroid Build Coastguard Worker   {
393*61c4878aSAndroid Build Coastguard Worker     "pw": {
394*61c4878aSAndroid Build Coastguard Worker       "pw_env_setup": {
395*61c4878aSAndroid Build Coastguard Worker         "root_variable": "EXAMPLE_ROOT",
396*61c4878aSAndroid Build Coastguard Worker         "cipd_package_files": [
397*61c4878aSAndroid Build Coastguard Worker           "pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/pigweed.json",
398*61c4878aSAndroid Build Coastguard Worker           "pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/luci.json"
399*61c4878aSAndroid Build Coastguard Worker           "tools/myprojectname.json"
400*61c4878aSAndroid Build Coastguard Worker         ],
401*61c4878aSAndroid Build Coastguard Worker         "virtualenv": {
402*61c4878aSAndroid Build Coastguard Worker           "gn_root": ".",
403*61c4878aSAndroid Build Coastguard Worker           "gn_targets": [
404*61c4878aSAndroid Build Coastguard Worker             ":python.install",
405*61c4878aSAndroid Build Coastguard Worker           ],
406*61c4878aSAndroid Build Coastguard Worker           "system_packages": false
407*61c4878aSAndroid Build Coastguard Worker         },
408*61c4878aSAndroid Build Coastguard Worker         "pw_packages": [],
409*61c4878aSAndroid Build Coastguard Worker         "optional_submodules": [
410*61c4878aSAndroid Build Coastguard Worker           "optional/submodule/one",
411*61c4878aSAndroid Build Coastguard Worker           "optional/submodule/two"
412*61c4878aSAndroid Build Coastguard Worker         ],
413*61c4878aSAndroid Build Coastguard Worker         "gni_file": "tools/environment.gni",
414*61c4878aSAndroid Build Coastguard Worker         "json_file": "tools/environment.json",
415*61c4878aSAndroid Build Coastguard Worker         "rosetta": "allow"
416*61c4878aSAndroid Build Coastguard Worker       }
417*61c4878aSAndroid Build Coastguard Worker     }
418*61c4878aSAndroid Build Coastguard Worker   }
419*61c4878aSAndroid Build Coastguard Worker
420*61c4878aSAndroid Build Coastguard WorkerOnly the packages necessary for almost all projects based on Pigweed are
421*61c4878aSAndroid Build Coastguard Workerincluded in the ``cipd_setup/pigweed.json`` file. A number of other files are
422*61c4878aSAndroid Build Coastguard Workerpresent in that directory for projects that need more than the minimum.
423*61c4878aSAndroid Build Coastguard WorkerInternal-Google projects using LUCI should at least include ``luci.json``.
424*61c4878aSAndroid Build Coastguard Worker
425*61c4878aSAndroid Build Coastguard WorkerIn case the CIPD packages need to be referenced from other scripts, variables
426*61c4878aSAndroid Build Coastguard Workerlike ``PW_${BASENAME}_CIPD_INSTALL_DIR`` point to the CIPD install directories,
427*61c4878aSAndroid Build Coastguard Workerwhere ``${BASENAME}`` is ``"PIGWEED"`` for
428*61c4878aSAndroid Build Coastguard Worker``"pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/pigweed.json"`` and
429*61c4878aSAndroid Build Coastguard Worker``"LUCI"`` for
430*61c4878aSAndroid Build Coastguard Worker``"pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/luci.json"``. This example
431*61c4878aSAndroid Build Coastguard Workerwould set the following environment variables.
432*61c4878aSAndroid Build Coastguard Worker
433*61c4878aSAndroid Build Coastguard Worker- ``PW_LUCI_CIPD_INSTALL_DIR``
434*61c4878aSAndroid Build Coastguard Worker- ``PW_MYPROJECTNAME_CIPD_INSTALL_DIR``
435*61c4878aSAndroid Build Coastguard Worker- ``PW_PIGWEED_CIPD_INSTALL_DIR``
436*61c4878aSAndroid Build Coastguard Worker
437*61c4878aSAndroid Build Coastguard WorkerThese directories are also referenced in the gni_file specified by the
438*61c4878aSAndroid Build Coastguard Workerenvironment config file as ``dir_cipd_${BASENAME}``. This allows the GN build to
439*61c4878aSAndroid Build Coastguard Workerreliably reference these directories without using GN ``getenv()`` calls or
440*61c4878aSAndroid Build Coastguard Workerhardcoding paths.
441*61c4878aSAndroid Build Coastguard Worker
442*61c4878aSAndroid Build Coastguard WorkerIn addition, ``PW_${BASENAME}_CIPD_INSTALL_DIR`` and
443*61c4878aSAndroid Build Coastguard Worker``PW_${BASENAME}_CIPD_INSTALL_DIR/bin`` are both added to ``PATH`` for each
444*61c4878aSAndroid Build Coastguard Workerpackage directory.
445*61c4878aSAndroid Build Coastguard Worker
446*61c4878aSAndroid Build Coastguard WorkerIf multiple packages install executables with the same name, the file mentioned
447*61c4878aSAndroid Build Coastguard Workerlast topologically takes priority. For example, with the file contents below,
448*61c4878aSAndroid Build Coastguard Worker``d.json``'s entries will appear in ``PATH`` before ``c.json``'s, which will
449*61c4878aSAndroid Build Coastguard Workerappear before ``b.json``'s, which will appear before ``a.json``'s.
450*61c4878aSAndroid Build Coastguard Worker
451*61c4878aSAndroid Build Coastguard Worker.. code-block:: json
452*61c4878aSAndroid Build Coastguard Worker   :caption: :octicon:`file;1em` pigweed.json
453*61c4878aSAndroid Build Coastguard Worker
454*61c4878aSAndroid Build Coastguard Worker   {
455*61c4878aSAndroid Build Coastguard Worker     "pw": {
456*61c4878aSAndroid Build Coastguard Worker       "pw_env_setup": {
457*61c4878aSAndroid Build Coastguard Worker         "cipd_package_files": [
458*61c4878aSAndroid Build Coastguard Worker           "a.json",
459*61c4878aSAndroid Build Coastguard Worker           "b.json",
460*61c4878aSAndroid Build Coastguard Worker           "d.json"
461*61c4878aSAndroid Build Coastguard Worker         ]
462*61c4878aSAndroid Build Coastguard Worker       }
463*61c4878aSAndroid Build Coastguard Worker     }
464*61c4878aSAndroid Build Coastguard Worker   }
465*61c4878aSAndroid Build Coastguard Worker
466*61c4878aSAndroid Build Coastguard Worker.. code-block:: json
467*61c4878aSAndroid Build Coastguard Worker   :caption: :octicon:`file;1em` a.json
468*61c4878aSAndroid Build Coastguard Worker
469*61c4878aSAndroid Build Coastguard Worker   {
470*61c4878aSAndroid Build Coastguard Worker     "package_files": [
471*61c4878aSAndroid Build Coastguard Worker       // ...
472*61c4878aSAndroid Build Coastguard Worker     ]
473*61c4878aSAndroid Build Coastguard Worker   }
474*61c4878aSAndroid Build Coastguard Worker
475*61c4878aSAndroid Build Coastguard Worker.. code-block:: json
476*61c4878aSAndroid Build Coastguard Worker   :caption: :octicon:`file;1em` b.json
477*61c4878aSAndroid Build Coastguard Worker
478*61c4878aSAndroid Build Coastguard Worker   {
479*61c4878aSAndroid Build Coastguard Worker     "included_files": ["c.json"],
480*61c4878aSAndroid Build Coastguard Worker     "package_files": [
481*61c4878aSAndroid Build Coastguard Worker       // ...
482*61c4878aSAndroid Build Coastguard Worker     ]
483*61c4878aSAndroid Build Coastguard Worker   }
484*61c4878aSAndroid Build Coastguard Worker
485*61c4878aSAndroid Build Coastguard Worker.. code-block:: json
486*61c4878aSAndroid Build Coastguard Worker   :caption: :octicon:`file;1em` c.json
487*61c4878aSAndroid Build Coastguard Worker
488*61c4878aSAndroid Build Coastguard Worker   {
489*61c4878aSAndroid Build Coastguard Worker     "package_files": [
490*61c4878aSAndroid Build Coastguard Worker       // ...
491*61c4878aSAndroid Build Coastguard Worker     ]
492*61c4878aSAndroid Build Coastguard Worker   }
493*61c4878aSAndroid Build Coastguard Worker
494*61c4878aSAndroid Build Coastguard Worker.. code-block:: json
495*61c4878aSAndroid Build Coastguard Worker   :caption: :octicon:`file;1em` d.json
496*61c4878aSAndroid Build Coastguard Worker
497*61c4878aSAndroid Build Coastguard Worker   {
498*61c4878aSAndroid Build Coastguard Worker     "package_files": [
499*61c4878aSAndroid Build Coastguard Worker       // ...
500*61c4878aSAndroid Build Coastguard Worker     ]
501*61c4878aSAndroid Build Coastguard Worker   }
502*61c4878aSAndroid Build Coastguard Worker
503*61c4878aSAndroid Build Coastguard Worker.. code-block::
504*61c4878aSAndroid Build Coastguard Worker   :caption: Effective File Loading Order
505*61c4878aSAndroid Build Coastguard Worker
506*61c4878aSAndroid Build Coastguard Worker   pigweed.json
507*61c4878aSAndroid Build Coastguard Worker   a.json
508*61c4878aSAndroid Build Coastguard Worker   b.json
509*61c4878aSAndroid Build Coastguard Worker   c.json
510*61c4878aSAndroid Build Coastguard Worker   d.json
511*61c4878aSAndroid Build Coastguard Worker
512*61c4878aSAndroid Build Coastguard WorkerPinning Python Packages
513*61c4878aSAndroid Build Coastguard Worker***********************
514*61c4878aSAndroid Build Coastguard WorkerPython modules usually express dependencies as ranges, which makes it easier to
515*61c4878aSAndroid Build Coastguard Workerinstall many Python packages that might otherwise have conflicting dependencies.
516*61c4878aSAndroid Build Coastguard WorkerHowever, this means version of packages can often change underneath us and
517*61c4878aSAndroid Build Coastguard Workerbuilds will not be hermetic.
518*61c4878aSAndroid Build Coastguard Worker
519*61c4878aSAndroid Build Coastguard WorkerTo ensure versions don't change without approval, Pigweed by default pins the
520*61c4878aSAndroid Build Coastguard Workerversions of packages it depends on using a `pip constraints file`_. To pin the
521*61c4878aSAndroid Build Coastguard Workerversions of additional packages your project depends on, run
522*61c4878aSAndroid Build Coastguard Worker``pw python-packages list <path/to/constraints/file>`` and then add
523*61c4878aSAndroid Build Coastguard Worker``pw_build_PIP_CONSTRAINTS = ["//path/to/constraints/file"]`` to your project's
524*61c4878aSAndroid Build Coastguard Worker``.gn`` file (see `Pigweed's .gn file`_ for an example).
525*61c4878aSAndroid Build Coastguard Worker
526*61c4878aSAndroid Build Coastguard Worker.. _pip constraints file: https://pip.pypa.io/en/stable/user_guide/#constraints-files
527*61c4878aSAndroid Build Coastguard Worker.. _default constraints: https://cs.pigweed.dev/pigweed/+/main:pw_env_setup/py/pw_env_setup/virtualenv_setup/constraint.list
528*61c4878aSAndroid Build Coastguard Worker.. _Pigweed's .gn file: https://cs.pigweed.dev/pigweed/+/main:.gn
529*61c4878aSAndroid Build Coastguard Worker
530*61c4878aSAndroid Build Coastguard WorkerTo update packages, set ``pw_build_PIP_CONSTRAINTS = []``, delete the
531*61c4878aSAndroid Build Coastguard Workerenvironment, and bootstrap again. Then run the ``list`` command from above
532*61c4878aSAndroid Build Coastguard Workeragain, and run ``pw presubmit``.
533*61c4878aSAndroid Build Coastguard Worker
534*61c4878aSAndroid Build Coastguard WorkerEnvironment Variables
535*61c4878aSAndroid Build Coastguard Worker*********************
536*61c4878aSAndroid Build Coastguard WorkerInput Variables
537*61c4878aSAndroid Build Coastguard Worker---------------
538*61c4878aSAndroid Build Coastguard WorkerThe following environment variables affect env setup behavior. Most users will
539*61c4878aSAndroid Build Coastguard Workernever need to set these.
540*61c4878aSAndroid Build Coastguard Worker
541*61c4878aSAndroid Build Coastguard Worker``CIPD_CACHE_DIR``
542*61c4878aSAndroid Build Coastguard Worker  Location of CIPD cache dir. Read by CIPD, but if unset will be defaulted to
543*61c4878aSAndroid Build Coastguard Worker  ``$HOME/.cipd-cache-dir``.
544*61c4878aSAndroid Build Coastguard Worker
545*61c4878aSAndroid Build Coastguard Worker``PW_NO_CIPD_CACHE_DIR``
546*61c4878aSAndroid Build Coastguard Worker  Disables the CIPD cache.
547*61c4878aSAndroid Build Coastguard Worker
548*61c4878aSAndroid Build Coastguard Worker``PW_ACTIVATE_SKIP_CHECKS``
549*61c4878aSAndroid Build Coastguard Worker  If set, skip running ``pw doctor`` at end of bootstrap/activate. Intended to
550*61c4878aSAndroid Build Coastguard Worker  be used by automated tools but not interactively.
551*61c4878aSAndroid Build Coastguard Worker
552*61c4878aSAndroid Build Coastguard Worker``PW_BANNER_FUNC``
553*61c4878aSAndroid Build Coastguard Worker  Command to print a banner at the beginning of bootstrap.
554*61c4878aSAndroid Build Coastguard Worker
555*61c4878aSAndroid Build Coastguard Worker``PW_BOOTSTRAP_PYTHON``
556*61c4878aSAndroid Build Coastguard Worker  Python executable to be used, for example "python3". Defaults to
557*61c4878aSAndroid Build Coastguard Worker  "python3" if that's in ``PATH``, then tries "python".
558*61c4878aSAndroid Build Coastguard Worker
559*61c4878aSAndroid Build Coastguard Worker``PW_CIPD_SERVICE_ACCOUNT_JSON``
560*61c4878aSAndroid Build Coastguard Worker  Value to pass as ``-service-account-json`` to CIPD invocations. This should
561*61c4878aSAndroid Build Coastguard Worker  point either to a service account JSON key file, or be the magical value
562*61c4878aSAndroid Build Coastguard Worker  ``:gce`` to tell the tool to fetch tokens from GCE metadata server.
563*61c4878aSAndroid Build Coastguard Worker
564*61c4878aSAndroid Build Coastguard Worker``PW_ENVIRONMENT_ROOT``
565*61c4878aSAndroid Build Coastguard Worker  Location to which packages are installed. Defaults to ``environment`` folder
566*61c4878aSAndroid Build Coastguard Worker  within the checkout root. This variable is cleared after environment setup is
567*61c4878aSAndroid Build Coastguard Worker  complete.
568*61c4878aSAndroid Build Coastguard Worker
569*61c4878aSAndroid Build Coastguard Worker``PW_ENVSETUP_DISABLE_SPINNER``
570*61c4878aSAndroid Build Coastguard Worker  Disable the spinner during env setup. Intended to be used when the output is
571*61c4878aSAndroid Build Coastguard Worker  being redirected to a log.
572*61c4878aSAndroid Build Coastguard Worker
573*61c4878aSAndroid Build Coastguard Worker``PW_ENVSETUP_DISABLE_SPINNER``
574*61c4878aSAndroid Build Coastguard Worker  Disable the console spinner that runs when waiting for env setup steps to
575*61c4878aSAndroid Build Coastguard Worker  complete.
576*61c4878aSAndroid Build Coastguard Worker
577*61c4878aSAndroid Build Coastguard Worker``PW_ENVSETUP_NO_BANNER``
578*61c4878aSAndroid Build Coastguard Worker  Skip printing the banner.
579*61c4878aSAndroid Build Coastguard Worker
580*61c4878aSAndroid Build Coastguard Worker``PW_ENVSETUP_QUIET``
581*61c4878aSAndroid Build Coastguard Worker  Disables all non-error output.
582*61c4878aSAndroid Build Coastguard Worker
583*61c4878aSAndroid Build Coastguard Worker``PW_PROJECT_ROOT``
584*61c4878aSAndroid Build Coastguard Worker  The absolute path of the project using Pigweed's env setup. For Pigweed this
585*61c4878aSAndroid Build Coastguard Worker  is the same as ``PW_ROOT``. This should be set by the project's bootstrap
586*61c4878aSAndroid Build Coastguard Worker  script.
587*61c4878aSAndroid Build Coastguard Worker
588*61c4878aSAndroid Build Coastguard Worker``PW_ROOT``
589*61c4878aSAndroid Build Coastguard Worker  The absolute path to the Pigweed repository within ``PW_PROJECT_ROOT``. This
590*61c4878aSAndroid Build Coastguard Worker  should be set by the project's bootstrap script.
591*61c4878aSAndroid Build Coastguard Worker
592*61c4878aSAndroid Build Coastguard WorkerOutput Variables
593*61c4878aSAndroid Build Coastguard Worker----------------
594*61c4878aSAndroid Build Coastguard WorkerThe following environment variables are set by env setup.
595*61c4878aSAndroid Build Coastguard Worker
596*61c4878aSAndroid Build Coastguard Worker``PATH``
597*61c4878aSAndroid Build Coastguard Worker  System executable search path. Many of the environment variables below are
598*61c4878aSAndroid Build Coastguard Worker  also added to this variable.
599*61c4878aSAndroid Build Coastguard Worker
600*61c4878aSAndroid Build Coastguard Worker``_PW_ACTUAL_ENVIRONMENT_ROOT``
601*61c4878aSAndroid Build Coastguard Worker  Location the environment was installed into. Separate from
602*61c4878aSAndroid Build Coastguard Worker  ``PW_ENVIRONMENT_ROOT`` because setting that implicitly and switching to
603*61c4878aSAndroid Build Coastguard Worker  another project directory causes unexpected behavior.
604*61c4878aSAndroid Build Coastguard Worker
605*61c4878aSAndroid Build Coastguard Worker``PW_CIPD_INSTALL_DIR``
606*61c4878aSAndroid Build Coastguard Worker  Top-level CIPD install directory. This is where the ``cipd`` executable is.
607*61c4878aSAndroid Build Coastguard Worker
608*61c4878aSAndroid Build Coastguard Worker``PW_*_CIPD_INSTALL_DIR``
609*61c4878aSAndroid Build Coastguard Worker  Each CIPD package file is installed into its own directory. This allows other
610*61c4878aSAndroid Build Coastguard Worker  tools to determine what those directories are. The ``*`` is replaced with an
611*61c4878aSAndroid Build Coastguard Worker  all-caps version of the basename of the package file, without the extension.
612*61c4878aSAndroid Build Coastguard Worker  (E.g., "path/foo.json" becomes ``PW_FOO_CIPD_INSTALL_DIR``.)
613*61c4878aSAndroid Build Coastguard Worker
614*61c4878aSAndroid Build Coastguard Worker``PW_PACKAGE_ROOT``
615*61c4878aSAndroid Build Coastguard Worker  Location that packages installed by ``pw package`` will be installed to.
616*61c4878aSAndroid Build Coastguard Worker
617*61c4878aSAndroid Build Coastguard Worker``VIRTUAL_ENV``
618*61c4878aSAndroid Build Coastguard Worker  Path to Pigweed's virtualenv.
619*61c4878aSAndroid Build Coastguard Worker
620*61c4878aSAndroid Build Coastguard WorkerNon-Shell Environments
621*61c4878aSAndroid Build Coastguard Worker**********************
622*61c4878aSAndroid Build Coastguard WorkerIf using this outside of bash—for example directly from an IDE or CI
623*61c4878aSAndroid Build Coastguard Workersystem—users can process the ``actions.json`` file that's generated in the
624*61c4878aSAndroid Build Coastguard Workerlocation specified by the environment config. It lists variables to set, clear,
625*61c4878aSAndroid Build Coastguard Workerand modify. An example ``actions.json`` is shown below. The "append" and
626*61c4878aSAndroid Build Coastguard Worker"prepend" actions are listed in the order they should be applied, so the
627*61c4878aSAndroid Build Coastguard Worker``<pigweed-root>/out/host/host_tools`` entry should be at the beginning of
628*61c4878aSAndroid Build Coastguard Worker``PATH`` and not in the middle somewhere.
629*61c4878aSAndroid Build Coastguard Worker
630*61c4878aSAndroid Build Coastguard Worker.. code-block:: json
631*61c4878aSAndroid Build Coastguard Worker
632*61c4878aSAndroid Build Coastguard Worker   {
633*61c4878aSAndroid Build Coastguard Worker       "modify": {
634*61c4878aSAndroid Build Coastguard Worker           "PATH": {
635*61c4878aSAndroid Build Coastguard Worker               "append": [],
636*61c4878aSAndroid Build Coastguard Worker               "prepend": [
637*61c4878aSAndroid Build Coastguard Worker                   "<pigweed-root>/environment/cipd",
638*61c4878aSAndroid Build Coastguard Worker                   "<pigweed-root>/environment/cipd/pigweed",
639*61c4878aSAndroid Build Coastguard Worker                   "<pigweed-root>/environment/cipd/pigweed/bin",
640*61c4878aSAndroid Build Coastguard Worker                   "<pigweed-root>/environment/cipd/luci",
641*61c4878aSAndroid Build Coastguard Worker                   "<pigweed-root>/environment/cipd/luci/bin",
642*61c4878aSAndroid Build Coastguard Worker                   "<pigweed-root>/environment/pigweed-venv/bin",
643*61c4878aSAndroid Build Coastguard Worker                   "<pigweed-root>/out/host/host_tools"
644*61c4878aSAndroid Build Coastguard Worker               ],
645*61c4878aSAndroid Build Coastguard Worker               "remove": []
646*61c4878aSAndroid Build Coastguard Worker           }
647*61c4878aSAndroid Build Coastguard Worker       },
648*61c4878aSAndroid Build Coastguard Worker       "set": {
649*61c4878aSAndroid Build Coastguard Worker           "PW_PROJECT_ROOT": "<pigweed-root>",
650*61c4878aSAndroid Build Coastguard Worker           "PW_ROOT": "<pigweed-root>",
651*61c4878aSAndroid Build Coastguard Worker           "_PW_ACTUAL_ENVIRONMENT_ROOT": "<pigweed-root>/environment",
652*61c4878aSAndroid Build Coastguard Worker           "PW_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd",
653*61c4878aSAndroid Build Coastguard Worker           "CIPD_CACHE_DIR": "<home>/.cipd-cache-dir",
654*61c4878aSAndroid Build Coastguard Worker           "PW_PIGWEED_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd/pigweed",
655*61c4878aSAndroid Build Coastguard Worker           "PW_LUCI_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd/luci",
656*61c4878aSAndroid Build Coastguard Worker           "VIRTUAL_ENV": "<pigweed-root>/environment/pigweed-venv",
657*61c4878aSAndroid Build Coastguard Worker           "PYTHONHOME": null,
658*61c4878aSAndroid Build Coastguard Worker           "__PYVENV_LAUNCHER__": null
659*61c4878aSAndroid Build Coastguard Worker       }
660*61c4878aSAndroid Build Coastguard Worker   }
661*61c4878aSAndroid Build Coastguard Worker
662*61c4878aSAndroid Build Coastguard WorkerMany of these variables are directly exposed to the GN build as well, through
663*61c4878aSAndroid Build Coastguard Workerthe GNI file specified in the environment config file.
664*61c4878aSAndroid Build Coastguard Worker
665*61c4878aSAndroid Build Coastguard Worker.. code-block::
666*61c4878aSAndroid Build Coastguard Worker
667*61c4878aSAndroid Build Coastguard Worker   declare_args() {
668*61c4878aSAndroid Build Coastguard Worker     pw_env_setup_CIPD_LUCI = "<environment-root>/cipd/packages/luci"
669*61c4878aSAndroid Build Coastguard Worker     pw_env_setup_CIPD_PIGWEED = "<environment-root>/cipd/packages/pigweed"
670*61c4878aSAndroid Build Coastguard Worker     pw_env_setup_PACKAGE_ROOT = "<environment-root>/packages"
671*61c4878aSAndroid Build Coastguard Worker     pw_env_setup_VIRTUAL_ENV = "<environment-root>/pigweed-venv"
672*61c4878aSAndroid Build Coastguard Worker   }
673*61c4878aSAndroid Build Coastguard Worker
674*61c4878aSAndroid Build Coastguard WorkerIt's straightforward to use these variables.
675*61c4878aSAndroid Build Coastguard Worker
676*61c4878aSAndroid Build Coastguard Worker.. code-block:: cpp
677*61c4878aSAndroid Build Coastguard Worker
678*61c4878aSAndroid Build Coastguard Worker   import("//build_overrides/pigweed_environment.gni")
679*61c4878aSAndroid Build Coastguard Worker
680*61c4878aSAndroid Build Coastguard Worker   deps = [ "$pw_env_setup_CIPD_PIGWEED/..." ]
681*61c4878aSAndroid Build Coastguard Worker
682*61c4878aSAndroid Build Coastguard WorkerImplementation
683*61c4878aSAndroid Build Coastguard Worker**************
684*61c4878aSAndroid Build Coastguard WorkerThe environment is set up by installing CIPD and Python packages in
685*61c4878aSAndroid Build Coastguard Worker``PW_ENVIRONMENT_ROOT`` or ``<checkout>/environment``, and saving modifications
686*61c4878aSAndroid Build Coastguard Workerto environment variables in setup scripts in those directories. To support
687*61c4878aSAndroid Build Coastguard Workermultiple operating systems this is done in an operating system-agnostic manner
688*61c4878aSAndroid Build Coastguard Workerand then written into operating system-specific files to be sourced now and in
689*61c4878aSAndroid Build Coastguard Workerthe future when running ``activate.sh`` instead of ``bootstrap.sh``. In the
690*61c4878aSAndroid Build Coastguard Workerfuture these could be extended to C shell and PowerShell. A logical mapping of
691*61c4878aSAndroid Build Coastguard Workerhigh-level commands to system-specific initialization files is shown below.
692*61c4878aSAndroid Build Coastguard Worker
693*61c4878aSAndroid Build Coastguard Worker.. grid:: 1
694*61c4878aSAndroid Build Coastguard Worker   :padding: 0
695*61c4878aSAndroid Build Coastguard Worker
696*61c4878aSAndroid Build Coastguard Worker   .. grid-item-card::
697*61c4878aSAndroid Build Coastguard Worker      :columns: 12
698*61c4878aSAndroid Build Coastguard Worker      :class-header: font-monospace
699*61c4878aSAndroid Build Coastguard Worker
700*61c4878aSAndroid Build Coastguard Worker      SET $PW_ROOT /home/$USER/pigweed
701*61c4878aSAndroid Build Coastguard Worker      ^^^
702*61c4878aSAndroid Build Coastguard Worker
703*61c4878aSAndroid Build Coastguard Worker      .. grid:: 2
704*61c4878aSAndroid Build Coastguard Worker         :margin: 0
705*61c4878aSAndroid Build Coastguard Worker         :padding: 0
706*61c4878aSAndroid Build Coastguard Worker
707*61c4878aSAndroid Build Coastguard Worker         .. grid-item:: **Windows**
708*61c4878aSAndroid Build Coastguard Worker
709*61c4878aSAndroid Build Coastguard Worker         .. grid-item:: **Linux & Mac (sh-compatible shells)**
710*61c4878aSAndroid Build Coastguard Worker
711*61c4878aSAndroid Build Coastguard Worker      .. grid:: 2
712*61c4878aSAndroid Build Coastguard Worker         :margin: 0
713*61c4878aSAndroid Build Coastguard Worker         :padding: 0
714*61c4878aSAndroid Build Coastguard Worker
715*61c4878aSAndroid Build Coastguard Worker         .. grid-item::
716*61c4878aSAndroid Build Coastguard Worker
717*61c4878aSAndroid Build Coastguard Worker            .. code-block:: dosbatch
718*61c4878aSAndroid Build Coastguard Worker
719*61c4878aSAndroid Build Coastguard Worker               set PW_ROOT /home/%USER%/pigweed
720*61c4878aSAndroid Build Coastguard Worker
721*61c4878aSAndroid Build Coastguard Worker         .. grid-item::
722*61c4878aSAndroid Build Coastguard Worker
723*61c4878aSAndroid Build Coastguard Worker            .. code-block:: shell
724*61c4878aSAndroid Build Coastguard Worker
725*61c4878aSAndroid Build Coastguard Worker               PW_ROOT="/home/$USER/pigweed"
726*61c4878aSAndroid Build Coastguard Worker               export PW_ROOT
727*61c4878aSAndroid Build Coastguard Worker
728*61c4878aSAndroid Build Coastguard Worker.. grid:: 1
729*61c4878aSAndroid Build Coastguard Worker   :padding: 0
730*61c4878aSAndroid Build Coastguard Worker
731*61c4878aSAndroid Build Coastguard Worker   .. grid-item-card::
732*61c4878aSAndroid Build Coastguard Worker      :columns: 12
733*61c4878aSAndroid Build Coastguard Worker      :class-header: font-monospace
734*61c4878aSAndroid Build Coastguard Worker
735*61c4878aSAndroid Build Coastguard Worker      PREPEND $PATH $PW_ROOT/.env/bin
736*61c4878aSAndroid Build Coastguard Worker      ^^^
737*61c4878aSAndroid Build Coastguard Worker      .. grid:: 2
738*61c4878aSAndroid Build Coastguard Worker         :margin: 0
739*61c4878aSAndroid Build Coastguard Worker         :padding: 0
740*61c4878aSAndroid Build Coastguard Worker
741*61c4878aSAndroid Build Coastguard Worker         .. grid-item:: **Windows**
742*61c4878aSAndroid Build Coastguard Worker
743*61c4878aSAndroid Build Coastguard Worker         .. grid-item:: **Linux & Mac (sh-compatible shells)**
744*61c4878aSAndroid Build Coastguard Worker
745*61c4878aSAndroid Build Coastguard Worker      .. grid:: 2
746*61c4878aSAndroid Build Coastguard Worker         :margin: 0
747*61c4878aSAndroid Build Coastguard Worker         :padding: 0
748*61c4878aSAndroid Build Coastguard Worker
749*61c4878aSAndroid Build Coastguard Worker         .. grid-item::
750*61c4878aSAndroid Build Coastguard Worker
751*61c4878aSAndroid Build Coastguard Worker            .. code-block:: dosbatch
752*61c4878aSAndroid Build Coastguard Worker
753*61c4878aSAndroid Build Coastguard Worker               set PATH=%PW_ROOT%/.env/bin;%PATH%
754*61c4878aSAndroid Build Coastguard Worker
755*61c4878aSAndroid Build Coastguard Worker         .. grid-item::
756*61c4878aSAndroid Build Coastguard Worker
757*61c4878aSAndroid Build Coastguard Worker            .. code-block:: shell
758*61c4878aSAndroid Build Coastguard Worker
759*61c4878aSAndroid Build Coastguard Worker               PATH="$(\
760*61c4878aSAndroid Build Coastguard Worker                 echo "$PATH" | \
761*61c4878aSAndroid Build Coastguard Worker                 sed "s|:$PW_ROOT/.env/bin:|:|g;" | \
762*61c4878aSAndroid Build Coastguard Worker                 sed "s|^$PW_ROOT/.env/bin:||g;" | \
763*61c4878aSAndroid Build Coastguard Worker                 sed "s|:$PW_ROOT/.env/bin$||g;")"
764*61c4878aSAndroid Build Coastguard Worker               PATH="$PW_ROOT/.env/bin;$PATH"
765*61c4878aSAndroid Build Coastguard Worker               export PATH
766*61c4878aSAndroid Build Coastguard Worker
767*61c4878aSAndroid Build Coastguard Worker.. grid:: 1
768*61c4878aSAndroid Build Coastguard Worker   :padding: 0
769*61c4878aSAndroid Build Coastguard Worker
770*61c4878aSAndroid Build Coastguard Worker   .. grid-item-card::
771*61c4878aSAndroid Build Coastguard Worker      :columns: 12
772*61c4878aSAndroid Build Coastguard Worker      :class-header: font-monospace
773*61c4878aSAndroid Build Coastguard Worker
774*61c4878aSAndroid Build Coastguard Worker      ECHO "Setup Complete!"
775*61c4878aSAndroid Build Coastguard Worker      ^^^
776*61c4878aSAndroid Build Coastguard Worker
777*61c4878aSAndroid Build Coastguard Worker      .. grid:: 2
778*61c4878aSAndroid Build Coastguard Worker         :margin: 0
779*61c4878aSAndroid Build Coastguard Worker         :padding: 0
780*61c4878aSAndroid Build Coastguard Worker
781*61c4878aSAndroid Build Coastguard Worker         .. grid-item:: **Windows**
782*61c4878aSAndroid Build Coastguard Worker
783*61c4878aSAndroid Build Coastguard Worker         .. grid-item:: **Linux & Mac (sh-compatible shells)**
784*61c4878aSAndroid Build Coastguard Worker
785*61c4878aSAndroid Build Coastguard Worker
786*61c4878aSAndroid Build Coastguard Worker      .. grid:: 2
787*61c4878aSAndroid Build Coastguard Worker         :margin: 0
788*61c4878aSAndroid Build Coastguard Worker         :padding: 0
789*61c4878aSAndroid Build Coastguard Worker
790*61c4878aSAndroid Build Coastguard Worker         .. grid-item::
791*61c4878aSAndroid Build Coastguard Worker
792*61c4878aSAndroid Build Coastguard Worker            .. code-block:: dosbatch
793*61c4878aSAndroid Build Coastguard Worker
794*61c4878aSAndroid Build Coastguard Worker               echo Setup Complete!
795*61c4878aSAndroid Build Coastguard Worker
796*61c4878aSAndroid Build Coastguard Worker         .. grid-item::
797*61c4878aSAndroid Build Coastguard Worker
798*61c4878aSAndroid Build Coastguard Worker            .. code-block:: shell
799*61c4878aSAndroid Build Coastguard Worker
800*61c4878aSAndroid Build Coastguard Worker               echo "Setup Complete!"
801*61c4878aSAndroid Build Coastguard Worker
802*61c4878aSAndroid Build Coastguard Worker
803*61c4878aSAndroid Build Coastguard Worker.. _Requirements Files documentation: https://pip.pypa.io/en/stable/user_guide/#requirements-files
804*61c4878aSAndroid Build Coastguard Worker.. _Constraints Files documentation: https://pip.pypa.io/en/stable/user_guide/#constraints-files
805