xref: /aosp_15_r20/external/cronet/third_party/protobuf/README.chromium (revision 6777b5387eb2ff775bb5750e3f5d96f37fb7352b)

Name: Protocol Buffers
Short Name: protobuf
URL: https://github.com/google/protobuf
License: BSD
License File: LICENSE
Version: 21.12
CPEPrefix: cpe:/a:google:protobuf:21.12
Revision: f0dc78d7e6e331b8c6bb2d5283e06aa26883ca7c
Security Critical: yes
Shipped: yes

Steps used to create the current version (or roll a newer one):
0. Nuke third_party/protobuf from orbit (rm -rf `third_party/protobuf`). Note
   that this removes this README.chromium so you may want it open in code search
   or remember the steps up until 2. where you get it back.

1. Pull the release from https://github.com/google/protobuf/releases (Source
   code .tar.gz). Extract to `third_party/protobuf` (without version suffix),
   i.e. you may `$ mv protobuf-21.12 third_party/protobuf` or similar.

2. Add back chromium's non-upstream files (from this directory):

   $ git checkout main BUILD.gn DEPS DIR_METADATA OWNERS README.chromium \
     gen_chromium_file_lists.py gen_extra_chromium_files.py proto_library.gni \
     patches/

   Note: Ideally we wouldn't do this but instead have a
   `third_party/protobuf/src/` subdirectory where upstream code goes directly
   into. That will require updating things outside this repository (at least
   `third_party/webrtc`) to agree on where the source code lives. I wouldn't be
   surprised if that also required Google-internal uses of protobuf from webrtc
   to also agree on `third_party/protobuf/src` so maybe this'll never happen?
   Here be dragons. Probably.

   Probably worth making a local `git commit` here before applying patches (next
   step).

3. Apply patches in patches/ (see the description below):

   $ for patch in patches/*; do patch -s -p1 < $patch; done

   When a patch fails, it's probably best to check out previous state (from 3)
   for any changes not inside the patches/ folder, fix the patch and then try to
   reapply everything. It's not a good idea to fix the error, forget to generate
   a patch and leave patches/ in a bad state.

   If a patch no longer applies it may not be needed, if so remove it and its
   description in this README.chromium file. Similarly try to eventually
   upstream any patches out of band from unblocking the roll so we can
   eventually remove it here as well.

   Probably worth committing locally here too.

4. Generate `proto_sources.gni` using the script `./gen_chromium_file_lists.py`.

   Add and commit `proto_sources.gni`,

5. Generate descriptor_pb2.py using the script "gen_extra_chromium_files.py" in
   the same directory as this file.

   $ ./gen_extra_chromium_files.py -C ../../out/Default

   Add and commit `python/google/protobuf/descriptor_pb2.py` and
   `python/google/protobuf/compiler/plugin_pb2.py` as well.

6. Update Version, CPEPrefix and Revision in this file.

Notes (hints) for the next roll(er):

* This is probably harder to roll than most things you've rolled in the past.
  Good luck! Also please reach out for help proactively. The current roll as of
  writing this required help from multiple very-senior people to even figure out
  what was broken in the first place. `protobuf` rolling is not in a good place.
* Update README.chromium as you go so you don't forget (after step 2. above, so
  you don't nuke those changes).
* Rolling one version of protobuf likely introduces a large amount of changes.
  We should possibly figure out if it's reasonable to stay close to tip of tree
  and roll frequently rather than use released versions. Sharp corners here
  likely include that the version is encoded in protobuf output and I (pbos@)
  don't know if that version tag changes immediately on format incompatibilities
  or get bumped every time something format incompatible is introduced.
* After v3.20 protobuf-javascript/ split out from upstream protobuf/. You may
  need to be aware of both repositories. Also upstream protobuf/ seems to move
  faster / out of sync with protobuf-javascript/. Hopefully that doesn't cause
  format incompatibilities in the future (protobuf/ supports something we need
  that protobuf-javascript/ doesn't).
* Any argument changes to `protoc_wrapper` in `third_party/protobuf/BUILD.gn`
  may require updating `build/config/siso/protoc_wrapper.star` as well.
* We've seen instances where instead of rolling protobuf a local copy of a new
  built-in type has been checked in. When that happens you may see ODR
  violations as your built-in type is defined in multiple files. If that happens
  you may want to move the copy that should never have been checked in into its
  own namespace, complete the roll and then cleanup the local copy.
* third_party/mediapipe checks in a weird modified copy of
  `third_party/protobuf/src/google/protobuf/any.pb.cc` called `any_lite.pb.cc`.
  This precompiled protobuf will get invalidated on any protobuf version bumps
  and you'll need to either painstakingly generated a new one (do a diff of the
  checked-in `any_lite.pb.{cc,h}` and current `any.pb.{cc,h}` to aid you in the
  process.
  WARNING: This item is a lot larger than it looks. We should really
  either fix mediapipe upstream to build with `MEDIAPIPE_PROTO_LITE` and
  `MEDIAPIPE_PROTO_THIRD_PARTY` both set, but it looks like that configuration
  has bit rotted.

Note about vpython:

Some Python scripts end up mixing protoc output from this copy of protobuf with
the google.protobuf module from vpython's protobuf. If Python scripts break due
to the vpython copy of protobuf, you may need to update the version in
//.vpython3. See https://crbug.com/1320047.

Description of the patches:

- 0004-fix-shared-library-exports.patch

  This patch allows exporting protobuf symbols in Linux .so libraries, so
  that protobuf can be built as a component (see http://crrev.com/179806).

- 0008-uninline_get_empty_string.patch
- 0010-uninline-generated-code.patch

  These patches uninline some functions, resulting in a significant reduction
  (somewhere between 500 KB and 1 MB) of binary size.

- 0021-Fix-protobuf-s-library-.gitinore-file.patch

  Un-ignores python/google/protobuf/descriptor_pb2.py and
  python/google/protobuf/compiler/plugin_pb2.py

- 0022-Allow-deprecated-fields.patch

  Allows deprecated fields to be used without extra C++ compiler warnings.

- 0026-remove-sprintf.patch

  Imports
  https://github.com/protocolbuffers/protobuf/commit/c0fc2e881bc36aafb0bf539bf41889611370f60c
  to remove use of sprintf.

- 0027-no-noreturn.patch

  Removes an instance of [[noreturn]]. The attribute is correct, but the way
  protobuf's GOOGLE_LOG(FATAL) is defined, the compiler can't see this and it
  trips -Winvalid-noreturn. See https://github.com/protocolbuffers/protobuf/issues/9817

- 0029-make-initializers-optimizable.patch

  Makes the InitProtobufDefaults() static initializer optimizable by Clang when
  built with libc++. It patches out the OnShutdownDestroyString call, which we
  do not need, and removes the thread-safe initialization. Thread safety is only
  needed if a static initializer spawns a thread which then calls
  InitProtobufDefaults() without synchronizing with the start of main().
  (Anything which happens after main() starts can rely on the initializer
  running.)

- 0030-workaround-window-constinit.patch

  Disables PROTOBUF_CONSTINIT in generated code in Windows shared library
  builds. Protobuf's default instances take pointers to a dllimport variable,
  fixed_address_empty_string. This is not constinit on Windows. This is a bug in
  protobuf as the default instance was intended to be constant-initialized. But
  the components build in Chromium is a developer configuration, so we tolerate
  an initializer as long as the build works, until protobuf has a proper fix.

  See https://github.com/protocolbuffers/protobuf/issues/10159.

- 0031-workaround-cfi-unrelated-cast.patch

  A workaround for Clang's Control Flow Integrity check for casting pointers to
  memory that his not yet initialized to be of that type for empty arrays, does
  not work, and still fails. This patch removes the workaround and instead
  disables cfi-unrelated-cast for the affected methods and simplifies them.

  See https://github.com/protocolbuffers/protobuf/issues/10186.
  See https://bugs.chromium.org/p/chromium/issues/detail?id=1294200#c26.

- 0032-cxx20.patch

  Fixes necessary to build in --std=c++20 mode.

  Imports https://critique.corp.google.com/cl/451177197 (a portion of
  https://github.com/protocolbuffers/protobuf/commit/6dd8af4ecfa7987bddb309862932886b84f1e4ef
  ).

- 0033-no-enum-conversion-warn.patch

  Avoid hitting the clang error -Wenum-constexpr-conversion by specifying width
  of the enum.

  Imports https://critique.corp.google.com/cl/466986872.

- 0034-change-macro-to-avoid-pedantic-warning.patch

  Avoid hitting -Wextra-semi.

  Imports (rebased):
  https://github.com/protocolbuffers/protobuf/commit/def602dd07b7eae1cac6823705975317b5607fc3

- 0035-fix-shared-library-constants

  Fixes for component build when building with MediaPipe. AnyMetadata::PackFrom
  is called which require string constants to be exported.

- 0036-fix-undefined-memcpy-call.patch

  Fixes an undefined call to memcpy which may be passed NULL, 0. NULL, 0 is
  forbidden in C, due to a mistake in the C language specification. This
  partially integrates
  https://github.com/protocolbuffers/protobuf/commit/b2e1d7d8a1b1959c782595cd815c22fb690ac8e2
  from upstream and may be removed when protobuf is updated past that commit.

- 0037-fix-wextra-semi.patch

  Removes an extra semicolon, fixed in upstream
  https://github.com/protocolbuffers/protobuf/commit/3e6f8ee152db71ae49cc95649feab50d2faacdfa

- 0038-silence-implicit-precision-loss.patch

  Silences -Wshorten-64-to-32 for now. Fixed in upstream
  https://github.com/protocolbuffers/protobuf/commit/d85c9944c55fb38f4eae149979a0f680ea125ecb

- 0039-generate-mediapipe-anylite.patch

  Workaround to allow generating protobuf code compatible with
  third_party/mediapipe. This replaces checked-in precompiled .pb.cc files. See
  https://crbug.com/332939935, which when/if resolved should make this patch
  obsolete.

- 0040-fix-ndk-major.patch

  Fix Android compile error that affects webrtc. Fixed in upstream
  https://github.com/protocolbuffers/protobuf/commit/9471a882256eaf01027524e8b6c15cbb5e42bef6