1Name: Protocol Buffers 2Short Name: protobuf 3URL: https://github.com/google/protobuf 4License: BSD 5License File: LICENSE 6Version: 21.12 7CPEPrefix: cpe:/a:google:protobuf:21.12 8Revision: f0dc78d7e6e331b8c6bb2d5283e06aa26883ca7c 9Security Critical: yes 10Shipped: yes 11 12Steps used to create the current version (or roll a newer one): 130. Nuke third_party/protobuf from orbit (rm -rf `third_party/protobuf`). Note 14 that this removes this README.chromium so you may want it open in code search 15 or remember the steps up until 2. where you get it back. 16 171. Pull the release from https://github.com/google/protobuf/releases (Source 18 code .tar.gz). Extract to `third_party/protobuf` (without version suffix), 19 i.e. you may `$ mv protobuf-21.12 third_party/protobuf` or similar. 20 212. Add back chromium's non-upstream files (from this directory): 22 23 $ git checkout main BUILD.gn DEPS DIR_METADATA OWNERS README.chromium \ 24 gen_chromium_file_lists.py gen_extra_chromium_files.py proto_library.gni \ 25 patches/ 26 27 Note: Ideally we wouldn't do this but instead have a 28 `third_party/protobuf/src/` subdirectory where upstream code goes directly 29 into. That will require updating things outside this repository (at least 30 `third_party/webrtc`) to agree on where the source code lives. I wouldn't be 31 surprised if that also required Google-internal uses of protobuf from webrtc 32 to also agree on `third_party/protobuf/src` so maybe this'll never happen? 33 Here be dragons. Probably. 34 35 Probably worth making a local `git commit` here before applying patches (next 36 step). 37 383. Apply patches in patches/ (see the description below): 39 40 $ for patch in patches/*; do patch -s -p1 < $patch; done 41 42 When a patch fails, it's probably best to check out previous state (from 3) 43 for any changes not inside the patches/ folder, fix the patch and then try to 44 reapply everything. It's not a good idea to fix the error, forget to generate 45 a patch and leave patches/ in a bad state. 46 47 If a patch no longer applies it may not be needed, if so remove it and its 48 description in this README.chromium file. Similarly try to eventually 49 upstream any patches out of band from unblocking the roll so we can 50 eventually remove it here as well. 51 52 Probably worth committing locally here too. 53 544. Generate `proto_sources.gni` using the script `./gen_chromium_file_lists.py`. 55 56 Add and commit `proto_sources.gni`, 57 585. Generate descriptor_pb2.py using the script "gen_extra_chromium_files.py" in 59 the same directory as this file. 60 61 $ ./gen_extra_chromium_files.py -C ../../out/Default 62 63 Add and commit `python/google/protobuf/descriptor_pb2.py` and 64 `python/google/protobuf/compiler/plugin_pb2.py` as well. 65 666. Update Version, CPEPrefix and Revision in this file. 67 68Notes (hints) for the next roll(er): 69 70* This is probably harder to roll than most things you've rolled in the past. 71 Good luck! Also please reach out for help proactively. The current roll as of 72 writing this required help from multiple very-senior people to even figure out 73 what was broken in the first place. `protobuf` rolling is not in a good place. 74* Update README.chromium as you go so you don't forget (after step 2. above, so 75 you don't nuke those changes). 76* Rolling one version of protobuf likely introduces a large amount of changes. 77 We should possibly figure out if it's reasonable to stay close to tip of tree 78 and roll frequently rather than use released versions. Sharp corners here 79 likely include that the version is encoded in protobuf output and I (pbos@) 80 don't know if that version tag changes immediately on format incompatibilities 81 or get bumped every time something format incompatible is introduced. 82* After v3.20 protobuf-javascript/ split out from upstream protobuf/. You may 83 need to be aware of both repositories. Also upstream protobuf/ seems to move 84 faster / out of sync with protobuf-javascript/. Hopefully that doesn't cause 85 format incompatibilities in the future (protobuf/ supports something we need 86 that protobuf-javascript/ doesn't). 87* Any argument changes to `protoc_wrapper` in `third_party/protobuf/BUILD.gn` 88 may require updating `build/config/siso/protoc_wrapper.star` as well. 89* We've seen instances where instead of rolling protobuf a local copy of a new 90 built-in type has been checked in. When that happens you may see ODR 91 violations as your built-in type is defined in multiple files. If that happens 92 you may want to move the copy that should never have been checked in into its 93 own namespace, complete the roll and then cleanup the local copy. 94* third_party/mediapipe checks in a weird modified copy of 95 `third_party/protobuf/src/google/protobuf/any.pb.cc` called `any_lite.pb.cc`. 96 This precompiled protobuf will get invalidated on any protobuf version bumps 97 and you'll need to either painstakingly generated a new one (do a diff of the 98 checked-in `any_lite.pb.{cc,h}` and current `any.pb.{cc,h}` to aid you in the 99 process. 100 WARNING: This item is a lot larger than it looks. We should really 101 either fix mediapipe upstream to build with `MEDIAPIPE_PROTO_LITE` and 102 `MEDIAPIPE_PROTO_THIRD_PARTY` both set, but it looks like that configuration 103 has bit rotted. 104 105Note about vpython: 106 107Some Python scripts end up mixing protoc output from this copy of protobuf with 108the google.protobuf module from vpython's protobuf. If Python scripts break due 109to the vpython copy of protobuf, you may need to update the version in 110//.vpython3. See https://crbug.com/1320047. 111 112Description of the patches: 113 114- 0004-fix-shared-library-exports.patch 115 116 This patch allows exporting protobuf symbols in Linux .so libraries, so 117 that protobuf can be built as a component (see http://crrev.com/179806). 118 119- 0008-uninline_get_empty_string.patch 120- 0010-uninline-generated-code.patch 121 122 These patches uninline some functions, resulting in a significant reduction 123 (somewhere between 500 KB and 1 MB) of binary size. 124 125- 0021-Fix-protobuf-s-library-.gitinore-file.patch 126 127 Un-ignores python/google/protobuf/descriptor_pb2.py and 128 python/google/protobuf/compiler/plugin_pb2.py 129 130- 0022-Allow-deprecated-fields.patch 131 132 Allows deprecated fields to be used without extra C++ compiler warnings. 133 134- 0026-remove-sprintf.patch 135 136 Imports 137 https://github.com/protocolbuffers/protobuf/commit/c0fc2e881bc36aafb0bf539bf41889611370f60c 138 to remove use of sprintf. 139 140- 0027-no-noreturn.patch 141 142 Removes an instance of [[noreturn]]. The attribute is correct, but the way 143 protobuf's GOOGLE_LOG(FATAL) is defined, the compiler can't see this and it 144 trips -Winvalid-noreturn. See https://github.com/protocolbuffers/protobuf/issues/9817 145 146- 0029-make-initializers-optimizable.patch 147 148 Makes the InitProtobufDefaults() static initializer optimizable by Clang when 149 built with libc++. It patches out the OnShutdownDestroyString call, which we 150 do not need, and removes the thread-safe initialization. Thread safety is only 151 needed if a static initializer spawns a thread which then calls 152 InitProtobufDefaults() without synchronizing with the start of main(). 153 (Anything which happens after main() starts can rely on the initializer 154 running.) 155 156- 0030-workaround-window-constinit.patch 157 158 Disables PROTOBUF_CONSTINIT in generated code in Windows shared library 159 builds. Protobuf's default instances take pointers to a dllimport variable, 160 fixed_address_empty_string. This is not constinit on Windows. This is a bug in 161 protobuf as the default instance was intended to be constant-initialized. But 162 the components build in Chromium is a developer configuration, so we tolerate 163 an initializer as long as the build works, until protobuf has a proper fix. 164 165 See https://github.com/protocolbuffers/protobuf/issues/10159. 166 167- 0031-workaround-cfi-unrelated-cast.patch 168 169 A workaround for Clang's Control Flow Integrity check for casting pointers to 170 memory that his not yet initialized to be of that type for empty arrays, does 171 not work, and still fails. This patch removes the workaround and instead 172 disables cfi-unrelated-cast for the affected methods and simplifies them. 173 174 See https://github.com/protocolbuffers/protobuf/issues/10186. 175 See https://bugs.chromium.org/p/chromium/issues/detail?id=1294200#c26. 176 177- 0032-cxx20.patch 178 179 Fixes necessary to build in --std=c++20 mode. 180 181 Imports https://critique.corp.google.com/cl/451177197 (a portion of 182 https://github.com/protocolbuffers/protobuf/commit/6dd8af4ecfa7987bddb309862932886b84f1e4ef 183 ). 184 185- 0033-no-enum-conversion-warn.patch 186 187 Avoid hitting the clang error -Wenum-constexpr-conversion by specifying width 188 of the enum. 189 190 Imports https://critique.corp.google.com/cl/466986872. 191 192- 0034-change-macro-to-avoid-pedantic-warning.patch 193 194 Avoid hitting -Wextra-semi. 195 196 Imports (rebased): 197 https://github.com/protocolbuffers/protobuf/commit/def602dd07b7eae1cac6823705975317b5607fc3 198 199- 0035-fix-shared-library-constants 200 201 Fixes for component build when building with MediaPipe. AnyMetadata::PackFrom 202 is called which require string constants to be exported. 203 204- 0036-fix-undefined-memcpy-call.patch 205 206 Fixes an undefined call to memcpy which may be passed NULL, 0. NULL, 0 is 207 forbidden in C, due to a mistake in the C language specification. This 208 partially integrates 209 https://github.com/protocolbuffers/protobuf/commit/b2e1d7d8a1b1959c782595cd815c22fb690ac8e2 210 from upstream and may be removed when protobuf is updated past that commit. 211 212- 0037-fix-wextra-semi.patch 213 214 Removes an extra semicolon, fixed in upstream 215 https://github.com/protocolbuffers/protobuf/commit/3e6f8ee152db71ae49cc95649feab50d2faacdfa 216 217- 0038-silence-implicit-precision-loss.patch 218 219 Silences -Wshorten-64-to-32 for now. Fixed in upstream 220 https://github.com/protocolbuffers/protobuf/commit/d85c9944c55fb38f4eae149979a0f680ea125ecb 221 222- 0039-generate-mediapipe-anylite.patch 223 224 Workaround to allow generating protobuf code compatible with 225 third_party/mediapipe. This replaces checked-in precompiled .pb.cc files. See 226 https://crbug.com/332939935, which when/if resolved should make this patch 227 obsolete. 228 229- 0040-fix-ndk-major.patch 230 231 Fix Android compile error that affects webrtc. Fixed in upstream 232 https://github.com/protocolbuffers/protobuf/commit/9471a882256eaf01027524e8b6c15cbb5e42bef6 233