1# Development Workflow 2 3## Test Running 4 5### Unit Tests 6 7To run the unit tests in `gapic-generator-java` submodule, first build all 8modules with `mvn -pl '!gapic-generator-java' install -DskipTests` at the root 9directory, 10then `cd` into `gapic-generator-java` submodule for the following commands: 11 12- Run all unit tests: 13 14 ```sh 15 # In gapic-generator-java submodule 16 mvn test 17 ``` 18 19- Run a single or multiple unit tests: 20 21 ```sh 22 # In gapic-generator-java submodule 23 mvn test -Dtest=JavaCodeGeneratorTest 24 25 mvn test "-Dtest=Basic*, !%regex[.*.Unstable.*], !%regex[.*.MyTest.class#one.*|two.*], %regex[#fast.*|slow.*]" 26 ``` 27 28- Update all unit test golden files: 29 30 ```sh 31 # In gapic-generator-java submodule 32 mvn test -DupdateUnitGoldens 33 ``` 34 35- Update a single unit test golden file, for example `JavaCodeGeneratorTest.java`: 36 37 ```sh 38 # In gapic-generator-java submodule 39 mvn test -DupdateUnitGoldens -Dtest=JavaCodeGeneratorTest 40 ``` 41 42Note that `mvn -pl '!gapic-generator-java' install -DskipTests` 43at the root directory is needed for `mvn test` commands, 44because the gapic-generator-java submodule depends on the "test jars" of 45gax-java. The test jars are absent until Maven's "package" phase, which is later 46than the "test" phase. 47 48### Integration Tests 49 50To run integration test for gapic-generator-java, run this Bazel command in the 51root of the repository (where you have WORKSPACE file for Bazel.) 52 53```sh 54# In the repository root directory 55bazelisk test //... # integration tests 56``` 57 58 59- Run a single integration test for API like `Redis`, it generates Java source 60 code using the Java microgenerator and compares them with the goldens files 61 in `test/integration/goldens/redis`. 62 63 ```sh 64 # In the repository root directory 65 bazelisk test //test/integration:redis 66 ``` 67 68- Update integration test golden files, for example `Redis`. This clobbers all the 69 files in `test/integration/goldens/redis`. 70 71 ```sh 72 # In the repository root directory 73 bazelisk run //test/integration:update_redis 74 ``` 75 76## Running the Plugin under googleapis with local gapic-generator-java 77 78For running the Plugin with showcase protos and local gapic-generator-java, see 79[Showcase Integration Testing](../showcase/README.md). 80 81To generate a production GAPIC API: 82 831. Clone [googleapis](https://github.com/googleapis/googleapis). 84 852. Modify `googleapis/WORKSPACE` to point to local gapic-generator-java 86 87 Normally, googleapis's build pulls in gapic-generator-java from Maven Central. 88 For a local run, we first need to build a local SNAPSHOT jar of the generator. Then we point googleapis to 89 both the local SNAPSHOT jar and the local copy of the generator. 90 91 Replace the following section in googleapis 92 ``` 93 _gapic_generator_java_version = "2.13.0" 94 95 maven_install( 96 artifacts = [ 97 "com.google.api:gapic-generator-java:" + _gapic_generator_java_version, 98 ], 99 #Update this False for local development 100 fail_on_missing_checksum = True, 101 repositories = [ 102 "m2Local", 103 "https://repo.maven.apache.org/maven2/", 104 ] 105 ) 106 107 http_archive( 108 name = "gapic_generator_java", 109 strip_prefix = "gapic-generator-java-%s" % _gapic_generator_java_version, 110 urls = ["https://github.com/googleapis/gapic-generator-java/archive/v%s.zip" % _gapic_generator_java_version], 111 ) 112 113 # gax-java is part of gapic-generator-java repository 114 http_archive( 115 name = "com_google_api_gax_java", 116 strip_prefix = "gapic-generator-java-%s/gax-java" % _gapic_generator_java_version, 117 urls = ["https://github.com/googleapis/gapic-generator-java/archive/v%s.zip" % _gapic_generator_java_version], 118 ) 119 ``` 120 121 to 122 ``` 123 _gapic_generator_java_version = "2.15.4-SNAPSHOT" 124 125 maven_install( 126 artifacts = [ 127 "com.google.api:gapic-generator-java:" + _gapic_generator_java_version, 128 ], 129 #Update this False for local development 130 fail_on_missing_checksum = False, 131 repositories = [ 132 "m2Local", 133 "https://repo.maven.apache.org/maven2/", 134 ] 135 ) 136 137 local_repository( 138 name = "gapic_generator_java", 139 path = "/absolute/path/to/your/local/gapic-generator-java", 140 ) 141 142 # gax-java is part of gapic-generator-java repository 143 local_repository( 144 name = "com_google_api_gax_java", 145 path = "/absolute/path/to/your/local/gapic-generator-java/gax-java", 146 ) 147 ``` 148 149 Note: At the time of writing, the gapic-generator version was `2.13.0`. Update the version to the latest version in the pom.xml 150 1513. Build the new target. 152 153 You can generate any client library based on the protos within googleapis. 154 You just need the name of the service within the `java_gapic_assembly_gradle_pkg` 155 rules within the service's `BUILD.bazel` file. 156 For instance, to run your local generator on the `speech`'s v2 service, you can 157 run: 158 159 ``` 160 bazelisk build //google/cloud/speech/v2:google-cloud-speech-v2-java 161 ``` 162 163 Note: If you are running into bazel build issues, you can try to remove gapic-generator-java cached in your local m2 164 Try running this command: 165 ``` 166 rm -rf ~/.m2/repository/com/google/api/ 167 ``` 168 and then rebuild gapic-generator-java (`mvn clean install`). 169 170## Debugging the gapic-generator-java running on protobuf compiler 171 1721. In [googleapis](https://github.com/googleapis/googleapis) root directory, run the following commands: 173 1. Set `JVM_DEBUG_PORT` environment variable. 174 ```shell 175 export JVM_DEBUG_PORT=5005 176 ``` 177 The protobuf compiler runs the `protoc-gen-java_gapic` shell script (a wrapper of java command with an option to specify JVM debugging). 178 We set the environment variable `JVM_DEBUG_PORT` to enable the debug option. 179 180 2. Run the command to build a client library, e.g., `java-monitoring`, with `--subcommands` flag to output commands spawned by `bazel build`. 181 ```shell 182 bazel build --subcommands //google/monitoring/v3:google-cloud-monitoring-v3-java 183 ``` 184 3. In the output, find the command of protobuf compiler invocation with gapic-generator-java plugin (the `darwin_arm64-opt-exec-2B5CBBC6` part may be different on your environment). 185 186 Note that different client libraries have different proto files. 187 ```shell 188 bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/com_google_protobuf/protoc --experimental_allow_proto3_optional '--plugin=protoc-gen-java_gapic=bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/gapic_generator_java/protoc-gen-java_gapic' '--java_gapic_out=metadata:bazel-out/darwin_arm64-fastbuild/bin/google/monitoring/v3/monitoring_java_gapic_srcjar_raw.srcjar.zip' '--java_gapic_opt=transport=grpc,rest-numeric-enums,grpc-service-config=google/monitoring/v3/monitoring_grpc_service_config.json,gapic-config=google/monitoring/v3/monitoring_gapic.yaml,api-service-config=google/monitoring/v3/monitoring.yaml' '-Igoogle/monitoring/v3/alert.proto=google/monitoring/v3/alert.proto' '-Igoogle/monitoring/v3/alert_service.proto=google/monitoring/v3/alert_service.proto' '-Igoogle/monitoring/v3/common.proto=google/monitoring/v3/common.proto' '-Igoogle/monitoring/v3/dropped_labels.proto=google/monitoring/v3/dropped_labels.proto' '-Igoogle/monitoring/v3/group.proto=google/monitoring/v3/group.proto' '-Igoogle/monitoring/v3/group_service.proto=google/monitoring/v3/group_service.proto' '-Igoogle/monitoring/v3/metric.proto=google/monitoring/v3/metric.proto' '-Igoogle/monitoring/v3/metric_service.proto=google/monitoring/v3/metric_service.proto' '-Igoogle/monitoring/v3/mutation_record.proto=google/monitoring/v3/mutation_record.proto' '-Igoogle/monitoring/v3/notification.proto=google/monitoring/v3/notification.proto' '-Igoogle/monitoring/v3/notification_service.proto=google/monitoring/v3/notification_service.proto' '-Igoogle/monitoring/v3/query_service.proto=google/monitoring/v3/query_service.proto' '-Igoogle/monitoring/v3/service.proto=google/monitoring/v3/service.proto' '-Igoogle/monitoring/v3/service_service.proto=google/monitoring/v3/service_service.proto' '-Igoogle/monitoring/v3/snooze.proto=google/monitoring/v3/snooze.proto' '-Igoogle/monitoring/v3/snooze_service.proto=google/monitoring/v3/snooze_service.proto' '-Igoogle/monitoring/v3/span_context.proto=google/monitoring/v3/span_context.proto' '-Igoogle/monitoring/v3/uptime.proto=google/monitoring/v3/uptime.proto' '-Igoogle/monitoring/v3/uptime_service.proto=google/monitoring/v3/uptime_service.proto' '-Igoogle/api/annotations.proto=google/api/annotations.proto' '-Igoogle/api/http.proto=google/api/http.proto' '-Igoogle/protobuf/descriptor.proto=bazel-out/darwin_arm64-fastbuild/bin/external/com_google_protobuf/_virtual_imports/descriptor_proto/google/protobuf/descriptor.proto' '-Igoogle/api/client.proto=google/api/client.proto' '-Igoogle/api/launch_stage.proto=google/api/launch_stage.proto' '-Igoogle/protobuf/duration.proto=bazel-out/darwin_arm64-fastbuild/bin/external/com_google_protobuf/_virtual_imports/duration_proto/google/protobuf/duration.proto' '-Igoogle/api/distribution.proto=google/api/distribution.proto' '-Igoogle/protobuf/any.proto=bazel-out/darwin_arm64-fastbuild/bin/external/com_google_protobuf/_virtual_imports/any_proto/google/protobuf/any.proto' '-Igoogle/protobuf/timestamp.proto=bazel-out/darwin_arm64-fastbuild/bin/external/com_google_protobuf/_virtual_imports/timestamp_proto/google/protobuf/timestamp.proto' '-Igoogle/api/field_behavior.proto=google/api/field_behavior.proto' '-Igoogle/api/label.proto=google/api/label.proto' '-Igoogle/api/metric.proto=google/api/metric.proto' '-Igoogle/api/monitored_resource.proto=google/api/monitored_resource.proto' '-Igoogle/protobuf/struct.proto=bazel-out/darwin_arm64-fastbuild/bin/external/com_google_protobuf/_virtual_imports/struct_proto/google/protobuf/struct.proto' '-Igoogle/api/resource.proto=google/api/resource.proto' '-Igoogle/rpc/status.proto=google/rpc/status.proto' '-Igoogle/type/calendar_period.proto=google/type/calendar_period.proto' '-Igoogle/protobuf/empty.proto=bazel-out/darwin_arm64-fastbuild/bin/external/com_google_protobuf/_virtual_imports/empty_proto/google/protobuf/empty.proto' '-Igoogle/protobuf/field_mask.proto=bazel-out/darwin_arm64-fastbuild/bin/external/com_google_protobuf/_virtual_imports/field_mask_proto/google/protobuf/field_mask.proto' '-Igoogle/protobuf/wrappers.proto=bazel-out/darwin_arm64-fastbuild/bin/external/com_google_protobuf/_virtual_imports/wrappers_proto/google/protobuf/wrappers.proto' '-Igoogle/cloud/common_resources.proto=google/cloud/common_resources.proto' google/monitoring/v3/alert.proto google/monitoring/v3/alert_service.proto google/monitoring/v3/common.proto google/monitoring/v3/dropped_labels.proto google/monitoring/v3/group.proto google/monitoring/v3/group_service.proto google/monitoring/v3/metric.proto google/monitoring/v3/metric_service.proto google/monitoring/v3/mutation_record.proto google/monitoring/v3/notification.proto google/monitoring/v3/notification_service.proto google/monitoring/v3/query_service.proto google/monitoring/v3/service.proto google/monitoring/v3/service_service.proto google/monitoring/v3/snooze.proto google/monitoring/v3/snooze_service.proto google/monitoring/v3/span_context.proto google/monitoring/v3/uptime.proto google/monitoring/v3/uptime_service.proto google/cloud/common_resources.proto 189 ``` 190 4. Run the protoc command. 191 192 It shows nothing because it's waiting for a debugger to attach. 193 1942. In the IntelliJ that has gapic-generator-java, add [Remote JVM debug configuration](https://www.jetbrains.com/help/idea/tutorial-remote-debug.html). 195 1963. Set a breakpoint, e.g., in the 1st line in the main method in [Main](src/main/java/com/google/api/generator/Main.java) (this line always get called). 197 1984. Start the Remote JVM Debug configuration and you can debug the generator in Intellij. 199  200 201## FAQ 202 203### Error in workspace: workspace() got unexpected keyword argument 'managed_directories' 204 205Full Error: 206 207``` 208ERROR: Traceback (most recent call last): 209 File "/home/alicejli/googleapis/WORKSPACE", line 1, column 10, in <toplevel> 210 workspace( 211Error in workspace: workspace() got unexpected keyword argument 'managed_directories' 212ERROR: Error computing the main repository mapping: Encountered error while reading extension file 'tools/build_defs/repo/http.bzl': no such package '@bazel_tools//tools/build_defs/repo': error loading package 'external': Could not load //external package 213``` 214 215You may be using the latest version of bazel which this project does not support yet. Try installing bazelisk to force 216bazel to use the version specified in `.bazeliskrc` 217 218
