Support Spanner Emulator Host for SpannerChangeStreamsToPubSub Templates#3545
Support Spanner Emulator Host for SpannerChangeStreamsToPubSub Templates#3545hunsche wants to merge 4 commits intoGoogleCloudPlatform:mainfrom
Conversation
This CL updates SpannerConfig initialization in the SpannerChangeStreamsToPubSub template to explicitly respect the SPANNER_EMULATOR_HOST environment variable, restoring standard Google Cloud client behavior. The SPANNER_EMULATOR_HOST variable is the official standard across all Google Cloud client libraries (Go, Java, Python, Node) for discovering the emulator. By default, the underlying Spanner client automatically reads and uses this variable to establish plaintext connections natively. However, Apache Beam's SpannerIO inadvertently overrides this default fallback behavior. To fix this, this CL checks for the presence of SPANNER_EMULATOR_HOST. If present, it maps it directly to .withEmulatorHost() before falling back to the template's runtime options. This aligns the Beam pipeline's local development experience perfectly with the conventions of the broader Google Cloud ecosystem, eliminating the need for workarounds. TEST=Verified via local build
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
Codecov Report❌ Patch coverage is
❌ Your patch check has failed because the patch coverage (0.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #3545 +/- ##
============================================
- Coverage 52.25% 52.21% -0.04%
+ Complexity 6055 5641 -414
============================================
Files 1040 1040
Lines 62956 62990 +34
Branches 6893 6902 +9
============================================
- Hits 32895 32888 -7
- Misses 27843 27877 +34
- Partials 2218 2225 +7
🚀 New features to boost your workflow:
|
...cloud/src/main/java/com/google/cloud/teleport/v2/templates/SpannerChangeStreamsToPubSub.java
Show resolved
Hide resolved
.../src/main/java/com/google/cloud/teleport/v2/options/SpannerChangeStreamsToPubSubOptions.java
Show resolved
Hide resolved
45c8170 to
43d53e7
Compare
43d53e7 to
96a2da9
Compare
This CL updates SpannerConfig initialization in the SpannerChangeStreamsToPubSub template to explicitly respect the SPANNER_EMULATOR_HOST environment variable, restoring standard Google Cloud client behavior.
The SPANNER_EMULATOR_HOST variable is the official standard across all Google Cloud client libraries (Go, Java, Python, Node) for discovering the emulator. By default, the underlying Spanner client automatically reads and uses this variable to establish plaintext connections natively.
However, Apache Beam's SpannerIO inadvertently overrides this default fallback behavior. To fix this, this CL checks for the presence of SPANNER_EMULATOR_HOST. If present, it maps it directly to .withEmulatorHost() before falling back to the template's runtime options.
This aligns the Beam pipeline's local development experience perfectly with the conventions of the broader Google Cloud ecosystem, eliminating the need for workarounds.
TEST=Verified via local build