Add support for spanner table options in import/export by n-d-joshi · Pull Request #3552 · GoogleCloudPlatform/DataflowTemplates

n-d-joshi · 2026-03-25T05:24:57Z

This PR adds:

Support for Import/Export of table options.
Adds an allowlist for the table options which currently only has fulltext_dictionary_table.

Syncing branch

gemini-code-assist · 2026-03-25T05:25:23Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the Spanner import/export functionality by enabling the handling of table options. This allows for a more complete and accurate representation of Spanner table definitions during data migration and schema synchronization processes, ensuring that critical table-level configurations are preserved.

Highlights

Spanner Table Options Support: Implemented comprehensive support for importing and exporting Spanner table options, ensuring that table-level configurations are preserved during data migration and schema synchronization.
Table Option Allowlist: Introduced a new TableOptionAllowlist to explicitly define and manage supported table options, with fulltext_dictionary_table as the initial entry.
DDL Conversion Enhancements: Updated AvroSchemaToDdlConverter and DdlToAvroSchemaConverter to correctly parse and generate DDL with table options from and to Avro schemas.
Information Schema Integration: Enhanced the InformationSchemaScanner to query and extract table options directly from Spanner's information_schema for both Google Standard SQL and PostgreSQL dialects.
DDL Representation Update: Modified the internal Table DDL representation to include tableOptions and updated the DDL pretty-printing logic to correctly incorporate these options.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

n-d-joshi · 2026-03-25T06:03:32Z

R: @darshan-sj

darshan-sj

The changes look good overall, I have left a few comments.

Please address these as well:

Please build the Import and export template with your code changes and run an export from Spanner database with your table option. Import the exported data into a new and empty Spanner database and make sure that the table option is created properly in the new database. Please document the results of the test here.
Please add integration test cases in ExportPipelineIT and ImportPipelineIT.

darshan-sj · 2026-03-26T05:17:36Z

v1/src/main/java/com/google/cloud/teleport/spanner/ddl/InformationSchemaScanner.java

+        String quoteChar =
+            dialect == Dialect.POSTGRESQL ? POSTGRESQL_LITERAL_QUOTE : GSQL_LITERAL_QUOTE;
+        options.add(
+            optionName + "=" + quoteChar + OPTION_STRING_ESCAPER.escape(optionValue) + quoteChar);


Why is NameUtils' quoteIdentifier or quoteLiteral not used here? - https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/main/v1/src/main/java/com/google/cloud/teleport/spanner/common/NameUtils.java

darshan-sj · 2026-03-26T05:23:09Z

v1/src/main/java/com/google/cloud/teleport/spanner/ddl/InformationSchemaScanner.java

+  @VisibleForTesting
+  Statement listTableOptionsSQL() {
+    switch (dialect) {
+      case GOOGLE_STANDARD_SQL:


Are all these columns and tables available in every production databases? Please confirm.

Please get these 2 queries reviewed and LGTMed by one of your team members.

darshan-sj · 2026-03-26T05:26:47Z

v1/src/main/java/com/google/cloud/teleport/spanner/ddl/Table.java

+    if (!tableOptions().isEmpty()) {
+      appendable.append("\nWITH (").append(String.join(", ", tableOptions())).append(")");
+    }


Is the relative positioning of "with" clause correct in this CREATE TABLE statement?

Also, please add a test case in DdlTest.java covering all the clauses here including Table options.

darshan-sj · 2026-03-26T05:28:07Z

v1/src/main/java/com/google/cloud/teleport/spanner/ddl/Table.java

+    if (!tableOptions().isEmpty()) {
+      appendable.append(",\nOPTIONS (").append(String.join(", ", tableOptions())).append(")");
+    }


Same comment as PG. I hope the relative ordering of OPTIONS clause is correct here. And Please add test case in DdlTest.java covering all the clauses currently present here.

darshan-sj · 2026-03-26T05:29:50Z

v1/src/main/java/com/google/cloud/teleport/spanner/AvroUtil.java

  public static final String SPANNER_PARENT = "spannerParent";
  public static final String SPANNER_INTERLEAVE_TYPE = "spannerInterleaveType";
  public static final String SPANNER_PRIMARY_KEY = "spannerPrimaryKey";
+  public static final String SPANNER_TABLE_OPTIONS = "spannerTableOptions_";


Other numbered properties are singular in this file.

Please change to Singular

SPANNER_TABLE_OPTION = "spannerTableOption_"

darshan-sj · 2026-03-26T05:32:05Z

v1/src/test/java/com/google/cloud/teleport/spanner/AvroSchemaToDdlConverterTest.java

                + " ) PRIMARY KEY (`id` ASC)"));
  }

+  @Test


I see some escaping is being done for the Table options in InformationSchemaScanner class. Please add test cases where escaping is exercised and escaping is not exercised.

codecov · 2026-03-26T05:55:32Z

Codecov Report

❌ Patch coverage is 81.81818% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 52.22%. Comparing base (b8c4776) to head (fb34a87).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
...oud/teleport/spanner/ddl/TableOptionAllowlist.java	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #3552      +/-   ##
============================================
+ Coverage     52.19%   52.22%   +0.02%     
+ Complexity     6048     5644     -404     
============================================
  Files          1040     1041       +1     
  Lines         62985    62995      +10     
  Branches       6901     6904       +3     
============================================
+ Hits          32877    32897      +20     
+ Misses        27882    27874       -8     
+ Partials       2226     2224       -2

Components	Coverage Δ
spanner-templates	`72.24% <81.81%> (+0.04%)`	⬆️
spanner-import-export	`69.06% <81.81%> (+0.15%)`	⬆️
spanner-live-forward-migration	`80.47% <ø> (-0.02%)`	⬇️
spanner-live-reverse-replication	`77.87% <ø> (-0.02%)`	⬇️
spanner-bulk-migration	`89.18% <ø> (-0.01%)`	⬇️
gcs-spanner-dv	`85.32% <ø> (-0.03%)`	⬇️

Files with missing lines	Coverage Δ
...oud/teleport/spanner/AvroSchemaToDdlConverter.java	`86.80% <100.00%> (+0.02%)`	⬆️
...va/com/google/cloud/teleport/spanner/AvroUtil.java	`93.75% <ø> (ø)`
...oud/teleport/spanner/DdlToAvroSchemaConverter.java	`97.53% <100.00%> (+0.01%)`	⬆️
...a/com/google/cloud/teleport/spanner/ddl/Table.java	`89.76% <100.00%> (+0.41%)`	⬆️
...oud/teleport/spanner/ddl/TableOptionAllowlist.java	`0.00% <0.00%> (ø)`

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

n-d-joshi added 4 commits January 26, 2025 22:40

Merge pull request #1 from GoogleCloudPlatform/main

f3298e6

Syncing branch

Merge branch 'GoogleCloudPlatform:main' into main

1fbdbb8

Add support for spanner table options in import/export.

5f67b5e

Merge branch 'GoogleCloudPlatform:main' into main

fb34a87

n-d-joshi requested a review from a team as a code owner March 25, 2026 05:24

n-d-joshi requested review from manitgupta and rohitwali March 25, 2026 05:24

pull-request-size bot added the size/L label Mar 25, 2026

darshan-sj reviewed Mar 26, 2026

View reviewed changes

darshan-sj added the improvement Making existing code better label Mar 26, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for spanner table options in import/export#3552

Add support for spanner table options in import/export#3552
n-d-joshi wants to merge 4 commits intoGoogleCloudPlatform:mainfrom
n-d-joshi:table-options

n-d-joshi commented Mar 25, 2026

Uh oh!

gemini-code-assist bot commented Mar 25, 2026

Uh oh!

n-d-joshi commented Mar 25, 2026

Uh oh!

darshan-sj left a comment

Uh oh!

darshan-sj Mar 26, 2026

Uh oh!

darshan-sj Mar 26, 2026

Uh oh!

darshan-sj Mar 26, 2026

Uh oh!

darshan-sj Mar 26, 2026

Uh oh!

darshan-sj Mar 26, 2026

Uh oh!

darshan-sj Mar 26, 2026

Uh oh!

codecov bot commented Mar 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

n-d-joshi commented Mar 25, 2026

Uh oh!

gemini-code-assist bot commented Mar 25, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

n-d-joshi commented Mar 25, 2026

Uh oh!

darshan-sj left a comment

Choose a reason for hiding this comment

Uh oh!

darshan-sj Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

darshan-sj Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

darshan-sj Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

darshan-sj Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

darshan-sj Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

darshan-sj Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Mar 26, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants