add DDL propagation wait for SharedMergeTree migrations#86
Merged
alvarogar4 merged 7 commits intomainfrom Mar 19, 2026
Merged
Conversation
alvarogar4
force-pushed
the
alvaro/num-6759-obsession-sharedmergetree-migration-race-condition
branch
from
fc4d2c6 to
6217e17
Compare
alvarogar4
changed the title
On ClickHouse Cloud (SharedMergeTree), DDL statements acknowledged on one node may not yet be visible on other nodes. This caused CREATE MATERIALIZED VIEW to fail when it referenced a table created moments earlier in the same migration. After each DDL statement, the migrate command now polls system.tables or system.columns to confirm the object has propagated before executing the next statement. - Extract polling utilities from e2e-testkit into production module packages/clickhouse/src/ddl-propagation.ts - Export waitForDDLPropagation dispatcher from @chkit/clickhouse - Add E2E test: table + MV in single migration on ClickHouse Cloud Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
alvarogar4
force-pushed
the
alvaro/num-6759-obsession-sharedmergetree-migration-race-condition
branch
from
6217e17 to
036cc7d
Compare
KeKs0r
reviewed
Mar 18, 2026
| ): Promise<void> { | ||
| const { timeoutMs = DEFAULT_TIMEOUT_MS, intervalMs = DEFAULT_INTERVAL_MS } = options | ||
| const deadline = Date.now() + timeoutMs | ||
| while (Date.now() < deadline) { |
Contributor
There was a problem hiding this comment.
maybe directly use p-retry it has exponential backoff and all configurations easy in there. (and cleanaer syntax, imho)
Replace manual while-loop polling with p-retry for cleaner syntax and built-in exponential backoff support. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
alvarogar4
force-pushed
the
alvaro/num-6759-obsession-sharedmergetree-migration-race-condition
branch
from
0d918b4 to
7d15e6f
Compare
…ing hoisted node_modules The emitZod test symlinked packages/plugin-codegen/node_modules into a temp dir, but zod may be hoisted to the workspace root depending on the lockfile format and Bun version. Use require.resolve to find the actual zod location and symlink only that package. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…uild
Zod v4 hoisted to root conflicts with Astro/Starlight which requires
zod v3. The codegen only emits `import { z } from 'zod'` — the devDep
is only used in tests and works fine with v3.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
alvarogar4
deleted the
alvaro/num-6759-obsession-sharedmergetree-migration-race-condition
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
system.tables/system.columnsto confirm propagation before proceedinge2e-testkit.tsinto a new production module@chkit/clickhouse(ddl-propagation.ts)Test plan
bun run typecheckpasses for@chkit/clickhouseandchkitbun run buildpasses for all packages🤖 Generated with Claude Code