sequin-red-black (srb)

NOT-RELEASED

Zero-downtime red-black deployments for Sequin CDC sinks backed by OpenSearch.

SRB is a pure YAML transformer. It takes a base Sequin config with unsuffixed index names and produces a suffixed config ready to apply. It does not call any external APIs in its core commands — that's the caller's job.

How it works

You write Sequin configs with base index names (jobs, clients). SRB adds red/black suffixes (jobs_red, clients_black) and manages the dual-write pattern for zero-downtime reindexing.

STEADY STATE          DUAL-WRITE (deploy)        AFTER PROMOTE
jobs_red (alias)  →   jobs_red (alias)        →  jobs_black (alias)
                      jobs_black (backfill)       jobs_red dropped

State is tracked via annotations on Sequin sinks (red-black/role, red-black/base) and verified against OpenSearch aliases as ground truth.

Install

Download a prebuilt binary from GitHub Releases:

# Linux x64
gh release download --repo meetsmore/sequin-red-black --pattern "srb-linux-x64"
chmod +x srb-linux-x64 && mv srb-linux-x64 /usr/local/bin/srb

# macOS arm64
gh release download --repo meetsmore/sequin-red-black --pattern "srb-darwin-arm64"
chmod +x srb-darwin-arm64 && mv srb-darwin-arm64 /usr/local/bin/srb

Or build from source (requires Bun):

bun install
bun run build     # produces dist/srb

Commands

SRB separates offline commands (pure YAML transforms, no network) from online commands (connect to Sequin/OpenSearch).

Offline Commands

srb offline transform

Apply red-black suffixes to a base config.

# First deploy (defaults to red)
srb offline transform base.yaml > suffixed.yaml

# With known state
srb offline transform base.yaml --state state.json > suffixed.yaml

# With backfill (dual-write)
srb offline transform base.yaml --state state.json --backfill jobs > suffixed.yaml

Input: base.yaml — a Sequin config with unsuffixed names:

sinks:
  - name: jobs_sink
    destination:
      type: elasticsearch
      index_name: jobs            # no suffix
    transform: jobs-transform

Output: suffixed YAML to stdout:

sinks:
  - name: jobs_red_sink
    destination:
      type: elasticsearch
      index_name: jobs_red        # suffixed
    transform: jobs-red-transform
    annotations:
      red-black/role: active
      red-black/base: jobs

Options:

Flag	Description
--state <file>	JSON file: {"jobs":"red","clients":"black"}. Defaults to all-red if omitted.
--backfill idx1,idx2	Indexes that need backfill. Creates dual-write sinks with the opposite suffix.

srb offline promote

Generate a config with promoted sinks. Takes the live config (from sequin config export) and removes old active sinks, promoting pending sinks to active.

sequin config export > live.yaml
srb offline promote live.yaml --indexes jobs > promoted.yaml
# stderr: "Indexes to drop: jobs_red"

srb offline rollback

Generate a config with pending sinks removed. Aborts a dual-write.

sequin config export > live.yaml
srb offline rollback live.yaml --indexes jobs > rolledback.yaml
# stderr: "Indexes to drop: jobs_black"

Online Commands

These connect to Sequin and/or OpenSearch. Connection details come from flags or environment variables (OPENSEARCH_URL, SEQUIN_URL, SEQUIN_TOKEN).

srb online opensearch-state

Get the active suffix for each managed index from OpenSearch aliases (ground truth).

srb online opensearch-state --opensearch-url http://localhost:9200 \
  --sequin-url http://localhost:7376 --sequin-token $TOKEN
# {"jobs":"red","clients":"black"}

srb online sequin-state

Get the active suffix from Sequin sink annotations.

srb online sequin-state --sequin-url http://localhost:7376 --sequin-token $TOKEN
# {"jobs":"red","clients":"black"}

srb online status

Combined status with doc counts, alias targets, and dual-write detection.

srb online status --json

Deployment Flows

First deploy

srb offline transform base.yaml > suffixed.yaml
sequin config apply suffixed.yaml

Deploy with backfill

# 1. Get current state from OpenSearch aliases
srb online opensearch-state > state.json

# 2. Transform with dual-write sinks
srb offline transform base.yaml --state state.json --backfill jobs > suffixed.yaml

# 3. Create the new OpenSearch index
curl -XPUT "$OPENSEARCH_URL/jobs_black"

# 4. Apply — Sequin starts dual-writing to both indexes
sequin config apply suffixed.yaml

Promote

# 1. Generate promoted config
sequin config export > live.yaml
srb offline promote live.yaml --indexes jobs > promoted.yaml

# 2. Swap the alias atomically
curl -XPOST "$OPENSEARCH_URL/_aliases" -H 'Content-Type: application/json' -d '{
  "actions": [
    {"remove": {"index": "*", "alias": "jobs"}},
    {"add": {"index": "jobs_black", "alias": "jobs"}}
  ]
}'

# 3. Apply — Sequin stops dual-writing
sequin config apply promoted.yaml

# 4. Drop the old index
curl -XDELETE "$OPENSEARCH_URL/jobs_red"

Rollback

# 1. Generate rollback config
sequin config export > live.yaml
srb offline rollback live.yaml --indexes jobs > rolledback.yaml

# 2. Apply — Sequin stops dual-writing
sequin config apply rolledback.yaml

# 3. Drop the pending index
curl -XDELETE "$OPENSEARCH_URL/jobs_black"

State Format

The state file is a simple JSON mapping of index base name to active suffix:

{
  "jobs": "red",
  "clients": "black"
}

On first deploy, if no --state is provided, all indexes default to red.

The state comes from OpenSearch aliases (ground truth). Use srb online opensearch-state to generate it, or produce it yourself.

GitHub Actions

Composite actions are provided in actions/ for CI/CD integration:

Action	Description
actions/plan	Transform config, run sequin config plan, comment on PR
actions/deploy	Fetch state, transform config, create indexes, apply
actions/promote	Generate promoted config, swap aliases, apply, drop old
actions/rollback	Generate rollback config, apply, drop pending
actions/status	Report current state

Example usage in a consuming repo:

- uses: meetsmore/sequin-red-black/actions/deploy@v1
  with:
    config: /tmp/base.yaml
    backfill-indexes: jobs
    sequin-url: ${{ secrets.SEQUIN_URL }}
    sequin-token: ${{ secrets.SEQUIN_TOKEN }}
    opensearch-url: ${{ secrets.OPENSEARCH_URL }}

Development

bun install
bun run test              # unit tests
bun run build             # compile binary to dist/srb

Architecture

SRB's core is a set of pure functions with no I/O:

Module	Purpose
src/transform.ts	transformConfig, promoteConfig, rollbackConfig
src/suffixer.ts	Suffix application and dual-sink injection
src/state.ts	State derivation from sinks/configs
src/types.ts	TypeScript interfaces and State type

Online utilities for convenience:

Module	Purpose
src/opensearch.ts	getAlias, getDocCount
src/sequin.ts	getSinks