feat: implement live sync infrastructure with pub/sub support and centralized configuration

[adiagarwalrock]

This pull request introduces several important improvements across the codebase, focusing on enhanced documentation, environment and deployment configuration, database provider flexibility, and dependency updates. The most significant changes include the introduction of a required code map for code navigation and ownership, updates to the Prisma database configuration for better provider handling, improvements to the Docker and environment setup, and documentation updates for AI model support.

Documentation and Workflow Improvements

• Added a comprehensive, mandatory plan/code-map.md file, establishing a code map workflow that requires developers to update the map with every structural or ownership change. This ensures accurate file routing and reduces unnecessary searching.
• Updated AGENTS.md to enforce the code map workflow and clarified the structure and update requirements for plan/code-map.md. [1] [2] [3]

Database and Prisma Configuration

• Refactored Prisma configuration to resolve the database URL using a shared utility, supporting flexible provider selection (PostgreSQL, MySQL, SQLite) and aligning with new schema practices. [1] [2]
• Updated prisma/schema.prisma to set the provider to PostgreSQL by default and enabled the fullTextSearchPostgres preview feature, while removing redundant search vector indexes. [1] [2] [3]
• Updated prisma/schema.template.prisma to allow dynamic preview features via template substitution.

Deployment and Environment Configuration

• Improved docker-compose.yml by setting explicit user IDs, adding AI and live sync environment variables, defining custom networks and volumes, and updating health checks for reliability. [1] [2] [3]
• Added an autogenerated .codex/environments/environment.toml for environment setup automation.
• Enhanced .env.example with live sync transport and polling interval options for better configuration clarity.
• Updated .dockerignore to exclude additional configuration, documentation, and data directories, ensuring cleaner Docker builds. [1] [2]

AI Model and Documentation Updates

• Updated Gemini AI model defaults and documentation to use gemini-3.1-pro-preview and added support for the GEMINI_API_VERSION environment variable, with corresponding changes in user-facing docs. [1] [2] [3] [4]

Tooling and Dependency Updates

• Upgraded several dependencies, including mermaid, next, prisma, and added pino, pino-pretty, and recharts for enhanced logging and charting. [1] [2]
• Introduced a new GitHub Actions workflow for linting and formatting to enforce code quality on pull requests and main branch pushes.

These changes collectively improve project maintainability, developer onboarding, deployment flexibility, and documentation accuracy.

[Copilot]

Pull request overview

This PR adds a live-sync (SSE “socket” + polling fallback) infrastructure backed by a new pub/sub layer, centralizes sync configuration via environment variables, and updates related caching/DB/UX plumbing to support near-realtime collaboration.

Changes:

• Introduces server-side pub/sub + SSE endpoints for live sync events, plus client hooks to consume them with polling fallback.
• Adds snapshot helpers + ETag support for doc GET endpoints and publishes list/doc sync events on create/update/delete.
• Refactors settings mutations to a single updateSettings, adds a chart UI component for Settings, and updates DB URL/provider utilities + docs/tooling.

Reviewed changes

Copilot reviewed 50 out of 51 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
src/lib/useLiveSync.ts	Adds SSE-driven doc sync with ETag-based refresh + polling fallback.
src/lib/useListSync.ts	Adds SSE-driven list sync option alongside polling.
src/lib/types.ts	Simplifies store API to updateSettings; removes live sync settings fields.
src/lib/store.ts	Implements updateSettings; removes per-setting setters and live-sync defaults.
src/lib/pubsub.ts	Adds pub/sub backends (redis/file/memory) for sync event fanout.
src/lib/prisma.ts	Uses shared DB URL resolver + sqlite path parsing.
src/lib/notes-data.ts	Updates searchVector filtering for Postgres full-text search.
src/lib/logger.ts	Introduces pino-based logger and structured API error logging.
src/lib/live-sync-config.ts	Centralizes live-sync transport + polling interval via env vars.
src/lib/database-url.ts	Wraps scripts/database-url for shared URL/path resolution.
src/lib/data.ts	Updates diagram searchVector filtering for Postgres full-text search.
src/lib/cache.ts	Adds doc snapshot helpers for diagrams/notes.
src/lib/ai/config.ts	Centralizes AI model + Gemini API version defaults.
src/components/ui/chart.tsx	Adds reusable Recharts wrapper components (ChartContainer/Tooltip/etc.).
src/components/notes/NoteWorkspace.tsx	Adds draft publish + SSE live updates; wires x-client-id into writes.
src/components/notes/NotesContext.tsx	Switches notes list sync to LIVE_SYNC_CONFIG + SSE topics.
src/components/DiagramGrid.tsx	Switches diagrams list sync to LIVE_SYNC_CONFIG + SSE topics; settings updates.
src/components/DiagramEditor.tsx	Adds draft publish + SSE live updates; wires x-client-id into writes.
src/components/DashboardHeader.tsx	Migrates to updateSettings for AI settings hydration.
src/app/settings/page.tsx	Migrates to updateSettings, adds activity chart, shows sync transport status.
src/app/api/sync/stream/route.ts	Adds SSE stream endpoint subscribing to pub/sub topics.
src/app/api/sync/publish/route.ts	Adds CSRF-protected publish endpoint with optional draft de-dupe via snapshots.
src/app/api/settings/sync-status/route.ts	Exposes pub/sub backend status for UI/ops visibility.
src/app/api/settings/db-status/route.ts	Uses shared DB URL resolver for DB status reporting.
src/app/api/settings/ai-key/route.ts	Uses centralized AI model defaults from src/lib/ai/config.ts.
src/app/api/notes/route.ts	Publishes list sync events on note create; refactors caching flow.
src/app/api/notes/[id]/route.ts	Adds snapshot + ETag support; publishes doc/list sync events on update/delete.
src/app/api/diagrams/route.ts	Publishes list sync events on diagram create.
src/app/api/diagrams/[id]/route.ts	Adds snapshot + ETag support; publishes doc/list sync events on update/delete.
src/app/api/ai/assist/route.ts	Uses centralized AI config; supports configurable Gemini API version.
scripts/prepare-prisma-schema.js	Adds template substitution for previewFeatures + index behavior by provider.
scripts/database-url.js	Adds robust DB URL + writable sqlite path resolution.
scripts/bootstrap.js	Uses shared DB URL resolver + sqlite path parsing; refactors bootstrap flow.
scripts/backfill-tag-counts.js	Uses shared DB URL resolver + sqlite path parsing.
public/sw.js	Disables service worker caching to avoid interfering with SSE/live sync.
prisma/schema.template.prisma	Adds @@PREVIEW_FEATURES@@ token for provider-specific generation.
prisma/schema.prisma	Defaults to Postgres + enables fullTextSearchPostgres preview feature.
prisma/prisma.config.ts	Uses shared DB URL resolver for prisma config datasource URL.
prisma.config.ts	Uses shared DB URL resolver for prisma config datasource URL.
plan/code-map.md	Adds mandatory code map and intent/ownership routing.
package.json	Updates deps (Next, Mermaid, Prisma) and adds pino/recharts; removes next-pwa.
docs/settings.md	Updates Gemini default model + documents GEMINI_API_VERSION.
docs/CONTAINER_STARTUP.md	Updates Gemini default model + documents GEMINI_API_VERSION.
docs/AI.md	Updates AI provider docs (Gemini model reference).
docker-compose.yml	Adds live sync env vars, AI key, explicit UID/GID, networks/volumes, healthcheck.
AGENTS.md	Adds mandatory code-map workflow instructions.
.github/workflows/lint.yml	Adds CI lint + prettier check workflow.
.env.example	Documents live-sync env vars (transport + poll interval).
.dockerignore	Expands ignored files/dirs for cleaner Docker builds.
.codex/environments/environment.toml	Adds autogenerated environment setup metadata.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.