Skip to content

feat: migrate database to postgres#67

Open
Copilot wants to merge 28 commits intomainfrom
copilot/add-postgresql-driver
Open

feat: migrate database to postgres#67
Copilot wants to merge 28 commits intomainfrom
copilot/add-postgresql-driver

Conversation

Copy link
Copy Markdown

Copilot AI commented Jan 12, 2026

This PR migrates the database layer to Postgres, enabling better control over data storage.

Required changes:

  • Updated drivers for Postgres
  • Changed queries to use Postgres types
  • Updated test infrastructure to run containerised Postgres instance
  • Updated API handlers to appropriately condition Postgres database responses for the frontend
  • Added graceful shutdown mechanism in main.go to prevent locked database connections
  • Updated readme

Copilot AI and others added 2 commits January 12, 2026 16:43
Co-authored-by: reesvarney <22414669+reesvarney@users.noreply.github.com>
Co-authored-by: reesvarney <22414669+reesvarney@users.noreply.github.com>
Copilot AI changed the title [WIP] Add PostgreSQL database connector with environment variable support Add PostgreSQL database connector with environment-based driver selection Jan 12, 2026
Copilot AI requested a review from reesvarney January 12, 2026 16:49
@reesvarney reesvarney marked this pull request as ready for review January 19, 2026 16:37
Copilot AI review requested due to automatic review settings January 19, 2026 16:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds PostgreSQL database support alongside the existing SQLite implementation, allowing runtime selection via environment variables. However, the implementation has several critical issues that prevent the PostgreSQL driver from functioning correctly.

Changes:

  • Added new PostgreSQL driver in lib/database/drivers/postgres/ with environment-based configuration
  • Modified database wrapper to support runtime driver selection via DB_DRIVER environment variable
  • Updated database migrations to use cross-compatible data types (BYTEA, TIMESTAMP, single-quoted strings)

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
lib/database/wrapper.go Added driver selection logic based on DB_DRIVER environment variable, supporting "sqlite", "postgres", and "postgresql" values
lib/database/drivers/postgres/postgres.go New PostgreSQL driver implementation with environment-based connection configuration
lib/database/query.sql.go SQLC version bump from v1.29.0 to v1.30.0
lib/database/models.go SQLC version bump from v1.29.0 to v1.30.0
lib/database/db.go SQLC version bump from v1.29.0 to v1.30.0
database/migrations/0001_init.up.sql Changed BLOB→BYTEA, DATETIME→TIMESTAMP, fixed string literals from double to single quotes
database/migrations/0002_add_fields.up.sql Changed BLOB→BYTEA, DATETIME→TIMESTAMP, fixed foreign key reference from sample_tags to tags
go.mod Added github.com/lib/pq v1.10.9 dependency for PostgreSQL driver
go.sum Added dependencies for PostgreSQL driver and related testing libraries
README.md Added PostgreSQL configuration documentation with environment variable examples

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@reesvarney
Copy link
Copy Markdown
Collaborator

@copilot Implement some form of local-file postgres (embedded) that stores to a local file in place of trying to support both postgresSQL and sqlite. Only use the embedded if an external postgres is not specified. Also setup a migrations table to store the down queries and execute them if a rollback is performed (compare latest migration vs the migrations table).

Copy link
Copy Markdown
Author

Copilot AI commented Feb 20, 2026

@copilot Implement some form of local-file postgres (embedded) that stores to a local file in place of trying to support both postgresSQL and sqlite. Only use the embedded if an external postgres is n...

I need clarification on the embedded PostgreSQL requirement. True embedded PostgreSQL (like embedded-postgres-go) still runs a full PostgreSQL server process in the background - it doesn't work like SQLite with a simple file-based storage.

Would you prefer:

  1. Keep SQLite as the embedded/local-file database (default), with external PostgreSQL as an option
  2. Use embedded-postgres library that downloads and runs a local PostgreSQL instance
  3. Something else?

For the migration rollback functionality, I can implement a custom migrations table to store down queries regardless of which approach we choose.

@reesvarney
Copy link
Copy Markdown
Collaborator

@copilot is there not a solution where the data for the embedded process can be stored in the database directory that currently stores the db.sqlite?

Copy link
Copy Markdown
Author

Copilot AI commented Feb 20, 2026

@copilot is there not a solution where the data for the embedded process can be stored in the database directory that currently stores the db.sqlite?

Yes, there are solutions that can store data in the database directory:

  1. embedded-postgres-go: Downloads PostgreSQL binaries and runs them locally. You can configure it to store data files in database/postgres_data/. However, it still runs a full PostgreSQL server process (just locally).

  2. ferretdb: PostgreSQL-compatible API but uses SQLite/file storage underneath. This would give you PostgreSQL compatibility while storing data in a file.

  3. Keep current approach: Use SQLite for local storage (already file-based in database/db.sqlite) and PostgreSQL for external connections.

Given the bot review identified that current PostgreSQL support doesn't work (queries use SQLite syntax), I recommend:

  • Use SQLite as the default (already file-based)
  • Add proper PostgreSQL support with correct query syntax for external databases
  • Implement the migration rollback feature you requested

Would you like me to proceed with this approach, or try embedded-postgres-go with data in the database directory?

Copilot AI and others added 4 commits February 20, 2026 14:11
…SQL syntax

Co-authored-by: reesvarney <22414669+reesvarney@users.noreply.github.com>
Co-authored-by: reesvarney <22414669+reesvarney@users.noreply.github.com>
Co-authored-by: reesvarney <22414669+reesvarney@users.noreply.github.com>
…l PostgreSQL

Co-authored-by: reesvarney <22414669+reesvarney@users.noreply.github.com>
Copilot AI changed the title Add PostgreSQL database connector with environment-based driver selection Replace SQLite with embedded PostgreSQL and correct query syntax Feb 20, 2026
Co-authored-by: reesvarney <22414669+reesvarney@users.noreply.github.com>
Base automatically changed from feature/51-unit-tests to main February 23, 2026 11:56
@reesvarney reesvarney changed the title Replace SQLite with embedded PostgreSQL using DATABASE_URL feat: move to postgres Mar 25, 2026
@reesvarney reesvarney changed the title feat: move to postgres feat: migrate database to postgres Mar 25, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 24 changed files in this pull request and generated 13 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

reesvarney and others added 5 commits March 26, 2026 15:09
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Author

Copilot AI commented Mar 26, 2026

@reesvarney Unfortunately I hit an unexpected error while processing your comment. I've automatically reported this to GitHub.

You can ask me to try again later by mentioning me in a new comment.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: b2cffb8b-7191-4dfd-8f4a-0011382faf85

Sorry for the inconvenience!

2 similar comments
Copy link
Copy Markdown
Author

Copilot AI commented Mar 26, 2026

@reesvarney Unfortunately I hit an unexpected error while processing your comment. I've automatically reported this to GitHub.

You can ask me to try again later by mentioning me in a new comment.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: b2cffb8b-7191-4dfd-8f4a-0011382faf85

Sorry for the inconvenience!

Copy link
Copy Markdown
Author

Copilot AI commented Mar 26, 2026

@reesvarney Unfortunately I hit an unexpected error while processing your comment. I've automatically reported this to GitHub.

You can ask me to try again later by mentioning me in a new comment.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: b2cffb8b-7191-4dfd-8f4a-0011382faf85

Sorry for the inconvenience!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants