verify that postgres is up-and-running on startup by njaard · Pull Request #29 · boustrophedon/pgtemp

njaard · 2026-01-22T18:11:17Z

Atomically acquire the port by retaining the TcpListener until after postgres starts up. We verify postgres is running by connecting to it via its unix domain socket, and then send the raw minimum bytes to verify.

I have confirmed this works because I can run extensive automated tests on my own software with cargo t -j<very large number> and it no longer fails sporadically by either postgres saying that it's not yet ready, or the weirder message where it gets the wrong port number by chance.

coveralls · 2026-01-22T18:14:11Z

Pull Request Test Coverage Report for Build 22128353563

Details

70 of 73 (95.89%) changed or added relevant lines in 2 files are covered.
3 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.08%) to 89.104%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/run_db.rs	54	57	94.74%

Files with Coverage Reduction	New Missed Lines	%
src/run_db.rs	3	85.33%

Totals
Change from base Build 21737054921:	0.08%
Covered Lines:	507
Relevant Lines:	569

💛 - Coveralls

boustrophedon · 2026-01-22T23:13:41Z

src/run_db.rs

+    use std::os::unix::net::UnixStream;
+
+    let Ok(mut stream) =
+        UnixStream::connect(format!("{unix_socket_directory}/.s.PGSQL.{tcp_port}"))


Do you know if this is publicly documented i.e. won't change based on the pg version?

It's documented right for unix_socket_directories. I expect it to be stable because you can connect to the server with different versions of psql

boustrophedon · 2026-01-22T23:15:25Z

src/lib.rs

    /// and then set it as the current port.
+    ///
+    /// If you don't set the port in advance, then a port will be automatically selected
+    /// atomically, and can prevent a race condition, therefor you should avoid calling this function.


Can you clarify this comment? Why should you avoid calling it?

This function gets a random port "right now" and puts it into the Builder. I don't want to change the API for the builder, so I can't have it hold the TcpListener. Therefor, use of this function causes pgtemp to retain the previous behavior.

Can you proposed improved wording for that comment?

boustrophedon · 2026-01-22T23:19:26Z

src/run_db.rs

+    let _ = stream.set_write_timeout(Some(Duration::from_millis(500)));
+
+    // simply send the SSL handshake
+    let ssl_request = [0u8, 0, 0, 8, 4, 210, 22, 47];


Can you add some documentation to this? eg. when I search for "ssl handshake bytes", because I do not have that data structure memorized, I see that it seems like the first byte of the client hello should be 0x16.

boustrophedon · 2026-01-22T23:21:48Z

This is a neat solution, thanks for digging into this. I left a few comments and there also appears to be a lint issue. Thanks again!

njaard · 2026-01-22T23:41:50Z

This is a neat solution, thanks for digging into this. I left a few comments and there also appears to be a lint issue. Thanks again!

The lint issue pertains to a piece of python code apparently unrelated to my change

boustrophedon · 2026-02-06T03:01:34Z

Sorry for the delay, had some personal issues as well as work. Can you rebase on master? It should fix any lint issues.

njaard · 2026-02-06T19:35:55Z

Sorry for the delay, had some personal issues as well as work. Can you rebase on master? It should fix any lint issues.

I did so, but now the CI tests are failing in some other way, which may be a spurious or github change. The tests pass on my machine.

boustrophedon · 2026-02-07T03:40:41Z

Yeah, this seems flaky still. I've rerun it a few times and master doesn't error but random tests with this change are:


test create_table_and_insert ... ok
test check_database_name ... FAILED
test builder_setters ... ok

failures:

---- check_database_name stdout ----

thread 'check_database_name' (6514) panicked at tests/basic_operations.rs:16:10:
failed to connect to db: Io(Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" })

boustrophedon · 2026-02-07T03:41:00Z

sorry, accidentally closed somehow

njaard · 2026-02-07T22:40:53Z

Well, the reason that my solution seemed to work is that it would bind to the ipv4 port, and postgres would bind to the ipv6 port. Then when you connect to "localhost", you'd get the ipv6 port. SO_REUSEPORT was never actually used, and postgres would always fail to bind to the ipv4 port.

So, back to the drawing board for random port selection, I guess.

Maybe an alternate approach is to keep on trying to start postgres on a randomly selected port until it succeeds.

boustrophedon · 2026-02-08T00:23:24Z

It might be useful to simply email the postgres mailing list or perhaps see how their own CI is set up. I might take a look today or tomorrow.

njaard · 2026-02-08T00:26:09Z

I had a crazy scheme of just proxying the Unix domain socket

boustrophedon · 2026-02-08T00:35:23Z

Ah, bind to either a random tcp addr/port or the user provided addr/port and then have pgtemp just proxy the traffic to the pg server's local unix socket? That would probably work (and we're already proxying in the daemon anyway, so it's not that crazy).

I wonder if we could simplify/unify the "base" pgtemp code with the daemon code somehow by doing this.

boustrophedon · 2026-02-08T23:31:53Z

It looks like there's actually a small standalone binary called "pg_isready" that basically implements this functionality. We could probably just shell out to that.

boustrophedon · 2026-02-09T01:13:09Z

Actually, sorry, I'm not sure what the underlying issue is currently. Is it that we're connecting too quickly before the pg server is ready, or is it that when running a large suite of tests sometimes we end up trying to start a server on a port that already has one running?

njaard · 2026-02-18T05:50:58Z

Take a look again! I think I got it this time.

Atomically acquire the port by building the TcpListener before postgres even starts. Tell postgres to not even bind to a TCP port; it only creates a Unix Domain Socket. It still names that filename with "5432", but it doesn't bind to a port. Then, proxy the Unix Domain Socket over the TcpListener we previously acquired. We can now be sure which postgres we are talking to and on what port. Finally, as our last trick, we can verify postgres is up and running by running the `pg_isready` CLI tool. We tell pg_isready to check the unix domain socket path (though we could just as well have it go over our proxied TCP listener)

njaard · 2026-03-01T02:33:42Z

@boustrophedon bump :)

boustrophedon reviewed Jan 22, 2026

View reviewed changes

njaard force-pushed the master branch from 38ae6e4 to 94e0cb5 Compare January 22, 2026 23:40

njaard force-pushed the master branch from 94e0cb5 to 5abfedc Compare February 6, 2026 19:32

boustrophedon closed this Feb 7, 2026

boustrophedon reopened this Feb 7, 2026

njaard force-pushed the master branch 3 times, most recently from f0526d1 to ae7860e Compare February 7, 2026 22:05

njaard marked this pull request as draft February 7, 2026 22:41

njaard force-pushed the master branch from ae7860e to 8e0ae31 Compare February 18, 2026 05:50

njaard marked this pull request as ready for review February 18, 2026 05:50

njaard force-pushed the master branch from 8e0ae31 to 5aaf392 Compare February 18, 2026 05:52

Conversation

njaard commented Jan 22, 2026

Uh oh!

coveralls commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 22128353563

Details

💛 - Coveralls

Uh oh!

boustrophedon Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

njaard Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

boustrophedon Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

njaard Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

boustrophedon Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

njaard Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

boustrophedon commented Jan 22, 2026

Uh oh!

njaard commented Jan 22, 2026

Uh oh!

boustrophedon commented Feb 6, 2026

Uh oh!

njaard commented Feb 6, 2026

Uh oh!

boustrophedon commented Feb 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boustrophedon commented Feb 7, 2026

Uh oh!

njaard commented Feb 7, 2026

Uh oh!

boustrophedon commented Feb 8, 2026

Uh oh!

njaard commented Feb 8, 2026

Uh oh!

boustrophedon commented Feb 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boustrophedon commented Feb 8, 2026

Uh oh!

boustrophedon commented Feb 9, 2026

Uh oh!

njaard commented Feb 18, 2026

Uh oh!

njaard commented Mar 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coveralls commented Jan 22, 2026 •

edited

Loading

boustrophedon commented Feb 7, 2026 •

edited

Loading

boustrophedon commented Feb 8, 2026 •

edited

Loading