src/lib/socket.c: fix build with libressl >= 3.5.0

[ffontaine]

Fix the following build failure with libressl >= 3.5.0:

/nvmedata/autobuild/instance-26/output-1/build/lua-cqueues-20200726/src/lib/socket.c: In function 'compat_SSL_is_server':
/nvmedata/autobuild/instance-26/output-1/build/lua-cqueues-20200726/src/lib/socket.c:188:12: error: invalid use of incomplete typedef 'SSL' {aka 'struct ssl_st'}
  188 |  return ssl->server != NULL;
      |            ^~
/nvmedata/autobuild/instance-26/output-1/build/lua-cqueues-20200726/src/lib/socket.c: At top level:
/nvmedata/autobuild/instance-26/output-1/build/lua-cqueues-20200726/src/lib/socket.c:2543:1: error: variable 'bio_methods' has initializer but incomplete type
 2543 | static BIO_METHOD bio_methods = {
      | ^~~~~~

Fixes:

• http://autobuild.buildroot.org/results/823c9e27cdb8e01a048cb751287c01c5dc70f860

Signed-off-by: Fabrice Fontaine fontaine.fabrice@gmail.com

[daurnimator]

What libressl feature is this checking for?

It should be more explicit.

[mischief]

i just ran into this as well, and it's preventing cqueues from working on OpenBSD. i took an alternate approach that still calls bio_methods_init, with an approach lifted from public domain code in exim (https://github.com/Exim/exim/blob/13835a3c1e057efad7da269c0f93bf2eac850205/src/src/dane-openssl.c#L1684).

i can send a different PR for this if it looks ok, @daurnimator.

side note, i tried to run make check but it wasn't clear how to get the desired 'openssl' module into the path..

diff --git a/src/lib/socket.c b/src/lib/socket.c
index 10b230d..959ee9b 100644
--- a/src/lib/socket.c
+++ b/src/lib/socket.c
@@ -135,7 +135,7 @@ int socket_v_api(void) {
 #endif

 #ifndef HAVE_SSL_IS_SERVER
-#define HAVE_SSL_IS_SERVER OPENSSL_PREREQ(1,1,0)
+#define HAVE_SSL_IS_SERVER (OPENSSL_PREREQ(1,1,0) || LIBRESSL_PREREQ(2,7,0))
 #endif

 #ifndef HAVE_SSL_UP_REF
@@ -2539,7 +2539,7 @@ static int bio_destroy(BIO *bio) {
        return 1;
 } /* bio_destroy() */

-#if !OPENSSL_PREREQ(1,1,0)
+#if !OPENSSL_PREREQ(1,1,0) && !LIBRESSL_PREREQ(2,8,0)
 static BIO_METHOD bio_methods = {
        BIO_TYPE_SOURCE_SINK,
        "struct socket*",
@@ -2557,6 +2557,36 @@ static BIO_METHOD *so_get_bio_methods() {
        return &bio_methods;
 } /* so_get_bio_methods() */
 #else
+
+/* LibreSSL lacks CRYPTO_ONCE and friends */
+#ifndef CRYPTO_ONCE
+#define CRYPTO_ONCE volatile int
+#define CRYPTO_ONCE_STATIC_INIT 0
+#define CRYPTO_THREAD_run_once run_once
+static void
+run_once(volatile int * once, void (*init)(void))
+{
+int wlock = 0;
+
+CRYPTO_r_lock(CRYPTO_LOCK_SSL_CTX);
+if (!*once)
+  {
+  CRYPTO_r_unlock(CRYPTO_LOCK_SSL_CTX);
+  CRYPTO_w_lock(CRYPTO_LOCK_SSL_CTX);
+  wlock = 1;
+  if (!*once)
+    {
+    *once = 1;
+    init();
+    }
+  }
+if (wlock)
+  CRYPTO_w_unlock(CRYPTO_LOCK_SSL_CTX);
+else
+  CRYPTO_r_unlock(CRYPTO_LOCK_SSL_CTX);
+}
+#endif
+
 static BIO_METHOD *bio_methods = NULL;

 static CRYPTO_ONCE bio_methods_init_once = CRYPTO_ONCE_STATIC_INIT;

[daurnimator]

This was added in libressl/openbsd@27c7f5c

Which was present in libressl 2.7.0.

[daurnimator]

I'm committing the fixes (minus the CRYPTO_ONCE polyfill) now.

@mischief something like your fix might work; I don't love the exact form though. Send it as a PR and we can iterate a bit