The function returned the literal string "ok" for every input, including real verification failures. Consumers such as curl, wget and libssh2 render SSL_get_verify_result() through this function, so an expired or self-signed certificate was reported to the user as "ok" -- both misleading and a violation of the shim's no-fabrication rule. Return the real OpenSSL 3.x text for all X509_V_ERR_* codes (copied verbatim from crypto/x509/x509_vfy.c, verified byte-for-byte against system libcrypto), and OpenSSL's "unknown certificate verification error" fallback for unrecognised codes. Also expand the bundled include/openssl/x509.h, which defined only X509_V_OK and X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT, to the full X509_V_ERR_* set: a program compiled against the shim's headers could not previously name the code it was reporting. Tested: go test, go vet, make testlibs, make crypto-run, ./scripts/test.sh (6/6), plus a C program linked against the shim whose output matches system libcrypto for X509_V_OK / expired / hostname-mismatch / unspecified.
openssl-gocrypto
An implementation of the OpenSSL C API on top of Go's standard library
crypto/tls, built as a shared library with cgo -buildmode=c-shared.
The goal is a drop-in OpenSSL replacement: a program compiled against the
OpenSSL headers (or any binary linked against libssl.so.3 / libcrypto.so.3)
should run unchanged against this library, with the TLS work done by Go.
That goal is not yet reached. The TLS data path is complete and curl
works when the shim supplies both libssl and libcrypto; what is still
missing is the asymmetric-crypto and cipher surface that larger OpenSSL
consumers link against. Current status and
TODO.md describe exactly how far it gets and what is left.
Contents
- How it works
- Repository layout
- Current status
- API vs ABI compatibility
- Building
- Using the shim
- Connection flow
- Testing
- Known limitations
- TODO — planned work and remaining gaps
How it works
Every OpenSSL object is an opaque pointer. In real OpenSSL that pointer refers
to a heap allocation whose layout is private to libssl; in this shim it is a
key into a Go map that holds the real object. The C-facing functions are Go
functions exported with //export and mapped onto Go's crypto primitives:
| OpenSSL | Go |
|---|---|
SSL_CTX |
tls.Config wrapped in an sslCtx |
SSL |
tls.Conn wrapped in an ssl |
SSL_connect / SSL_accept |
tls.Client / tls.Server + Handshake() |
SSL_read / SSL_write |
tls.Conn.Read / Write |
X509 |
crypto/x509.Certificate |
EVP_MD |
the hash functions in crypto/md5, crypto/sha1, crypto/sha256, … |
BIGNUM, RSA public n/e |
math/big.Int |
RAND_bytes |
crypto/rand.Read |
Handle maps exist because Go's cgo rules forbid passing Go pointers to C.
Design decisions that matter
TLS runs over the caller's transport. libcurl builds its own BIO_METHOD
(a connection-filter BIO) and installs it with SSL_set0_rbio /
SSL_set0_wbio. The shim dispatches that method's read/write/ctrl callbacks
and runs crypto/tls over the caller's socket, so libcurl's poll loop and
deadlines stay authoritative. An earlier version ignored the caller's BIO and
dialled its own socket, which produced "data arrives but the connection hangs".
A would-block is reported, not swallowed. When application data cannot
proceed the shim returns SSL_ERROR_WANT_READ / SSL_ERROR_WANT_WRITE so the
caller's event loop resumes, rather than blocking inside SSL_read.
No fabrication. Where a capability is not implemented the function returns
failure (0 / NULL) rather than a plausible-looking success. A caller that
would have detected an error gets one.
Objects the shim did not create are rejected, not guessed at. Every
function takes an opaque handle and looks it up; if the handle is not one of
ours, the call fails (0 / NULL) rather than dereferencing memory whose
layout is private to whoever allocated it. This is what makes the no-fabrication
rule above enforceable: an unrecognised handle can never be mistaken for a valid
one.
Why two libraries
The shim currently provides libssl and the parts of libcrypto that libssl
consumers use. Replacing libcrypto as well is the point: almost every libssl
consumer also links libcrypto, because libssl's ABI is expressed in terms of
libcrypto types. A real drop-in replacement has to provide both.
Repository layout
Go sources (one package, split by API area)
| File | Contents |
|---|---|
openssl_shim.go |
Object structs behind the opaque handles, handle maps, alloc helpers, init hooks |
ssl.go |
SSL_CTX / SSL, cert/key loading, the synchronous TLS connection logic |
ssl_compat.go |
SSL accessors/state functions libcurl calls during a handshake (kept here so they never fall through to system libssl) |
bio.go |
BIO objects, BIO_METHOD, callback dispatch |
x509.go |
X.509 / ASN.1 / X509_STORE, PEM and DER certificate handling |
crypto.go |
EVP_MD / EVP_MD_CTX digests, BIGNUM, RSA values |
evp_pkey.go |
EVP_PKEY public-key objects (RSA, ECDSA, Ed25519) |
evp_sign.go |
EVP_DigestSign/EVP_DigestVerify (RSA PKCS#1 v1.5, ECDSA, Ed25519 one-shot) |
evp_pkey_ctx.go |
EVP_PKEY_CTX + raw-digest EVP_PKEY_sign/EVP_PKEY_verify (RSA PKCS#1 v1.5 + PSS, ECDSA, Ed25519) |
hash_legacy.go |
Legacy MD5_* / SHA*_* Init/Update/Final API |
session.go |
SSL_SESSION |
error.go |
Error queue and ERR_* functions |
provider.go |
OSSL_LIB_CTX / OSSL_PROVIDER stubs |
misc.go |
OCSP, ENGINE, PKCS#12, PEM, OSSL_STORE, UI, OBJ and other stubs |
bn.go |
BIGNUM documentation and aliases (implementation is in crypto.go) |
testhelpers.go |
Small C callbacks used only by the Go tests (cgo is not allowed in _test.go) |
Together these export 499 C functions via //export directives.
Other files
| Path | Purpose |
|---|---|
include/openssl/ |
OpenSSL-compatible headers for C programs |
include/openssl-system/openssl/ |
System OpenSSL headers, used to prove system-header compatibility |
openssl_versions.map |
Linker version script exporting symbols as @@OPENSSL_3.0.0 |
cgo-version-script.sh |
Wrapper that deduplicates the repeated -Wl,--version-script flags cgo emits |
cmd/test_client/ |
Example C client (make run) |
cmd/shim_test/ |
C checks for semantics easy to get wrong (error codes, shutdown state) |
cmd/crypto_test/ |
C checks for libcrypto primitives (EVP, BIGNUM, RSA) |
cmd/dlsym_test/ |
Checks dlopen + dlsym symbol resolution |
scripts/test.sh |
End-to-end integration harness (tests A–F) |
curl-po/ |
The curl proof-of-concept build tree |
Current status
Works
| Area | Status |
|---|---|
| TLS handshake + application data to a real host | Works |
| Client built against our headers (test A) | Works |
| Client built against system headers (test B) | Works |
curl linked against our libssl.so (test C) |
Works — TLS handshake, HTTP 2xx |
| curl POC with our libssl and our libcrypto | Works — HTTP/2 200 and HTTP/1.1 200, full body |
Stock curl with LD_PRELOAD of our shim (test D) |
Works — fetches and exits cleanly |
| mod_ssl loads and configures with our libssl + libcrypto | Works — 0 unresolved symbols (was 141) |
ALPN (SSL_set_alpn_protos, SSL_get0_alpn_selected) |
Works |
SNI (SSL_set_tlsext_host_name) |
Works |
TLS protocol bounds (SSL_CTX_set_{min,max}_proto_version, SSL_CTX_ctrl) |
Works — stored, mapped onto crypto/tls |
Certificate chains (SSL_CTX_add0_chain_cert, _add_extra_chain_cert, _clear_chain_certs, _get0_chain_certs) |
Works — sent in the Certificate message |
SSL_CTX_use_certificate / SSL_CTX_set_options semantics |
Works — options accumulate, return the previous word |
SSL_get_cipher / SSL_version / SSL_get_version |
Report the negotiated values; NULL/0 before the handshake |
SSL_ERROR_* numbering, shutdown state (test E) |
Matches OpenSSL |
BIO: memory BIOs, caller BIO_METHOD, refcounting |
Works |
Server-side TLS (SSL_accept) over an fd or caller BIO |
Works — tested by TestSSLServerHandshakeOverSocket |
SSL_set_accept_state / SSL_is_server / role dispatch |
Works |
| EVP digests, MD5/SHA-1/SHA-2, known-answer tests (test F) | Real hash values |
Legacy MD5_* / SHA*_* Init/Update/Final |
Works |
BIGNUM values and RSA n/e accessors |
Real math/big.Int |
| Certificate decode/encode (PEM + DER) | Works via crypto/x509 |
Cert/key loading into SSL_CTX |
Works (cert, chain, key, match check) |
X.509 subject/issuer/serial/validity, X509_NAME walking |
Works |
EVP_PKEY public keys from certificates (RSA/EC/Ed25519) |
Works |
Symbol versioning (@@OPENSSL_3.0.0) |
Works — 483 exported symbols |
dlopen/dlsym resolution |
Works |
The two-curl test
The curl POC can run with the shim supplying both libraries and nothing
forwarded to the system OpenSSL. Verified against https://example.com/: the
handshake negotiates h2, curl receives HTTP/2 200 plus the complete
response body; with --http1.1 it receives HTTP/1.1 200 and the same body.
make testlibs
mkdir -p /tmp/shimboth
cp libssl_versioned.so /tmp/shimboth/libssl.so.3
cp libssl_versioned.so /tmp/shimboth/libcrypto.so.3
LD_LIBRARY_PATH=/tmp/shimboth:curl-po/curl/lib/.libs \
curl-po/curl/src/.libs/curl -k -sS --max-time 10 -v https://example.com/
One copy of the library serves both SONAMEs: it is self-contained and carries
no NEEDED entry for libcrypto.
Caveat — the binary still links the system libcurl.so.4. That is what
pulls in libssh2, libldap and friends, and it is why replacing both libraries
is enough for the TLS path but not yet enough for a stock binary. The POC
demonstrates that the shim's own code runs with no OpenSSL present, not that a
stock curl can do so.
Not yet
- Asymmetric private-key operations —
EVP_DigestSign/EVP_DigestVerifyandEVP_PKEY_sign/EVP_PKEY_verifywork (RSA PKCS#1 v1.5 + PSS, ECDSA over SHA-1/SHA-2, Ed25519), theEVP_PKEY_CTX_ctrlfamily libssh2 uses is implemented, and the legacyECDSA_*/EC_KEYhelpers work. Loading a private key works (PEM_read_bio_PrivateKey,d2i_PrivateKey_bio,PKCS12_parse). Still missing: keygen, derive, andEVP_PKEY_encrypt/decrypt. This is the largest remaining block and is what libssh2 still needs. - Symmetric ciphers —
EVP_CipherInit, AES/RC4/DES/Blowfish/CAST, HMAC, and theEVP_MACAPI. PEM_read_bio_PrivateKey/d2i_PrivateKey_bio/PKCS#12— implemented for unencrypted PKCS#8 / PKCS#1 / SEC1 PEM and DER private keys, and for PKCS#12 PFX containers (MAC-verified, decrypted via go-pkcs12). Encrypted PEM private keys (Proc-Type: 4,ENCRYPTED/BEGIN ENCRYPTED PRIVATE KEY) are not yet decrypted and yield failure.SSL_CTX_use_PrivateKey_filestill loads and checks a key via its own PEM parsing.- QUIC / HTTP-3 —
SSL_new_stream,SSL_poll,SSL_set_blocking_modeare not implemented, socurl --http3cannot work. Ordinary TLS over TCP does. - X.509 store validation —
X509_LOOKUP_hash_diris missing andX509_LOOKUP_fileis a stub, soX509_STORE_add_lookupcannot build a real store; verification goes throughSSL_CTX_load_verify_locationsinstead. - Session caching is stubbed. DH/DSA/EC_KEY are stubs.
Server-side TLS
SSL_accept now runs a real server handshake, over an SSL_set_fd descriptor
or a caller-supplied BIO transport. The role is tracked on the SSL object
(SSL_set_accept_state / SSL_set_connect_state / SSL_is_server), and
SSL_do_handshake dispatches on it rather than always being a client
handshake — a caller that uses SSL_set_accept_state in place of SSL_accept
still gets the server side.
establishConnection was previously hardcoded to tls.Client at every call
site, which is why the BIO path could never work for a server: a server accepts
a connection and reads it through filters, it never dials. handshakeOver now
picks the direction from the role.
Verified by TestSSLServerHandshakeOverSocket: a shim server accepts a
connection, completes the handshake against a shim client, and echoes a payload
in both directions.
apache2 / mod_ssl
scripts/test_apache.sh runs the system apache2 with mod_ssl against the
shim's libssl and libcrypto. No apache source tree is needed: mod_ssl is a
separate module that links both libraries, so LD_LIBRARY_PATH is enough.
The load-time symbol surface is now complete — 141 missing symbols reduced to
0 — and mod_ssl loads, configures its vhosts and loads the server certificate.
Startup does not yet complete: apache2 spins inside a Go cgo callback
(cgocallbackg) during post-config init. That is a runtime-level issue, not a
missing symbol, and it is the remaining blocker. The script bounds every step
with timeout and kills by pidfile, so a hang is reported rather than hanging
the harness.
A caveat on process exit
Over a caller-supplied BIO, curl receives the full response body but does
not always exit on its own: it keeps the connection open until --max-time.
Transfers are judged on the received status and body. This affects test C and
the two-curl run, not test D (stock curl with LD_PRELOAD exits cleanly).
API vs ABI compatibility
This distinction determines what the shim can and cannot do.
API-compatible — all types are opaque and all functions have correct
signatures, so any C program that compiles against these headers (or the
system OpenSSL headers) and links against our libssl.so works. Tests A, B
and C pass this way.
ABI-compatible — a binary that was linked against system OpenSSL 3.x and
loads ours at runtime. LD_PRELOAD works for stock curl (test D) because of
the two mechanisms below; the remaining gap is QUIC and the asymmetric-crypto
surface, not a crash.
Where it fails: programs that allocate an OpenSSL object themselves and then read its struct fields directly. Our handles are opaque map keys, so the struct layout is not observable — that is inherent to the design, not a bug to be fixed incrementally.
Building
make # libssl.so
make testlibs # libssl.so + libssl_versioned.so (used by the harness)
make test_client # example client
make run # run example client against example.com:443
make dlsym-run # check dlopen/dlsym
make shim-run # error codes / shutdown state checks
make crypto-run # EVP digest / BIGNUM / RSA checks
make integration # full A/B/E/F/C/D suite
make test # Go unit tests
make clean # remove build artifacts
To emit OPENSSL_3.0.0 symbol versions:
make VERSION_SCRIPT=openssl_versions.map LIB=libssl_versioned.so
nm -D libssl_versioned.so | grep 'SSL_connect@@'
To name the library something else, override LIB:
make LIB=libopenssl_shim.so
Using the shim
Compile a C program against the bundled headers and link the shim:
gcc -Iinclude my_program.c -L. -lssl -Wl,-rpath,'$ORIGIN' -lpthread
LD_LIBRARY_PATH=. ./my_program
#include <openssl/ssl.h>
SSL_library_init();
SSL_CTX *ctx = SSL_CTX_new(TLS_client_method());
SSL_CTX_set_default_verify_paths(ctx);
SSL *ssl = SSL_new(ctx);
SSL_set_tlsext_host_name(ssl, "example.com");
SSL_connect(ssl);
SSL_write(ssl, request, len);
SSL_read(ssl, buf, sizeof(buf));
SSL_shutdown(ssl);
SSL_free(ssl);
SSL_CTX_free(ctx);
Symbol versioning
Real OpenSSL 3.x exports version-tagged symbols such as
SSL_connect@@OPENSSL_3.0.0. A binary linked against a system libssl.so.3
looks up those versioned names, so the shim must provide them. Use the
version script:
make VERSION_SCRIPT=openssl_versions.map LIB=libssl_versioned.so
cgo-version-script.sh is required because cgo emits the
-Wl,--version-script flag more than once, which ld rejects as a duplicate
version tag.
Binaries linked against newer OpenSSL releases reference higher version nodes
(OPENSSL_3.2.0, OPENSSL_3.3.0, OPENSSL_3.4.0, OPENSSL_3.0.3); the
shipped script only defines OPENSSL_3.0.0, so such binaries need those nodes
added (the blocks must list the base node last — ld rejects a script that
references a dependency defined later).
Connection flow
SSL_CTX_new()creates ansslCtxwrapping atls.Config.SSL_new()creates ansslwrapper.SSL_set_tlsext_host_name()records the SNI hostname.SSL_connect()establishes TLS, in order of preference:- over a caller-supplied BIO transport (the libcurl path);
- over an
SSL_set_fddescriptor, adopted withnet.FileConn; - otherwise by dialling the SNI host and handshaking.
SSL_write()/SSL_read()go through thetls.Conn; a would-block is surfaced asSSL_ERROR_WANT_READ/SSL_ERROR_WANT_WRITE.SSL_shutdown()/SSL_free()release state and any owned BIO references.
The handshake is driven from the first SSL_read / SSL_write, not from
SSL_connect. crypto/tls's Handshake is not resumable — once it returns an
error the connection is permanently dead — so it is run to completion inside
the first call the caller makes after SSL_connect, where the caller's own
poll loop can wait for it.
Testing
make integration # all tests, default host example.com
./scripts/test.sh # same
./scripts/test.sh www.google.com
SKIP_CURL=1 ./scripts/test.sh # skip the curl build test
SKIP_PRELOAD=1 ./scripts/test.sh # skip the LD_PRELOAD test
| Test | What it does | Compatibility exercised |
|---|---|---|
| A | Builds cmd/test_client against our headers, checks handshake + data read |
API (our headers) |
| B | Builds the same client against the system OpenSSL headers | API (system headers) |
| E | cmd/shim_test: SSL_ERROR_* numbering, SSL_SENT_SHUTDOWN, SSL_get_shutdown, SSL_get_error |
API / semantics |
| F | cmd/crypto_test: EVP digest vectors, EVP_MD_get_size, name/NID lookup, BIGNUM conversion, RSA borrowed handles, DER/PEM certificates, X509_NAME walking |
libcrypto API |
| C | curl POC built against our libssl.so, fetches https://<host>/, checks for HTTP 2xx |
API (real program) |
| D | System curl with our shim LD_PRELOAD'd, fetches the page, checks success and clean exit |
ABI (real program, LD_PRELOAD) |
Current result: 6 passed, 0 failed.
The curl POC lives in curl-po/curl and is already configured against the
shim. To rebuild it:
make testlibs
cd curl-po/curl
make clean
LDFLAGS="-L$(pwd)/../.. -lssl -Wl,-rpath,$(pwd)/../.." \
LIBS="-lssl -lpthread -ldl -lz" \
./configure --with-openssl=/usr --without-ca-fallback \
--enable-shared --disable-static --disable-ldap --disable-ldaps \
--disable-rtsp --disable-ipfs --disable-ipns --disable-mqtt \
--disable-tftp --disable-imap --disable-imaps --disable-pop3 \
--disable-pop3s --disable-smtp --disable-smtps --disable-telnet \
--disable-dict --disable-file --disable-ftp --disable-ftps \
--disable-gopher --disable-gophers --disable-ws --disable-wss \
--disable-smb --disable-ntlm-wb --disable-spnego --disable-crypto-auth \
--disable-ntlm --disable-tls-srp --disable-unix-sockets --disable-alt-svc \
--disable-hsts --disable-cookies --disable-http2 --disable-https-proxy \
--disable-progress-meter --disable-verbose --disable-shared-passwords \
--disable-sspi --disable-curldebug --disable-debug
make -j4
(--disable-http2 is in the list because the POC links system libcurl.so.4,
which provides HTTP/2 via nghttp2 independently of this build.)
Known limitations
- No asymmetric private-key operations beyond the digest and raw-digest
sign/verify paths:
EVP_DigestSign/EVP_DigestVerifyandEVP_PKEY_sign/EVP_PKEY_verifywork (RSA PKCS#1 v1.5 + PSS, ECDSA, Ed25519), theEVP_PKEY_CTX_ctrlfamily and the legacyECDSA_*/EC_KEYhelpers work, and private-key loading works (PEM_read_bio_PrivateKey,d2i_PrivateKey_bio,PKCS12_parse), but keygen, derive andEVP_PKEY_encrypt/decryptare missing, so libssh2 cannot run yet. - No symmetric ciphers or MACs:
EVP_CipherInit, AES/RC4/DES/Blowfish/CAST, HMAC,EVP_MAC. - apache2 does not finish starting (see above): mod_ssl loads and configures, but startup spins inside a Go cgo callback. Client-side TLS and the server role both work; this is specific to apache's init sequence.
- No QUIC / HTTP-3 stream API.
- X.509 store validation is not usable:
X509_LOOKUP_hash_diris missing andX509_LOOKUP_fileis a stub, soX509_STORE_add_lookupcannot build a store. - DH/DSA/EC_KEY and session caching are stubs; many functions return defaults.
- A stock binary with LDAP, SSH, Kerberos or similar linked in will fail at load time on unresolved libcrypto symbols.
- Transfers complete but the process may not exit on its own over a caller-supplied BIO (see above).
See TODO.md for the plan to close these.
License
See the repository for the project license.