Skip to content

Phase 2: the connection phase — the driver logs in - #5

Merged
m13m merged 5 commits into
masterfrom
phase-2-handshake
Oct 29, 2026
Merged

m13m merged 5 commits into
masterfrom
phase-2-handshake

Conversation

@m13m

@m13m m13m commented Oct 29, 2026

Copy link
Copy Markdown
Member

The driver connects to a real MySQL server and authenticates. First time this project has completed anything end to end.

49 tests with --include integration; 46 and 3 excluded without, so mix test still needs no database.

The exchange

seq 0  →  HandshakeV10, announcing caching_sha2_password
seq 1  ←  HandshakeResponse41, naming mysql_native_password
seq 2  →  AuthSwitchRequest: 0xFE + "mysql_native_password" + a fresh 20-byte scramble
seq 3  ←  AuthSwitchResponse: the reply to that scramble
seq 4  →  OK

What is here

  • Protocol.Handshake — parses the server greeting
  • Auth.NativePassword — SHA1(password) XOR SHA1(scramble <> SHA1(SHA1(password)))
  • Protocol.HandshakeResponse — builds the reply, and owns which capabilities the driver claims
  • Client.connect/1 and Client.handshake/3 — the conversation, including method switching
  • Transport grows connect/4, send/2, close/1; the fake records what was written
  • test/fixtures/handshake_v10_8.4.4.bin — 73 bytes captured from a real server

The 2018 bugs, as the fix

{server_version, rest} = Codec.decode_string_nul(rest)    # not binary-size(6)

tail_length = max(13, auth_plugin_data_len - 8)           # not a hardcoded 13

server_version is NUL-terminated. The old pattern read it as six fixed bytes, which matches MySQL 5.7.21 — exactly six characters — and crashes on anything else. Verified both ways against the captured bytes: the 2018 pattern raises MatchError on 8.4.4 and parses cleanly when the version is six characters. Eight years of breakage caused by a coincidence between a guessed width and one version string.

What the real server corrected

I had the client refuse when the greeting named a plugin it cannot speak. That was wrong. The greeting names the server's default plugin, chosen before it knows which account is connecting; the account's actual method is settled afterwards by an AuthSwitchRequest. The check now applies to the switch, where the server has said what it needs.

No scripted test would have caught this — the fake only replays what I already believed. Worth weighing when deciding how much of a protocol can be proven without a server.

What the integration test proves that no unit test could

  • The scramble is right. Its outer SHA1 and XOR had only cross-implementation agreement behind them; the inner stages were checkable against mysql.user.authentication_string, the outer ones were not. A server accepting the login is the first real evidence.
  • The capability flags are acceptable to a server — claimed and withheld both.
  • The 23-byte filler, charset id and max packet size are correct, none of which a unit test can validate.
  • auth_plugin_data is joined correctly from two fields nine bytes apart. If that join were wrong the login would fail as "access denied" and look like a password problem.

On the capability flags

Claimed: PROTOCOL_41, PLUGIN_AUTH, SECURE_CONNECTION, TRANSACTIONS, and CONNECT_WITH_DB when a database is named.

Refused, with reasons in the source: LOCAL_FILES (a hostile server can then request any file from the client), MULTI_STATEMENTS (turns one injected query into several), SESSION_TRACK and DEPRECATE_EOF (both change packet shapes this driver does not parse yet), CONNECT_ATTRS (metadata nothing here needs).

capabilities/1 is public because the value sent is the value the Ok, Err and Eof parsers must be given. A mismatch there silently misparses every response.

Running the integration tests

docker run -d --name mysqlex-dev -p 3307:3306 \
  -e MYSQL_ROOT_PASSWORD=secret -e MYSQL_DATABASE=mysqlex_test mysql:8.0

docker exec mysqlex-dev mysql -uroot -psecret -e "
  CREATE USER 'mysqlex'@'%' IDENTIFIED WITH mysql_native_password BY 'secret';
  GRANT ALL ON mysqlex_test.* TO 'mysqlex'@'%';"

mix test --include integration

Pinned to 8.0 because MySQL 8.4 ships mysql_native_password disabled. --mysql-native-password=ON is an 8.4-only option and makes an 8.0 server refuse to start.

One expectation of mine was wrong and the server was right: a database the account cannot reach reports 1044, not 1049 — granted a single schema, the server says access denied rather than unknown database.

Not here

  • caching_sha2_password, so modern servers with default settings cannot be reached yet
  • TLS, which is the same transport seam with :ssl behind it
  • Anything in the command phase; COM_QUERY is next

m13m added 5 commits October 29, 2026 22:14
The packet this project stopped at in 2018, rewritten against the two
codecs that make its variable-length fields tractable.

Both of the original bugs are here as the fix. server_version is
NUL-terminated rather than six bytes wide: the old pattern happened to
match MySQL 5.7.21, whose version string is exactly six characters, and
crashes on anything else. The scramble arrives in two pieces nine bytes
apart, and the second is MAX(13, auth_plugin_data_len - 8) bytes with a
terminator at the end, so the scramble is auth_plugin_data_len - 1
bytes once joined.

Tested against bytes captured from a real MySQL 8.4.4, saved as a
fixture. A fixture built by hand would have encoded the same misreading
as the parser and passed.
SHA1(password) XOR SHA1(scramble <> SHA1(SHA1(password))). The password
never crosses the wire, and because the scramble is per-connection the
reply cannot be replayed.

This is the one piece so far with no byte-level example in the spec, so
the vectors come from elsewhere. The inner stages are checked against
MySQL itself: SHA1(SHA1("secret")) is what the server keeps in
mysql.user.authentication_string, and MySQL 8.4.4 computing it with its
own functions returns the value asserted here. The full reply is checked
against an independent implementation.

Neither of those verifies the outer XOR against a real exchange. The
test that does is a server accepting a login, which needs the handshake
response next.

An empty password answers with no bytes rather than a hash of the empty
string, which is a rule of the method rather than an optimisation.
The first packet the driver sends. Its capability flags are a claim
rather than a question: what is set here decides the shape of nearly
every packet afterwards, so capabilities/1 is public and the response
parsers are meant to be given the same value.

Claimed: PROTOCOL_41, PLUGIN_AUTH, SECURE_CONNECTION, TRANSACTIONS, and
CONNECT_WITH_DB only when a database is named.

Left off on purpose, with reasons in the source: LOCAL_FILES, because a
malicious server can then ask the client for any file it likes;
MULTI_STATEMENTS, because it turns one injected query into several;
SESSION_TRACK and DEPRECATE_EOF, because both change packet shapes this
driver does not parse yet; CONNECT_ATTRS, because it sends metadata
nothing here needs.

The tests assert the exact byte layout, since for a packet the driver
emits there is nothing else to check it against until a server accepts
it.
Client.handshake/3 runs the conversation: read the greeting, check the
server wants a method we can speak, answer its scramble, read what it
makes of the answer. Dispatching on the reply's first byte — 0x00 for
success, 0xFF for refusal, 0xFE for an auth switch we do not yet handle.

Split from connect/1 so the conversation is testable over a scripted
transport, which is what makes the failure branches reachable at all:
a refused login and an unsupported plugin are both tested without a
server.

The plugin check refuses rather than guesses. Sending a native-password
reply to a server asking for caching_sha2 produces a rejection that
looks exactly like a wrong password, which is a bad hour to spend.

Transport grows connect/4, send/2 and close/1 now that the client needs
them, and the fake records what was written so a test can assert the
exact bytes of the response as sequence id 1.
The greeting names the server's default plugin, chosen before it knows
which account is connecting. A real MySQL 8.0 offering
caching_sha2_password still accepts a native-password account — it
answers the client's response with an AuthSwitchRequest naming the
method it actually wants and a fresh scramble, then validates the reply
to that.

So refusing on the plugin named in the greeting was wrong: it rejected
connections the server would have completed. The check now applies to
the switch instead, where the server has stated what it needs.

Found by running against a real server, which no scripted test would
have revealed: the fake only ever replays what I already believed.

Integration tests arrive with it, excluded from mix test by default and
run with --include integration. They cover a real login, a wrong
password, and a database the account cannot reach. The last one reports
1044 rather than 1049 — the account is granted a single schema, so the
server says access denied rather than unknown database.

switch_auth carries a partial client struct rather than nine positional
arguments, which credo rightly objected to.
@m13m
m13m merged commit 8a6e21b into master Oct 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant