test(etcdtest): share the embedded etcd and bind it to port 0 (#673, #509)

[tdakkota]

Item 7 (etcd half) of #673. Also fixes #509's port race, by removing the race rather than retrying
around it.

A new cluster/etcd/etcdtest package replaces the 4 embedded-etcd setups and the 3 freeAddr copies:
cluster/etcd membership and rejoin, cluster/router, and the root cluster_test.go with its
startEtcd/tryStartEtcd/etcdStartAttempts. −31 lines net.

API

• Start(tb) string: the client endpoint.
• NewServer(tb) *Server with:
  • Endpoint();
  • Stop() (idempotent);
  • Restart() (same client URL and data dir, for the rejoin test);
  • Dial() (a client closed with the test).
• It imports nothing from this module, so package etcd's internal tests can use it.

#509: the race is gone by construction

The old helpers probed a free port, closed it, then handed the address to etcd, so another process
could take the port in between. embed.StartEtcd can't take a bound listener, but it does
accept port 0. etcdtest binds both the client and peer URLs to 127.0.0.1:0, and reads the real
client port from e.Clients[0].Addr(). The kernel only ever hands out a free port, so there's
nothing to retry.

Supporting changes:

• The peer URL is advertised as :0. A single-node cluster never dials it.
• The gRPC gateway is off: it would dial the :0 address, and clients don't use it.
• The root package's 5-attempt retry existed only for this collision, so it is deleted.
• cluster/router's two HTTP test servers listen on 127.0.0.1:0 and read their address back,
  instead of probing with freeAddr.

Still exposed: Server.Restart must rebind its known port so running clients find etcd again. That
window predates this PR and affects one test (TestMembershipRejoinsAfterEtcdRestart).

Relation to #513. #513 hardens probe-then-bind: it holds the listeners, retries 5×, and
classifies EADDRINUSE by errno and message text. This PR removes probe-then-bind instead, so it
supersedes #513's first half. The two conflict in cluster/etcd/membership_test.go and
rejoin_test.go. #513's second half, which replaces the Rejoins() assertion with
SelfAbsent/memberKeys checks, is independent and would rebase onto etcdtest.

Evidence

• -count=20, before and after: cluster/etcd 680/680, cluster/router 320/320, root -run Cluster
  1220 passed / 20 skipped / 0 failed. No run logged "address already in use".
• A port-churn stress run (256 goroutines binding random ephemeral ports) produced no bind errors,
  before or after. The old race is too narrow to reproduce on demand; this PR removes it by
  construction.
• go test ./... green. -race on cluster/etcd, etcdtest and cluster/router green. -count=5 on
  cluster/etcd and cluster/router green. golangci-lint run --concurrency=2 ./...: 0 issues.
• cluster/ARCH.md and the ARCHITECTURE.md package map are updated.

The other half of item 7, a shared fetch-and-drain helper, is deliberately not in this PR. See #673.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KcLq1G8KpWBvM53JaQb1tS

[codecov]

Codecov Report

❌ Patch coverage is 95.55556% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
cluster/etcd/etcdtest/etcdtest.go	95.55%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!