feat: Add optional headless browser login

.scratch/headless-auth/diagnosis.md

@@ -0,0 +1,128 @@
+# Login investigation — October 9, 2026
+
+Status: Code 366 is explained; authentication now uses one browser workflow
+with headless terminal prompts or a visible window. Real-account password and
+MFA success remain unverified against live X.
+
+## Current account-login failure
+
+An instrumented run of the actual CLI with the supplied username reached
+`POST /onboarding/web/actions/begin_login` with both `username_or_email` and a
+browser-generated `$castle_token`. X returned HTTP 200 with an application
+error, and its visible form displayed:
+
+> We’ve temporarily limited your login. Please try again later.
+
+The page never advanced to password entry. This is evidence of a temporary
+login limit, not evidence that headless login requires a visible browser.
+The response did not identify the scope of the limit or when it expires.
+No further account submissions were made after identifying it.
+
+The CLI previously collapsed this text into `AUTHENTICATION_FAILED` with a
+headed-login suggestion. It now reports `AUTH_RATE_LIMITED` and preserves
+the instruction to try later. Generic page failures use
+`AUTH_LOGIN_UNAVAILABLE`; hidden error text is ignored. Local Chromium
+regressions exercise these rendered error states without contacting X.
+
+The earlier phone attempt also used the wrong entry path: the current
+`username_or_email` field is labelled “Email or username,” while phone login
+has a separate “Continue with phone” control. The terminal prompt now matches
+that field. The separate phone-login route is not implemented by this driver.
+Unknown form timeouts indicate a limitation in form recognition, not proof of
+an X requirement to use a visible browser.
+
+The former experimental `auth login-api` implementation called the retired
+classic onboarding flow. Updating its endpoint alone could not implement the
+replacement: Jetfuel uses URL-encoded actions, binary responses, changing form
+fields, and Castle device tokens. The implementation and public command have
+been removed in favor of X's own web login.
+
+## Final implementation decision
+
+`twt auth login` defaults to a visible Chromium window for manual login.
+`--headed` selects that default explicitly; `--headless` uses terminal prompts
+for identity, password, and supported verification codes. Both
+modes share one managed browser instance and application profile through
+authentication and operation-contract capture. The chosen mode never changes
+automatically, and Chromium is not relaunched between login and capture.
+
+This lets X's frontend generate device tokens and submit its current login
+forms. The CLI does not maintain a parallel implementation of the private
+login API. Passwords and verification codes are hidden during terminal entry,
+are not accepted in flags or environment variables, and are never saved.
+Unsupported challenges, including CAPTCHA and security keys, fail with
+instructions to rerun `twt auth login --headed`.
+
+Contract refresh uses the existing profile headlessly. When authentication has
+expired, it directs the user to log in explicitly rather than opening a
+visible browser. After capture, the command saves authentication and candidate
+contracts before validating the read operations. Successful validation promotes
+the candidate contracts to active contracts. Validation failure preserves the
+previous active contracts; the newly captured authentication is already saved.
+
+## Historical reproduction
+
+Before removal of the legacy implementation,
+`go run ./.scratch/.sandbox/authprobe` called it without supplying account
+credentials. The probe exited successfully only if the flow reached a
+terminal input prompt. Repeated runs returned `AUTH_API_UNAVAILABLE` with X
+code 366 before any prompt. The helper at
+`.scratch/.sandbox/authprobe/main.go` was removed together with the legacy
+`authapi` implementation. The command above records the historical experiment
+and no longer runs in the current checkout.
+
+## Live observations
+
+- Direct HTTP `GET https://jf.x.com/onboarding/web?mode=login` returned HTTP 200.
+  The binary response exposed `username_or_email` and `begin_login`.
+- Playwright with Chromium 156 loaded
+  `https://x.com/i/jf/onboarding/web?mode=login`. X rejected the default
+  `HeadlessChrome` user agent with HTTP 403; the corresponding standard Chrome
+  user agent loaded the page. A separate HTTP comparison reproduced this.
+- One dummy identifier, `twt-login-probe@invalid.example`, was used throughout.
+  No real account identifier, password, email code, or session credentials were
+  provided. The dummy address uses the reserved `.example` domain.
+- X's own frontend submitted
+  `POST https://jf.x.com/onboarding/web/actions/begin_login` with form fields
+  `username_or_email` and `$castle_token`. The generated token was about 5.9 KB;
+  its value was neither printed nor saved.
+- A controlled comparison intercepted a fresh browser submission and replayed
+  it twice through the same HTTP client with the same headers and identifier.
+  Removing only `$castle_token` returned HTTP 200 with an 82-byte binary
+  “Something went wrong” response. Including the browser-generated token
+  returned HTTP 200 and X's normal “Get the app to finish signing up using email”
+  branch. HTTP success alone is therefore insufficient to validate login.
+
+This establishes the active endpoint and the device-token dependency at the
+identifier step. It does not establish successful password login or MFA for a
+real account, nor prove that every headless-generated token is accepted.
+A short headless token-generation step was considered during the investigation.
+The final decision keeps the whole login in the same browser as contract
+capture. A reliable browser-free token source was not established.
+
+## Unified browser implementation probes
+
+The final browser workflow was exercised through the CLI with both managed
+Chromium and current Chromium, using isolated `XDG_CONFIG_HOME` and
+`XDG_STATE_HOME` directories. These probes used only a dummy identifier in the
+reserved `.example` domain. No real account password or verification code was
+supplied.
+
+The live identifier form contains an inert password input hidden by
+`opacity: 0`. Login-field detection now filters that input so the CLI prompts
+for the visible identifier. The current probe reaches that prompt and then
+returns `AUTH_CHALLENGE_UNSUPPORTED`; it does not establish successful password
+login or MFA. No specific post-identifier branch is claimed from this probe.
+
+## Current primary-source corroboration
+
+- [Twifork migration evidence, July 30](https://github.com/PawiX25/twifork/commit/3294c1ebf020761c95c877dc36ed06c24a944e88)
+- [Mautrix Jetfuel route update, September 30](https://github.com/mautrix/twitter/pull/153)
+- [Mautrix login implementation, October 6 revision](https://github.com/mautrix/twitter/blob/fea0bb26af1a5b7529588dee9ab4fa1f6e10be3b/pkg/twittermeow/login_jetfuel.go)
+- [Mautrix's browser/webview token requirement](https://github.com/mautrix/twitter/blob/fea0bb26af1a5b7529588dee9ab4fa1f6e10be3b/pkg/twittermeow/castle_token.go)
+- [Emusks login documentation and limitations](https://emusks.tiago.zip/jetfuel/login)
+
+Mautrix's current implementation carries browser/webview-generated Castle tokens
+for Jetfuel form submissions. Emusks also describes a browser token dependency;
+its documentation and shipped source differ on routes, and its statements about
+initial stages should not be read as proof of full browser-free account login.

CONTEXT.md

@@ -9,7 +9,7 @@ A CLI command that reads account or public X data. It requires valid authenticat
 _Avoid_: Browser command, API command
 
 **Authentication state**:
-Locally persisted browser credentials established through interactive X login and loaded by later data commands. It changes independently from operation contracts.
+Locally persisted X credentials established through a visible browser login or headless browser login with terminal prompts and loaded by later data commands. It changes independently from operation contracts.
 _Avoid_: Auth headers, copied cURL, session
 
 **Application profile**:

README.md

@@ -140,13 +140,35 @@ Then authenticate:
 twt auth login
 ```
 
-Authentication uses a dedicated Chromium profile owned by `twt`, not the
-user's everyday browser profile. The command first checks that profile
-headlessly. If X requires a login or interactive challenge, it opens a headed
-Chromium window for the user to complete the flow. Once authenticated, it
-captures the current Search, Bookmarks, and View request contracts, verifies that
-the four read operations work, and activates the new local state only
-after validation succeeds.
+Authentication opens managed Chromium in a visible window for manual login.
+Use `--headed` to select the default explicitly, or `--headless` to enter your
+username, password, and supported verification codes through terminal prompts.
+Passwords and codes are hidden during terminal entry and are never saved.
+Credentials are not accepted through command flags or environment variables:
+
+```bash
+twt auth login --headless
+twt auth login --headed
+```
+
+Use a username or email for the initial headless prompt. X's separate phone
+login route is not automated. If X temporarily limits login attempts, the CLI
+reports `AUTH_RATE_LIMITED`; try again later.
+
+Both modes use one Chromium instance and the same dedicated application
+profile throughout login and contract capture. The selected mode remains in
+effect for the entire command. After authentication, `twt` captures the current
+Search, Bookmarks, and View request contracts and saves the captured
+authentication. It verifies that the four read operations work before replacing
+the active operation contracts. If validation fails, the previous active
+contracts remain and the newly captured authentication is already saved.
+
+Headless login supports terminal prompts for identity, password, and
+verification codes. CAPTCHA, security-key, and other unsupported challenges
+return an actionable failure; rerun `twt auth login --headed` to complete them
+in the visible browser. Real-account password and MFA success have not yet
+been verified against live X. The [login investigation](.scratch/headless-auth/diagnosis.md)
+records the evidence behind using X's own web login in both modes.
 
 By default, sensitive and generated state is stored at:
 
@@ -176,8 +198,10 @@ and then rerun the original command:
 twt contract refresh
 ```
 
-Refresh follows the same headless-first authentication behavior. Data commands
-never refresh contracts, open Chromium, log in, or retry themselves.
+Refresh uses the existing application profile headlessly. If X requires login,
+it returns an authentication failure directing you to `twt auth login`; it
+does not open a visible browser automatically. Data commands never refresh
+contracts, open Chromium, log in, or retry themselves.
 
 ## Command documentation
 

docs/commands.md

@@ -126,13 +126,39 @@ revisions is offered separately and defaults to no.
 ## Authentication
 
 ```text
-twt auth login
+twt auth login [--headless | --headed]
 ```
 
-Checks the isolated application profile headlessly and opens managed Chromium
-only when X requires interactive login or a challenge. After authentication,
-the command captures and validates the Search, Bookmarks, and View contracts before
-activating the new local authentication and contract state.
+`login` defaults to a visible managed Chromium window for manual login. It
+reuses the isolated application profile. `--headless` uses terminal prompts
+when X requests an identity, password, or supported verification code. Passwords and verification codes are
+hidden during terminal entry and are never saved or included in command
+output. Account credentials are not accepted through flags or environment
+variables. An already authenticated profile proceeds directly to contract
+capture.
+
+The current initial form accepts a username or email. X's separate phone-login
+route is not automated. `AUTH_RATE_LIMITED` means X temporarily limited login
+attempts; try later. `AUTH_LOGIN_UNAVAILABLE` reports an X login-page error.
+Neither failure implies that changing browser display mode will resolve it.
+
+`--headed` selects the default visible-browser mode explicitly. The flags are mutually exclusive. Both modes use one
+browser instance and the same profile for login and capture; the command does
+not switch modes or relaunch Chromium after authentication.
+
+After authentication, the command captures the Search, Bookmarks, and View
+contracts, saves the captured authentication and candidate contracts, then
+validates the four read operations. Only successful validation replaces the
+active contracts. Successful output reports `status: "authenticated"`,
+`authentication_path`, and `contract_path`.
+
+CAPTCHA, security-key, and other unsupported headless challenges return an
+actionable failure suggesting `twt auth login --headed`. A failure before
+capture leaves the saved authentication and active contract files unchanged.
+If contract validation fails, the previous active contracts remain, but the
+newly captured authentication has already been saved.
+Real-account password and MFA success have not yet been verified against live
+X; see the [login investigation](../.scratch/headless-auth/diagnosis.md).
 
 ## Contract maintenance
 
@@ -144,10 +170,11 @@ twt contract refresh
 `status` validates the local authentication and contract files without opening
 Chromium or contacting X. It does not prove that X will still accept them.
 
-`refresh` explicitly recaptures and validates current operation contracts. It
-uses the existing application profile headlessly when possible and opens
-Chromium only if X requires interactive authentication. Data commands never
-refresh contracts or retry themselves.
+`refresh` explicitly recaptures and validates current operation contracts using
+the existing application profile headlessly. If X requires login, it returns
+an authentication failure directing you to `twt auth login`. Refresh never
+switches to a visible browser. Data commands never refresh contracts or retry
+themselves.
 
 ## Output, warnings, and failures
 

go.mod

@@ -6,6 +6,7 @@ require (
 	github.com/ach968/x-client-transaction-id-go v0.1.0
 	github.com/go-rod/rod v0.116.2
 	github.com/santhosh-tekuri/jsonschema/v6 v6.0.3
+	golang.org/x/term v0.43.0
 )
 
 require (
@@ -15,5 +16,6 @@ require (
 	github.com/ysmood/gson v0.7.3 // indirect
 	github.com/ysmood/leakless v0.9.0 // indirect
 	golang.org/x/net v0.55.0 // indirect
+	golang.org/x/sys v0.45.0 // indirect
 	golang.org/x/text v0.37.0 // indirect
 )

go.sum

@@ -22,5 +22,9 @@ github.com/ysmood/leakless v0.9.0 h1:qxCG5VirSBvmi3uynXFkcnLMzkphdh3xx5FtrORwDCU
 github.com/ysmood/leakless v0.9.0/go.mod h1:R8iAXPRaG97QJwqxs74RdwzcRHT1SWCGTNqY8q0JvMQ=
 golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
 golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
+golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
+golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
+golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4=
+golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk=
 golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
 golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=

internal/app/browser/capture.go

@@ -1,6 +1,7 @@
 package browser
 
 import (
+	"context"
 	"encoding/json"
 	"errors"
 	"fmt"
@@ -48,6 +49,9 @@ func (step ContractCaptureStep) CapturedOperations() []app.OperationName {
 }
 
 type ContractCaptureOptions struct {
+	Context     context.Context
+	Login       bool
+	LoginPrompt LoginPromptFunc
 	ProfilePath string
 	Headless    bool
 	URLs        []string
@@ -261,7 +265,10 @@ func CaptureOperationContracts(options ContractCaptureOptions) (result app.Captu
 	if options.Timeout == 0 {
 		options.Timeout = 10 * time.Second
 	}
-	client, err := newBrowserClient(options.ProfilePath, options.Headless, options.Timeout)
+	if options.Login && options.Headless && options.LoginPrompt == nil {
+		return result, loginFailure("Headless login requires terminal input")
+	}
+	client, err := newBrowserClient(options.ProfilePath, options.Headless, 0)
 	if err != nil {
 		return result, err
 	}
@@ -270,6 +277,17 @@ func CaptureOperationContracts(options ContractCaptureOptions) (result app.Captu
 			err = closeErr
 		}
 	}()
+	if options.Context == nil {
+		options.Context = context.Background()
+	}
+	client.page = client.page.Context(options.Context)
+	if options.Login {
+		if err := authenticateBrowser(client, options.LoginPrompt, loginURL); err != nil {
+			return result, err
+		}
+	}
+	client.page = client.page.Timeout(options.Timeout)
+	defer client.page.CancelTimeout()
 	page := client.page
 	var mutex sync.Mutex
 	operations := map[app.OperationName]app.OperationContract{}
@@ -331,6 +349,8 @@ func CaptureOperationContracts(options ContractCaptureOptions) (result app.Captu
 				return nil
 			}
 			select {
+			case <-options.Context.Done():
+				return options.Context.Err()
 			case <-notify:
 			case <-authenticationTicker.C:
 			case <-timer.C:

internal/app/browser/client.go

@@ -3,6 +3,7 @@ package browser
 import (
 	"fmt"
 	"os"
+	"strings"
 	"time"
 
 	"github.com/go-rod/rod"
@@ -78,6 +79,17 @@ func newBrowserClient(profilePath string, headless bool, timeout time.Duration)
 		_ = release()
 		return nil, err
 	}
+	// Use the installed browser's normal user agent in both display modes.
+	// X rejects the HeadlessChrome product name before rendering the login page.
+	version, err := launched.browser.Version()
+	if err == nil {
+		err = page.SetUserAgent(&proto.NetworkSetUserAgentOverride{UserAgent: strings.ReplaceAll(version.UserAgent, "HeadlessChrome/", "Chrome/"), AcceptLanguage: "en-US,en;q=0.9"})
+	}
+	if err != nil {
+		_ = launched.Close()
+		_ = release()
+		return nil, err
+	}
 	if timeout > 0 {
 		page = page.Timeout(timeout)
 	}