From d9af8c6662ef87624f2b0b1ee6c7ad8c5a8d67ff Mon Sep 17 00:00:00 2001 From: Kingsley Faulkner Date: Sat, 10 Oct 2026 23:36:14 +0100 Subject: [PATCH 1/4] fix: load Mixcloud New Shows and fix counts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to #344: load Mixcloud's website **New Shows** feed instead of approximating the following stream by merging uploads from followed creators. The provider entry is now **Stream (New Shows)**. The website uses the `viewer.uploadNotifications` GraphQL connection. cliamp reads it using the browser session selected by `cookies_from`, preserves its ordering, and follows its pagination cursors up to the existing `max_items` limit. Playback still uses the existing yt-dlp pipeline and stable show-page URLs. **Suggested review order:** 1. `external/mixcloud/new_shows.go` and its tests — the feed request, ordering, pagination and failure handling. 2. `internal/ytdlcookies/browser.go` and its tests — offline browser-cookie export and cookie scope. 3. `external/mixcloud/provider.go`, config, setup and docs — replacement of the old aggregation and removal of `stream_creators`. 4. `ui/model/view.go` and the playlist-label regression — the Favorites count correction below. No new dependencies; `go.mod` and `go.sum` are unchanged. - Reads the signed-in browser account's New Shows feed, rather than sorting and merging followed creators' public uploads. - Uses fixed 20-show requests, bounded by `max_items`; removes the obsolete creator-limit setting without adding another tuning option. - Loads cookies through yt-dlp without visiting a website or downloading media. The private temporary cookie file is removed afterwards. Cookie domain, host-only, path, expiry and secure restrictions are retained. - Keeps website cookies separate from REST/OAuth requests and rejects redirects for the feed request. - Reports missing/expired sessions, API failures and invalid pagination rather than silently returning an approximate or partial feed. Public discovery and playback behavior are otherwise unchanged. - Hides an unknown provider Favorites count instead of displaying `Favorites · 0 tracks`. The genuinely empty local Favorites playlist still displays zero. Not included. 1. Install yt-dlp and FFmpeg, sign into Mixcloud in your browser, and configure: ```toml [mixcloud] enabled = true username = "yourname" cookies_from = "firefox" max_items = 100 ``` 2. `make build && ./cliamp --provider mixcloud`, then open **Stream (New Shows)**. Compare the shows and their order with that same browser account's website New Shows page. 3. Remove `cookies_from` or use an expired session: loading Stream should report the missing session, not fall back to merged public uploads. A subsequent-page failure must not populate a partial result. 4. Confirm provider Favorites with an unknown count renders simply as `Favorites`, while empty local Favorites still renders `Favorites · 0 tracks`. Focused offline checks passed: ```sh CLIAMP_LIVE_MIXCLOUD=0 go test -count=1 ./internal/ytdlcookies ./external/mixcloud ./config ./cmd go test -count=1 docs_links_test.go ``` The playlist-label regression also passed in isolation using the exact source helpers and `TestPlaylistLabel`. Full UI/native-dependent builds were unavailable locally because the audio development libraries are missing; CI needs to validate those paths. No authenticated feed comparison is claimed. - The website GraphQL API is undocumented and may change independently of Mixcloud's public REST API. REST `/feed/` represents account activity; equivalence with website New Shows is not established. - Stream now requires a signed-in `cookies_from` browser session. A public username or developer OAuth token alone is insufficient; Stream uses the browser account, while REST account views continue to use the configured username/token. - Existing `stream_creators` configuration is obsolete and can be removed. `max_items` remains the total result limit. - No matching open issue was found in the Mixcloud/New Shows/Favorites searches. #388 concerns provider availability in an older installation and is already resolved; this PR does not claim to close it or unrelated Favorites issues. - [x] Focused offline tests and documentation-anchor checks pass - [x] Playlist-label regression passes in isolation - [x] `git diff --check` passes - [x] Matching documentation and example configuration updated; the website does not enumerate these playlist labels and needs no change - [ ] Full `make check` / native UI validation (not run locally; missing audio development libraries) - [ ] Authenticated website/feed comparison --- cmd/setup.go | 12 +- cmd/setup_test.go | 9 +- config.toml.example | 3 +- config/config.go | 23 +-- config/config_golden_test.go | 31 ++-- config/mixcloud_test.go | 3 +- docs/configuration.md | 3 +- docs/mixcloud.md | 25 ++- external/mixcloud/client.go | 5 + external/mixcloud/live_test.go | 12 +- external/mixcloud/new_shows.go | 155 +++++++++++++++++ external/mixcloud/new_shows_test.go | 246 +++++++++++++++++++++++++++ external/mixcloud/provider.go | 135 +++------------ external/mixcloud/provider_test.go | 75 +------- internal/ytdlcookies/browser.go | 98 +++++++++++ internal/ytdlcookies/browser_test.go | 113 ++++++++++++ providers.go | 17 +- ui/model/view.go | 2 +- ui/model/view_helpers_test.go | 8 +- 19 files changed, 709 insertions(+), 266 deletions(-) create mode 100644 external/mixcloud/new_shows.go create mode 100644 external/mixcloud/new_shows_test.go create mode 100644 internal/ytdlcookies/browser.go create mode 100644 internal/ytdlcookies/browser_test.go diff --git a/cmd/setup.go b/cmd/setup.go index d9fe11750..3fe4785eb 100644 --- a/cmd/setup.go +++ b/cmd/setup.go @@ -589,7 +589,7 @@ func providers() []providerSpec { "Browse and play Mixcloud shows through its public API and yt-dlp.", "A username adds your public library, following stream, and creators.", "An optional developer access token adds /me and Listen Later.", - "Choose browser cookies for subscriber-only or signed-in playback.", + "Choose browser cookies for Stream and signed-in playback.", "Docs: cliamp.stream → docs/mixcloud.md", }, picker: &pickerSpec{ @@ -616,7 +616,6 @@ func providers() []providerSpec { onlyIf: func(v map[string]string) bool { return v[keyMixcloudBrowser] == "custom" }}, {key: "styles", label: "Music styles (optional)", help: "comma-separated; blank uses cliamp defaults"}, {key: "max_items", label: "Items per view", help: fmt.Sprintf("%d-%d", mixcloud.MinItems, mixcloud.MaxItemsLimit), defaultV: strconv.Itoa(mixcloud.DefaultMaxItems)}, - {key: "stream_creators", label: "Creators in Stream", help: fmt.Sprintf("%d-%d", mixcloud.MinItems, mixcloud.MaxStreamCreators), defaultV: strconv.Itoa(mixcloud.DefaultStreamCreators)}, }, extraValidate: func(v map[string]string) error { limits := []struct { @@ -624,7 +623,6 @@ func providers() []providerSpec { max int }{ {key: "max_items", label: "items per view", max: mixcloud.MaxItemsLimit}, - {key: "stream_creators", label: "stream creators", max: mixcloud.MaxStreamCreators}, } for _, field := range limits { n, err := strconv.Atoi(strings.TrimSpace(v[field.key])) @@ -637,7 +635,7 @@ func providers() []providerSpec { } return nil }, - owned: []string{"enabled", "username", "access_token", "cookies_from", "styles", "max_items", "stream_creators"}, + owned: []string{"enabled", "username", "access_token", "cookies_from", "styles", "max_items"}, body: func(v map[string]string) []config.KeyValue { kv := []config.KeyValue{rawKV("enabled", "true")} if username := strings.TrimSpace(v["username"]); username != "" { @@ -656,11 +654,7 @@ func providers() []providerSpec { if maxItems == "" { maxItems = strconv.Itoa(mixcloud.DefaultMaxItems) } - streamCreators := strings.TrimSpace(v["stream_creators"]) - if streamCreators == "" { - streamCreators = strconv.Itoa(mixcloud.DefaultStreamCreators) - } - return append(kv, rawKV("max_items", maxItems), rawKV("stream_creators", streamCreators)) + return append(kv, rawKV("max_items", maxItems)) }, }, { diff --git a/cmd/setup_test.go b/cmd/setup_test.go index cbfaf21bb..86502bb2a 100644 --- a/cmd/setup_test.go +++ b/cmd/setup_test.go @@ -854,7 +854,6 @@ func TestMixcloudSetupBody(t *testing.T) { "access_token": "token", "styles": "ambient, deep-house", "max_items": " 75 ", - "stream_creators": "15", } if err := spec.extraValidate(values); err != nil { t.Fatalf("extraValidate: %v", err) @@ -862,12 +861,11 @@ func TestMixcloudSetupBody(t *testing.T) { checkBody(t, spec.body(values), map[string]string{ "enabled": "true", "username": `"alice"`, "access_token": `"token"`, "cookies_from": `"firefox"`, "styles": `["ambient", "deep-house"]`, - "max_items": "75", "stream_creators": "15", + "max_items": "75", }) publicOnly := spec.body(map[string]string{ keyMixcloudBrowser: "none", "max_items": "100", - "stream_creators": "20", }) if _, ok := bodyValues(publicOnly)["cookies_from"]; ok { t.Fatalf("public-only session must not write cookies_from: %q", publicOnly) @@ -876,16 +874,15 @@ func TestMixcloudSetupBody(t *testing.T) { keyMixcloudBrowser: "custom", "cookies_from": "chrome:Profile 1", "max_items": "100", - "stream_creators": "20", }) checkBody(t, custom, map[string]string{"cookies_from": `"chrome:Profile 1"`}) if spec.validate != nil { t.Fatal("mixcloud setup should not claim a live validation probe") } - err := spec.extraValidate(map[string]string{"max_items": "bad", "stream_creators": "also bad"}) + err := spec.extraValidate(map[string]string{"max_items": "bad"}) if err == nil || !strings.Contains(err.Error(), "items per view") { - t.Fatalf("validation order error = %v, want items per view first", err) + t.Fatalf("validation error = %v, want items per view", err) } } diff --git a/config.toml.example b/config.toml.example index 305ddacb4..502385fd1 100644 --- a/config.toml.example +++ b/config.toml.example @@ -266,7 +266,7 @@ eq = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] # for your following stream, profile activity, uploads, read-only show favorites, # listening history, collections, and followed-creator browsing. A developer # OAuth access_token makes /me the account identity and adds Listen Later. -# cookies_from is used by yt-dlp for signed-in playback only. styles is the +# cookies_from enables Stream and signed-in playback through yt-dlp. styles is the # local genre-favorites list; toggle it with [f] in Mixcloud's Genres browser. # [mixcloud] # enabled = true @@ -275,7 +275,6 @@ eq = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] # cookies_from = "firefox" # chrome, brave, edge, chromium, opera, safari, vivaldi, whale… # styles = ["ambient", "deep-house", "house", "jazz", "techno"] # max_items = 100 -# stream_creators = 20 # --- # NetEase Cloud Music (optional, opt-in) diff --git a/config/config.go b/config/config.go index efb142d86..265c70fe6 100644 --- a/config/config.go +++ b/config/config.go @@ -565,18 +565,17 @@ func (s *SoundCloudConfig) set(key, val string) { // MixcloudConfig holds settings for the Mixcloud provider. Public discovery // works with only enabled=true. Username adds public account views; an access -// token adds /me and Listen Later; browser cookies are used only by yt-dlp for -// playback that needs the listener's signed-in Mixcloud session. +// token adds /me and Listen Later; browser cookies enable Stream and signed-in +// playback. // Enable rule: opt-in. type MixcloudConfig struct { - Enabled bool - Username string - AccessToken string - CookiesFrom string - Styles []string - StylesSet bool // distinguishes omitted styles (defaults) from an explicit empty list - MaxItems int - StreamCreators int + Enabled bool + Username string + AccessToken string + CookiesFrom string + Styles []string + StylesSet bool // distinguishes omitted styles (defaults) from an explicit empty list + MaxItems int } // IsSet reports whether the Mixcloud provider should be shown. @@ -602,10 +601,6 @@ func (m *MixcloudConfig) set(key, val string) { if v, ok := parseInt(val); ok { m.MaxItems = v } - case "stream_creators": - if v, ok := parseInt(val); ok { - m.StreamCreators = v - } } } diff --git a/config/config_golden_test.go b/config/config_golden_test.go index b37974a10..5850a8e7f 100644 --- a/config/config_golden_test.go +++ b/config/config_golden_test.go @@ -131,7 +131,6 @@ access_token = mc#token cookies_from = "brave" styles = ["ambient", "deep-house"] max_items = 50 -stream_creators = 10 [netease] enabled = true @@ -271,14 +270,13 @@ func TestLoadGolden(t *testing.T) { sectionsWant.Podcast = PodcastConfig{Country: "no"} sectionsWant.SoundCloud = SoundCloudConfig{Enabled: true, User: "sc-user", CookiesFrom: "chrome"} sectionsWant.Mixcloud = MixcloudConfig{ - Enabled: true, - Username: "mc-user", - AccessToken: "mc#token", - CookiesFrom: "brave", - Styles: []string{"ambient", "deep-house"}, - StylesSet: true, - MaxItems: 50, - StreamCreators: 10, + Enabled: true, + Username: "mc-user", + AccessToken: "mc#token", + CookiesFrom: "brave", + Styles: []string{"ambient", "deep-house"}, + StylesSet: true, + MaxItems: 50, } sectionsWant.NetEase = NetEaseConfig{Enabled: true, CookiesFrom: "chrome:Profile 1", CookiesFile: "/home/user/cookies.txt", UserID: "42"} sectionsWant.Yandex = YandexConfig{Enabled: true, Token: "y0_token"} @@ -363,14 +361,13 @@ func TestLoadGolden(t *testing.T) { } allWant.SoundCloud = SoundCloudConfig{Enabled: true, User: "yourname", CookiesFrom: "firefox"} allWant.Mixcloud = MixcloudConfig{ - Enabled: true, - Username: "yourname", - AccessToken: "mc-env-token", - CookiesFrom: "firefox", - Styles: []string{"ambient", "deep-house", "house", "jazz", "techno"}, - StylesSet: true, - MaxItems: 100, - StreamCreators: 20, + Enabled: true, + Username: "yourname", + AccessToken: "mc-env-token", + CookiesFrom: "firefox", + Styles: []string{"ambient", "deep-house", "house", "jazz", "techno"}, + StylesSet: true, + MaxItems: 100, } allWant.NetEase = NetEaseConfig{ Enabled: true, diff --git a/config/mixcloud_test.go b/config/mixcloud_test.go index 5df761ada..60d77453c 100644 --- a/config/mixcloud_test.go +++ b/config/mixcloud_test.go @@ -22,7 +22,6 @@ access_token = "${CLIAMP_TEST_MIXCLOUD_TOKEN}" cookies_from = "firefox" styles = ["ambient", "deep-house"] max_items = 75 -stream_creators = 15 `) if err := os.WriteFile(path, data, 0o600); err != nil { t.Fatal(err) @@ -35,7 +34,7 @@ stream_creators = 15 if !cfg.Mixcloud.IsSet() || cfg.Mixcloud.Username != "alice" || cfg.Mixcloud.AccessToken != "secret-token" || cfg.Mixcloud.CookiesFrom != "firefox" { t.Fatalf("Mixcloud config = %+v", cfg.Mixcloud) } - if !slices.Equal(cfg.Mixcloud.Styles, []string{"ambient", "deep-house"}) || cfg.Mixcloud.MaxItems != 75 || cfg.Mixcloud.StreamCreators != 15 { + if !slices.Equal(cfg.Mixcloud.Styles, []string{"ambient", "deep-house"}) || cfg.Mixcloud.MaxItems != 75 { t.Fatalf("Mixcloud limits/styles = %+v", cfg.Mixcloud) } if !cfg.Mixcloud.StylesSet { diff --git a/docs/configuration.md b/docs/configuration.md index 3853b1248..12df3d3f9 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -473,7 +473,7 @@ Add `username` for your following stream, activity, uploads, read-only show favorites, listening history, collections, and followed-creator browsing. An optional developer `access_token` sets `/me/` as the account identity and adds Listen Later. `cookies_from` gives yt-dlp your signed-in browser session for -playback that requires it. +Stream and playback that requires it. ```toml [mixcloud] @@ -483,7 +483,6 @@ access_token = "${MIXCLOUD_ACCESS_TOKEN}" cookies_from = "firefox" styles = ["ambient", "deep-house", "jazz", "techno"] max_items = 100 -stream_creators = 20 ``` The `styles` list is also the local genre-favorites list for the provider. In diff --git a/docs/mixcloud.md b/docs/mixcloud.md index b4a435c16..b95d41b61 100644 --- a/docs/mixcloud.md +++ b/docs/mixcloud.md @@ -13,7 +13,7 @@ pipeline. Put `yt-dlp` and `ffmpeg` on `PATH`. | Live category catalogue and Latest/Popular genre charts | Provider enabled | **Genres** in the provider pane or `N` browser | | Genre/tag search and local genre favorites | Provider enabled and writable config file | `/`, `Enter`, and `f` in **Genres** | | Public profile activity, uploads, show favorites, listening history, and collections | Public profile `username`, or an `access_token` | **Your Mixcloud** and **Collections** sections | -| Following stream and followed-creator browser | Public profile `username`, or an `access_token` | **Stream (Following Releases)** and **Creators** | +| Following stream and followed-creator browser | Public profile `username`, or an `access_token`; Stream also needs `cookies_from` | **Stream (New Shows)** and **Creators** | | Open a highlighted show's creator Uploads/Favorites | Provider enabled; account not required | Press `N` on a Mixcloud show | | Listen Later | Developer OAuth `access_token` | **Your Mixcloud** section | | Signed-in or subscriber-gated playback | `cookies_from` for a supported browser with a Mixcloud session | Playback through yt-dlp | @@ -65,7 +65,7 @@ Press `X` to select Mixcloud. The provider pane has these sections. | Section | Entry | Behaviour | |---|---|---| -| **Your Mixcloud** | **Stream (Following Releases)** | Merges recent uploads from followed creators. See [Limits and tuning](#limits-and-tuning). | +| **Your Mixcloud** | **Stream (New Shows)** | Reads the signed-in browser account's feed. See [Limits and tuning](#limits-and-tuning). | | **Your Mixcloud** | **Favorites** | Reads the configured public account or the token owner `/me/` connections | | **Your Mixcloud** | **Creators** | Lists the configured account and followed creators. Separates each creator **Uploads** and **Favorites**. | | **Your Mixcloud** | **Uploads**, **Profile Activity**, **Listening History** | Reads the configured public account or the token owner `/me/` connections | @@ -202,15 +202,15 @@ configuration levels: | `enabled = true` | Public discovery, shows, genres, genre/tag search, and show search | | `username` | Public account connections: following, activity, uploads, show favorites, listening history, and collections | | `access_token` | Resolves the authorized user through `/me/` and adds Listen Later | -| `cookies_from` | Gives yt-dlp a signed-in browser session for playback only | +| `cookies_from` | Enables Stream and signed-in playback using the browser session | If both `username` and `access_token` are set, the access-token owner `/me/` -identity controls all account views. This prevents a stale or different public +identity controls REST account views. This prevents a stale or different public username from mixing two accounts in one menu. -Browser cookies do not authenticate the JSON API, reveal private library data, +Browser cookies do not authenticate the public JSON API or replace a developer token. cliamp has no generic "logged in" badge for -cookie playback. It reads cookies only when yt-dlp starts a show. +cookie playback. Stream uses the browser account; yt-dlp also uses its cookies. ## Access token and Listen Later @@ -319,10 +319,9 @@ the player silent. | `enabled` | `false` | Registers the Mixcloud provider | | `username` | empty | Public profile URL username for account views when no token is set | | `access_token` | empty | Developer OAuth token for `/me/` and Listen Later | -| `cookies_from` | empty | Browser/profile sent to yt-dlp for signed-in playback | +| `cookies_from` | empty | Browser/profile used for Stream and signed-in playback | | `styles` | cliamp default style list | Local genre favorites. Each produces Latest and Popular provider rows and show sort modes. | | `max_items` | `100` | Maximum shows, creators, or collections per view. Setup accepts 1 to 500. Manual values at or below zero use the default. Values above 500 use 500. | -| `stream_creators` | `20` | Maximum followed creators for the following stream. Setup accepts 1 to 100. Manual values at or below zero use the default. Values above 100 use 100. | Use this complete example: @@ -336,17 +335,13 @@ access_token = "${MIXCLOUD_ACCESS_TOKEN}" cookies_from = "firefox" styles = ["chillout", "deep-house", "drum-bass", "electronica", "house", "techno"] max_items = 100 -stream_creators = 20 ``` ## Limits and tuning -The API does not expose the exact personalized Mixcloud website home feed. -**Stream (Following Releases)** is an approximation. It gets the first -`stream_creators` followed accounts, merges their newest uploads, sorts by -publication time, removes duplicates, and returns at most `max_items` shows. -Lower these values if you encounter API rate limits. cliamp includes the -Mixcloud `Retry-After` value in the error it shows. +**Stream (New Shows)** reads the website's New Shows feed in order, using +`cookies_from`. `max_items` limits the total. Lower it if you encounter API rate limits. +cliamp includes the Mixcloud `Retry-After` value in the error it shows. Mixcloud does not provide audio streams through its JSON API. The official third-party playback method is the visible web widget. Terminal playback depends diff --git a/external/mixcloud/client.go b/external/mixcloud/client.go index 6326ebae2..2a594b782 100644 --- a/external/mixcloud/client.go +++ b/external/mixcloud/client.go @@ -13,6 +13,7 @@ import ( "time" "github.com/bjarneo/cliamp/internal/httpclient" + "github.com/bjarneo/cliamp/internal/ytdlcookies" ) const ( @@ -49,6 +50,8 @@ type client struct { baseURL string accessToken string httpClient *http.Client + feedURL string + loadCookies func(context.Context, string) (http.CookieJar, error) } func newClient(accessToken string) *client { @@ -56,6 +59,8 @@ func newClient(accessToken string) *client { baseURL: defaultAPIBase, accessToken: strings.TrimSpace(accessToken), httpClient: httpclient.NewAPI(30 * time.Second), + feedURL: "https://app.mixcloud.com/graphql", + loadCookies: ytdlcookies.FromBrowser, } } diff --git a/external/mixcloud/live_test.go b/external/mixcloud/live_test.go index e4b8f04cd..8726052e6 100644 --- a/external/mixcloud/live_test.go +++ b/external/mixcloud/live_test.go @@ -75,11 +75,10 @@ func TestLivePublicAccount(t *testing.T) { } p := NewFromConfig(Config{ - Enabled: true, - Username: username, - StylesSet: true, - MaxItems: 3, - StreamCreators: 2, + Enabled: true, + Username: username, + StylesSet: true, + MaxItems: 3, }) lists, err := p.Playlists() if err != nil { @@ -92,6 +91,9 @@ func TestLivePublicAccount(t *testing.T) { } } for _, id := range wantLists { + if id == streamID { + continue // New Shows requires browser cookies, not public REST access. + } if _, err := p.Tracks(id); err != nil { t.Errorf("account view %q: %v", id, err) } diff --git a/external/mixcloud/new_shows.go b/external/mixcloud/new_shows.go new file mode 100644 index 000000000..2ad19bfa2 --- /dev/null +++ b/external/mixcloud/new_shows.go @@ -0,0 +1,155 @@ +package mixcloud + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "net/http" + "net/url" + "strconv" + "strings" + "time" +) + +const newShowsPageSize = 20 + +// This is the connection used by the website's NewUploadsQuery. Keep its edge +// order: sorting uploads or merging creator catalogs is not the same feed. +// Reading it must not invoke markUploadNotificationsRead or any other mutation. +const newShowsQuery = `query CliampNewShows($count: Int!, $cursor: String) { + viewer { + me { username } + uploadNotifications(first: $count, after: $cursor) { + edges { node { slug name audio_length: audioLength is_exclusive: isExclusive created_time: publishDate user: owner { username name: displayName } } } + pageInfo { endCursor hasNextPage } + } + } +}` + +var errNewShowsSession = errors.New("mixcloud: Stream requires cookies_from with a signed-in Mixcloud browser session") + +type newShowsResponse struct { + Data struct { + Viewer *struct { + Me *struct { + Username string `json:"username"` + } `json:"me"` + Uploads *struct { + Edges []struct { + Node *struct { + apiCloudcast + Slug string `json:"slug"` + } `json:"node"` + } `json:"edges"` + PageInfo *struct { + EndCursor string `json:"endCursor"` + HasNextPage bool `json:"hasNextPage"` + } `json:"pageInfo"` + } `json:"uploadNotifications"` + } `json:"viewer"` + } `json:"data"` + Errors []json.RawMessage `json:"errors"` +} + +func (c *client) newShows(ctx context.Context, browser string, limit int) ([]apiCloudcast, error) { + if browser == "" { + return nil, errNewShowsSession + } + jar, err := c.loadCookies(ctx, browser) + if err != nil { + return nil, fmt.Errorf("mixcloud: load New Shows session: %w", err) + } + endpoint, err := url.Parse(c.feedURL) + if err != nil { + return nil, errors.New("mixcloud: invalid New Shows endpoint") + } + if jar == nil || len(jar.Cookies(endpoint)) == 0 { + return nil, errNewShowsSession + } + // Isolate website cookies from REST requests and never follow a redirect + // with them. OAuth credentials are not sent to this endpoint. + session := *c.httpClient + session.Jar = jar + session.CheckRedirect = func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse } + + limit = max(limit, 1) + shows := make([]apiCloudcast, 0, min(limit, newShowsPageSize)) + var cursor *string + seenCursors := make(map[string]bool) + username := "" + for len(shows) < limit { + variables, _ := json.Marshal(map[string]any{"count": min(newShowsPageSize, limit-len(shows)), "cursor": cursor}) + u := *endpoint + q := u.Query() + q.Set("query", newShowsQuery) + q.Set("variables", string(variables)) + q.Set("operationName", "CliampNewShows") + u.RawQuery = q.Encode() + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil) + if err != nil { + return nil, fmt.Errorf("mixcloud: create New Shows request: %w", err) + } + req.Header.Set("Accept", "application/json") + req.Header.Set("User-Agent", "cliamp-mixcloud/1") + resp, err := session.Do(req) + if err != nil { + return nil, fmt.Errorf("mixcloud: request New Shows: %w", redactRequestURL(err, "Mixcloud New Shows")) + } + if resp.StatusCode != http.StatusOK { + resp.Body.Close() + retry, _ := strconv.Atoi(resp.Header.Get("Retry-After")) + return nil, &APIError{StatusCode: resp.StatusCode, Message: http.StatusText(resp.StatusCode), RetryAfter: time.Duration(max(retry, 0)) * time.Second} + } + var page newShowsResponse + err = json.NewDecoder(io.LimitReader(resp.Body, maxErrorBody)).Decode(&page) + resp.Body.Close() + if err != nil { + return nil, fmt.Errorf("mixcloud: decode New Shows: %w", err) + } + if len(page.Errors) != 0 { + // GraphQL errors can accompany partial data. Do not silently return + // a truncated feed, or echo server text that might contain secrets. + return nil, errors.New("mixcloud: New Shows GraphQL request failed; no partial feed loaded") + } + viewer := page.Data.Viewer + if viewer == nil || viewer.Me == nil || viewer.Me.Username == "" || viewer.Uploads == nil { + return nil, errNewShowsSession + } + if viewer.Uploads.Edges == nil || viewer.Uploads.PageInfo == nil { + return nil, errors.New("mixcloud: New Shows response is incomplete; no partial feed loaded") + } + if username != "" && username != viewer.Me.Username { + return nil, errors.New("mixcloud: New Shows account changed during pagination; reload the stream") + } + username = viewer.Me.Username + for _, edge := range viewer.Uploads.Edges { + if edge.Node == nil { + continue // Deleted shows can have null nodes, with no playable identity. + } + node := edge.Node + if node.User.Username == "" || node.Slug == "" || strings.ContainsAny(node.User.Username+node.Slug, "/?#") { + return nil, errors.New("mixcloud: New Shows returned an invalid show identity") + } + node.Key = "/" + node.User.Username + "/" + node.Slug + "/" + if !validAPIKey(node.Key) { + return nil, errors.New("mixcloud: New Shows returned an invalid show key") + } + shows = append(shows, node.apiCloudcast) + if len(shows) == limit { + return shows, nil + } + } + info := viewer.Uploads.PageInfo + if !info.HasNextPage { + return shows, nil + } + if info.EndCursor == "" || seenCursors[info.EndCursor] { + return nil, errors.New("mixcloud: New Shows pagination did not advance; no partial feed loaded") + } + seenCursors[info.EndCursor] = true + cursor = &info.EndCursor + } + return shows, nil +} diff --git a/external/mixcloud/new_shows_test.go b/external/mixcloud/new_shows_test.go new file mode 100644 index 000000000..1997ed71a --- /dev/null +++ b/external/mixcloud/new_shows_test.go @@ -0,0 +1,246 @@ +package mixcloud + +import ( + "context" + "encoding/json" + "errors" + "net/http" + "net/http/cookiejar" + "net/http/httptest" + "net/url" + "slices" + "strconv" + "strings" + "testing" + "time" +) + +func feedNode(slug, name, date string) map[string]any { + return map[string]any{"slug": slug, "name": name, "audio_length": 3600, "is_exclusive": false, "created_time": date, + "user": map[string]any{"username": "Sam_Maclaren", "name": "Broadcasting"}} +} + +func feedPage(username string, nodes []any, cursor string, next bool) map[string]any { + edges := make([]any, len(nodes)) + for i, node := range nodes { + edges[i] = map[string]any{"node": node} + } + return map[string]any{"data": map[string]any{"viewer": map[string]any{ + "me": map[string]any{"username": username}, + "uploadNotifications": map[string]any{"edges": edges, "pageInfo": map[string]any{"endCursor": cursor, "hasNextPage": next}}, + }}} +} + +func providerWithFeed(t *testing.T, handler http.HandlerFunc) (*Provider, *httptest.Server) { + t.Helper() + server := httptest.NewTLSServer(handler) + p := NewFromConfig(Config{Enabled: true, Username: "configured-public-user", AccessToken: "rest-secret", CookiesFrom: "firefox", MaxItems: 3}) + p.client.httpClient = server.Client() + p.client.feedURL = server.URL + "/graphql" + p.client.baseURL = server.URL + "/rest" + p.client.loadCookies = func(_ context.Context, browser string) (http.CookieJar, error) { + if browser != "firefox" { + t.Errorf("cookie source = %q", browser) + } + u, _ := url.Parse(server.URL) + jar, _ := cookiejar.New(nil) + jar.SetCookies(u, []*http.Cookie{ + {Name: "sessionid", Value: "browser-session", Path: "/", Secure: true, HttpOnly: true}, + {Name: "unrelated", Value: "other-secret", Domain: ".other.example", Path: "/"}, + }) + return jar, nil + } + return p, server +} + +func TestNewShowsPreservesWebsiteOrderAndCursorPagination(t *testing.T) { + var cursors []string + var counts []int + p, server := providerWithFeed(t, func(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodGet || r.URL.Path != "/graphql" { + t.Errorf("unexpected request: %s %s (no aggregation or mutations allowed)", r.Method, r.URL.Path) + http.NotFound(w, r) + return + } + if c, err := r.Cookie("sessionid"); err != nil || c.Value != "browser-session" { + t.Error("website session cookie missing") + } + if strings.Contains(r.Header.Get("Cookie"), "other-secret") || r.URL.Query().Get("access_token") != "" || r.Header.Get("Authorization") != "" { + t.Error("unrelated cookies or OAuth credentials leaked to website feed") + } + if q := r.URL.Query().Get("query"); !strings.Contains(q, "uploadNotifications(first: $count, after: $cursor)") || strings.Contains(q, "mutation") { + t.Errorf("query does not read the website connection: %s", q) + } + var variables struct { + Count int `json:"count"` + Cursor *string `json:"cursor"` + } + if err := json.Unmarshal([]byte(r.URL.Query().Get("variables")), &variables); err != nil { + t.Error(err) + } + counts = append(counts, variables.Count) + cursor := "" + if variables.Cursor != nil { + cursor = *variables.Cursor + } + cursors = append(cursors, cursor) + if cursor == "" { + // The first item is intentionally older: a date-sort shortcut fails. + nodes := []any{ + feedNode("electric-drift-6", "Electric Drift 6", "2026-09-01T00:00:00Z"), + feedNode("newer", "Newer but second", "2026-10-01T00:00:00Z"), + } + for i := len(nodes); i < newShowsPageSize; i++ { + nodes = append(nodes, feedNode(strconv.Itoa(i), strconv.Itoa(i), "2026-10-01T00:00:00Z")) + } + writeJSON(t, w, feedPage("browser-user", nodes, "opaque+/cursor=", true)) + } else { + writeJSON(t, w, feedPage("browser-user", []any{ + feedNode("third", "Third", "2026-10-02T00:00:00Z"), + feedNode("extra", "Beyond limit", "2026-10-03T00:00:00Z"), + }, "end", true)) + } + }) + defer server.Close() + p.maxItems = newShowsPageSize + 1 + + tracks, err := p.Tracks(streamID) + if err != nil { + t.Fatal(err) + } + var titles []string + for _, track := range tracks { + titles = append(titles, track.Title) + } + wantTitles := []string{"Electric Drift 6", "Newer but second"} + for i := 2; i < newShowsPageSize; i++ { + wantTitles = append(wantTitles, strconv.Itoa(i)) + } + wantTitles = append(wantTitles, "Third") + if !slices.Equal(titles, wantTitles) { + t.Fatalf("titles = %v, want website edge order", titles) + } + if tracks[0].Artist != "Broadcasting" || tracks[0].Path != "https://www.mixcloud.com/Sam_Maclaren/electric-drift-6/" || tracks[0].DurationSecs != 3600 || tracks[0].Year != 2026 { + t.Fatalf("first track metadata = %+v", tracks[0]) + } + if !slices.Equal(cursors, []string{"", "opaque+/cursor="}) || !slices.Equal(counts, []int{20, 1}) { + t.Fatalf("cursors = %v, counts = %v", cursors, counts) + } + if p.client.httpClient.Jar != nil { + t.Error("REST HTTP client was changed to carry website cookies") + } +} + +func TestNewShowsRejectsPartialOrInvalidFeed(t *testing.T) { + for _, mode := range []string{"graphql-error", "logged-out", "null-feed", "cursor-cycle", "missing-cursor", "account-changed", "bad-identity", "missing-edges", "missing-page-info", "malformed-json", "rate-limit", "redirect"} { + t.Run(mode, func(t *testing.T) { + requests := 0 + p, server := providerWithFeed(t, func(w http.ResponseWriter, r *http.Request) { + requests++ + page := feedPage("browser-user", []any{feedNode("one", "One", "2026-10-01T00:00:00Z")}, "same", true) + viewer := page["data"].(map[string]any)["viewer"].(map[string]any) + switch mode { + case "graphql-error": + if requests > 1 { + page["errors"] = []any{map[string]any{"message": "session-secret"}} + } + case "logged-out": + viewer["me"] = nil + case "null-feed": + viewer["uploadNotifications"] = nil + case "missing-cursor": + viewer["uploadNotifications"].(map[string]any)["pageInfo"] = map[string]any{"hasNextPage": true} + case "account-changed": + if requests > 1 { + viewer["me"] = map[string]any{"username": "other"} + } + case "bad-identity": + page = feedPage("browser-user", []any{feedNode("../hidden", "Invalid", "2026-10-01T00:00:00Z")}, "", false) + case "missing-edges": + delete(viewer["uploadNotifications"].(map[string]any), "edges") + case "missing-page-info": + delete(viewer["uploadNotifications"].(map[string]any), "pageInfo") + case "malformed-json": + w.Write([]byte(`{"data":`)) + return + case "rate-limit": + w.Header().Set("Retry-After", "12") + w.WriteHeader(http.StatusTooManyRequests) + return + case "redirect": + w.Header().Set("Location", "/unexpected") + w.WriteHeader(http.StatusFound) + return + } + writeJSON(t, w, page) + }) + defer server.Close() + tracks, err := p.Tracks(streamID) + if err == nil || len(tracks) != 0 { + t.Fatalf("tracks=%v error=%v, want no partial feed", tracks, err) + } + if strings.Contains(err.Error(), "session-secret") { + t.Error("server error text exposed session data") + } + if mode == "rate-limit" { + var apiErr *APIError + if !errors.As(err, &apiErr) || apiErr.RetryAfter != 12*time.Second { + t.Fatalf("rate-limit error = %v", err) + } + } + if mode == "redirect" && requests != 1 { + t.Fatalf("followed redirect: %d requests", requests) + } + }) + } +} + +func TestNewShowsRequiresSessionWithoutPublicFallback(t *testing.T) { + p := NewFromConfig(Config{Enabled: true, Username: "alice", AccessToken: "token"}) + p.client.loadCookies = func(context.Context, string) (http.CookieJar, error) { + t.Fatal("unexpected cookie extraction") + return nil, nil + } + _, err := p.Tracks(streamID) + if !errors.Is(err, errNewShowsSession) { + t.Fatalf("error = %v", err) + } + + p.cookiesFrom = "firefox" + p.client.loadCookies = func(context.Context, string) (http.CookieJar, error) { return nil, context.Canceled } + _, err = p.Tracks(streamID) + if !errors.Is(err, context.Canceled) { + t.Fatalf("cancellation = %v", err) + } +} + +func TestNewShowsEmptyAndNullNodes(t *testing.T) { + for _, empty := range []bool{false, true} { + t.Run(strconv.FormatBool(empty), func(t *testing.T) { + requests := 0 + p, server := providerWithFeed(t, func(w http.ResponseWriter, r *http.Request) { + requests++ + if empty { + writeJSON(t, w, feedPage("browser-user", []any{}, "", false)) + } else if requests == 1 { + writeJSON(t, w, feedPage("browser-user", []any{nil}, "null-node-page", true)) + } else { + node := feedNode("same", "Same", "2026-10-01T00:00:00Z") + writeJSON(t, w, feedPage("browser-user", []any{node, node}, "", false)) + } + }) + defer server.Close() + tracks, err := p.Tracks(streamID) + want := 2 + if empty { + want = 0 + } + if err != nil || len(tracks) != want { + t.Fatalf("tracks=%v error=%v", tracks, err) + } + if !empty && requests != 2 { + t.Fatalf("requests=%d, stopped on null-node page", requests) + } + }) + } +} diff --git a/external/mixcloud/provider.go b/external/mixcloud/provider.go index 45d194be2..04fb7f6e9 100644 --- a/external/mixcloud/provider.go +++ b/external/mixcloud/provider.go @@ -8,7 +8,6 @@ import ( "fmt" "net/url" "slices" - "sort" "strings" "sync" "time" @@ -35,18 +34,15 @@ const ( stylePrefix = "style:" creatorUploadsID = "creator:uploads:" creatorFavoritesID = "creator:favorites:" - streamConcurrency = 6 accountSection = "Your Mixcloud" ) // Setup and provider limits are exported so the onboarding UI and runtime // normalization cannot drift apart. const ( - MinItems = 1 - DefaultMaxItems = 100 - MaxItemsLimit = 500 - DefaultStreamCreators = 20 - MaxStreamCreators = 100 + MinItems = 1 + DefaultMaxItems = 100 + MaxItemsLimit = 500 ) var ( @@ -73,15 +69,14 @@ var DefaultStyles = []string{ // Config holds settings for the Mixcloud provider. type Config struct { - Enabled bool - Username string - AccessToken string - CookiesFrom string - Styles []string - StylesSet bool - MaxItems int - StreamCreators int - SaveStyles func([]string) error + Enabled bool + Username string + AccessToken string + CookiesFrom string + Styles []string + StylesSet bool + MaxItems int + SaveStyles func([]string) error } // IsSet reports whether Mixcloud should be registered. @@ -91,12 +86,12 @@ func (c Config) IsSet() bool { return c.Enabled } // stable Mixcloud page URLs rather than extracted media URLs; yt-dlp resolves // the current stream only when playback begins, so queue entries do not expire. type Provider struct { - client *client - username string - styles []string - maxItems int - streamCreators int - saveStyles func([]string) error + client *client + username string + cookiesFrom string + styles []string + maxItems int + saveStyles func([]string) error mu sync.Mutex resolvedUsername string @@ -113,23 +108,18 @@ func NewFromConfig(cfg Config) *Provider { maxItems = DefaultMaxItems } maxItems = min(maxItems, MaxItemsLimit) - streamCreators := cfg.StreamCreators - if streamCreators <= 0 { - streamCreators = DefaultStreamCreators - } - streamCreators = min(streamCreators, MaxStreamCreators) styles := normalizeStyles(cfg.Styles) if len(styles) == 0 && !cfg.StylesSet { styles = slices.Clone(DefaultStyles) } return &Provider{ - client: newClient(cfg.AccessToken), - username: strings.TrimSpace(cfg.Username), - styles: styles, - maxItems: maxItems, - streamCreators: streamCreators, - saveStyles: cfg.SaveStyles, + client: newClient(cfg.AccessToken), + username: strings.TrimSpace(cfg.Username), + cookiesFrom: strings.TrimSpace(cfg.CookiesFrom), + styles: styles, + maxItems: maxItems, + saveStyles: cfg.SaveStyles, } } @@ -189,7 +179,7 @@ func (p *Provider) Playlists() ([]playlist.PlaylistInfo, error) { } lists := []playlist.PlaylistInfo{ - {ID: streamID, Name: "Stream (Following Releases)", Section: accountSection}, + {ID: streamID, Name: "Stream (New Shows)", Section: accountSection}, {ID: favoritesID, Name: "Favorites", Section: accountSection}, {ID: uploadsID, Name: "Uploads", Section: accountSection}, {ID: activityID, Name: "Profile Activity", Section: accountSection}, @@ -237,7 +227,7 @@ func (p *Provider) Tracks(playlistID string) ([]playlist.Track, error) { case popularID: shows, err = p.client.cloudcasts(ctx, "/discover/all/popular/", p.maxItems) case streamID: - shows, err = p.followingStream(ctx) + shows, err = p.client.newShows(ctx, p.cookiesFrom, p.maxItems) case activityID: shows, err = p.profileActivity(ctx) case uploadsID: @@ -618,81 +608,6 @@ func (p *Provider) profileActivity(ctx context.Context) ([]apiCloudcast, error) return dedupeCloudcasts(shows, p.maxItems), nil } -// followingStream approximates Mixcloud's website stream using the documented -// API: fetch followed creators, then merge each creator's newest uploads. -func (p *Provider) followingStream(ctx context.Context) ([]apiCloudcast, error) { - username, err := p.accountUsername(ctx) - if err != nil { - return nil, err - } - users, err := p.client.users(ctx, p.accountConnection(username, "following"), p.streamCreators) - if err != nil { - return nil, fmt.Errorf("mixcloud: load followed creators for stream: %w", err) - } - if len(users) == 0 { - return nil, nil - } - perCreator := max(2, (p.maxItems+len(users)-1)/len(users)) - - type result struct { - shows []apiCloudcast - err error - } - results := make(chan result, len(users)) - sem := make(chan struct{}, streamConcurrency) - var wg sync.WaitGroup - for _, user := range users { - if user.Username == "" { - continue - } - wg.Add(1) - go func(username string) { - defer wg.Done() - select { - case sem <- struct{}{}: - case <-ctx.Done(): - results <- result{err: ctx.Err()} - return - } - defer func() { <-sem }() - shows, err := p.client.cloudcasts(ctx, userPath(username, "cloudcasts"), perCreator) - results <- result{shows: shows, err: err} - }(user.Username) - } - go func() { - wg.Wait() - close(results) - }() - - var ( - shows []apiCloudcast - firstErr error - ) - for result := range results { - if result.err != nil { - if errors.Is(result.err, context.Canceled) || errors.Is(result.err, context.DeadlineExceeded) { - return nil, result.err - } - var apiErr *APIError - if errors.As(result.err, &apiErr) && apiErr.StatusCode == 404 { - // A followed creator may have disappeared between listing and - // loading. Treat only that expected race as an empty source. - continue - } - if firstErr == nil { - firstErr = result.err - } - continue - } - shows = append(shows, result.shows...) - } - if firstErr != nil { - return nil, fmt.Errorf("mixcloud: build following stream: %w", firstErr) - } - sort.SliceStable(shows, func(i, j int) bool { return shows[i].CreatedTime.After(shows[j].CreatedTime) }) - return dedupeCloudcasts(shows, p.maxItems), nil -} - func (p *Provider) albumSortPath(sortType string) (string, bool) { switch sortType { case "", "latest": diff --git a/external/mixcloud/provider_test.go b/external/mixcloud/provider_test.go index 088ad0dd0..cfd94564c 100644 --- a/external/mixcloud/provider_test.go +++ b/external/mixcloud/provider_test.go @@ -9,7 +9,6 @@ import ( "strconv" "strings" "testing" - "time" "github.com/bjarneo/cliamp/playlist" "github.com/bjarneo/cliamp/provider" @@ -46,8 +45,8 @@ func TestNewFromConfigAndStyles(t *testing.T) { if got, want := p.styles, []string{"deep-house", "drum-bass"}; !slices.Equal(got, want) { t.Fatalf("styles = %v, want %v", got, want) } - if p.maxItems != DefaultMaxItems || p.streamCreators != DefaultStreamCreators { - t.Fatalf("defaults = max %d stream %d", p.maxItems, p.streamCreators) + if p.maxItems != DefaultMaxItems { + t.Fatalf("max items = %d, want %d", p.maxItems, DefaultMaxItems) } } @@ -96,11 +95,6 @@ func TestBrowseEntriesExposeShowsCreatorsAndGenres(t *testing.T) { if entries[0].AfterID != favoritesID { t.Fatalf("creators placement = %+v, want immediately after Favorites", entries[0]) } - - publicEntries := NewFromConfig(Config{Enabled: true}).BrowseEntries() - if len(publicEntries) != 3 || publicEntries[0].ID != browseCreatorsID || !publicEntries[0].OpenInPlaylist { - t.Fatalf("public browse capabilities = %+v, want config-independent creator behavior", publicEntries) - } } func TestAlbumListFillsShortAPIPage(t *testing.T) { @@ -417,71 +411,6 @@ func TestAccountPlaylistsUseMeWithToken(t *testing.T) { } } -func TestFollowingStreamMergesNewestAndDeduplicates(t *testing.T) { - handler := func(w http.ResponseWriter, r *http.Request) { - switch r.URL.Path { - case "/alice/following/": - writeJSON(t, w, map[string]any{"data": []any{ - map[string]any{"username": "one"}, map[string]any{"username": "two"}, - }}) - case "/one/cloudcasts/": - writeJSON(t, w, map[string]any{"data": []any{ - map[string]any{"key": "/one/old/", "name": "Old", "created_time": "2026-01-01T00:00:00Z", "user": map[string]any{"username": "one"}}, - map[string]any{"key": "/shared/show/", "name": "Shared", "created_time": "2026-03-01T00:00:00Z", "user": map[string]any{"username": "shared"}}, - }}) - case "/two/cloudcasts/": - writeJSON(t, w, map[string]any{"data": []any{ - map[string]any{"key": "/two/new/", "name": "New", "created_time": "2026-04-01T00:00:00Z", "user": map[string]any{"username": "two"}}, - map[string]any{"key": "/shared/show/", "name": "Shared", "created_time": "2026-03-01T00:00:00Z", "user": map[string]any{"username": "shared"}}, - }}) - default: - http.NotFound(w, r) - } - } - p, server := providerWithServer(t, Config{Enabled: true, Username: "alice", MaxItems: 10, StreamCreators: 2}, handler) - defer server.Close() - - tracks, err := p.Tracks(streamID) - if err != nil { - t.Fatalf("Tracks(stream): %v", err) - } - if len(tracks) != 3 { - t.Fatalf("tracks = %d, want 3 merged and deduplicated shows", len(tracks)) - } - if got := []string{tracks[0].Title, tracks[1].Title, tracks[2].Title}; !slices.Equal(got, []string{"New", "Shared", "Old"}) { - t.Fatalf("titles = %v", got) - } -} - -func TestFollowingStreamPreservesPartialRateLimit(t *testing.T) { - handler := func(w http.ResponseWriter, r *http.Request) { - switch r.URL.Path { - case "/alice/following/": - writeJSON(t, w, map[string]any{"data": []any{ - map[string]any{"username": "working"}, map[string]any{"username": "limited"}, - }}) - case "/working/cloudcasts/": - writeJSON(t, w, map[string]any{"data": []any{ - map[string]any{"key": "/working/show/", "name": "Show", "user": map[string]any{"username": "working"}}, - }}) - case "/limited/cloudcasts/": - w.Header().Set("Retry-After", "12") - w.WriteHeader(http.StatusForbidden) - writeJSON(t, w, map[string]any{"error": map[string]any{"type": "RateLimitException", "message": "slow down"}}) - default: - http.NotFound(w, r) - } - } - p, server := providerWithServer(t, Config{Enabled: true, Username: "alice", MaxItems: 10, StreamCreators: 2}, handler) - defer server.Close() - - _, err := p.Tracks(streamID) - var apiErr *APIError - if !errors.As(err, &apiErr) || apiErr.RetryAfter != 12*time.Second { - t.Fatalf("error = %T %v, want preserved rate limit", err, err) - } -} - func TestProviderRejectsMalformedRecordsAndKeys(t *testing.T) { if got := tracksFromCloudcasts([]apiCloudcast{{Name: "missing identity"}}); len(got) != 0 { t.Fatalf("tracks = %v, want malformed item omitted", got) diff --git a/internal/ytdlcookies/browser.go b/internal/ytdlcookies/browser.go new file mode 100644 index 000000000..3c2b565bc --- /dev/null +++ b/internal/ytdlcookies/browser.go @@ -0,0 +1,98 @@ +package ytdlcookies + +import ( + "bufio" + "context" + "fmt" + "net/http" + "net/http/cookiejar" + "net/url" + "os" + "os/exec" + "path/filepath" + "strconv" + "strings" + "time" + + "golang.org/x/net/publicsuffix" +) + +// FromBrowser exports cookies through yt-dlp without visiting a website or +// downloading media. The cookie file exists only inside a private temp directory. +func FromBrowser(ctx context.Context, browser string) (http.CookieJar, error) { + if strings.TrimSpace(browser) == "" { + return nil, fmt.Errorf("browser cookie source is required") + } + dir, err := os.MkdirTemp("", "cliamp-browser-cookies-*") + if err != nil { + return nil, fmt.Errorf("create private cookie directory: %w", err) + } + defer os.RemoveAll(dir) + path := filepath.Join(dir, "cookies.txt") + if err := os.WriteFile(path, []byte("# Netscape HTTP Cookie File\n"), 0600); err != nil { + return nil, fmt.Errorf("prepare cookie file: %w", err) + } + + ctx, cancel := context.WithTimeout(ctx, 30*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, "yt-dlp", + "--ignore-config", "--no-cache-dir", "--simulate", + "--cookies-from-browser", browser, "--cookies", path, + "--load-info-json", "-", + ) + cmd.Stdin = strings.NewReader(`[]`) + // Do not include yt-dlp output in errors: it may contain private cookie data. + if err := cmd.Run(); err != nil { + if ctx.Err() != nil { + err = ctx.Err() + } + return nil, fmt.Errorf("read browser cookies with yt-dlp (check browser profile and keyring): %w", err) + } + f, err := os.Open(path) + if err != nil { + return nil, err + } + defer f.Close() + jar, err := cookiejar.New(&cookiejar.Options{PublicSuffixList: publicsuffix.List}) + if err != nil { + return nil, err + } + scanner := bufio.NewScanner(f) + for scanner.Scan() { + line := scanner.Text() + if strings.HasPrefix(line, "#") && !strings.HasPrefix(line, "#HttpOnly_") { + continue + } + fields := strings.Split(line, "\t") + if len(fields) != 7 { + continue + } + expires, err := strconv.ParseInt(fields[4], 10, 64) + if err != nil { + continue + } + cookie := &http.Cookie{ + Domain: strings.TrimPrefix(fields[0], "#HttpOnly_"), + Path: fields[2], + Secure: fields[3] == "TRUE", + HttpOnly: strings.HasPrefix(fields[0], "#HttpOnly_"), + Name: fields[5], + Value: fields[6], + } + if expires != 0 { + cookie.Expires = time.Unix(expires, 0) + } + if cookie.Valid() != nil { + continue + } + origin := &url.URL{Scheme: "https", Host: strings.TrimPrefix(cookie.Domain, ".")} + if fields[1] == "FALSE" { + cookie.Domain = "" // Host-only cookies must not reach subdomains. + } + jar.SetCookies(origin, []*http.Cookie{cookie}) + } + if err := scanner.Err(); err != nil { + return nil, err + } + return jar, nil +} diff --git a/internal/ytdlcookies/browser_test.go b/internal/ytdlcookies/browser_test.go new file mode 100644 index 000000000..0335eaa43 --- /dev/null +++ b/internal/ytdlcookies/browser_test.go @@ -0,0 +1,113 @@ +package ytdlcookies + +import ( + "context" + "errors" + "net/url" + "os" + "path/filepath" + "runtime" + "strings" + "testing" +) + +const browserCookieFixture = "# Netscape HTTP Cookie File\n" + + "#HttpOnly_.mixcloud.com\tTRUE\t/\tTRUE\t0\tsessionid\tsession-value\n" + + ".mixcloud.com\tTRUE\t/private\tTRUE\t0\tprivate\tprivate-value\n" + + ".mixcloud.com\tTRUE\t/\tTRUE\t1\texpired\texpired-value\n" + + ".other.example\tTRUE\t/\tTRUE\t0\tother\tother-value\n" + + "mixcloud.com\tFALSE\t/\tTRUE\t0\troot-only\troot-value\n" + + "www.mixcloud.com\tFALSE\t/\tTRUE\t0\twww-only\twww-value\n" + + "app.mixcloud.com\tFALSE\t/\tTRUE\t0\tapp-only\tapp-value\n" + + "# not a cookie\nmalformed\n" + + ".mixcloud.com\tTRUE\t/\tTRUE\t0\tbad name\tvalue\n" + +func fakeCookieExporter(t *testing.T, script string) string { + t.Helper() + if runtime.GOOS == "windows" { + t.Skip("shell-based fake yt-dlp") + } + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "yt-dlp"), []byte("#!/bin/sh\n"+script), 0700); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", dir+string(os.PathListSeparator)+os.Getenv("PATH")) + return dir +} + +func TestFromBrowserExportsOfflineAndCleansUp(t *testing.T) { + dir := fakeCookieExporter(t, `printf '%s\n' "$@" > "$FAKE_ARGS" +cat > "$FAKE_STDIN" +while [ "$#" -gt 0 ]; do + if [ "$1" = '--cookies' ]; then shift; cookie_path="$1"; fi + shift +done +cp -p "$cookie_path" "$FAKE_INITIAL" +printf '%s' "$FAKE_COOKIES" > "$cookie_path" +`) + argsPath := filepath.Join(dir, "args") + stdinPath := filepath.Join(dir, "stdin") + initialPath := filepath.Join(dir, "initial") + t.Setenv("FAKE_ARGS", argsPath) + t.Setenv("FAKE_STDIN", stdinPath) + t.Setenv("FAKE_INITIAL", initialPath) + t.Setenv("FAKE_COOKIES", browserCookieFixture) + jar, err := FromBrowser(context.Background(), "firefox:profile") + if err != nil { + t.Fatal(err) + } + for host, want := range map[string]string{ + "mixcloud.com": "sessionid,root-only", + "www.mixcloud.com": "sessionid,www-only", + "app.mixcloud.com": "sessionid,app-only", + "sub.app.mixcloud.com": "sessionid", + } { + endpoint, _ := url.Parse("https://" + host + "/graphql") + var names []string + for _, cookie := range jar.Cookies(endpoint) { + names = append(names, cookie.Name) + } + if got := strings.Join(names, ","); got != want { + t.Errorf("cookies for %s = %s, want %s", host, got, want) + } + } + stdin, err := os.ReadFile(stdinPath) + if err != nil || string(stdin) != "[]" { + t.Fatalf("offline input = %s, error=%v", stdin, err) + } + args, err := os.ReadFile(argsPath) + if err != nil { + t.Fatal(err) + } + for _, want := range []string{"--ignore-config", "--no-cache-dir", "--simulate", "--cookies-from-browser\nfirefox:profile", "--load-info-json\n-"} { + if !strings.Contains(string(args), want) { + t.Errorf("missing cookie export option %s", want) + } + } + info, err := os.Stat(initialPath) + if err != nil || info.Mode().Perm() != 0600 { + t.Fatalf("initial cookie file permissions: %v, %v", info, err) + } + lines := strings.Split(string(args), "\n") + for i, line := range lines { + if line == "--cookies" { + if _, err := os.Stat(filepath.Dir(lines[i+1])); !errors.Is(err, os.ErrNotExist) { + t.Fatalf("private cookie directory not removed: %v", err) + } + } + } +} + +func TestFromBrowserFailureDoesNotExposeCookieOutput(t *testing.T) { + fakeCookieExporter(t, "echo synthetic-secret\necho synthetic-secret >&2\nexit 1\n") + _, err := FromBrowser(context.Background(), "firefox") + if err == nil || strings.Contains(err.Error(), "synthetic-secret") { + t.Fatalf("unsafe error: %v", err) + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + _, err = FromBrowser(ctx, "firefox") + if !errors.Is(err, context.Canceled) { + t.Fatalf("cancellation = %v", err) + } +} diff --git a/providers.go b/providers.go index 60f3140e6..3054ad2ae 100644 --- a/providers.go +++ b/providers.go @@ -224,15 +224,14 @@ func buildProviders(cfg config.Config, interactive bool) *providerSet { add("soundcloud", p) } if p := mixcloud.NewFromConfig(mixcloud.Config{ - Enabled: cfg.Mixcloud.Enabled, - Username: cfg.Mixcloud.Username, - AccessToken: cfg.Mixcloud.AccessToken, - CookiesFrom: cfg.Mixcloud.CookiesFrom, - Styles: cfg.Mixcloud.Styles, - StylesSet: cfg.Mixcloud.StylesSet, - MaxItems: cfg.Mixcloud.MaxItems, - StreamCreators: cfg.Mixcloud.StreamCreators, - SaveStyles: config.SaveMixcloudStyles, + Enabled: cfg.Mixcloud.Enabled, + Username: cfg.Mixcloud.Username, + AccessToken: cfg.Mixcloud.AccessToken, + CookiesFrom: cfg.Mixcloud.CookiesFrom, + Styles: cfg.Mixcloud.Styles, + StylesSet: cfg.Mixcloud.StylesSet, + MaxItems: cfg.Mixcloud.MaxItems, + SaveStyles: config.SaveMixcloudStyles, }); p != nil { add("mixcloud", p) } diff --git a/ui/model/view.go b/ui/model/view.go index d5d66ed7e..177b730ec 100644 --- a/ui/model/view.go +++ b/ui/model/view.go @@ -211,7 +211,7 @@ func (m Model) radioPlayingHere(id string) bool { func playlistLabel(prefix string, p playlist.PlaylistInfo) string { out := prefix + terminalText(p.Name) var parts []string - if p.TrackCount > 0 || p.Name == favorites.PlaylistName { + if p.TrackCount > 0 || p.ID == favorites.PlaylistName { parts = append(parts, fmt.Sprintf("%d tracks", p.TrackCount)) } if p.DirSourceCount == 0 { diff --git a/ui/model/view_helpers_test.go b/ui/model/view_helpers_test.go index 621bb4548..24bfd3636 100644 --- a/ui/model/view_helpers_test.go +++ b/ui/model/view_helpers_test.go @@ -165,9 +165,15 @@ func TestPlaylistLabel(t *testing.T) { { "favorites shows zero count", " ", - playlist.PlaylistInfo{Name: favorites.PlaylistName}, + playlist.PlaylistInfo{ID: favorites.PlaylistName, Name: favorites.PlaylistName}, " Favorites · 0 tracks", }, + { + "provider favorites with unknown count", + " ", + playlist.PlaylistInfo{ID: "account:favorites", Name: favorites.PlaylistName}, + " Favorites", + }, { "favorites with tracks", " ", From 58bad76b3c139be06c43a16301aa91bd8a5094b1 Mon Sep 17 00:00:00 2001 From: Kingsley Faulkner Date: Mon, 19 Oct 2026 13:23:29 +0100 Subject: [PATCH 2/4] docs: clarify Mixcloud feed requests Distinguish cliamp's New Shows GraphQL requests from yt-dlp cookie loading and playback in the configuration reference and example.\n\nVerified with go test -count=1 ./config, go test -count=1 docs_links_test.go, and git diff --check. --- config.toml.example | 5 +++-- docs/configuration.md | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/config.toml.example b/config.toml.example index 502385fd1..eda8adc2c 100644 --- a/config.toml.example +++ b/config.toml.example @@ -266,8 +266,9 @@ eq = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] # for your following stream, profile activity, uploads, read-only show favorites, # listening history, collections, and followed-creator browsing. A developer # OAuth access_token makes /me the account identity and adds Listen Later. -# cookies_from enables Stream and signed-in playback through yt-dlp. styles is the -# local genre-favorites list; toggle it with [f] in Mixcloud's Genres browser. +# cookies_from supplies browser cookies via yt-dlp for cliamp's New Shows GraphQL +# requests and signed-in playback. yt-dlp handles playback, not feed requests. +# styles is the local genre-favorites list; toggle it with [f] in Mixcloud's Genres browser. # [mixcloud] # enabled = true # username = "yourname" diff --git a/docs/configuration.md b/docs/configuration.md index 12df3d3f9..e15b66334 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -472,8 +472,9 @@ enabled = true Add `username` for your following stream, activity, uploads, read-only show favorites, listening history, collections, and followed-creator browsing. An optional developer `access_token` sets `/me/` as the account identity and adds -Listen Later. `cookies_from` gives yt-dlp your signed-in browser session for -Stream and playback that requires it. +Listen Later. For Stream (New Shows), cliamp makes GraphQL requests using browser +cookies loaded through yt-dlp via `cookies_from`. Playback uses yt-dlp, with the +same browser session when required. ```toml [mixcloud] From c33361350275539bade975eab89c2b13034f4607 Mon Sep 17 00:00:00 2001 From: Kingsley Faulkner Date: Mon, 19 Oct 2026 13:29:16 +0100 Subject: [PATCH 3/4] docs: correct Mixcloud username features Remove the obsolete following-stream claim from the username descriptions. New Shows uses the signed-in browser session instead. Verified with config tests, documentation-link checks, and git diff --check. --- config.toml.example | 2 +- docs/configuration.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/config.toml.example b/config.toml.example index eda8adc2c..29d348054 100644 --- a/config.toml.example +++ b/config.toml.example @@ -263,7 +263,7 @@ eq = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] # Mixcloud (optional, opt-in) # Public discovery, show search, the live genre catalogue, genre/tag search, # creator jumps, and Latest/Popular charts work with enabled=true. Set username -# for your following stream, profile activity, uploads, read-only show favorites, +# for profile activity, uploads, read-only show favorites, # listening history, collections, and followed-creator browsing. A developer # OAuth access_token makes /me the account identity and adds Listen Later. # cookies_from supplies browser cookies via yt-dlp for cliamp's New Shows GraphQL diff --git a/docs/configuration.md b/docs/configuration.md index e15b66334..166b6667c 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -469,7 +469,7 @@ native show search, direct creator jumps, and playback need no account: enabled = true ``` -Add `username` for your following stream, activity, uploads, read-only show +Add `username` for activity, uploads, read-only show favorites, listening history, collections, and followed-creator browsing. An optional developer `access_token` sets `/me/` as the account identity and adds Listen Later. For Stream (New Shows), cliamp makes GraphQL requests using browser From 9e67f03aa11af0090a63cc285497b2f1260c1a38 Mon Sep 17 00:00:00 2001 From: Kingsley Faulkner Date: Thu, 22 Oct 2026 21:08:37 +0100 Subject: [PATCH 4/4] mixcloud: address PR 613 feedback Hide Stream when browser cookies are not configured, retaining the other account views. Share the offline browser-cookie exporter between Mixcloud and NetEase, keeping private temporary storage and cleanup. Update regression tests and documentation. Verified with focused race tests and go vet for ytdlcookies, Mixcloud and NetEase, documentation-link tests, and git diff --check. Mixcloud UAT passed on macOS, including website ordering, username-only browsing, playback and invalid-profile errors. Live NetEase UAT was unavailable because no account was available. --- cmd/setup.go | 2 +- docs/mixcloud.md | 8 ++- external/mixcloud/live_test.go | 8 +-- external/mixcloud/provider.go | 17 +++-- external/mixcloud/provider_test.go | 68 ++++++++++++++++++- external/netease/provider.go | 52 ++++----------- external/netease/provider_test.go | 104 +++++++++++++++++++++++++---- internal/ytdlcookies/browser.go | 25 +++++-- nix/package.nix | 2 +- site/index.html | 2 +- 10 files changed, 210 insertions(+), 78 deletions(-) diff --git a/cmd/setup.go b/cmd/setup.go index 3fe4785eb..5638461eb 100644 --- a/cmd/setup.go +++ b/cmd/setup.go @@ -587,7 +587,7 @@ func providers() []providerSpec { section: "mixcloud", intro: []string{ "Browse and play Mixcloud shows through its public API and yt-dlp.", - "A username adds your public library, following stream, and creators.", + "A username adds your public library and followed creators.", "An optional developer access token adds /me and Listen Later.", "Choose browser cookies for Stream and signed-in playback.", "Docs: cliamp.stream → docs/mixcloud.md", diff --git a/docs/mixcloud.md b/docs/mixcloud.md index b95d41b61..a3ba2325f 100644 --- a/docs/mixcloud.md +++ b/docs/mixcloud.md @@ -65,7 +65,7 @@ Press `X` to select Mixcloud. The provider pane has these sections. | Section | Entry | Behaviour | |---|---|---| -| **Your Mixcloud** | **Stream (New Shows)** | Reads the signed-in browser account's feed. See [Limits and tuning](#limits-and-tuning). | +| **Your Mixcloud** | **Stream (New Shows)** | Appears only when `cookies_from` is set. Reads the signed-in browser account's feed. See [Limits and tuning](#limits-and-tuning). | | **Your Mixcloud** | **Favorites** | Reads the configured public account or the token owner `/me/` connections | | **Your Mixcloud** | **Creators** | Lists the configured account and followed creators. Separates each creator **Uploads** and **Favorites**. | | **Your Mixcloud** | **Uploads**, **Profile Activity**, **Listening History** | Reads the configured public account or the token owner `/me/` connections | @@ -76,8 +76,10 @@ Press `X` to select Mixcloud. The provider pane has these sections. | **Discover** | **Recent Releases**, **Popular** | Global public discovery. No account required. | | **Music Styles** | `