Skip to content

Commit f4931d8

Browse files
committed
[YouTube] Workaround 403s on streaming URLs of WEB client after some time
These changes work around an anti-bot token, for which its requirement is A/B tested on the WEB client. In this test, streaming URLs of this client return HTTP errors 403 if the token is not provided after some time. It also allows to not fetch the JavaScript player for non-age restricted videos, reducing data usage. The TVHTML5 embed client is now only fetched in the case of age-restricted videos. The methods forceFetchAndroidClient and forceFetchIosClient of YoutubeStreamExtractor have been removed, as they are now not needed anymore. These changes also break the extraction of appropriate error states for private and deleted videos and invalid video IDs.
1 parent 312e910 commit f4931d8

2 files changed

Lines changed: 139 additions & 177 deletions

File tree

extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/YoutubeParsingHelper.java

Lines changed: 37 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1301,31 +1301,49 @@ public static JsonBuilder<JsonObject> prepareTvHtml5EmbedJsonBuilder(
13011301
}
13021302

13031303
@Nonnull
1304-
public static byte[] createDesktopPlayerBody(
1304+
public static JsonObject getWebPlayerResponse(
1305+
@Nonnull final Localization localization,
1306+
@Nonnull final ContentCountry contentCountry,
1307+
@Nonnull final String videoId) throws IOException, ExtractionException {
1308+
final byte[] body = JsonWriter.string(
1309+
prepareDesktopJsonBuilder(localization, contentCountry)
1310+
.value(VIDEO_ID, videoId)
1311+
.value(CONTENT_CHECK_OK, true)
1312+
.value(RACY_CHECK_OK, true)
1313+
.done())
1314+
.getBytes(StandardCharsets.UTF_8);
1315+
final String url = YOUTUBEI_V1_URL + "player" + "?" + DISABLE_PRETTY_PRINT_PARAMETER
1316+
+ "&$fields=microformat,playabilityStatus,storyboards,videoDetails";
1317+
1318+
return JsonUtils.toJsonObject(getValidJsonResponseBody(
1319+
getDownloader().postWithContentTypeJson(
1320+
url, getYouTubeHeaders(), body, localization)));
1321+
}
1322+
1323+
@Nonnull
1324+
public static byte[] createTvHtml5EmbedPlayerBody(
13051325
@Nonnull final Localization localization,
13061326
@Nonnull final ContentCountry contentCountry,
13071327
@Nonnull final String videoId,
13081328
@Nonnull final Integer sts,
1309-
final boolean isTvHtml5DesktopJsonBuilder,
1310-
@Nonnull final String contentPlaybackNonce) throws IOException, ExtractionException {
1329+
@Nonnull final String contentPlaybackNonce) {
13111330
// @formatter:off
1312-
return JsonWriter.string((isTvHtml5DesktopJsonBuilder
1313-
? prepareTvHtml5EmbedJsonBuilder(localization, contentCountry, videoId)
1314-
: prepareDesktopJsonBuilder(localization, contentCountry))
1315-
.object("playbackContext")
1316-
.object("contentPlaybackContext")
1317-
// Signature timestamp from the JavaScript base player is needed to get
1318-
// working obfuscated URLs
1319-
.value("signatureTimestamp", sts)
1320-
.value("referer", "https://www.youtube.com/watch?v=" + videoId)
1331+
return JsonWriter.string(
1332+
prepareTvHtml5EmbedJsonBuilder(localization, contentCountry, videoId)
1333+
.object("playbackContext")
1334+
.object("contentPlaybackContext")
1335+
// Signature timestamp from the JavaScript base player is needed to get
1336+
// working obfuscated URLs
1337+
.value("signatureTimestamp", sts)
1338+
.value("referer", "https://www.youtube.com/watch?v=" + videoId)
1339+
.end()
13211340
.end()
1322-
.end()
1323-
.value(CPN, contentPlaybackNonce)
1324-
.value(VIDEO_ID, videoId)
1325-
.value(CONTENT_CHECK_OK, true)
1326-
.value(RACY_CHECK_OK, true)
1327-
.done())
1328-
.getBytes(StandardCharsets.UTF_8);
1341+
.value(CPN, contentPlaybackNonce)
1342+
.value(VIDEO_ID, videoId)
1343+
.value(CONTENT_CHECK_OK, true)
1344+
.value(RACY_CHECK_OK, true)
1345+
.done())
1346+
.getBytes(StandardCharsets.UTF_8);
13291347
// @formatter:on
13301348
}
13311349

0 commit comments

Comments
 (0)