Skip to content

Commit 1c8dafe

Browse files
committed
Lightweight method to get the client id on SoundCloud
- Instead of downloading the whole ~1MB file, download just the first 16KB. With fallback to the old way in case of fail - I've come across some technical difficulties as to how to limit the download (current implementation is too basic/simple for that), but in the end, a simple http header (Range) was enough
1 parent 77a74b8 commit 1c8dafe

1 file changed

Lines changed: 16 additions & 8 deletions

File tree

extractor/src/main/java/org/schabi/newpipe/extractor/services/soundcloud/SoundcloudParsingHelper.java

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
import java.text.ParseException;
2323
import java.text.SimpleDateFormat;
2424
import java.util.Date;
25+
import java.util.HashMap;
2526

2627
import static org.schabi.newpipe.extractor.utils.Utils.replaceHttpWithHttps;
2728

@@ -35,18 +36,25 @@ public static String clientId() throws ReCaptchaException, IOException, RegexExc
3536
if (clientId != null && !clientId.isEmpty()) return clientId;
3637

3738
Downloader dl = NewPipe.getDownloader();
38-
3939
String response = dl.download("https://soundcloud.com");
40-
Document doc = Jsoup.parse(response);
4140

42-
// TODO: Find a less heavy way to get the client_id
43-
// Currently we are downloading a 1MB file (!) just to get the client_id,
44-
// youtube-dl don't have a way too, they are just hardcoding and updating it when it becomes invalid.
45-
// The embed mode has a way to get it, but we still have to download a heavy file (~800KB).
41+
Document doc = Jsoup.parse(response);
4642
Element jsElement = doc.select("script[src^=https://a-v2.sndcdn.com/assets/app]").first();
47-
String js = dl.download(jsElement.attr("src"));
4843

49-
return clientId = Parser.matchGroup1(",client_id:\"(.*?)\"", js);
44+
final String clientIdPattern = ",client_id:\"(.*?)\"";
45+
46+
try {
47+
final HashMap<String, String> headers = new HashMap<>();
48+
headers.put("Range", "bytes=0-16384");
49+
String js = dl.download(jsElement.attr("src"), headers);
50+
51+
return clientId = Parser.matchGroup1(clientIdPattern, js);
52+
} catch (IOException | RegexException ignored) {
53+
// Ignore it and proceed to download the whole js file
54+
}
55+
56+
String js = dl.download(jsElement.attr("src"));
57+
return clientId = Parser.matchGroup1(clientIdPattern, js);
5058
}
5159

5260
public static String toDateString(String time) throws ParsingException {

0 commit comments

Comments
 (0)