Application crashes when user adds large music collection

[Giggeli]

I researched that when I add more than 61 cd albums (there is a few ep's) unplayer crashes every time when I select "update library". Note that whole my music collection is in flac format aka. lossless codec. After this situation I runned unplayer with sailfish command line as follows command: harbour-unplayer. So here is the output. I also tried to separate my music collection to few folders but same behavior; crash. I even tried to unselect "prefer cover art located as separate file…", no effect.

I was using sailfish 3.0.0.8 and unplayer was 1.4.1 release.

Afterall unfortunately this bug make unplayer for me show stopper. Let me know if you need some logs about unplayer. And of course if something more information needed!

Kind regards: Joakim

[equeim]

Unfortunately, there is no easy way to get such crash logs in Sailfish OS, but it would help me a lot if you did this.
Here is the instructions:

1. Run devel-su in terminal and enter password that you set for SSH access in developer settings
2. Run pkcon install gdb to install a debugger
3. Install both packages from archive using file manager or pkcon install-local from terminal
4. Exit to normal shell (exit or CTRL+D)
5. Run gdb -ex run harbour-unplayer, then reproduce the crash. When it will crash, app's window won't close, instead it will become unresponsive. In terminal you will see GDB's command prompt. Run bt in it to get a backtrace and post it here.

rpms.zip

[Giggeli]

It's long time ago when I already found solution (without debugger) for this issue. The cause of this is one of my album what I ripped from cd in windows software "Cueripper". I will check it out in another player and check is it corrupted or what. Note that almost all of my music is ripped by this "Cueripper".

I'm sorry that I was away so long, also for this probably unnecessary issue. So I will be back soon!

[Giggeli]

I tried different players and most of them acts weird like starting play middle of song (tried to play same album what causes unplayer crash). For now I still will test with debugger to see actual error and give output here, "stay tuned".

Thanks so far for instructions they might be useful also in other situations in future.

[FabriceColin]

I meant to file a report on this topic, so I'll comment here.
I have around 160GB of MP3s I rsync regularly to a SD card. Every time, I have to get unplayer to update the library. From scratch, it's last taken 2 days on my original Jolla phone.
I have the following suggestions:

• provide visual feedback on the library building process, in terms of the number of directories left to process or maybe which alphabetical letter is being imported if directories can be imported in alphabetical order?
• provide incremental upgrade, i.e. import files newer that the latest file recorded in the current library
• provide a command-line utility to reset/recreate/update the library. This could be the basis for tools to import libraries from third-party music players. I'd be interested into I could further automate my iTunes-rsync process.
  Thanks!

[equeim]

provide visual feedback on the library building process, in terms of the number of directories left to process or maybe which alphabetical letter is being imported if directories can be imported in alphabetical order?

I'm not sure how this could be done. Unplayer interates over directories recursively, which means it doesn't know how many files or directories it will iterate over beforehand. One way is to scan filesystem first to put supported files in the list and then iterate over that list and do necessary checks, etc. I'm afraid that it could slow down scanning even more, but I need to test it to determine it for sure.

provide incremental upgrade, i.e. import files newer that the latest file recorded in the current library

Unplayer already does this. Do you use latest version? Previous versions had a couple of bugs regarding library scanning which prevented incremental update from working properly. Unplayer currently stores last modification timestamp from filesystem in the SQLite database (alongside extracted tags) and when updating the library it checks it agaist the new timestamp. If the new timestamp is different, it extracts tags again. Maybe it's just iteration over large number of files which takes a long time, but I can't imagine how it can take 2 days even on Jolla 1 (I myself test it only on my Jolla 1) and FAT32 filesystem (you mean 2 days only for initial scan, or for subsequent scans too?). Anyway currently Unplayer is not optimised for collections this large, I should test it myself to determine where the bottleneck is.

provide a command-line utility to reset/recreate/update the library.

This could easily be done.

[FabriceColin]

I guess to start with, it doesn't have to know how many files there are. Showing what it's doing might be enough, i.e. what album is being imported.
I think it was a full update with the latest version. I will try "Update Library" with new media later on.
Thanks for considering a command line utility!

[equeim]

I have around 160GB of MP3s I rsync regularly to a SD card. Every time, I have to get unplayer to update the library. From scratch, it's last taken 2 days on my original Jolla phone.

Okay, I did some tests with 20 GB of 10000 mp3 files and Unplayer scans them for about 15-20 minutes on my Jolla 1. I managed to do some minor optimizations, but the bottleneck was I/O performance. How slow is your SD card (maybe it is dying)? Also, does directory tree that is configured for scanning in Unplayer contains only music or also a lot of other files too? Currently Unplayer is not very good in such cases. These optimizations improve this, but there is not much I can do.

[FabriceColin]

Okay, I did some tests with 20 GB of 10000 mp3 files and Unplayer scans them for about 15-20 minutes on my Jolla 1. I managed to do some minor optimizations, but the bottleneck was I/O performance. How slow is your SD card (maybe it is dying)? Also, does directory tree that is configured for scanning in Unplayer contains only music or also a lot of other files too? Currently Unplayer is not very good in such cases. These optimizations improve this, but there is not much I can do.

That's good to hear, in any case. I haven't tested that SD card for speed. The tree only contains MP3s straight from iTunes. I look forward to trying these optimisations in the next version! Thanks!