what is the scope?

[andrewrk]

What is the extent of the functionality that this API intends to cover?

Does it intend to be a read-only interface? Does it intend to support importing into the library or modifying tags?

[sampsyo]

Good question. We should try to be more explicit about this in the introduction.

The intention is to cover everything that a "generic" music library would want to do. I've intentionally tried to keep the "core" small, but I plan to add extensions to cover more general functionality. To answer your specific question, the core will be read-only, but I would like to (relatively soon) add writability as an extension. My intuition currently says that importing should not be specified (since the behavior, at least for beets, is so idiosyncratic), but maybe there's a good design that multiple projects can agree in somewhere.

Does that help? What should we add to index.rst to clarify the scope?

[sampsyo]

Oops, I also meant to link to the Potential Features wiki page, where I've been collecting ideas (and would love to collect more, if you have any!).

[andrewrk]

Potential features wiki page is what I was looking for. I added a section: https://github.com/sampsyo/aura/wiki/Potential-Features#real-time-data

I suppose we can keep the issue open so we have somewhere to discuss the potential features.

[sampsyo]

Awesome; thanks for the addition. Change pushes are an important feature; you're absolutely right. I can't tell you how frustrating MPD's state-polling can be…

I'm adding you to the project in case you ever want to write up initial thoughts on this kind of thing. I'd also be interested in reading more about how this works in Groove Basin—is there some source file (or even documentation) that would be illuminating?

[andrewrk]

No documentation yet; I want to get a few more things done on the roadmap before releasing a stable version of the protocol. But here are a couple source files:

code that handles a client subscription. Send them the data immediately (if the version they have is outdated), and then set them up so that they get all updates. Client can decide if they want to receive deltas or not.
https://github.com/andrewrk/groovebasin/blob/fb57f91dc875a29cad32a644faba9db871be5a22/lib/player_server.js#L174

code that server calls when it wants to set up something that you can subscribe to. Give it the name that you subscribe to, and a function that serializes the information being subscribed to into a javascript object or value. It returns a function that you call when the information has been updated.
https://github.com/andrewrk/groovebasin/blob/fb57f91dc875a29cad32a644faba9db871be5a22/lib/player_server.js#L681

examples of things being set up just above that: https://github.com/andrewrk/groovebasin/blob/fb57f91dc875a29cad32a644faba9db871be5a22/lib/player_server.js#L551

client code that subscribes to the server code:
https://github.com/andrewrk/groovebasin/blob/fb57f91dc875a29cad32a644faba9db871be5a22/src/client/playerclient.js#L159

client code that responds to a subscription message: https://github.com/andrewrk/groovebasin/blob/fb57f91dc875a29cad32a644faba9db871be5a22/src/client/playerclient.js#L83

[sampsyo]

Fantastic; thanks for the links. I see you're using an existing JSON diff/patch algorithm for arbitrary objects, which seems like a great idea. I'll keep looking into this—I'm excited about the prospect of real-time metadata updates.

[Afterster]

Real time updates with WebSocket / SSE is indeed something interesting. I was only aware about this use with web streaming based solution on Plex API to notify about administrative / server running tasks, but I didn't realized GrooveBasin was working this way for the complete music collection.
Any reason you didn't provided also a query interface (code duplication, ...)?

[andrewrk]

Any reason you didn't provided also a query interface

The official groove basin client downloads all the music metadata to the browser so that the user can type in a filter query and quickly see results without making a round trip to the server. (see example at demo.groovebasin.com) Further, this simplifies the code because everything deals with indexes into the database structure instead of caching tags and remembering when to clear/update the cache.

It does have a weakness of being slow with very large libraries. My library is about 7100 songs and the web client takes about 3 seconds to load the index, depending on connection of course. Some people I hear have libraries upwards of 127,000 songs which could get quite slow trying to download the entire index.

groovebasin does support querying via the MPD protocol. But ideally applications which want to use groove basin will not use the MPD protocol and instead upgrade to the groovebasin protocol instead (or perhaps the aura protocol if it becomes powerful enough). So if a user needed querying ability, I would add that to the protocol. It would still be a subscription model, however. You subscribe to a search, you get the search results, and if the result to that search ever change, you get an update pushed.

[Afterster]

Thanks for your clarification. The way GrooveBasin is working is really interesting (and a bit unusual).
But I don't think this should be the default and mandatory implementation for a standard API. Simply because it will not work for several servers and clients.
For instance, Ampache is a PHP application and even if it has some WebSocket feature to broadcast a listening song to other web player, only few usesr use it as it requires complex web server configuration that cannot be achieve on most web server.
It could be an extension but then we should be careful to not split again the community and to impose a mandatory set of API calls that must be implemented by server and clients to always be Aura compatible.

[andrewrk]

But I don't think this should be the default and mandatory implementation for a standard API

I agree. So what I think this means is that the groove basin protocol is unique enough that it makes sense for it to exist independently from other protocols such as MPD and aura.

[sampsyo]

I agree that streaming will be optional, but I don't see a huge gap, fundamentally, between the base API and a potential streaming extension. It appears possible to gracefully add streaming in a way that works well for both kinds of clients.

I 100% understand that Groove Basin may do best with its own, native protocol, but it would be useful to build a protocol that could be comfortably employed by something Groove-Basin-like. 😃 So I'm still interested in learning whatever lessons we can from how GB succeeds already.

[andrewrk]

If aura became good enough to fit groove basin's needs, I would use it in lieu of a special protocol.