Bugzilla – Bug 11127
SqueezePlay protocol optimizations for touch screen devices
Last modified: 2009-09-08 09:27:06 UTC
I have no idea, how the protocol formerly known as JiveMLON is supposed to be called now, so I will just stick to referring to it as "MLON". This is an enhancement request that I file as a basis for discussion around how the SqueezePlay menu protocol (MLON) could be optimized to be able to support other classes of devices than the SBC. I cam across a few issues while implementing a MLON renderer for iPeng but I believe the same will apply for FAB4. It will cover context menus, under "Suggestions"->4., if you want the shortcut :-) I did already file an earlier ER on a sub-issue, the scope of this bug is included here but it is more concrete in it's formulation: https://bugs-archive.lyrion.org/show_bug.cgi?id=11005 Issues to be addressed: 1. Implementation dependance. The MLON protocol as it is today is deeply bound to the implementation it runs on - notably SBC. This has implications in a few areas, such as: - Commands are bound to the hardware keys on the device and the actions on these keys, not on the functionality ("play", "add", "add-hold"). - MLON assumes a single-screen, single-thread interface. This means, it assumes any action taken on the actually visible screen will never have implications on other screens unless the screen is left. It always assumes a new screen, even if on a higher level, will always be refreshed when the screen is being left in spite of the fact that there are concrete refresh commands in the "nextWindow" parameter. This especially applies to "input" and "slider" controls, an application like iPeng that does NOT provide a separate screen for these controls will find them difficult to render. This is especially true for pages fully included in other pages, that is without a calling command. - On some elements MLON makes assumptions on the implementation, e.g. for input fields it assumes the input field will be in a separate window, even though it does not specify one. For input fields, this means that all "parent" actions act on the menu level where they are defined instead of the level above. This is IMHO even an implementation error. 2. Command syntax and prioritisation is horribly complex and illogical I am not sure I already fully understood this issue to depth since the documentation is incomplete and wrong but from what I can see, the command syntax goes like: IF there is an embedded, lower level page, go to that page ELSE IF there is an "action" command for the key pressed, use that ELSE IF there is a "do" command, use that ELSE IF there is a "go" command, use that ELSE use "do" or "go" from "base" fields for the page OK, so far so straightforward, however, determining what has to happen next is more complex: IF the command is to use an embedded page, open it, ELSE IF there is a "do" or an "action" command other than "go", execute a command THEN, IF there is a "nextWindow" parameter, use that parameter, ELSE IF there is one of ("radio", "choice", "checkbox" or "ipnput") refresh the current window (equivalent to "nextWindow" => "refresh") ELSE IF there is a "go" command, use it to open a new page 3. The protocol does not provide globality/locality Due to 2., a lot of processing has to be done before all the consequences of an action for a field are clear, especially whether a new window will be openend and which control elements will be needed for an item. While this is fine on the SBC where you scroll through a list and can process the information for an element while doing so, on a touch screen you will want to show these for all elements at once which will create a lot of processing overhead 4. The protocol requires a renderer to browse the whole table before rendering the context The same as 3. goes for "global" control elements. If you want to provide command chices that apply to the whole screen, you can only know which commands will be needed when the full list has been rendered. 5. Icon and Icon-id are mixed up Suggestions I would suggest to improve the protocol in the following way. I am aware that this will be an incompatible change and thus will requite a protocol version indicator (and I would have to re-implement the renderer in iPeng myself) but I believe it's a good basis for discussion. 1. Name commands independent of key or functional hierarchy. This is more or less what I proposed in bug 1105. actions : { <command1> : {...} <command2> : {...} ... } 2. Attribute Titles and Icons for commands so that they can be displayed expalinig their functionality <command> : { ... text : "...", icon : "...", } 3. Attribute "nextWindow" to the command instead of the item, make "go" a parameter of "nextWindow" <command> : { ... nextWindow : <whatever> } 4. Attribute default actions to categories, mandatory (I understand it is possible today, yet undocumented) defaultAction : <command> contextMenuAction : <command1> helpAction : <command2> playAction: <command> ... This way, the renderer can decide how to process actions. For example, it could display all actions in a context menu (note: this is something else than the "contextMenuAction" example above) or it could use function buttons. The categorization is not meant to pre-empt what happens but to help the renderer categorize the commands. 5. Announce all commands present in the page in "base" base : { defaultActions : [<command1>, <command2>,...], helpActions : [...], ... otherActions : [...] } This way, a renderer will know upfront what has to be displayed on the screen and will be able to lay out the content accordingly. 6. Announce the elements contained in a page base : { elementsUsed : [slider, icon, input,...] } Rationale: same as 5. So, lots of text, hope you are ready to read and discuss this.
As the original author of JiveMLON, I feel qualified enough to comment - even though the protocol has evolved quite a bit since I last touched it. So you can blame for most of the original inconsistencies. As a general design rule, the protocol tried to solve the general problem of browsing music, not the general problem of rendering stuff on a screen - i.e. it expected the renderer to "know" a lot of things, if possible in C so it would be too slow. Time moves on, and given Fab4, I agree the original design will probably not scale. I think I agree to most of your proposed changes, however there are two things I do not understand from your proposal: * How is command priority expressed ? * How are commands mapped to physical buttons (in SBC) ? Your proposal 5 and 6 may be hard to implement, in particular in the light of paging: the server must always go through the complete table before returning results.
Fred, thanks for commenting. I see, just rendering music is a quite different task. Today, however, you've got things like "Alarms" that render pretty complex menu screens with little content (compared to music tables) and quite a lot of plugins use MLON this way. On your two questions: * How is command priority expressed ? * How are commands mapped to physical buttons (in SBC) ? There is no implied priority. The protocol should add a "class" to the command, equivalent of giving information on what kind of command this is. This is similar to what is defined today with the "addAction" etc. parameters, but without mapping actions to buttons. The decision on how a command is mapped is then up to the client. The big advantage is, that you can have very different mappings for different clients. SBC could map actions to appropriate buttons, FAB4 could probably present icons or textual buttons. Any command that has a class that is not known to the client would then only show up in a context menu or be ignored. The server can be agnostic of the UI model of the client. Proposals 5 and 6: I don't fully get your comment. It's true that this would mean some work on the side of the server, BUT You mention paging: For paging it will be especially important to have those suggestions implemented if you want global layout decisions since the client would have no other chance to know which content will be on the pages that are not already loaded. Also, thinking about it, I'm not sure paging applies here. Typically, you only have lots of paging for data with similar content. The server module or plugin should know which kind of content a page will present, pages typically only page between similar content. To give examples: - I don't know if you know iPeng. To "emulate" the "play", "add" and "insert" actions on the SBC, iPeng has a "mode" for a "current command", that is, you can toggle through these commands and whichever is active can then be applied to any element in the list by a gesture (simple tap for lowest level, touch-and-hold for higher level menus). It also uses this paradigm for the button mappings when rendering MLON. To do this, it needs information on the content of the page. Today, it renders the whole list, for pages data, additional buttons will be added later, with suggestions 5 and 6 this would be much more efficient. On FAB4, this could be used to generate a "dictionary" of gestures or to provide appropriate layout options, columns, row sizes for buttons, you name it.
I'd like to add proposal #7 re issue 1: Currently, all menus are player dependent, which is fine in a single threaded environment where you also typically only control one player. However, on more complex interfaces (this may me more of an issue for desktop SP than FAB4, it is one for iPeng), you might want to quickly switch between players. Today, this means that all menus will be closed and you have to re-start browsing because the whole menu structure is built from scratch. I can't see a need for this at least for the big browse tables (Artists, Albums...) and it's definitely annoying if you browse through a long list, find the track you want to play, then find out that you are connected to the wrong player and have to re-browse again. A solution would be to allow menus that have player-independent content to be delivered for all players. I believe the functionality is already in the subscriptions, it's just not being used. Proposal: Allow "all" identifier for menus that are player independent and do not discard these menus whenever the player but not the music source is changing. A distinguising could be done using the "menustatus" subscription by returning two sets of menus, one specific for the player information is being subscribed for and one for items valid on all players. Query: { "id" : <n>, "data" : { "request" : ["<player>", [ "menustatus"]], "response" : "/<client-id>/slim/menustatus/<player>" }, "channel" : "/slim/subscribe" } Response (on channel "/<client-id/slim/menustaus/<player>", no need to distinguish here since the client is still only connected to one player at a time): [ { "data" : [ "menustatus", [<menus for for all players>], "add", "all" ], "id" : <n>, "channel" : "/<client-id>/slim/menustatus/<player>", "ext" : {...} }, { "data" : [ "menustatus", [<menus for for specific player>], "add", "<player>" ], "id" : <n>, "channel" : "/<client-id>/slim/menustatus/<player>", "ext" : {...} } ] As said before, this seems to be in the protocol already (Erland's plugins use it) but it should be used for the main browse menus to be able to stay within a menu hierarchy while switching players.
let's talk about this on friday
OK :-)
Moving to the product SqueezePlay because this bug appears to apply to any player based on that application code. Feel free to move it back if it's specific to the original product.