Mon Havre de Liberté

This ticket is a proposition of the adviced way of using “Feature Discovery and Knowledge” efficiently. Indeed the current reality in Jabber/XMPP implementations and my work on my PubSub plugin showed me some really bad behaviours of both servers and clients which mostly lead to possible data loss. The principal reason is probably that the current XEPs were focalized on the idea that only the client had to know what the server can do, hence acting according to this knowledge. But the reality is that the server should also know what the client can do before sending anything.

I began this ticket a few days ago and finalized it after a discussion on the Standards mailing list. It is mostly based on 3 existing XEPs: Service Discovery, Entity Capabilities and Resource Application Priority

Constatations of wrong Software Behaviours

The current state of the art of many Jabber softwares (let’s call them Entity, whether they are clients, servers or services) exchanging with other softwares is simply to “try and test”: an entity sends something and “sees” if the other entity understands. This can be ok sometimes — depending on the feature — but this is not the most efficient way. The worse case is when some of your messages and queries are lost and that you never know about it! Let’s have some examples:

1. Imagine you use a Jabber client with only Instant Messaging possibilities (which is nearly the case for most Jabber client currently). Then you receive a very important PubSub notification from a node you are subscribed at. But your client does not know how to handle it, hence discards it. You will never know you have received such notification.
   This is what does currently ejabberd. I saw in an XML console it sends me PubSub notifications from nodes I am subscribed at, even though my client didn’t advertize pubsub support through Entity Capabilities.
2. At the opposite, you may have a feed client which can subscribe and receive Jabber notifications. But this is a feed-only client and it does not have any IM feature. Then your wife/husband does not see you in his/her roster (in the best case. In the worst, it sees you and think you are online for discussion. I will discuss this point later) but she/he sends you anyway an (offline, so she thinks) message, because she knows you will connect some time during the day and receive it. And indeed you connect later… but never receive the message! So your wife/husband is mad at you tonight because she thinks you ignored her message.
   And why did you not receive it? Simply becauce the connected feed client received it on your behalf and discarded this message as it does not process IM messages.

And so how will you avoid these cases, and so many others which will happen as the number of features of Jabber will increase? Service discovery is already often used, but mainly from client towards server or services (indeed this is why it is called Service Discovery and not Feature/Capabilities Discovery). And even this is not well used (for instance, using Psi and its XML console feature, I noticed it keeps on sending ping query every minute even though my server answers continuously in error because it does not know this feature! But maybe is it just as some kind of keep-alive query?).
Moreover this is barely used in the opposite way: from the server to the client. So currently most servers just send anything they get destined to a connected and available JID with positive priority.

Reference XEPs

Nevertheless even though the Service Discovery uses always examples of a client to server query, it can probably be extended in the opposite direction server-to-client (or client-to-client, or maybe some day — why not? — service-to-client if we imagined evoluted services with presence support), as I could notice such a use in the Ad-Hoc Command feature.

There is an even a better feature: the Entity Capabilities, which is globally the same feature as the Service Discovery, except that it is initiated by the client. Hence it avoids some “disco query” flood because you (thus your server) don’t have to ask their implemented features to all your contacts, they directly send them when they connect and a good server would probably keep this information in cache memory when someone else needs. So it is probably the better choice.

And finally the last interesting XEP is Resource Application Priority (RAP) which enables a client to set “Application Priorities” (which is nothing else than a Per Feature priority system).

As a conclusion, I will propose a Best Practice solution where all clients should advertise their implemented capabilities and treat other clients/servers/services accordingly to their known possibilities and where servers treats all other clients/servers according to their known possibilities too.

Proposed Practices

IM features vs all other features

As XMPP is an Instant Messaging (IM) protocol on a first basis, the only feature which is treated differently as others are IM features (mostly basic “chat”), in particular because there is no declared <feature> element to advertise basic IM capabilities, nor is there any Extended Namespace in a simple chat message (which is defined in the main jabber:client namespace). For this reason, when all XEPs discuss a way to advertise extended features, none seems to tell the way to advertise absence of IM or presence feature, as though it is obviously present in the “jabber:client” namespace. Probably at the beginning of such specifications, specs writers did not foresee that the protocol would extend so much that clients without IM would be a good idea. Yet one can read that it has been imagined (section 12 of rfc3921) at least once, the case of a client without presence OR instant messaging support:

Note also that a server or client MAY support only presence or instant messaging, and is not required to support both if only a presence service or an instant messaging service is desired.

Of course one could imagine to treat IM and presence as any other feature and add some namespace/feature name for this, like “jabber:client:im”, “jabber:client:presence” or simply “http://www.ietf.org/rfc/rfc3920.txt” for both. But doing this has a major drawback: it will break all current servers and clients’ IM implementation (if a server does not forward a message to a client because this latter has not updated itself with the new IM namespace, this is a big issue), thus will never be implemented. A proposed solution is:

• It is not necessary to advertise non-support of presence because if you don’t send any presence, anyway you will never receive any stanza by another entity on its own (you are considered as unavailable by default). So any exchange will begin from your initiative.
• To advertize non-support for IM, you simply set a negative priority in your presence stanza.

The consequence for this is that this priority MUST correspond only to the basic jabber:client namespace, and especially only to basic IM, not to other extended features. These extended feature SHOULD have priorities on their own. This is where the Resource Application Priority concept is used, which corresponds to a system of per-extended-namespaces priority system.
I think that it is not necessary to send a RAP child element in the presence stanza in most case because the associated capability is already announced through Entity Capabilities and we can consider that if no RAP element is specified for an advertized capability, then the default RAP value for the feature is 0 (as is the default priority in the default namespace, used now for IM only). But a RAP child element must be added in the presence stanza when the user wants to disable a feature (negative RAP), or when he uses several clients providing same feature and wants to set clients priorities depending on the functionality.

Receiving Entities

Server behaviour when receiving a presence with negative Priority

Changing the meaning of the normal priority element changes one Server Rule for Handling XML Inbound Stanzas: the point 4.1 of section 11.1 when the JID is of the form “user @domain”, there is at least one available resource available for the user, but all resource have a negative priority. In such a case, a stanza MUST be sent anyway to this resource if it has at least one child element with an extended namespace which has been advertized as being featured by the client and which has a positive RAP. Else it is processed as though there were no available resource.

This modification in the meaning of priority has the advantage not breaking the current implementations of IM clients, whether the server follows these recommandations or not.

Client behaviour when receiving a presence with negative Priority

Here is how should behave a client which receives a presence element from a remote client with a negative priority and a list of supported feature.

• If the remote client has a negative priority and the current client has itself only IM capacities, then the current client MUST NOT display the availability of the remote client, as though it was unavailable (so looking not connected in the roster typically).
• If the remote client has a negative priority and the current client has other capabilities than IM but none of them corresponds to one of the remote client’s capabilities, the current client MUST NOT display the availability of the remote client, as though it was unavailable.
• If the remote client has a negative priority and the current client has other capabilities than IM and some (at least one) of them corresponds to some of its own capabilities, then it MAY display the availability of the remote client, showing explicitely the absence of IM capabilities with this resource if necessary (by instance by graying it).

Connecting Client

In the next examples, the client may or not have IM capabilities. Priority is set to X in order to show that the exchange would be the same in both case, except for the priority being set to a positive or negative number. From now on, IM capability is considered as any other feature (even though it is advertized differently).

First case: the client knows “Entity Capabilities” and aknowledges the server thanks to this

1/ The current client has just authenticated, and now sends its presence with entity capability (c) element:

<presence from='romeo@montague.lit/orchard'>
    <c xmlns='http://jabber.org/protocol/caps' hash='sha-1'
        node='http://code.google.com/p/exodus'
        ver='QgayPKawpkPSDYmwT/WM94uAlu0='/>
    <show>dnd</show>
    <status>Wooing Juliet</status>
    <status xml:lang='cz'>Ja dvořím Juliet</status>
    <priority>X</priority>
    <rap xmlns='http://www.xmpp.org/extensions/xep-0168.html#ns'
     ns='http://www.xmpp.org/extensions/xep-0167#ns'
     num='5'/>
</presence>

This presence elements advertizes that the client is “http://code.google.com/p/exodus” (with version signature “QgayPKawpkPSDYmwT/WM94uAlu0=”) which implies a set of known features, that the IM feature has a priority of X (IM activated depending of the sign of X), that the specific “http://www.xmpp.org/extensions/xep-0167#ns” namespace (Voice chat feature) is supported with a priority of 5, and that all other supported features (as implied by the previous node/version) are activated with a priority of 0.

2/ Several feature discovery exchanges may occure between the current client and the remote client (and with the server of both) following the XEP about Entity Capabilities.

3/ In this context, not only the remote client, but also the current client’s server SHOULD know by now this client’s capabilities. It therefore knows the implemented features and can forward the right informations (messages, pubsub notifications, etc.) to the right resources if necessary.

If some third party entity tries to send a stanza to the current entity:

1. If this stanza has at least one child element without any extended namespace (thus in the default namespace), the server MUST act accordingly to the normal Server Rules for Handling XML Stanzas. This will imply the stanza possibly sent to an available resource as long as the priority is positive because the corresponding client SHOULD be able to interpret at least part of the stanza (the default “xmpp:client” one), even though it MAY not be able to interpret other parts (extended namespace parts).
2. If this stanza has only child elements with extended namespaces and at least one of these namespaces is supported by an available resource of the recipient entity, the server MUST act accordingly to the normal Server Rules for Handling XML Stanzas with the priority being replaced by the maximum Resource Application Priority of all common namespaces. This will imply the stanza possibly sent as long as one of the RAP of the common features is positive, because the client SHOULD be able to interpret at least part of the stanza (the common extended namespaces with positive RAP), even though it MAY not be able to interpret other parts (other extended namespaces).
3. If this stanza has only child elements with extended namespaces and none of these namespaces is supported by the recipient entity, then the server MUST handle the stanza as though there is no available resource able to handle it.

   In other words, if it is an IQ, the server MUST answer on behalf of the user (with result or error); if it is a message, the server MUST handle the message as though there were no available resource (for instance by offline message storage until a featured resource gets available); and if it is a presence stanza, the server MUST handle it as though there were no available resource (storing subscription but discarding other presence usually).

Second case: client does NOT know “Entity Capabilities

1/ The client has just authenticated, and now sends its presence element without c element:

<presence from='romeo@montague.lit/orchard'>
    <show>dnd</show>
    <status>Wooing Juliet</status>
    <status xml:lang='cz'>Ja dvořím Juliet</status>
    <priority>X</priority>
</presence>

2/ The server thus sends a disco info:

<iq type='get'
    from='montague.lit'
    to='romeo@montague.lit/orchard'
    id='disco1'>
    <query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>

3/ The client knows “Service Discovery” and answers with its implemented features:

<iq from='romeo@montague.lit/orchard'
    id='disco1'
    to='montague.lit'
    type='result'>
    <query xmlns='http://jabber.org/protocol/disco#info'>
        <identity category='client' name='Exodus 0.9.1' type='pc'/>
        <feature var='http://jabber.org/protocol/caps'/>
        <feature var='http://jabber.org/protocol/disco#info'/>
        <feature var='http://jabber.org/protocol/disco#items'/>
        <feature var='http://jabber.org/protocol/muc'/>
    </query>
</iq>

3 bis/ The client does not know “Service Discovery” and sends:

<iq type='error'
    from='plays.shakespeare.lit'
    to='romeo@montague.net/orchard'
    id='info1'>
    <query xmlns='http://jabber.org/protocol/disco#info'/>
    <error type='cancel'>
        <service -unavailable xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    </error>
</iq>

Note: it could also be a “feature-not-implemented” error.

In this last case, the server cannot know what the client does implement. So in the doubt, it will send all messages and IQ query destined to this JID. This is up to the user to get another client with a better implementation.

Already connected clients knowing each other capabilities

Considering a client knows the supported features of another entity, it should not send any stanza requiring the recipient being a full JID (with a resource) if this specific recipient client did not advertize the capacity of the sent stanza (but it can send a stanza with several extended namespaces, then the recipient must support at least one of the namespace).

If a client sends a stanza supporting a bare recipient JID (without resource), then it is up to the server to decide whether to forward it, store it for latter forward, etc.

Conclusion

Note that — though I think a common Jabber client may support the whole possible features — I noticed many people considered that different features should, or at least could, be dealt by different programs (for instance I was once told on a Jabber mailing list by a person who would prefer its IM client not receiving the pubsub notifications, but rather its feed client which already deals with RSS). And somehow they are right too. But for this, there need to be a process for the clients and the server to know what to send to which resource (here this is not only about priority, but especially about implemented features!).

As a conclusion, the best case is when any entity knows the “entity capabilities” feature with a mix of “service discovery” and “resource application priority”, with the standard “priority” being redefined into “priority for default namespace only”. From then, the server will know the known features by any other entities, hence it can forward the right information (chat messages, presences and all sort of queries like pubsub notifications or Ad-Hoc commands, etc.). If some other client tries to send a stanza unfeatured by this client, the server can directly answer with a result or an error (”<service-unavailable/>”), forward the stanza to a resource which can process this, keep in a waiting queue for the time when such a resource connects, etc.
This whole proposition will enable users having many different clients with different features, and enabling them to move easily from a computer to another with the same JID and no data loss.

It can certainly be improved. But this version or another, I am really certain some final recommandation and/or change in XEPs will have to be issued on this topic if we want Jabber becoming more than just an Instant Messaging system with some fun gadgets (which most extended features are currently with present implementations: just barely usable gadgets).

This entry was posted on Jeudi, juillet 3rd, 2008 at 4:03 and is filed under Jabber/xmpp, Réflexion. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.