Minutes of the 2018 Summit: Day one
1. XMPP 2.0
1.1. Message Routing
Matt asks whether servers couldn't just send full-JID messages to all clients, regardless of whether the client connected used BIND2 (i.e. asking for XMPP2) or not.
Kev: suggests annotation to full-JID messages, so that the above could be made to work.
Guus: what happens to clients that expected
Talking about rewriting JID from full-JID to bare-JID.
jonasw: if we stick to carbon rules, we wouldn't degrade in the XMPP 1.0 case. Kev: doing this rewriting is hard, because we'd need to know the original intention.
Discussion ensues on how many clients do resource locking versus sending to bare JIDs.
- Dino, Conversations and converse.js send to bare JID already.
- Gajim seems to be doing resource locking.
1.2 Stable IDs
Discussion on entropy ensues.
dwd: <stanza-ids> don't work for unique addressing because we don't trust clients to do them properly.
Kev: lots of possible attacks with spoofable stanza IDs.
1.3 Unread state
Problem of unread state synchronizing.
When a client comes online, it wants to know what to show the user immediately. So we want to know about unread chats.
Kev: I want a single stanza to tell me from whom I have unread messages. I can then query those JIDs to get the messages (when it considers it appropriate).
Discussion on whether this can be done with PEP ensued.
Kev talked himself into thinking it's perhaps a good idea and then out of it again, saying that the mechanics are different enought to warrant some other solution.
1.4 Inbox (Open Chats)
Michal's "Inbox" idea is to similar to (what I would describe as) "open chats", i.e. which conversations are currently ongoing.
mattw: Private PEP node with a list of JIDs
jonasw: like the inbox idea, but don't think we need server-side info. I can think of usecases where you not necessarily want to synchronize across devices.
Kev: well, if you do want to sync, it makes sense to have a pubsub node.
Pubsub node solves the case where you don't want certain conversations to be in your "inbox", i.e. a conversation from a spammer.
Client is responsible for setting this pubsub node, although nothing stops server from populating if required in some cases.
Although, removing chats from the inbox requires user input and thus the client involvement.
Daniel: if someone writes a ProtoXEP, I'll implement in Conversations soon.
1.5 Presence Sorting
I missed the discussion around this, sorry!
1.6 Metadata (for showing read/delivery receipts)
Kev: suggesting when you query the archive, you get besides the usual some metadata:
- This was delivered to X
- This was ready by X
This is to avoid making additional MAM queries in order to figure out delivery or read state of messages.
Backwards compatibility issue:
Currently only Conversations (AFAWK) has read receipts functionality based on MAM messages.
By default read receipts should not be archived, but hints could have been added to store them or the server could decide to store them anyway.
Sam: what if you on-the-fly injected the read receipts for older clients?
Kev: could cause trouble when querying, either by amount of messages or when querying on the id of an "injected" receipt which is not actually archived (but only injected on the fly).
Kev: this functionality of Conversations is not dependent on standard usage, we're "in the weeds" here. So is probably fine to not care about backwards compatibility here. Conversations will be updated soon after in any case.
1.7 Room synchronisation
MUC - synchronisation amongst different clients.
Kev: Could add bookmark with auto-join on join and remove on leave.
Daniel: ultimate solution is PEP. Workaround that Conversations currently does is to send direct invite to own JID. Doesn't cover leaving of MUC though.
I missed the discussion around this, sorry!
Kev: if you think of something like Slack or Discord... you have the ability to set per-channel notifications of messages.
- It changes what's displayed in the UI as unread messages.
- Affects what you get push notifications for.
We should do this for XMPP.
Kev: I think we should define a fairly simple set of (extensible) filter rules.
Kev: probably want global default and overrides.
Push notifications need to go out based on these.
Encrypted messages will break "push" because the server can't read the message content.
Could make E2E work with mentions, by adding references to who is being mentioned. See XEP-0372.
Edwin: does mean that you're leaking plaintext.
Discussion ensues on "profiles", so that you can sync your notification preferences across devices.
1.10 Fast Connections (Roundtrips)
dwd: Sasl2 on its own reduces one roundtrip and allows us to combine other roundtrips.
dwd: We are making a lot of headway in this. Cold-reconnections need work however.
Kev: BIND2 might help by binding everything in a single query.
dwd: need to be careful of initial requests coming through before you're authenticated.
dwd: initial parts might also fall under TLS 1.3 early data dwd: in other words, we might be getting sufficiently fast that we're outpacing security.
dwd: SASL does not allow pipelining (i.e. opportunistic sending of responses).
Discussion went a bit over my head
dwd: there are risks there, but believes them to be manageable.
Guus: how much does this save?
Kev: I think 3 round trips.
dwd: could put in bind request. Could put in information of the unread state. condensing as many roundtrips as we can into potentially a single roundtrip.
Kev: we also have the post-nominally setup, but not actually setup stage, of having an authenticated stream, but there is a lot of data to receive until the client is set up.
For example, the disco stuff... one block of data that can be saved (see discussion around that). Also, vCard checking, XEP-0153, XEP-0049 or equivalents. Fetching security label catalogs (XEP-0258).
There is quite a lot of stuff to fetch on startup.
dwd: SASL2 can save us all the stuff that's hard or impossible to pipeline. For all the data-retrieval stuff (e.g. XEP-0049), I think as long as we can assure that that can be pipelined, then we can get it to a single roundtrip.
Kev: I want to retrieve as little data as possible, i.e. less data. XEP-0198 session management only helps when it's available (shorter disconnects), we need a solution without it.
daniel: if you put more and more stuff into PEP and put that into archive, then you can catch up only with those things that have changed.
Kev: all the XEP-0049 stuff can go into PEP.
1.11 BIND2 and SASL2
By doing these, you sign up for all the features bundled under "XMPP2"
Kev: not much further to say except noting that they're part of the solution.
1.12 Disco caching
Daniel: before sending first message, need to look through disco items. Costs at least one unnesssary roundtrip, to do disco#items and then feature request for each item.
Idea: Have data form which
Kev: why can't you send a message without doing disco query.
- e.g. want to send image, then need to know which XEPs are supported.
- e.g. figuring out whether external components (like for stickers) are available
Sam: Querying a MUC, the items change every time.
Daniel: concerned mostly about server items being cached.
Disco extension form that contains hashes of cached values, as part of CAPS.
Disco items query, jid/name/type. Idea is to create form, variables names are the jids, then has two values. 1st value is name, 2nd is type.
Use hash or inject data directly?
Dave: rather try and make this generic than not.
Ralph: Usecase that I was talking about. Separates out MAM as separate component from server for scalability reasons. First thing you want to do when connecting to server is to discover where that MAM component lives.
Kev: if you want your MAM not to be hosted on the server directly, we already have a mechanism for saying : let this component ..
Link Mauve: Salut a Toi and Moving using disco for microblogging and pubsub stuff. So an argument towards being generic.
Sam: independent of path... showing items or hash... did (recently deferred) XEP for HipChat (general topic is lists, Entity Versioning: 0366)... this was the problem we tried to solve as well.
Idea was that you query a list once, in future you only get the diff.
Daniel: Remember that, but it was only traffic minimization and not roundtrip minimization.
Sam: think it would work for either case as the hashing and diffing mechanism.
Kev: define a form that we can put in disco#info, could write a ProtoXEP for that.
Daniel offers to write the ProtoXEP.
Kev: nice thing about putting a form into disco#info is that it gets bound into the CAPS hash. If we serve the CAPS hash as part of XMPP2, that means you can avoid roundtrips.
Daniel: when you put the items in the form (instead of hash), means you don't have to do the disco#items query.
Kev: still issue when having lots of items.
Daniel: maybe we can come up with a solution for feature hashes?
Kev: The more we put in, the more complete we make it, the more often we invalidate the hash. We can put the hash of the disco#info into the hash of the disco#items, that go into the disco#info query.
A. Server has to query these things B. Means you have to go recursively through to each item. C. Top-level hash would change often.
Matt: Approach seems sound. Some entity would eventually ask not to be cached, or could decide how deep to recursively hash.
Daniel, Kev: As a client you'd have to query to leaves of tree, which is not desired.
Kev: By not bothering to require recursiveness, makes implementation simpler.
Daniel: let's do the disco#items hashing first, and leave disco#features (and recursive hashing) until we know whether this is required or not.
1.13 Not IM
New stuff is opt-in. So if old way of doing is fine, then can stick to that.
1.14 Low Power
Kev: for low power stuff, relatively controversial suggestion.
Once we've got push working sensibly, let the low power devices just disconnect, instead of using CSI.
Because once you get online again, you'll get to the right state.
dwd: Re-establishing TLS connection is not slow, but does use battery, so not convinced it would be better.
Holger: could be left optional which approach to take.