Wednesday, May 28, 2008

Converging Problems and Solutions

Kellan has a nice addition to the Twitter (and social software in general) scaling discussion. Interestingly, he's giving a talk at OSCON entitled "Beyond REST? Building Data Services with XMPP PubSub."

XMPP PubSub is (part of) what makes collaboration work in Sugar as well, and that's got its own scaling issues on a school server. It isn't the "how do we handle hundreds of thousands of users with a rack of blades (or whatever)" problem, but the "how do we maintain very chatty collaboration between hundreds of kids on an inexpensive school server that's also doing 10 other things at the same time" problem. It seems to me that OLPC probably needs a heavily optimized server to do what they need, and finding someone to do that won't be easy. It is one reason I think Sugar needs a couple years and not a few months of love.

XMPP Pubsub is also the best route to a modern, open source SIF implementation (not that I expect to ever see one).

Regardless... more attention to XMPP PubSub will be a good thing for everyone.


kellan said...

Very interesting, I didn't realize Sugar was using PubSub. Got a link on that somewhere, love to read more.

Tom Hoffman said...


clofresh said...

Hey Tom,

"XMPP Pubsub is also the best route to a modern, open source SIF implementation"

Can you elaborate more on this? I suggested to the guys here who are working with SIF to use XMPP for their messaging queue, but I couldn't really back up my reasons for it besides the it seemed like the right thing to do.


Tom Hoffman said...


By my reading, nearly all of the SIF-protocol specific architecture, the ZIS and the parts of clients that move bits to it, could be tossed out and replaced with standard XMPP PubSub components. The only thing standing in the way of this is (last time I checked) that most of the existing XMPP clients and servers don't (yet) support PubSub.

A SIF ZIS is a single purpose publish/subscribe XML message router. A XMPP server with PubSub is a general purpose publish/subscribe XML message router. It would be a very clean replacement, unlike just saying "You could use SOAP for this," which would (as far as I can tell) require you to re-write your spec in SOAP. With XMPP PubSub, you'd just be able to get rid of a huge chunk of the spec.

Rafael Ferreira said...


I couldn't agree with you more. I think SIF and XMPP are a perfect match... and like you said we could throw all the infrastructure pieces right out the window.