Wednesday, January 09, 2008

XO Performance Problems and/or Sugar Performance Problems

If you've read this blog long, you'll note I regularly lapse into hopelessly technical comments without making much effort to explain the issues to my less geeky readers. Well, I'm making a rare attempt to do so here, and I'm reminded of why I generally don't bother. This took me three days, it is incredibly long, and I'm not very confident it makes any sense. If you want, you can skip to the bullet points at the end.

I've been trying to stick to using Sugar and the standard sugarized apps on my XO, but in order to subjectively experiment with the sources of XO's admittedly show performance in some areas, I installed Xfce, a lightweight window manager used in Xubuntu and other Linux distribtions, and Firefox, a popular, if porcine by XO standards, web browser. This did not require a flash of insight on my point, it is well trod and reasonably well documented territory.

The XO has the horsepower of a computer from around the turn of the century, and Xfce has all the flash and dash of Windows 2000 or Mac OS 9, so they work together pretty well. The performance is, well, pretty much what you'd expect from a consumer laptop. That is to say, unremarkable, but significantly more responsive than Sugar. Firefox on Xfce is pretty much its normal self, although it maxes out the processor and hangs at times. I'd bet this is Javascript-related. Maybe it can't plow through inefficient scripts as easily as most of today's high-powered processors. Also, I know Firefox regularly chews up over a gig of real and virtual memory on my Ubuntu systems, so I'm sure that would become a problem eventually and would require some attention.

That Xfce is more responsive than Sugar is not in the least surprising. The most obvious reason for this is that Sugar is written in Python, whereas Xfce is apparently written in C++. Put more generally, Sugar uses an interpreted language, and Xfce is written in a compiled language. If a computer executing a compiled program is like reading a foreign novel in translation; then running a program in an interpreted language is like reading a novel in a foreign language and translating it in your head. In most cases, it is a lot slower to translate in your head on the fly; programs in interpreted languages are usually a lot slower than the equivalent in a compiled language.

If you'd gotten 10 programmers together two years ago and told them you wanted to write a new desktop interface and you wanted to do it in Python, probably eight of them would tell you you're crazy, Python would be too slow, and you should do it in some variant on C. That's how it has been done in every case I'm aware of (Macs = Objective C, Windows = C++ & C#, Linux = C & C++).

So why use Python for Sugar? There are some reasons that fit with the philosophy of the project: it is completely free and unencumbered from IP threat or confusion; it hits a sweet spot where it is a good first language for kids and used for big professional projects; Red Hat in particular has experienced Python programmers.

But in the wider case, you use a scripting or agile language like Python to trade development time for performance. That is, all things being equal, your developers will be able to write, modify and improve the programs much more quickly, but the software will generally run more slowly. With fast processors, this is a good tradeoff. If Audacity (written in Python) takes 6 seconds to launch instead of three, but it has the features you want as a user, you're ok with that. With Sugar controlling the whole UI and a low-power processor, it is kind of painful right now, though.

But another way of looking at the development strategy OLPC is using is to apply the principle of avoiding what developers call "premature optimization." As no less than Donald Knuth said:

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

The idea is that it is very difficult to anticipate ahead of time what parts of the code are going to cause performance problems. There are so many moving parts and variables, not to mention users doing unanticipated things, that if you try to guess which parts to optimize, you'll probably be wrong. And in a sense, in 2007, writing all your code in C is itself a kind of premature optimization. The lines of C you write are more expensive to create and maintain than the equivalent in Python. Since the two can be combined easily, you can avoid premature optimization with this basic process:

  1. Quickly achieve the basic functionality you want in Python without regard for performance.
  2. If the application is too slow, use profiling tools to determine where the bottlenecks are.
  3. Optimize your Python code to eliminate bottlenecks.
  4. If you can't fix the bottlenecks in Python, write enough C to get rid of them.

This is the basic procedure we use for SchoolTool, although we've never had to resort to writing C. In particular, our developers at Programmers of Vilnius are real sticklers for this concept. It is not unusual for me to find some new feature takes, say, 20 seconds to generate a web response. I say "OK, this does what we need but it is way too slow." Usually by the next day it is 10 or 20 times faster. In particular, this approach pays off if the initial implementation doesn't do what we need it to do, or we discover our initial design was wrong. Then we've avoided spending time optimizing a design which is going to be discarded or heavily modified. It is really a strategy for continuous prototyping and tight feedback loops.

My interpretation of Sugar, writ large, is that it is primarily still in step one of the above process. It is, essentially, unoptimized. If you look at the OLPC bugtracker, there are only a handful of bugs tagged with "performance." Bug 5228, "launching activities is slow" gives you an idea of what the process of optimizing these issues is like (in this case for the Update.2 release, due in March.

Beyond that, if you look at the development roadmap, you see additional feature-oriented releases, but not a performance-oriented one. It isn't clear when they're going to stop making new stuff and make what they've got faster. I think, and I can't say I've actually dug around the code to prove this (and I'm not competent to determine this myself anyhow), that there is a lot of potential for speeding up Sugar. I just don't know who is going to do it and when. It is also important to note that the relative slowness of Sugar is probably less of an issue to OLPC's target audience. If Sugar is kind of slow, but tolerable, I don't think kids in the developing world will reject it. The same may not be true in the developed world, where kids have more alternatives.

One more specific observation from experimenting with Firefox and Xfce is that Firefox runs noticeably more slowly when launched from the terminal in Sugar than it does in Xfce, suggesting that Sugar slows down everything when it is being used, even compiled C programs that aren't "Sugarized." Bugs 4680 (fix) and 4912 are examples of bugs that would have that effect.

So... wrapping this up:

  • Sugar is slow.
  • The XO hardware doesn't have any particular performance problems. It is as fast (and slow) as you'd expect from its specification. Given a more typical UI, it performs typically.
  • There is good reason to expect that the performance of Sugar can and will improve dramatically, if time and money is put into that goal.


Miguel Guhlin (@mGuhlin) said...

Thanks for explaining it so simply. I appreciated learning about the diff between Sugar and XFCE, Python and other approaches.

I'll be sure to reference this post as I try to understand it better. Is it safe to say that XFCE should have been loaded by default on the machine for regular, speedy use, esp if the python-ish aspects of Sugar hadn't been needed for OLPC philosophy?


Tom Hoffman said...

You could argue that if you were trying to sell XO's as a commercial product in the US, you'd want to pre-install both window managers and make it easy to choose. Either way, we'll know in a year if Sugar works or not.