Pundit's Lamentations on Closing OS X Kernel Misguided

There's some interesting (although quite misguided) discussion of Apple's decision to keep closed the source code for the Intel version of xnu, the kernel that underlies Dawin and Mac OS X.

Infoworld columnist Tom Yager (whose piece from this morning also appears on the British MacWorld site) claims that Apple has turned its back on the open source initiative that brought it here.

Thanks to pirates, or rather the fear of them, the Intel edition of Apple's OS X is now a proprietary operating system.

Mac developers and power users no longer have the freedom to alter, rebuild, and replace the OS X kernel from source code. Stripped of openness, it no longer possesses the quality that elevated Linux to its status as the second most popular commercial OS.

Mac OS X just became a proprietary operating system? And only in the Intel version? That's news to me. Last I checked, the only vendor of OS X is Apple itself, and only a small portion of the operating system was ever open in the first place.

(It's also news to me that Linux is the second-most popular commercial OS, unless he means "for use in commerce." OS X has made Apple a lot more money than Linux has made for anyone).

This comes up a lot, but Darwin is not OS X, and it's also not Apple's open-source efforts (maybe it's time to rename the project Dinox?); it's simply the open-source portion of OS X that Apple has allowed developers to play with, including until recently, the kernel, xnu.

Now, it's certainly bad that Apple hasn't released their code for the OSX86 kernel. Lots of people are upset about it. For a pretty thorough and very informative account, visit Rob Braun's site.

On the other hand, the utopia that Yager is lamenting the loss of has never been present under OS X in the first place. While it certainly sounds like a shame that Apple will no longer make it possible to muck around in the kernel, this is over the top.

Users in demanding fields such as biosciences or meteorology do hack OS kernels to slim them down, alter the balance between throughput and computing, and to open them to the resources of a massive grid. The availability of Intel's top-shelf compilers, debuggers, libraries, and profilers create unprecedented opportunities to optimise OS X for specific applications.

As a developer friend of mine said when I asked him about it, "people custom-compile Linux and Solaris kernels, but honestly xnu is fundamentally slower."

The reason xnu is slower is because it's a Mach/BSD hybird. Mach messaging is slower than the schemes used in the pure BSD kernels, Linux or Solaris. It's been rumored Apple will pull Mach out of xnu for 10.5 to improve performance.

Another thing to recognize is that building xnu is fundamentally more difficult than recompiling a Linux kernel. Braun has some thoughts on this:

Building the source was also very difficult as the internal Apple build system, XBS, is very large, very involved, and was not documented outside Apple.

For building Darwin standalone releases, XBS could not be used because it was proprietary and because it was just much too large and complex for an average person to use. Not all of us have a team of people and a farm of machines dedicated to just building our software. So, a system was devised that attempted to mimic XBS as much as possible. This was much smaller and easier to use than XBS, but it was still monstrously complex, and only a few people were able to set it up and build a reasonable number of Darwin projects. I believe only 1 or 2 people outside Apple have ever been able to build a full Darwin release.

The point of a microkernel is to be small enough that there is no point to performing the kinds of customization seen on monoliths. It's why we have kernel extensions on OS X that can be loaded or unloaded on the fly.

The System X G5 supercomputer in Virginia was brought online without kernel hacks and runs OS 10.3.9, using MVAPICH to handle the grid-computing functions Yager talks about. The hackability of the PPC xnu had nothing to do with it.

The real issue is not that Apple suddenly closed down Mac OS X, it's that it hasn't supported Darwin the way it needs to in order to be a functional standalone platform. That means the hype around OS X as an open-source operating system was just hype after all.

(Thanks for the help, Andrew!)