Check out my latest product, BuildFactory
Archives: July 2006
30/07 FreeBSD vs. DragonFlyBSD
There's been a reasonable amount of speculation about the split of Matt Dillon's from the FreeBSD project, shortly after which he founded the DragonFlyBSD. While the DragonFlyBSD project has lofty goals it's hard to really judge their progress since...not much has really been finished, this of course makes it difficult to gauge how "successful" the project has been since Dillon forked from the FreeBSD 4.xx branch. In the interest of preventing more politics and flamewars between DragonFly and FreeBSD, I'm not going to attribute the following quote, but it is certainly the best explaination of the benefit of DragonFly's place in the open source ecosystem:Dillon needs his own project where he can be the guy who says yes, and no, and both he and we are much happier with him having his own project. He can take freebsd stuff, we can take his stuff, but there's never any question in DFBSD when dillon says "No, I just won't let it happen that way" whether he's right. In DFBSD, he's always right, and that makes it easy.
[tags: freebsd, dragonflybsd, opensource, unix]
29/07 Found: OpenBSD NFS Kernel Bug
I was recently given the task to track down, and annihilate a bug in OpenBSD's NFS implementation that was affecting a friend of mine's hosting operation. His setup seems relatively basic and solid for a small hosting company, a main NFS server packed with disks, running FreeBSD, serving the home directories as well as some other shares to a good heterogeneous mix of both OpenBSD and FreeBSD mail, web and shell servers. The bug, which had been previously reported in pr #3210, would occur in certain programs when their PID (process ID) would receive a signal of any kind (I tested with only SIGINT and SIGKILL since my machine for testing is 6 years old).After a good three or four days of testing, kernel recompiles, AnonCVS checkouts, locking up the machine, examining the kernel via ddb(4) and several cans of Coke, I had narrowed the bug down to a specific set of test cases, and had all the information to start writing code. I am by no means a kernel hacker (it is one of those life long goals, next to running for president though), I have never researched this small amount of code for so long before. The code that contained the bug was within about ~200 lines of code deep in the VFS layer upon which all of OpenBSD's NFS code sits (as well as their msdosfs and ext2fs code). Because of the similarities between both OpenBSD and NetBSD's kernels, the bug may also exist in NetBSD's VFS layer. I have posted to the tech@openbsd.org mailing list with my findings, and have uploaded my patch as well: vfs_bio.c.diff. Thanks to Thomas E. Spanjaard (TGEN on Freenode) for creating a patch for NetBSD (although he could not test it without setting up an NFS server): vfs_bio.c.nbsd.diff
21/07 fighting the good fight, or, enterprising open source
I had the good fortune at some time earlier this year to cross paths with a fellow by the name of William Hurley (whurley), who although most likely has questionable taste and an optometrist who's playing jokes on him, has a knack for stepping into the ring with the big enterprise software vendors with some of the open source projects he is involved with. Whurley (it rolls off the tongue nicely doesn't it?) works for one of the newer movers and shakers in the open source, Qlusters Inc.. Qlusters has been behind projects like openQRM, and most recently the Open Management Consortium which has already been covered by c|net in articles like this.One of the Open Management Consortium's goals is to help promote and advanced the development of open source systems and network management software, with members like the Webmin project, the Nagios project, and corporate members like opengear, and Emu Software, the Open Management Consortium is already composed of a respectable set of companies and projects that are doing everything within their power to push open source into data centers across the globe.
20/07 it's a jump...to conclusions mat, get it?
I think I finaly understand how poets feel who bemoan their existence in a soceity that doesn't understand them. My previous post is getting some interesting comments on digg and I wish to make it clear that I am in no way against the "open source" idea, I am a firm believer that above all else, be practical.There is a large proportion of the "open source movement" that has spent so long shouting "OPEN SOURCE RULEZZ!1" that they've become blinded by their own zealotry. Take the GNU Compiler Collection, I use it daily, it's a wonderful toolset to have essentially "for free," but it is by no means the best compiler out there. Anybody who has done any reasonable amount of work with C++ knows that before gcc 4, compiling C++ code (Qt anybody?) with gcc was slow and painful at best. With the 4 branch, C++ code generation has greatly improved, but I still don't believe it's even near the level of Intel's compilers, nor can anything touch the Plan 9 compilers yet. Does this mean that gcc SUCKS! No, of course not, but it does mean that like most software, gcc is not perfect, and we shouldn't be afraid to point out it's imperfections.
19/07 i hate open source, sort of.
Whisper it with me one time "open source." Now think about that, is that not the most miserable term you've ever heard? You thought that was bad, how about a question I've read enough times to kill a moose: "is it free as in speech, or free as in beer?" The entire concept of "open source" is a concept I thought of right after Al Gore invented the internet, and it has led to more headaches, more flamewars, and more phosphor green light emitted from my monitors than anything else. I joyfully participate in projects that can be considered "open source" in my spare time, time that is when I should probably be doing actual work, but I still inevitably come to the conclusion "GAWD! OPEN SOURCE SUCKS!!!!" after shouting this in a hushed voice in my office, I come to the scariest realization a young programmer will ever have:this is the best we've got
Does that send shivers down your spine too? But let's face reality here, you know what happens when we let corporations close up development of their systems? Mac OS 9 (yuck!), Windows XP SP2 Lite Beta Professional++ (double yuck!), and then more commercial unix-y systems like SCO Unix (I've had to do development on OpenServer before, it's fucking sick), and before a year or so ago, Solaris (I remember SunOS, and I still remember when Solaris sucked, some might argue it still does). I remember having to zap my PowerMac 6100's PRAM to fix some wonky bug, I remember having to reinstall Windows on my PC to get it to...work, I remember having to recompile the SCO Unix OpenServer kernel because I wanted the network interface to receive a DHCP lease instead of a static IP address, I remember using university Solaris servers for some CS projects; they all have something in common, they all fucking suck (expletives have been left in for emphasis, and rhythmic effect).
17/07 CentOS, at least it's free, right?
I'm somewhat forced to use CentOS for testing on some code I've got to fix for a client, CentOS in case you don't know, is the open and free version of Red Hat Enterprise Linux (RHEL, as in rhel stupid, ok, so that reference didn't work). I currently work on FreeBSD, OpenBSD, and Mac OS X (10.4.7) and gleefully use ssh(1) and various terminals, I'm no stranger to bash(1) and he's no stranger to me. In my customary fashion for setting up a new machine, I plug it into the network, and then just use secure shell for the rest of my work, there's something magical about using vim(1) on a remote machine that just makes me feel all warm and fuzzy inside. I've not used a Linux distribution in a year or so, and I haven't used a redhat distribution in over 3 years, so this was admittedly going to be different, but I was expecting it to...suck.The machine I have to use for this is an older dual Pentium II (2x450Mhz) with 512MB RAM, a few ethernet cards, USB, etc. It's an older machine, but definitely no 386. I grabbed the latest CentOS, 4.3, from the website, and tried to install...shortly after the install CD's kernel was loaded, it kernel paniced. I trying adding some boot loader arguments, such as 'linux noapic', still kernel paniced. Of course this left me with one place to go, backwards. When in doubt, install an older version and hope to god that it works.
This is a rant, just in case you were expecting something else :)
14/07 Holy smokes, I was dugg, here are the stats.
At the time I'm writing this, the story of mine that got dugg for all it's worth is at 454 diggs (what a lovely number). I've been a digg user for quite a while, hell I've been subscribed to the diggnation podcast since maybe, episode 20 or so? I've watched Kevin go to the Mac, ditch the Mac for a Thinkpad, and return to the Mac(Book) in the course of around 15 episodes...needless to say, I'm familiar with this confounded 'digg' thing.Anyways, I was quite interested to find the number of Firefox users, and Windows users that pounded my blog (ugh, I hate that word) for a few hours straight after the aforementioned story found itself on the front page of Digg. The statistics I'll cite in the following paragraphs are from today, July 14th, and are restricted only to digg.com referrals.
The first thing that amazed me was the sheer number of people who actually clicked through to read the blog entry about kernel_task, I mean, the topic certainly interests me (I've probably made that clear), but to think that it interested thousands more who bothered to actually read what I had to say on the topic.
14/07 Amit Singh puts kernel_task in it's place
Phil Aaronson of BuildFactory and cute daughters fame, pointed out last night that in his magical Mac OS X Internals (A Systems Approach) book, Amit Singh explains kernel_task more in some detail.In my own defense, I really think my speculation/weak-research was relatively close on, the kernel_task process you see in Activity Monitor is somewhat similar to the sigma0 concept on L4-based systems (L4 X.2 Manual (PDF). That is, sigma0 does handle (from the best of my understanding of course), basic page allocations of memory for the userland processes. kernel_task however does so much more, more along the lines of sigma0 plus the generic root task that you create to run on top of the L4 microkernel (I should really start making use of the µ symbol). From Amit's book here's what he describes kernel_task much more accurately:
The kernel uses a single task - the kernel task - with multiple threads that perform kernel operations such as scheduling, thread reaping, callout management, paging and Unix exception handling. Thus xnu is a monolithic kernel containing markedly different components such as Mach, BSD, and the I/O Kit, all running as gorups of threads in a single task in the same address space.
This is where Apple, as I pointed out previously, made a microkernel, I mean, µkernel not a µkernel. For example, when you fire up Darbat they do something different, more in tune with the µkernel design, with all that BSD, Mach and IOKit fluff. They run it...in userland...where all that fluffy non-kernel space stuff should really be imprisoned. What this means of course, is that IOKit is running in it's own chunk of address space outside of the kernel which, besides being very cool, means that in Darbat 0.2 they could accomplish KEXT compatibility with Mac OS 10.4, while not messing up their lovely µkernel design.
If I had a Mr. Rogers sweater, I'd probably pull that on right now and say I'm glad we could spend this time together to learn a little bit more about the world, thanks for stopping by my neighborhood.
[tags: apple, darwin, xnu, mach, l4]
13/07 kernel_task, it is as if i barely know you
Recently I've become a big fan of using Activity Monitor for fun and profit. Activity Monitor, just in case you aren't already aware, is like top(1) with pretty graphs and charts, and a swanky aqualicious interface, and will show you things like virtual memory usage, thread counts, a process' user id, CPU usage, and a lot of other things in 8-bit technicolor.
Fernando, infamous graphics designer (who just so happens to be responsible for the BuildFactory icon, and about anything else about bleep. that makes you say "man, that looks cool"), noted today that kernel_task has an obscene amount of threads, and takes up a decent chuck of virtual memory. So in my lack of any sort of infinite wisdom, I feel the need to explain this, since I of course, know everything about Mac OS X and mach internals (that's a facetious statement folks, look closely, I attempt to weave humour into my writing quite often).
13/07 i suck, don't hire me.
I've done a lot of stupid things, most purely accidental, but there the ever so sporadic an occasion where I do something as stupid as this:
I'm not a strong guy either, I haven't a clue how this happened. I am about 6'4", 160lbs (relatively underweight for my height as well), and I drive a blue 2001 Volkswagen Jetta, in short, i'm no muscle head.
A general rule of thumb to follow however, if there's a mosquito in your car, clap him between your hands, there's less chance of collateral damage.
11/07 Darbat 0.2 Released, Let's Celebrate
The guys over at NICTA have released version 0.2 of their L4/Darwin project a.k.a Darbat (Release Notes). (The following overview borrowed from the project page) "The L4/Darwin project is an experimental port of Darwin to the L4 microkernel to study the characteristics of a large-scale microkernel-based system. It includes a port of IOKit to the L4 microkernel, a modified libc to communicate with the Darbat Server, and of course XNU with many of the machine-dependent parts heavily modified (pmap, thread/task creation, etc) but much left unchanged (most of BSD, and large parts of OSFMK work without modification). The aim is for Darbat to be fully binary compatible with existing userlevel Darwin binaries. There are also plans to integrate L4 Linux (Wombat) to run side-by-side with Darbat on the same machine and allow them to communicate via L4 IPC."Darbat v(0.2) Supported Hardware
- Intel Mac Mini (Core Solo)
- Intel iMac (17", 1.83GHz Core Duo)
- Intel iMac (20", 2GHz Core Duo)
- MacBook (White, 1.83GHz Core Duo)
- MacBook Pro (15.4", 1.83GHz Core Duo)
The Darbat 0.2 release has overcome quite a few hurdles because of Apple's recent actions in regards to limiting Darwin source code. Charles Gray mentioned that it was difficult for them to acheive the binary compatibility for this release, noting that dyld(1) would randomly crash on release binaries, but would work for DTK transition binaries. After recompiling dyld(1) maybe 100 times between Charles and Tom Birch, Darbat 0.2 is binary compatible with 10.4.6 release binaries. Geoffrey Lee also managed to fix the IOKit binary loading, but Geoffrey has pulled off an impressive bit of engineering getting Apple's IOKit running in userland mode on top of the microkernel independent of whether or not Darbat is actually running.
So who cares?
Apple has done something, in my opinion, that only Apple could do, and that's (a) take a microkernel, and turn it into a monolithic kernel ;) and (b) base an entire operating system around a microkernel-esque based OS. Mach however has it's own pitfalls, speed being notably one of them, which is one of the things the L4 microkernel is one of the few microkernels that has such blazingly fast IPC, making the L4/Darwin project quite promising. If you'll note my very intelligent looking table:| Syscall | Darbat | Mac OS X 10.4.6 |
| mach_msg (null args) | 2393.86 | 2523.19 |
| pid_for_task | 4011.82 | 4327.65 |
| flock (null args) | 3515.86 | 3678.16 |
| mach_msg (atomic send & recv) | 5956.32 | 5897.97 |
The Darbat project has the potential of making something that doesn't really exist at the moment possible, a high performance, truly microkernel based operating system, which is 'l337' to say the least ;)
Unnecessary Graphic
While you cannot run Mail.app, Xcode, etc on top of Darbat just yet, they've documented it to some extent. The Darbat project is one of the more interesting open source projects going on at the moment in my opinion. While there is the WebKit project, there really aren't too many open source projects that are so tied to Apple technology in such a way that they can benefit greatly from current sources of any form (open the sources already Apple!).
Congrats to Tom, Chuck, and Geoffrey, and the rest of the guys at NICTA, they're doing some interesting stuff over there, and keep up the good work!
Digg this!
[tags: apple, l4, darwin, darbat, opensource]
10/07 The CoreFoundation Bug Revisited.
The bug I outlined with this post has been not only been updated, i.e. I'm closer to pegging the bug down, but my previous bug report (rdar://problem/4570035) now has been "split" into two bug reports by the Apple engineering folks, so now rdar://problem/4563964 exists as well.
So, what's changed? For about a week, I was running iChat without Chax, but lacking tabs was killing me...so I reinstalled Chax, and BAM! I hit the bug again except this time with iPhoto, instead of just Preview, and Mail.app, I also found a crash log from cc1obj, which crashed at the same time as iPhoto.
The temporary solution has been to use the latest Adium beta instead of Chax, which has been uninstalled from the system. But the question still remains...can you find this bug?
Check out today's crash logs: iPhoto.crash.log & cc1obj.crash.log
As well as the previous crash logs: Mail.crash.log & Preview.crash.log
[tags: apple, iphoto, chax, ichat, preview, mail, corefoundation]
09/07 Did you know...
A couple of unknown facts about this route between my house, and my sister's house:- There are exactly 34 Waffle Houses that dot the I-10 corridor.
- As of today, there are exactly 36 broken down, abandoned cars on I-10 West from Pensacola to Houston.
It's a good thing my car doesn't have cruise control...d'oh.
[tags: pensacola, roadtrip, wafflehouse, jetta]
07/07 Your Plug-in Ate My Application
A good lot of Apple applications implement either a private or public plug-in architectures, such as iPhoto's export plug-in architecture, or iTunes' Visual plug-in architecture. Of course, Safari implements one as well, which lovely, well known banes of my existence, such as Flash and Java, use...in excess.
I'm certain I got off on the wrong foot with some of the guys in the #webkit channel when, after trying to debug a vicious memory leak in Safari that was eating up almost 1/4 of my fancy smancy iMac's memory (2GB) as well as 2GB of virtual memory (see Figure 1). Ouch. They were all very patient in helping me debug my issue, but in the end it seems basically came down to Flash sucks, Java sucks, so if you visit websites that do any sort of...post-Y2k media, Safari's going to bloat, and how. Of course, this brings me to my point, the ever so elusive "point" that usually lacks from so many of my own rants.
Figure 1
06/07 Lost: One CoreFoundation Bug
Neither Apple engineers, nor my own bug hunting skills are adequate for pinning down this bug (4570035). (Rentzsch says use rdar:// urls, so it is so)
I've blamed various deitys, Microsoft, El Niño, and anything else I could find in a newspaper from 2001, but I cannot for the life of me reproduce this blasphemous bug. David Young hit the same bug (it seems) once upon a time as well: http://www.cocoabuilder.com/archive/message/cocoa/2004/8/4/113693 but under different circumstances.
Basically the details of the bug are as follows, it always comes late at night, sometimes when I'm asleep, but mostly when my internet connection is flakey. Colloquy spews the following errors to console.log, and then CoreFoundation crashes, usually taking Preview.app and Mail.app with it:
2006-06-10 07:21:01.853 Colloquy[393] connection error: NSError "POSIX error: Operation timed out" Domain=NSPOSIXErrorDomain Code=60
2006-06-10 07:22:46.885 Colloquy[393] connection error: NSError "POSIX error: Operation timed out" Domain=NSPOSIXErrorDomain Code=60
2006-06-10 07:24:31.917 Colloquy[393] connection error: NSError "POSIX error: Operation timed out" Domain=NSPOSIXErrorDomain Code=60
Jun 10 07:26:07 intellian crashdump[28569]: Mail crashed
Jun 10 07:26:08 intellian crashdump[28570]: crashdump crashed
Jun 10 07:26:08 intellian crashdump[28570]: crash report written to: /Library/Logs/CrashReporter/crashdump.crash.log
Jun 10 07:26:10 intellian crashdump[28569]: crash report written to: /Users/tyler/Library/Logs/CrashReporter/Mail.crash.log
2006-06-10 07:26:16.950 Colloquy[393] connection error: NSError "POSIX error: Operation timed out" Domain=NSPOSIXErrorDomain Code=60
Wacky huh? I thought so too.
You can check out my Mail.crash.log and my Preview.crash.log
[tags: apple, cocoa, mail, preview, itunes, bugsbugsbugs]
06/07 Dear South Texas
You're welcome for the rain.Tuesday night I left my windows down, and my beloved sandals in my car. Tuesday night we got probably the largest amount of rainfall all summer.
This morning, I opened up the windows to help dry it out...it rained again.
damnit.