Read and despair. Or be amused. It's your choice, it's a free world, unless you're with either EDS, the US Navy, or the USMC. This is a classic example of how NOT to do it -- from both sides.
March 24th, 2004
Last night, I decided it would be a good idea to upgrade from Perl-5.8.1 to Perl-5.8.3. Thus turned out to be a bad idea, because I ended up with a broken Perl installation that didn't believe in gethostbyname(). This caused many things to break. So I tried reinstalling 5.8.1, and found that I couldn't build a working Perl-5.8.1 any more either.
"OK," I thought, "time to reinstall a known good configuration. Say, the differential backup from Sunday night."
So I fired up Bacula, went to restore all the Perl-related files, and .... surprise! Bacula wasn't working. It wouldn't recursively select or deselect directories, and was generating bad paths for cwd. Turns out there was a bug in the Bacula code in CVS sometime around March 18-20 (and yes, I had downloaded, built and run that version) that damaged Bacula's MySQL catalog database. Some of the damage was fixable by a fairly simple SQL query, but it turned out that didn't completely solve the problem.
As a result, I've spent virtually all of today digging into the Bacula source code to find out why it still wasn't working. I discovered a couple of performance speedups and code optimizations along the way, and eventually came up with some modifications to one key file that both make restore tree building slightly more efficient, and allow Bacula to properly walk the filesystem tree and restore files from either a correct, undamaged catalog database or one damaged by the March 18 bug. Built and fired up the new Bacula, restored my Perl-5.8.1 installation, and hey, suddenly all my Perl stuff works again.
I hate days like this. Now I need to figure out why I can't currently build a working perl-5.8.3.... (or perl-5.8.anything for that matter.)
But at least everything works again, and I nailed several important items in Bacula.