So, on Friday, for various reasons, I decided to upgrade Perl from v5.8.something to v5.10. Upgrading Perl itself proved relatively straightforward, with a few rounds of removing old Perl directories and filling in missing non-standard modules. Some improvements were immediately evident — for instance, using Perl 5.8, ICBM (my threaded Perl ICB client) sits and thinks for about five to eight seconds during startup while threads start and structures initialize, while using Perl 5.10 it initializes almost instantly.
Of course, having upgraded Perl, I had to reinstall mod_perl. And this, well ...
To start with, I was a few minor revs behind on Apache 1.3.
Yes, 1.3. I was [gasp] still running Apache 1.3.33. Basically because I never saw any reason to upgrade. The first time I tried installing Apache 2.0, it wasn't ready for prime time, I couldn't get it to play nice, and I wasn't using any of the new functionality it offered anyway (and if memory serves, there were a few things Apache 1.2 did at the time that 2.0 didn't do yet). So I just kept running 1.2.x, then 1.3.x.
So anyway, I grabbed the latest 1.3 release, Apache 1.3.41, and the latest mod_perl and mod_ssl, and started building. mod-ssl is actually two minor revs behind, it only supports up to 1.3.39, but it was the work of a few minutes to fix that. mod_perl was a little more troublesome, and took rather more time. Anyway, to cut a long story short, I finally decided that mod_perl 1.30 and Perl 5.10 just wasn't going to work, so I decided it was time to upgrade to Apache 2.
So I pulled down Apache 2.2.8 and mod_perl 2.0.3, messed around a bit, found that 2.0.3 didn't quite work with Perl 5.10 either, tried mod_perl-2.0.4-dev out of svn and it worked fine, installed, and started migrating my configuration. And that's where the fun really started.
Right now, I have most things working again. DSPAM isn't working yet, and I'm probably going to have to rebuild Apache one more time with suexec enabled to get it working. Mailgraph isn't working, though MRTG is. (Installing the latest version may fix that.) But the oddest thing is error handling.
You see, I had a custom 403/404 handler CGI installed on Apache 1.3. The CGI still works under Apache 2.2.8, if I set up a ScriptAlias to call it directly. But if the exact same cgi is called by Apache to handle a 404 error, Apache generates a 500 internal sefver error — and doesn't even log the error. In fact, Apache 2.2.8 doesn't ever log ANYTHING on 500 errors.
I've been banging my head against this one since about this time yesterday, and I'm baffled. As far as I can tell, it SHOULD work. The CGI DOES work, if loaded directly. It only fails when invoked the way it's supposed to be invoked.
I have little doubt this comes down to Something Crucial I Don't Know About Apache 2.x. But so far, I'm drawing a complete blank for clues as to what the problem may be. And because Apache 2.2.8 doesn't log ANYTHING when it hits a 500 error, I have absolutely nothing to go on. It's bloody hard to diagnose a problem when Apache insists on pretending it never happened.
