It wasn't that long ago that I was writing about the pain and suffering of trying to get Win98SE to work with 1 GB of RAM. Windows 98, like Windows 95 before it, was supposedly a 32-bit operating system (OS), at least according to Microsoft's promotional literature, and any true 32-bit OS should cope with a full 4 GB of memory because that's the maximum amount directly addressable by 32 bits. But as is so often the case with Microsoft software, it didn't work properly because of stupid design decisions. Adding "too much memory" literally caused the cache management code to suck up all free selectors and starve the rest of the system. I thought I had put all that crap behind me in updating to Windows XP, but I recently discovered just how wrong I was.
When I first conceived Gorthaur the Cruel, I wanted him to have a full 2 GB of memory right out of the gate. The reason was simple: I was tired of fighting my previous desktop computer. No computer can be reasonably future-proofed for less than about $10,000, which I'm simply not going to spend, but one can make a few compromises and achieve about 80% of the performance at about 20% - 30% of the cost. That's the route I like to take, and it worked quite nicely as I've already shared.
Or at least it did until I really started digging into my EastWest/Quantum Leap Symphonic Choirs sample library. For the non-musically inclined I'll say merely that it's the choral library to have for anyone wishing to do orchestral music. Its sounds are magnificently sampled and produced, its library presets are a joy to work with, and its flexibility is simply unparalleled from what I've seen. The problem is that such beauty comes at a price: no less than 38 GB of disk space for all the samples. Loading even the smallest choir gave me low-memory warnings, even with 2 GB of physical RAM!
I was pretty surprised at that, but I figured I could always address it later. When an opportunity arose to beef Gorthaur up a bit—I've been working to produce video clips of my son, and a larger hard drive was needed for the raw, DV Cam footage—I figured I would stuff another 2 GB of memory into the motherboard and be done with it. I expected no issues in doing so; after all, I was running Windows XP, a true 32-bit OS, that should handle up to 4 GB. I figured as long as the new memory was detected by the motherboard, I should be good to go.
The first problem I encountered was, not surprisingly, with the BIOS settings. The first time I powered up Gorthaur after adding the new memory modules, the BIOS did its counting thing and reported a total of 2.75 GB of RAM. That threw me for a bit of a loop. I mean, every other time I've added two to two, I've come back with four. How adding 2 GB to a system that already had 2 GB could possibly sum to 2.75 GB was quite beyond me.
After a fair bit of digging around in various chat forums—how did we every cope with this kind of technical crap pre-Internet?!—I discovered that the BIOS in my DFI LanParty board had a setting for a "memory hole" . I have no idea what that's all about, but I can say that changing it did what I needed. Once I toggled the setting, the BIOS would properly report 4 GB of memory at startup. I figured I was done at this point; if the motherboard found it, then surely Windows XP would be able to use it. Right?
Wrong! When I booted the machine into Windows XP for the first time, I was surprised to find that it was reporting a total of 2.75 GB of physical RAM. At first I wondered whether perhaps I had misread the BIOS screen, but a quick reboot and check of the BIOS settings and startup screen confirmed that I had not.
My next thought was that maybe I needed to add the /3GB and/or the /PAE switches to the BOOT.INI file. For those who haven't a clue what I'm talking about, those switches allow Windows XP, Windows 2000, and Windows 2003 to do special things with their memory addressing. To be more specific, the /3GB switch tells the operating system to support loading applications with an uneven division of the logical process space, leaving the low 3 GB for the application and using only the high 1 GB for the OS.
In contrast, the /PAE switch enables support for Intel's physical address extensions to allow a 32-bit CPU to address multiple 32-bit address spaces, a trick similar in method to the "expanded memory" of the old DOS days (though without the screwy 64k page frame). But I digress. Regardless of what those switches actually do, they did nothing to help my cause. No matter how I set switches in the BOOT.INI file, I couldn't get Windows XP to recognize more than 2.75 GB of memory.
Then I made an awful discovery: my problem was Windows XP itself, or Windows XP service pack two (SP2) to be more precise. Remember all that hullabaloo about additional security features with SP2? Well, one of those nifty "features" is called Data Execution Prevention (DEP), a feature that is intended to block buffer exploits. Don't worry if that's all gibberish; just realize that it's a feature of SP2 which helps prevent malicious code from executing.
The problem is that Microsoft's DEP technology broke certain device drivers. So rather than actually fix their own code, or work with the vendors to fix their drivers, they took the impossibly stupid way out: they screwed their users. Or to be more precise, they changed the Windows memory handling code to ditch the top 1 GB of memory altogether! Yes, folks, you read that right. Microsoft arrogantly decided that you didn't really need that last gigabyte you bought.
It wouldn't even piss me off so much if DEP could be disabled and allow all 4 GB to be used, but it doesn't work that way. Even with DEP disabled the OS can't see more than 3 GB of RAM. Even more stupid is the fact that plenty of CPUs in the field don't even support DEP. In other words, Microsoft added a security feature that part of their user base can't use, which just happens to ignore an entire quarter of the memory a user might possess. Un-freaking-believably stupid.
So what's my solution? I had two: (1) use the 64-bit version of Windows XP, or (2) drop back to a Windows XP build prior to SP2. The former isn't really a solution for me, much as I'd like it to be, because 64-bit drivers simply do not exist for my pro-audio gear and other devices. So my only viable option was to create an installation of Windows XP and patch it only as far as service pack one (SP1a).
Isn't it just bloody ridiculous in this day and age that I have to fuss around with such crap? Microsoft, operating in full village-idiot mode, forces its users to choose between the largely unsupported 64-bit world and forgoing important security updates to run an older build, all just to use the memory the OS should be able to use. It's a wonder I'm still sane.
As it turned out, I had problems with only one of the drivers on my system, which surprised me a bit. What didn't surprise me was that the driver in question was the software from Creative Labs for my high-end, X-Fi audio card. Creative Labs has always made good hardware with crappy drivers, and it seems the X-Fi product is no exception. No matter what I did, merely having the drivers installed would cause the system to lock up, squeal horrible digital noise, and do all sorts of other unpleasant stuff.
In the final analysis, though, that wasn't such a big deal. I was building that OS to do audio/video production, and that all works great with my pro-audio gear. It's a little irritating not to have any of the basic Windows sounds playing properly, but I can live with it. I also seem to be unable to burn CD/DVD media reliably under SP1a, but I haven't yet given up on finding an update that fixes that issue. It's not such a big deal to use a different machine or reboot to burn discs, though it is irritating.
At any rate, the net result is that Gorthaur is now a dual-boot system. His primary drive has Windows XP SP2 installed with all my games and general working tools, while his secondary drive has Windows XP SP1a installed with all my audio/music production software. The former install sees only 2.75 GB of RAM, while the latter sees 3.75 GB of RAM. It's not perfect, but it's good enough. It's just such a pity that Microsoft thinks it's smarter to screw their customers than write code that works. It's pathetic, but that's the Microsoft way.
05/23/2006