Have you ever heard that old chestnut? You know, the one that goes "you can never be too thin"? The computer industry's version of that has long been "you can never have too much memory". Or at least, that's what it has been; the times just might be a changin'. As usual, with any story worth telling, this one begins with a girl. Uh, no, that's not right. Scratch that. Apparently, I've watched my Spiderman DVD one too many times in the recent past.
This sad, frustrating tale begins not with a girl, but with the addition of memory to my desktop system. More to the point, it begins with a simple experiment to determine whether the out-of-memory errors I was getting were really out-of-memory errors or not. I was having a hard time believing that the errors could be memory-related because my system had 512 MB of RAM at the time, along with a paging file that could grow as large as 1.5 GB. In short, I didn't think I could possibly be running low on memory.
As is so often the case, however, I was wrong. When I saw that my local Fry's Electronics had RAM on sale, I went down and bought another DIMM, a PC 2700 333 MHz. DDR DIMM with a heat-dissipation jacket from Mushkin to be more precise. As an aside, I can't believe it cost me a mere $109. I can still remember when I paid $100 for a 1 MB stick. Nowadays, I'm getting roughly 2.5 orders of magnitude more for my dollar. That amazes me. I wish other industries could keep up with computers; i.e., I would love to drive to work everyday at 300+ MPH in my own anti-gravity hovercar.
At any rate, when I got home and added the memory to my system, I was quite surprised with the results. Not only were the out-of-memory errors a thing of the past, I was pleased to see that several other operations were quite noticeably snappier as well. For example, a game I've been enjoying of late, Command & Conquer: Generals, would always take a good ten to twenty seconds to close down completely when I was done playing. That is, after exiting the game I would have to wait that long before my desktop refreshed and I could get back to work. Since the memory upgrade, the wait time is but a second or two at most. I've seen similar benefits with other games as well, launching, playing, and exiting them.
Gaming aside, I've also seen overall performance increases in other applications. Microsoft Access loads noticeably faster, as do Microsoft Word, Microsoft Excel, and especially Microsoft Visual C++. Switching tasks is faster, the start menu is now instantly available in all circumstances, the task manager is also instantly available in all circumstances, etc. In short, Windows XP Professional (WinXP) really seems to enjoy having 1 GB of RAM a lot more than 512 MB. This is to be expected, though, right? After all, as mentioned previously, you can never have enough memory.
My story with Windows 98 Second Edition (Win98SE) is itself worth an article, and maybe I'll write one someday soon. In the meantime, suffice it to say that I have a dual-boot system. My computer can boot into Win98SE or WinXP using the standard (i.e., Microsoft approved) boot menu at startup. This is necessary for my work for several reasons.
For example, plenty of programs that run under Win98SE won't run under WinXP. This is the case with various software development utilities, cracking tools, and a number of older games. To be clear, I don't generally endorse any kind of warez or piracy, but I do endorse cracking copy protection from legally purchased games. In several cases, I've had to crack games just run them (e.g., Worms World Party, Clive Barker's Undying), and in other cases they just run better or more conveniently (e.g., Sacrifice, Dungeon Keeper II).
As a software developer, it is also important for me to be able to test my applications and utilities on more than one platform. At one point, I had Win95, Win98, WinME, and WinXP available to me in my house on several different computers. I've since resigned myself to making do with Win98SE and WinXP. If an application runs on Win98SE, then it will typically run on WinME; similarly, if it runs on WinXP, then it will typically run on the various other flavors of WinNT/2K. Such a system isn't perfect, mind you, but it's the best I can do for my customers as a one-man shop.
In short, I need to have Win98SE available and working. The first harbinger of doom arose when I tried to update the audio drivers for my Audigy 2 sound card. Creative Labs has a spiffy new update feature on their web site, which works much like Windows Update. That is, you connect to the site, and it detects your card, checks for any available updates, and handles the download and installation for you. Personally, I prefer to have the files stored on my hard drive for future use, but the convenience of the update checking is quite nice.
Because I've been doing a lot of game troubleshooting of late (see my tips on gaming under WinXP for the results thereof), I was pleased to see that new updates from Creative Labs were available. I downloaded and installed them under WinXP and tried them out for a few days. They didn't seem to fix anything major, but they did bring some minor improvements, and, best of all, they didn't break anything else in the process—which is as refreshing as it is rare in my experience. After I felt pretty confident about the updates, I booted under Win98SE to download and apply them there.
And all hell broke loose. The updates downloaded and applied just fine, or so the Creative Labs software said, but my system was seriously sick after rebooting. I got these awful error messages about how my video card wasn't detected, the audio was completely non-functional, and despite my video card not being detected I was also getting an error about how it wasn't compatible with my monitor. That's a pretty good trick for a card that isn't even there, don't you think?
Nothing I did fixed the problem. If I tried to uninstall the Audigy 2 drivers, the installer crashed in the middle of the process. If I tried to uninstall the video drivers, for an ATI Radeon 9700 Pro, the uninstaller simply refused to run. If I tried to access any of the advanced display settings, I got a blue screen of death (BSOD) in one of the ATI VxDs. If I tried reinstalling or repairing the driver installations, I got all kinds of horrible errors along the way. In short, my Win98SE installation was screwed. Badly.
Normally, this wouldn't be such a big deal. As I've written far too many times elsewhere, I use GoBack almost religiously; i.e., I don't do actual sacrifices to the computing gods or anything like that, but I do consider the utility very high on my must-have list. Unfortunately, this particular problem caught me with my pants down, or at least napping—whichever metaphor you happen to prefer, it caught me. I had already evaluated the updates under WinXP for several days, and I had seen no issues. In the past, that's been an iron-clad guarantee that they will work just fine under Win98SE as well. As such, I wasn't worried about applying them without GoBack enabled. I had previously disabled GoBack to defragment my drives, you see, because they'd been without it for over a month, and I didn't think anything bad would happen.
Silly me. That one lousy update seemed to screw my machine completely. Those minor changes to the Audigy 2 software were enough to make my entire installation of Win98SE useless. Wonderful. The only saving grace to this whole scenario (thus far) was that I could reinstall Win98SE with relative ease. All I had to do was boot under WinXP, delete all the Win98SE crap from the C: drive, boot from a floppy, and reinstall Win98SE on the C: drive again. It's a relatively quick and painless process, since Win98SE (unlike the original Win98) actually respects the WinNT/2K/XP BOOT.INI file and doesn't screw it up.
Unfortunately, I wasn't yet aware of the root of all this evil, regarding which I'll say more shortly. Thus, you can imagine my surprise after installing Win98SE when I tried to reinstall the video drivers and had the entire system lock up. The audio drivers wouldn't install either. I tried all kinds of different combinations, using Windows Update to apply all the dozens of patches for Win98SE, installing video drivers first, installing audio drivers first, installing other drivers (e.g., mouse, network, etc.) in various orders, and so forth. Nothing worked.
To make matters worse, every time one of the installations failed, I had to reinstall Win98SE. For the first eight to twelve attempts, I didn't bother to worry about this. After all, I had made Win98SE functional previously, so if I just repeated what I had done before, it should all work just fine. Right? (sigh) I didn't know it at the time, but I was banging my head against the wall for no good reason. I kept trying different things, trying to reproduce exactly what I had done to get Win98SE working previously. I even tried a half dozen or so different versions of the various drivers involved. Nothing made any difference.
That is, nothing made any difference until I finally sat down and got scientific about it. Eventually, I satisfied myself that I had reproduced my previous steps exactly without success. Obviously, that makes no sense. They worked before, so they should work again. Granted, I'm still (perhaps stupidly) treating computers and software as fundamentally deterministic entities—more on my doubts about that can be found elsewhere—but I just don't think I can bring myself to treat them any other way. I'm too firmly committed to the inanimate elements of the world being governed by so-called "natural laws" to do otherwise.
What I realized (eventually) was that the only thing that had changed, since everything worked that is, was the addition of that new 512 MB of RAM to the system. But that simply couldn't be the problem, could it? I mean, you can't have too much memory, can you? The short answer, dear reader, is that yes, you can have too much memory, under Win98SE anyway.
You see, there are several different ways that Win98SE can screw itself badly if it has too much memory. Microsoft even admits to the "problems" in knowledgebase article 304943, though with sadly typical Microsoft arrogance, they do not consider it a problem. To quote directly from the "status" field of the issue: "This behavior is by design." In other words, Microsoft is claiming (in that particular article) that your computer's endless rebooting when it has more than 1.5 GB of installed RAM is deliberate on their part. One wonders: what could possibly motivate some jackass software developer to add that as a deliberate "feature"? Sheesh. I guess Microsoft just can't bring itself to admit mistakes; so much for "trustworthy computing".
The most salient way in which Win98SE can screw itself for my purposes, however, regards the amount of memory used for disk caching. Prior to Win98, users could tweak their cache settings with non-trivial benefit. Heck, I still remember all the research I did with Windows 95 to achieve the best performance from my system. With Win98, however, Windows had finally gotten good enough at managing such things that the user really didn't have to fuss with any tweaks. Typically, the caching system, VCache, would eat the majority of your free memory for better disk performance, releasing it as needed to run applications and so forth.
The problem is that although VCache will use a maximum of 800 MB, it can consume all the available addresses in the system area long before it hits that point. For those not familiar with the details of such memory stuff, memory addresses have been "virtualized" in various ways under Windows for some time. Win3.x veterans may remember occasionally having problems with the number of selectors being maxed out by making lots of calls to GlobalAlloc for small amounts of memory, which could prevent any additional memory allocations from succeeding. It's the same problem in different clothes under Win98SE.
Every time Windows has to allocate memory for something in the "system" area (i.e., the portion of the 4 GB address space for a 32-bit processor where applications run), it has to have a "handle" to that memory. Internally, that "handle" must be used so that the logical address of the memory can be translated correctly into the physical address on a memory chip somewhere in the system. If VCache is gobbling up huge amounts of memory, then the total number of "handles" can be exhausted. This results in memory allocation failures, which can be very confusing. The user can have a good 800+ MB of free memory, for example, and still get told he has no memory available when trying to launch an application. Note well: having an AGP video card exacerbates this problem insofar as its aperture (typically set at 128 MB in size by default) must also fall within the system area.
The fix is pretty simple, though it's annoying as all get out that users should still have to do this kind of crap. Remember how the introduction of the registry was supposed to take us away from the "bad old days" of the Win3.x code base? Remember how it was supposed to prevent us from having to muck around with "fragile" and "difficult-to-understand" initialization (*.ini) files? Well, to fix this problem with VCache, you have to add a line that says something like MaxFileCache=65536 (specified in kilobytes) to the [VCache] section of your SYSTEM.INI file. If you have more than 1 GB of physical RAM, you should also add MaxPhysPage=40000 to the [386Enh] section of your SYSTEM.INI file, for Win98SE cannot be trusted with more than that.
As soon as I added the VCache tweak to my SYSTEM.INI file, almost all of the bizarre problems I had been seeing vanished like a morning mist at sunrise. Previously, if I had tried to access the advanced settings for my Radeon video card, I would get a BSOD. That problem was resolved. I could also now install the drivers for my Audigy 2 card without any errors. There was only one problem remaining that needs to be mentioned.
To be more specific, I discovered that the ATI installation utility's prior crashing was not due to this memory problem. It was due to something else entirely. To be more specific, somewhere along the way, I had gotten in the habit of using the Windows Update site to update Windows itself prior to installing drivers. This is, in hindsight, A Bad Thing™. If I used Windows Update to update Internet Explorer to v6.0 service pack one (IE6SP1), then something would prevent the ATI driver-installation utility from working. I don't know what IE6SP1 changes, but it causes the ATI driver-installation utility to crash like clockwork. The moral of that story, of course, is to install all the latest drivers and such before attempting to update Windows or any other application.
The amount of pain and suffering this whole episode caused me is simply ridiculous. I was just about tearing my hair out trying to figure out why the exact same process that had worked previously was no longer working. Over the course of the last week, I easily installed Win98SE more than thirty times. I used GoBack to revert my drives during testing more than a few dozen times as well. All told, I spent in excess of a full twenty-four hours of working time on this stupid, niggling problem, which is solved by adding one lousy line to an initialization file.
Microsoft has done incredibly stupid things in the past, but this one just takes the cake as far as I'm concerned. What in God's name could they have been thinking? What idiot thought it was a good thing to design VCache so that it could exhaust all the addresses in the system space? It's not like Microsoft wasn't aware of the size of the complete address space; they had to design the Win95 kernel to deal with it. They should have had years of experience with it by the time Win98SE rolled off the factory-duplication line.
Worse, what other idiot thought it would be a good idea to claim that these sorts of problems are intended by design?! For crying out loud, when I do something stupid, I own up to it. I may not like it, mind you, but I don't make some dumb-ass excuse for it. But there we have it, in Microsoft's own words in their own knowledgebase article: "This behavior is by design." Sheesh. It's a bloody miracle any of their software even installs, let alone runs, with that kind of thinking running the show. Whatever else might be said, this much is clear: you can have too much memory, when you're running an operating system designed by idiots, that is.
02/28/2003