|Using Nonstandard Colors in Standard Collections|
The EditNotes Addendum #1 - Advanced Game-CLUT Work
Before you read any further folks, know two things:
One other note: this Addendum corrects for some mistakes found in the "Using non-standard colors in Standard collections" section of the v3 Anvil Masterclass, now updated to Anvil Masterclass V4. That stuff was written whilst I was still figuring all this stuff out, and I didn't discover the flaw in those theories until a bit later. I am correcting all that here)
So, anyway, here goes nothing.... : )
How Marathon Infinity handles colors on 8-bit displays (the definitive version<g>)
Well, for all you _complete_ editing maniacs out there (there must be at least one o' them, surely?;), here is how the system (appears to) work, as best as I can figure it:
Brighter folks amongst you may have spotted in the past that the Marathon Infinity engine contains a copy of the entire Marathon Infinity palette tucked in the resource fork (CLUT ID5002); you may even have guessed that this is the CLUT that Marathon Infinity refers to when it's running in 256 colors. (And if you don't already understand how 8-bit display mode works on a Mac then you shouldn't even be reading this, so I'm not going to start explaining it...) Sorry but no, it doesn't. You should have noticed this much though: all the Standard collections have their colors arranged in 'ramps' which run from a color through to black; but more on this later.
What I eventually deduced is that Marathon Infinity builds its 8-bit CLUT actively on-the-fly, as it loads up Shapes collections when starting a level. [See the following section regarding which collections get loaded and why.] I'd guess that it simply loads them in numerical order, although I may well be wrong (it might be equally likely it loads them in the order I've listed them in the next section; i.e. by "category" rather than numerical position). This is not especially important to know, other than to be aware that it's the Interface collection which is the first to load (presumably followed by weapons-in-hand). Anyway, in practice it adds colors to the game CLUT in the order it finds them in the Anvil CLUTs (excluding the first 3 colors in any collection which are seen as transparency colors by the game engine). Each time it finds a new color, into the in-game CLUT it goes (btw, for reasons that should be obvious enough it doesn't need to load each color more than once; nor would it want to). This goes on right throughout the entire collection-loading process, or until the in-game CLUT is full. And each time the engine encounters a black it assumes that this is where that particular 'ramp' of colors ends and the next one begins (or something vaguely like that). Bear in mind that if the game CLUT is full then the engine will have problems if it tries to display an additional color that isn't in the game CLUT - i.e. it can't. BTW, this is why Anvil contains color checking, to keep users from adding more colors than the engine can handle in 8-bit display (to do so will generally cause the engine to likely crash hard). And what all of this means is that you must take particular care if setting up new ramps, and even more so if you are actually changing the colors in your Marathon Infinity CLUT to something completely new. And _where_ you put the new colors is just as important - if you don't ensure that Marathon Infinity is loading the colors that make up each ramp in the correct order when it constructs its game palette then you are going to have some real visual ugliness when you run in 8-bit mode.
Why does this all matter anyway, and why is it only important when in 8-bit display mode, but not in 16/24-bit? Well, one good reason: it's _how_ Marathon Infinity uses these color ramps which counts. How these are constructed is very important to Marathon Infinity when it's running in 8-bit mode. Each pixel of each sprite/texture is mapped to a particular color in the CLUT, and each of those colors lies somewhere on a 'ramp'. In the shapes file you'll notice that each ramp starts with a color and ends in black. What the engine _effectively_ does is take the color of each pixel, then shift it to the right - i.e. towards the black - by a particular amount depending on the % light intensity that the map says that wall/sprite should be viewed at (for sprites the light value used is that of the floor or ceiling [depending where the sprite is positioned]). Well, OK, it probably doesn't work quite like that at all, but this hypothetical model illustrates the point well enough.
There is something else to realize about the way Marathon Infinity constructs its game CLUT, and that is how black is handled. Although each ramp must end in black in the shapes file, the Marathon Infinity game CLUT only includes it once (to include it at the end of each ramp would be a real waste of space). The only black you'll find in the game CLUT is the first one that loads - in the original Interface collection this means the black at the end of the white-black ramp. In the game CLUT this means the 18th color in (the transparency colors don't appear at the start of the game-CLUT as they do in Shapes CLUTs). So as far as in-game color handling goes, I guess that Marathon Infinity just assumes that each ramp still ends in black, even though black doesn't appear as such in the in-game CLUT (or some process analogous to that). And there appears to be one bit of bad news in all this; it appears that the Marathon Infinity engine is hard coded to look for that 18th color to use as its universal black (the black which appears when something is lit at 0% or is dark enough to appear as black). This is real important folks; it means that whatever else you do, don't change the black that lies in 21st position in the Interface CLUT (assuming the 20 colors before it are all unique). I did this myself whilst replacing the entire Marathon Infinity CLUT with ramps that were 32 steps in length and subsequently spent two days working out why all my shadows were mid-gray (the first ramp was a 32-step white-black) instead of good old fashioned coal black. Very, very ugly, in case you were wondering.
As for why I spent time working all this out; well, one of the things I've done a lot of is heavy-TC work. And eventually I got to the point where I wanted to change the game CLUT to give me colors not previously available (including some of the original Marathon 1 colors) so that I could give the new work a bit of a different look to the usual Marathon Infinity stuff. Also, Quartz presented me with a rather interesting Hoth-related problem waaay back (yea, and it's taken me all this time to finally figure a solution for that one!;) involving his addition of extra colors to the shapes file (even beyond the 256-color limit of 8-bit display mode - there's an interesting side effect in CLUT Modifier that seems to allow this to be done without actually crashing the Marathon Infinity app [sometimes:], although the results are generally pretty cheesy on screen if you do add >256 colors, so I do not recommend it at all if you want a scenario to look playable in 256 colors).
I've included below some screen shots taken via Forge (I could just as easily have taken them in Marathon Infinity, except the Shift-5 trick to allow Cmd-shit-3 to be used doesn't always work very well and I don't have Snapz). These are taken from a copy of the Hoth preview that I spent a little time on after downloading just to see if I could figure how to make it look more presentable in 256 colors (I know most folk have fast enough Macs to run in 1000s anyway, but if something like this can be done easily enough then maybe there's still a few folks out there who will appreciate it). I hope Quartz doesn't mind all this too much...<g> ; )
Well, here's a little test map using the original Hoth preview Shapes and viewed in 8-bit display mode. As you can see, the ground plane looks more or less ok as it's lit quite brightly, but the walls which are lit somewhat darker definitely don't look so good (the left one is 0% and the right one is 30%). To take a look at the CLUT the game engine had generated for itself, I took a screen shot, opened it in Photoshop and looked under Mode:color_Table. Here's a snapshot of what I saw:
Well, as you can see, Marathon Infinity has constructed the CLUT so that the new ice colors which were specially added for Hoth are sitting as a short little ramp at the end of the CLUT. And to the engine, this looks like it's a 9 color (+1 black at the end - somehow the engine is psychic enough to guess that this little ramp needs a black at the end, even if there wasn't one in the Shapes collection itself [so you see, there are _still_ a few mysteries left in Marathon Infinity for future generations to ponder...:]. Now, the new ice colors were only actually added to the walls1 collection (and were, in fact, tacked onto the end of its own CLUT). It's easy enough to spot the close relationship between the new ice colors and the darker ice/metallic blue ramp that appears near the top of the CLUT, so the first job was to arrange these 2 ramps to form a single long one. However, it still wasn't enough to have a long ice ramp in the walls1 collection only, because, as I mentioned above, the engine loads up colors in the order it finds them; thus by the time it reached the walls collection it had already loaded the regular 14-step metallic blue ramp having found it much earlier. Thus the new ice colors were still getting tacked onto the end of the game CLUT, and thus the ice textures still looked as bad in the game at 8-bit as before. So the next step was to ensure the engine was loading the colors in the order _I_ wanted. This meant changing the Interface CLUT (since this is where the loading process begins) to fit in the new ice-colors+metallic-blue super-long ramp to appear before any of the original shorter metallic-blue ones. Here's the new Interface CLUT:
Bear in mind that after changing the CLUT (btw, see the Anvil Masterclass notes on using CLUT Modifier for this sort of thing, and the important warnings) I had to stick all the bitmapped artwork back in again to get the artwork to remap to the new CLUT correctly - I simply exported it as PICT from an original copy of the shapes file (always keep backups and originals folks!) and imported it into the new shapes file. Sorted.
A quick pass through Forge's visual mode using the new Shapes and one snapshot later here's a picture of the new game CLUT. I'm sure you can see the obvious family resemblance to the above (note too how the last 2 ramps are being added in as Marathon Infinity finds then inside subsequent CLUTs).
[BTW, this snapshotting Forge/Marathon Infinity and examining the color Table of the resulting PICT is a terribly handy method for checking that everything is now tickety-boo following your nice new CLUT modifications. Just to be sure...]
Fortunately, although Quartz hadn't removed any existing colors from Marathon Infinity, the new ice colors still managed to squeeze in just on the 256 color limit. Whew! Just as well Bungie left some extra space in there. And here's the final result as seen in glorious 8-bit Technicolor - walls and floor now appear totally correctly.
The Landscape is still spotty, but this has nothing to do with the game CLUT; it's just that the Landscape collection hadn't had the new ice colors added to it to accommodate the new texture as well as it could have been. I stuck in the appropriate colors, made a new Standard landscape bitmap (using a copy of the Custom version as source material) and here's what it looks like once that's fixed too; perfect in 256:
And that was the story of how the Marathon engine handles colors in 8-bit mode. Hoorah!
How the Marathon Infinity engine loads Shapes collections at the start of a level
Bear in mind that Marathon only seems to load those collections which are required for that level, i.e: