Back to Form
|
by Siliconmaster (2022)
|
Part I: Back at Last
Well, we’ve done it again. Over a thousand bug fixes, visual upgrades, and painfully meticulous adjustments.
The idea of Patch 1.4 (italicized, as it’s a work of art in itself) had been around for years, slowly growing as we noticed little issues with version 1.3. For a while it was an ongoing joke: “So, when are we doing 1.4?” None of us really expected it to go anywhere. Even so, by December of 2020 COVID cabin fever had pushed us over the edge. We needed some sort of creative outlet to blow off steam. The initial discussion about patching TSC:E framed a small set of perhaps a dozen items, none of them terribly large-scale or complicated.
It never stays that way.
For a while I purposefully kept to the edges of the project, terrified that if I actually touched any of the lighting or environment visuals I would be sucked back into the deep morass that so completely ate my life six years ago. I helped do an EAX 3D audio pass and tested improvements and bug fixes from Teh Lag and Ifafudafi, such as our overhaul of the Hunter arena. And then, at the end of March 2021, I gave in.
The sensation of being able to open those tools, pull open my old files, and actually change things was nearly indescribable. Like getting to play with toys that have been locked behind a glass window, or eating the forbidden fruit. But really, it felt like coming home.
Now, having released 1.4 at the end of October, I look back both elated and horrified at what we’ve done. We didn’t just return to fix some holes. We tore the level apart.
Over 8 months, I probably spent more than 500 hours going over every single lightmap across 10 BSPs. Hundreds of UV errors, painted out by hand. And then I started re-UVing entire lightmap chunks, re-rendering them, and layering them back over the original files.
They say that most games would need an extra six months of development to get anywhere near the original vision. We got the rare opportunity to do just that.
It was exhilarating. It was fun. It was exhausting. And now here, post-release, it seems like a good time to look back at what we have wrought.
The idea of Patch 1.4 (italicized, as it’s a work of art in itself) had been around for years, slowly growing as we noticed little issues with version 1.3. For a while it was an ongoing joke: “So, when are we doing 1.4?” None of us really expected it to go anywhere. Even so, by December of 2020 COVID cabin fever had pushed us over the edge. We needed some sort of creative outlet to blow off steam. The initial discussion about patching TSC:E framed a small set of perhaps a dozen items, none of them terribly large-scale or complicated.
It never stays that way.
For a while I purposefully kept to the edges of the project, terrified that if I actually touched any of the lighting or environment visuals I would be sucked back into the deep morass that so completely ate my life six years ago. I helped do an EAX 3D audio pass and tested improvements and bug fixes from Teh Lag and Ifafudafi, such as our overhaul of the Hunter arena. And then, at the end of March 2021, I gave in.
The sensation of being able to open those tools, pull open my old files, and actually change things was nearly indescribable. Like getting to play with toys that have been locked behind a glass window, or eating the forbidden fruit. But really, it felt like coming home.
Now, having released 1.4 at the end of October, I look back both elated and horrified at what we’ve done. We didn’t just return to fix some holes. We tore the level apart.
Over 8 months, I probably spent more than 500 hours going over every single lightmap across 10 BSPs. Hundreds of UV errors, painted out by hand. And then I started re-UVing entire lightmap chunks, re-rendering them, and layering them back over the original files.
They say that most games would need an extra six months of development to get anywhere near the original vision. We got the rare opportunity to do just that.
It was exhilarating. It was fun. It was exhausting. And now here, post-release, it seems like a good time to look back at what we have wrought.
Part II: Recompressing the World
Near the start of development, one problem that had bothered all of us was the multicolored DDS compression artifacts caused by tool.exe’s bitmap importer. Unless you set the compression type to raw 32 bit (impossible for us due to size limitations), some level of distortion would creep in. Color shifts, clipped details, and jagged gradients were all accepted limitations of the engine, visible across weapons, vehicles, and especially the environment textures. In addition, some textures were based on old extracted bitmap tags. Certain extraction tools shifted the hue to the greenish side, changing the look of some legacy textures entirely. Check out the image below to see how tool.exe handled one of our textures in 1.3.
b30r_f_panels_generic-ext_base original source vs compressed TSC:E 1.3
We heard from fellow modding community member ShellyHerself that nVidia had released DDS texture tools that handled image compression far, far better. She demonstrated this process for us by fixing up our warthog textures, and developed nvdxt profiles for us to use. If we could find the original source textures, we could theoretically send the files through the nVidia tool and use Moses’s Mozzarilla program to import the resulting DDS file into a Halo .bitmap tag. Finding and cleaning up those original files would prove to be a more demanding task than expected.
On the bright side, we learned that Halo 1 stores the original raw image data in bitmap tags. We were able to extract much of this data with Mozzarilla - however, some textures had lost their source data after being extracted from compiled maps over the years. Some I still had access to through the massive texture repository I inherited from old CMT in 2012, but others were gone for good.
The problem was that the source textures we did have looked so different from the compressed 1.3 versions that reverting them actually changed the art style. In many cases, the compression artifacting added bright highlights in places that were not in the originals. While importing the raw, uncompressed textures certainly looked cleaner, it meant that some areas looked darker, or not as vibrant. Since our goal for 1.4 was to keep 1.3’s look, clearly a simple recompression wasn’t going to be enough.
My solution was to manually regrade every single environment texture.
In many cases I used the blend-if feature in Adobe Photoshop to extract only the compressed 1.3 highlights to place over the raw textures. In others, I layered adjustments to get just the right contrast and hue.
Some environment textures had multiple parts I needed to correct. Since the alpha channel in the diffuse texture controlled specular levels, it was very easy for a newly recompressed shader to look wildly different in shine-levels than it had before. I ended up having separate PSDs for each of my alpha channels where I could fine tune them. Through all of this I was constantly flipping back and forth between comparison shots with 1.3 to ensure consistency with all the newly recompressed textures.
In some cases where original source files were lost, I took extra steps to touch up or recreate work by Dano, a well-known texture artist from the old CMT days. In places where compression obliterated some finer details in favor of perceived grit, I hand traced the original shapes to add better dust and noise back into the raw source. And for the b30r_f_strips_wide-a_base and ext textures, I touched up a few small details to make them more seamless as they tiled along their respective walls.
At times I noticed some blatant errors we overlooked during 1.3’s development. The b30r_f_panels_ext-unearthed_base texture, resembling rusty sea corrosion, had its alpha channel completely offset from the diffuse, resulting in an unpleasant plasticky sheen. I shifted the alpha channel to match correctly, and then fine-tuned the specular levels. Now they look just as grimy and metallic as one would expect. Combined with the other fixes, this is most noticeable at the Landing Zone arch pillar indents
On the bright side, we learned that Halo 1 stores the original raw image data in bitmap tags. We were able to extract much of this data with Mozzarilla - however, some textures had lost their source data after being extracted from compiled maps over the years. Some I still had access to through the massive texture repository I inherited from old CMT in 2012, but others were gone for good.
The problem was that the source textures we did have looked so different from the compressed 1.3 versions that reverting them actually changed the art style. In many cases, the compression artifacting added bright highlights in places that were not in the originals. While importing the raw, uncompressed textures certainly looked cleaner, it meant that some areas looked darker, or not as vibrant. Since our goal for 1.4 was to keep 1.3’s look, clearly a simple recompression wasn’t going to be enough.
My solution was to manually regrade every single environment texture.
In many cases I used the blend-if feature in Adobe Photoshop to extract only the compressed 1.3 highlights to place over the raw textures. In others, I layered adjustments to get just the right contrast and hue.
Some environment textures had multiple parts I needed to correct. Since the alpha channel in the diffuse texture controlled specular levels, it was very easy for a newly recompressed shader to look wildly different in shine-levels than it had before. I ended up having separate PSDs for each of my alpha channels where I could fine tune them. Through all of this I was constantly flipping back and forth between comparison shots with 1.3 to ensure consistency with all the newly recompressed textures.
In some cases where original source files were lost, I took extra steps to touch up or recreate work by Dano, a well-known texture artist from the old CMT days. In places where compression obliterated some finer details in favor of perceived grit, I hand traced the original shapes to add better dust and noise back into the raw source. And for the b30r_f_strips_wide-a_base and ext textures, I touched up a few small details to make them more seamless as they tiled along their respective walls.
At times I noticed some blatant errors we overlooked during 1.3’s development. The b30r_f_panels_ext-unearthed_base texture, resembling rusty sea corrosion, had its alpha channel completely offset from the diffuse, resulting in an unpleasant plasticky sheen. I shifted the alpha channel to match correctly, and then fine-tuned the specular levels. Now they look just as grimy and metallic as one would expect. Combined with the other fixes, this is most noticeable at the Landing Zone arch pillar indents
Part III: Let There Be Light
Recompressing textures is all fine and great, but it doesn’t matter if the lighting has issues. And some of these issues had been bugging me for years.
Most egregious were the bright spots at the Cave view. I had a set of dark arches with lights on the inside edge, illuminating a long walkway. The silhouettes of the arches against the god rays of the cave view had always been marred by DLM errors: some lightmap UV clusters had been too small for 3ds Max to recognize, and so it skipped them during lightmap rendering. The result was perhaps only a dozen oddly lit surfaces, but it was enough that I tried not to look at them when I played this area. Now, unable to resist and without telling the rest of the team, I took the plunge and broke open the lightmap files for the first time in six years
Most egregious were the bright spots at the Cave view. I had a set of dark arches with lights on the inside edge, illuminating a long walkway. The silhouettes of the arches against the god rays of the cave view had always been marred by DLM errors: some lightmap UV clusters had been too small for 3ds Max to recognize, and so it skipped them during lightmap rendering. The result was perhaps only a dozen oddly lit surfaces, but it was enough that I tried not to look at them when I played this area. Now, unable to resist and without telling the rest of the team, I took the plunge and broke open the lightmap files for the first time in six years
Incorrectly lit faces in the Cave, taking their values from nearby UV clusters
|
An example in 3ds Max’s UV editor
|
For better or worse those errors turned out to be pretty darned easy to fix. With the help of 3ds Max’s UV editor window, I could load a lightmap and instantly find the spot where it should have a rendered RGB value. From there I could look around for a matching surface, copy the value, and paint that to the problem spot on the bitmap. Working with Directional Lightmaps meant that I was fixing 4 separate maps for each error, but the result was exactly what I had hoped for.
My success with the Cave bolstered my resolve to revisit some of the other areas which had bothered me. I figured I might as well go area by area and look for any issues that could be within scope to fix.
I found many, starting with the beach.
My success with the Cave bolstered my resolve to revisit some of the other areas which had bothered me. I figured I might as well go area by area and look for any issues that could be within scope to fix.
I found many, starting with the beach.
|
Our crates had separately spawning lids, as well as permutations with and without the lids covering the crates’ hollow centers. Apparently when I imported TSC:E‘s scenery using OS IDE for lightmapping purposes, it didn’t always grab the right variants.
I found that in many places on the beach we had sunlight shining through the middle of what should be a solid crate. Or, a solid shadow on a hollow crate. Out came the brush tool in photoshop, and I hand- painted all of them |
|
Part IV: Trial by Torchlight
While working on the beaches, I pondered an issue we had way back when we switched from Halo’s radiosity to 3ds Max DLMs. Halo’s stock lighting generates a set of 2D bitmaps containing RGB values which are then applied to the environment in real time. If the standard lightmap is vibrant blue, the environment will be that exact color of blue. Halo’s sunlight also doesn’t typically wash out color much, so things like teal Covenant torches on sunlit beaches still have pools of saturated color around them.
When we switched over to a more physically-based lighting renderer through 3ds Max, much of that exterior color was lost. Sunlight, just as it would in real life, completely washed out the torchlight, resulting in relatively bland beach areas. In addition, the three-directional DLMs blended the three light values - again more “realistic”, but much less vibrant and saturated than the normal game.
For example, a torch on the standard stock Halo lightmap would be vibrant teal all around it. With DLMs, a spot on the floor next to a torch would be 100% torch-lit on one DLM map, but the other two would show the ambient lighting shining from the opposite darker directions, away from the torch. At best, the blended torchlight would be a third of its stock value. Lag and I had always lamented this shift, but we accepted that there was nothing we could do about it when we shipped 1.3 in 2015.
The idea bouncing around in my head was to take just the torches, re-render them separately, and layer them back over the original lightmaps to get a vivid teal. I’d proposed it on a theoretical level a few times over the years, but now seemed like the perfect time to give it a shot. And it worked.
I found that by taking the resulting render, using it as a grayscale mask for a fully saturated teal layer in Photoshop, and layering it at 75% opacity, I could bring back a wonderful amount of color to the environment. For bright sunlit areas I was able to use the blend-if function to clip pure white values. This way the torches would be very visible in shadows, but wouldn’t be brighter than the sun itself. I showed Lag and Dafi and they agreed that if it was not too much trouble, it would be nice to do that across the entire island.
When we switched over to a more physically-based lighting renderer through 3ds Max, much of that exterior color was lost. Sunlight, just as it would in real life, completely washed out the torchlight, resulting in relatively bland beach areas. In addition, the three-directional DLMs blended the three light values - again more “realistic”, but much less vibrant and saturated than the normal game.
For example, a torch on the standard stock Halo lightmap would be vibrant teal all around it. With DLMs, a spot on the floor next to a torch would be 100% torch-lit on one DLM map, but the other two would show the ambient lighting shining from the opposite darker directions, away from the torch. At best, the blended torchlight would be a third of its stock value. Lag and I had always lamented this shift, but we accepted that there was nothing we could do about it when we shipped 1.3 in 2015.
The idea bouncing around in my head was to take just the torches, re-render them separately, and layer them back over the original lightmaps to get a vivid teal. I’d proposed it on a theoretical level a few times over the years, but now seemed like the perfect time to give it a shot. And it worked.
I found that by taking the resulting render, using it as a grayscale mask for a fully saturated teal layer in Photoshop, and layering it at 75% opacity, I could bring back a wonderful amount of color to the environment. For bright sunlit areas I was able to use the blend-if function to clip pure white values. This way the torches would be very visible in shadows, but wouldn’t be brighter than the sun itself. I showed Lag and Dafi and they agreed that if it was not too much trouble, it would be nice to do that across the entire island.
The dlm2 torch-only render for the LZ
|
|
So, I set out on a journey to re-render torches for every single exterior BSP. Along the way, I added exterior door holograms to the pass as well, so that the exteriors would look as vibrant as the interiors in all ways.
As you can imagine, checking every exterior lightmap meant I uncovered some other unfortunate oddities.
The end of the first beach, right before the path curves into the forested canyon, turned out to have a genuinely awfully low resolution, with jagged edges and unpleasant artifacting. I had already crossed the line of actually re-rendering lightmaps due to the torches, so I figured why not just re-render this area in higher resolution?
As you can imagine, checking every exterior lightmap meant I uncovered some other unfortunate oddities.
The end of the first beach, right before the path curves into the forested canyon, turned out to have a genuinely awfully low resolution, with jagged edges and unpleasant artifacting. I had already crossed the line of actually re-rendering lightmaps due to the torches, so I figured why not just re-render this area in higher resolution?
While I was here I also took the time to paint out some torchlight that didn’t make sense (visible in the mid-lower-right of the comparison). Removing the light spill reduced visual clutter and gave the player more obvious driving hints: right, along the shore to the hidden overshield, or left toward the rift and the forested canyon path.
When I hit the Cartographer field, I found that the underside of the Cartographer porch had a horrible, previously undetected lightmap UV error. I’d seen issues like it before.
When tool.exe initially runs radiosity on a BSP, it splits and flattens them into clusters, like peeling an orange. The DLM lightmapping process allows artists to modify those clusters to improve the mapping. Unfortunately, tool remembers the original clusters and will recombine split vertices by dragging one across the UVs to weld to the other. This manifests in the game as strangely colored lines across surfaces, or lighting from one spot appearing elsewhere.
In the case of the Cartographer porch, the more torchlight I shone on the underside the more clear it was that the light was appearing on the wrong side of the columns. Tool had decided that the entire column needed to be a single object with no seams at all. The resulting stretched faces are highlighted in red in the comparison to the right. It took a bit of painful UV work, but I got it figured out in the form of a top-down cylinder unwrap - visible as a set of concentric circles. The center of each is the bottom of the column, and the outside edge is the very top where the supports meet the porch.
When I hit the Cartographer field, I found that the underside of the Cartographer porch had a horrible, previously undetected lightmap UV error. I’d seen issues like it before.
When tool.exe initially runs radiosity on a BSP, it splits and flattens them into clusters, like peeling an orange. The DLM lightmapping process allows artists to modify those clusters to improve the mapping. Unfortunately, tool remembers the original clusters and will recombine split vertices by dragging one across the UVs to weld to the other. This manifests in the game as strangely colored lines across surfaces, or lighting from one spot appearing elsewhere.
In the case of the Cartographer porch, the more torchlight I shone on the underside the more clear it was that the light was appearing on the wrong side of the columns. Tool had decided that the entire column needed to be a single object with no seams at all. The resulting stretched faces are highlighted in red in the comparison to the right. It took a bit of painful UV work, but I got it figured out in the form of a top-down cylinder unwrap - visible as a set of concentric circles. The center of each is the bottom of the column, and the outside edge is the very top where the supports meet the porch.
(1.4) Bad UVs in red at the Cartographer porch.
|
(1.3) Fixed UVs at the Cartographer porch.
|
I’m very happy with the improved porch underside, especially with those areas that were particularly out of whack. The distant-middle-left pillar in the comparison above really shows the light direction shift. With luck, the increased teal lighting will help players notice the secret entrance better as well.
Part V: Increasing Complexity
Inevitably I ran into an issue that ground progress to a halt.
As I made my way around the exterior BSPs, layering torches as I went, I came across a very odd scratch-like lightmap error in the security pit. It was frustratingly obvious once noticed, and only got worse as I added torchlight to the scene. Bolstered by my success fixing UVs elsewhere, I opened up the file... only to discover that this small scratch was really a UV error stretching across much of the lightmap. Fixing it would mean re-UVing a large section of geometry, but I was fully invested by now and dug right in.
As I made my way around the exterior BSPs, layering torches as I went, I came across a very odd scratch-like lightmap error in the security pit. It was frustratingly obvious once noticed, and only got worse as I added torchlight to the scene. Bolstered by my success fixing UVs elsewhere, I opened up the file... only to discover that this small scratch was really a UV error stretching across much of the lightmap. Fixing it would mean re-UVing a large section of geometry, but I was fully invested by now and dug right in.
|
|
|
Instead of only fixing the problem clusters, I remapped everything for optimal lightmap detail. In addition, I foolishly decided to supersample the whole lightmap at 2048x2048 over its original 256x256 resolution. Even worse, I left for a trip and the re-render crashed only part way through. Lag and Dafi were ready to drag me off this area (rightfully so) but I managed to get it finished.
Even with the render completed, I had a lot of layering and manual painting to get it just right. All in all I spent over two weeks on this one task. In the end it was worth it, but that particular project threw up many warning bells from all of us that I was getting in over my head.
Even with the render completed, I had a lot of layering and manual painting to get it just right. All in all I spent over two weeks on this one task. In the end it was worth it, but that particular project threw up many warning bells from all of us that I was getting in over my head.
While in the area, I also took care of a few other lingering issues. Most obvious are the beam emitters, which suffered from the same sort of random brightness issues as the Cave arches. By choosing sections of nearby geometry that appeared correct and copying those RGB values over, I successfully hand painted more than a dozen of the errors. The end result is nothing short of transformative.
There is a ledge to the side of the small Security Pit bridge that allows players to engage the central turret from relative safety. However, it was also home to a rather frustrating lightmap UV error. I took a shot at this too while I dealt with the emitters.
A last minute hail-mary during this stressed period led to a fix for a mismatching Cave transition. In 1.3 the ground abruptly changed from grass to dirt as the player drove the warthog into the Cave. At the time we realized it had happened due to the wrong ground material being set on one side of the BSP switch, but didn’t have time or the ability to fix it before 1.3 shipped. Now however, I realized I could just paint the section of dirt green in the lightmaps, and the player would never notice the difference while driving through.
I added some additional fill lighting to the far side of the hog jump too, because I’d seen too many players aim into the darkness and fall to their deaths.
I added some additional fill lighting to the far side of the hog jump too, because I’d seen too many players aim into the darkness and fall to their deaths.
Part VI: Internal Struggles
Even as I dedicated myself to the exterior torch pass, I resolved not to mess with the interiors unless I noticed actual errors. The pain of the original process had faded, but I retained a sense of dread when considering any touch-ups involving the complex Forerunner geometry.
Nevertheless, I did take a few steps to fix lingering issues.
Shaft A, always a favorite area of mine, had some lightmap UV errors on the various struts holding up the holo panels. While fixing the small lightmap errors in the Cave and Security Pit had reacquainted me with the technical process, the struts proved one of the longer and drawn out fixes. At best I was copying RGB values from nearby faces. At worst I was guessing entirely, and fixing by trial and error. I ended up fixing issues like this across every single holo strut in the interior.
Nevertheless, I did take a few steps to fix lingering issues.
Shaft A, always a favorite area of mine, had some lightmap UV errors on the various struts holding up the holo panels. While fixing the small lightmap errors in the Cave and Security Pit had reacquainted me with the technical process, the struts proved one of the longer and drawn out fixes. At best I was copying RGB values from nearby faces. At worst I was guessing entirely, and fixing by trial and error. I ended up fixing issues like this across every single holo strut in the interior.
(1.3) Shaft A struts with lightmap UV errors.
|
(1.4) Shaft A struts with fixed lightmap UV errors.
|
I also took another shot at painting the BSP transition between the Cart entrance and the locked door room, since that had never quite looked right when walking back up the ramp. Now the lighting shift is still noticeable, but far less so than before.
|
During these other fixes, I noticed that some lights on the various Shaft A doors and ledges were missing or misaligned. This required some minor re-renders, which stepped past the degree of interior meddling I had so far tried to avoid.
And then, inevitably, I noticed that some of the holograms placed over the doorways were off-center. I am both proud and embarrassed to say that I painstakingly went down the list of every single door hologram and recentered them throughout the map. I’m honestly not sure how many, but probably upwards of a hundred. Once I had those adjusted, I mirrored the torch workflow from outside to add a new, fully re-centered glow over every door frame. |
|
By this point I was pretty confident regarding new lightmap renders, and went actively hunting for problems. The vertical supports in Shaft A had visible lightmap seams, so I redid them completely. Nearby, purple lighting underneath the locked door room also ended up washed out, so I hand painted that to a deeper, more saturated color.
As I continued through the interior, I tried to focus on blatant UV errors and exposure issues. In the Catacombs I brought back some purple glow from the overexposed door alcoves and re-rendered the entire area in the process. A short distance away in the Aquarium, I fixed an ugly UV error on the railing and some overexposure on the floor.
|
|
|
One fix that probably didn’t need to happen was the overexposure in the rafters from the Water Hall and the Lobby. I noticed that the warm lighting had blown out to full white. In the game, this looked slightly blue next to the less-exposed tan lighting. I re-rendered all the rafters in both areas with different light settings, and additionally painted all remaining white spots a very bright tan. Was it necessary? Likely not. Does it look better as a result? I’d like to think so.
Part VII: Fixing the Impossible
During development on 1.4 there were a few items that really pushed the envelope for what we considered possible, let alone within scope.
The first, and most impactful on gameplay, was overhauling the Security Complex Hunter arena's collision
The first, and most impactful on gameplay, was overhauling the Security Complex Hunter arena's collision
Our version of the Security Complex features two floors, a large ramp for the player to traverse from one to the other, and an elevator in the middle that our Hunters ride up and down to chase the player.
In the original TSC:E release, if one Hunter was killed, our intent was for the other to stop riding the elevator and try to chase you. However, in practice, the lone Hunter would often get stuck partially up the big ramp, leaving the player to run to safety. Our solution at the time was to instruct the Hunters to avoid the big ramp entirely.
In the original TSC:E release, if one Hunter was killed, our intent was for the other to stop riding the elevator and try to chase you. However, in practice, the lone Hunter would often get stuck partially up the big ramp, leaving the player to run to safety. Our solution at the time was to instruct the Hunters to avoid the big ramp entirely.
A Hunter stuck on the large ramp, a bug in 1.3. Unable to move, the unit can only fire at the player.
When evaluating potential fixes for Patch 1.4, making the Hunter arena as engaging as possible was a high-priority stretch goal. Teh Lag added new soft ping animations so the Hunters communicate their health status better to players. But really, we wanted to fix that ramp.
To explain, we have to look at Halo 1’s AI behavior. They’re notorious for not trusting holes in the floor, even when covered safely with glass or grates. That’s why enemies refuse to chase you from one side of the AotCR ziggurat to the other, since the central hallways all have grates on the floor.
We solved this in many places in TSC:E by creating a very shallow collision-only “bridge” over the affected areas. It didn’t always work, but it allowed units to walk around the Water Hall - which admittedly does still have a few places where units get stuck - and Server Room.
To explain, we have to look at Halo 1’s AI behavior. They’re notorious for not trusting holes in the floor, even when covered safely with glass or grates. That’s why enemies refuse to chase you from one side of the AotCR ziggurat to the other, since the central hallways all have grates on the floor.
We solved this in many places in TSC:E by creating a very shallow collision-only “bridge” over the affected areas. It didn’t always work, but it allowed units to walk around the Water Hall - which admittedly does still have a few places where units get stuck - and Server Room.
A player clip bridge covering the upper elevator hole
What puzzled me was that we had applied the same trick here, to both glass sections. Why did one work and the other didn’t?
And then I saw it. So blatantly obvious, and yet it eluded us for so many years. One player clip bridge covered the strips edging the glass. The other didn’t.
And then I saw it. So blatantly obvious, and yet it eluded us for so many years. One player clip bridge covered the strips edging the glass. The other didn’t.
An oversight which led to insane adventures
This clipping oversight intersected with another, related quirk of Halo's AI pathfinding: their navigation algorithm strongly preferred traversing along strip geometry, even if it was obstructed, if it technically got them closer to their destination. The result was that Hunters would run up the glass and immediately become stranded.
Teh Lag had the grandiose - and insane - idea to retopologize only the collision side of the BSP model to fix the player clipping placement. The trick would be to do it without recompiling the BSP, which would regenerate lightmap UVs and destroy all lighting work for the Cave.
No tool existed that could pull this off, so she made one. Using the MEK’s python library, Lag was able to create a BSP surgeon tool which could transplant another BSP's collision data, and then patch all of the existing render data into the updated BSP structure. We could make the fixes in 3ds Max, export the BSP normally, and then use the surgeon to swap it right in without losing all the lighting. This was a first in HCE modding history, and we have included this tool in our release.
Even with this fix, the Hunters still sometimes got stuck - this time on scenery that the ramp geometry didn't let them pathfind around. After days of attempts, we settled on an underhanded trick: an additional invisible gutter of player clipping geometry gently pushes the Hunter units' physics pills out of the way before they can get stuck. (We ended up adding similar gutters to other places in the Complex.)
Teh Lag had the grandiose - and insane - idea to retopologize only the collision side of the BSP model to fix the player clipping placement. The trick would be to do it without recompiling the BSP, which would regenerate lightmap UVs and destroy all lighting work for the Cave.
No tool existed that could pull this off, so she made one. Using the MEK’s python library, Lag was able to create a BSP surgeon tool which could transplant another BSP's collision data, and then patch all of the existing render data into the updated BSP structure. We could make the fixes in 3ds Max, export the BSP normally, and then use the surgeon to swap it right in without losing all the lighting. This was a first in HCE modding history, and we have included this tool in our release.
Even with this fix, the Hunters still sometimes got stuck - this time on scenery that the ramp geometry didn't let them pathfind around. After days of attempts, we settled on an underhanded trick: an additional invisible gutter of player clipping geometry gently pushes the Hunter units' physics pills out of the way before they can get stuck. (We ended up adding similar gutters to other places in the Complex.)
Invisible player clip gutters assisted the Hunters
Then, after fixing the ramp, we noticed similar problems - Hunters were taking paths all across the complex's strip geometry that got them stuck in obscure, nonsensical places. Already deeply invested, we did it again: we re-topologized the Complex floors to remove strip geometry in problematic areas so the Hunters could choose more intelligent paths. Lag paired this with an array of contingency scripting to secretly swap out some objects' pathfinding data when Hunters seemed like they were stuck.
Sketched proposals for collision simplification
It took a huge chunk of time, but the Hunters will now mercilessly chase players throughout the Complex. If one Hunter is killed, the other can and will follow you up the ramp. Even more importantly, if one falls down while the player is still battling the other on the upper level, the fallen Hunter will run back up the ramp to rejoin the fight. New and returning players should get a nice surprise.
The other gigantic project, accomplished with the same tools, was the ocean dropoff fix.
At one point during the level, the player drives across a beach next to a steep underwater dropoff. The giant archway looming overhead is actually the elevator the player later uses to descend down to the Water Hall, located on the ocean floor.
The other gigantic project, accomplished with the same tools, was the ocean dropoff fix.
At one point during the level, the player drives across a beach next to a steep underwater dropoff. The giant archway looming overhead is actually the elevator the player later uses to descend down to the Water Hall, located on the ocean floor.
The far beach, more integrated with the rest of the island than is obvious at first glance
In theory if a player pushes something off the beach cliff, it should land down on top of the Water Hall. In practice, our BSPs didn’t match in 1.3. Multiple players reported seeing Elites floating mid-ocean as they rode down the elevator. It turned out that the Elites were falling to the “bottom” of the ocean in the above-water BSP and tipping out of their vehicles. Then, when the player triggered the deeper underwater BSP, those same Elites would be stuck floating where the original ocean floor had been.
The depth of the ocean floor in the cart-capp BSP (highlighted in white) didn’t match that of int_shaft_b (purple)
|
Floating Elites visible from the Water Hall elevator
|
It had always been a dream of mine to find a way to match the upper and lower ocean floors, and Lag’s new BSP surgeon tool was the answer. I rigged up a very low-poly version of the water hall exterior and inserted it where the old ocean floor had been in the sapp-sec BSP.
The new collision model, with accurate approximations of the Water Hall and surrounding terrain
The good news is that items did indeed fall down just as I had hoped. The bad news was that the Elites and (if the target was your own hog with allies riding it) Marines stayed alive underwater. It was incredibly surreal to look out the Water Hall windows and see a very confused Elite looking around. On top of that, the collision import introduced new phantom BSPs: invisible geometry that blocks vehicles or weapon fire.
Some painstaking phantom BSPs fixes later, we turned to the AI issue. Lag decided the most logical solution was to kill any units that drop below the waterline. This would still let vehicles drift to the bottom, even if the game automatically cleaned up the actual bodies. Later Lag also prevented the game from removing vehicle debris, to give the illusion a little extra heft.
So, fling vehicles off the beach all you like- your resulting art will now be displayed as you move underwater later in the level.
Some painstaking phantom BSPs fixes later, we turned to the AI issue. Lag decided the most logical solution was to kill any units that drop below the waterline. This would still let vehicles drift to the bottom, even if the game automatically cleaned up the actual bodies. Later Lag also prevented the game from removing vehicle debris, to give the illusion a little extra heft.
So, fling vehicles off the beach all you like- your resulting art will now be displayed as you move underwater later in the level.
Part VIII: Final, Desperate Leaps
Some of my most complex fixes happened right before ship, which is miraculous and horrifying all at once.
When I re-rendered parts of the beach to try to get higher resolution shadows, the lighting shifted somehow and the waterline now drastically mismatched, far worse than in 1.3. Unfortunately I only noticed this after I had already sent Lag the “final” set of LZ lightmaps. I dug my feet in and hand painted every inch of the main beach, across 4 lightmaps and 3 directions of light. I’m still mortified I didn’t notice the issue sooner, but I’m glad I was able to fix it.
When I re-rendered parts of the beach to try to get higher resolution shadows, the lighting shifted somehow and the waterline now drastically mismatched, far worse than in 1.3. Unfortunately I only noticed this after I had already sent Lag the “final” set of LZ lightmaps. I dug my feet in and hand painted every inch of the main beach, across 4 lightmaps and 3 directions of light. I’m still mortified I didn’t notice the issue sooner, but I’m glad I was able to fix it.
One of the scariest last minute finds was realizing that all the mysterious locked door alcoves on the way down the Water Hall elevator had massive UV errors. In some cases entire sections were plastered with pitch black geometric shapes.
By this point I was nearly burnt out and very unhappy at this discovery. Nevertheless, I refused to ship something so blatantly incorrect.
By this point I was nearly burnt out and very unhappy at this discovery. Nevertheless, I refused to ship something so blatantly incorrect.
Along with that, the top of the elevator got way darker as the player rode the elevator down past the BSP switch, meaning the lights were different between int_shaft_a and int_shaft_b. I had to re-UV almost the entire elevator, and re-edit Shaft A to dim down the light as the player initially enters the room. Coincidentally this led to a spookier elevator reveal, as the player moves down the pitch black ramp and opens the door to find an elevator room lit almost entirely by the glowing holo panel.
Some of the elevator lightmaps overlapped into the Water Hall, leading to more work than I originally intended. While making sure my re-renders matched the rest of the Hall, I discovered that many of the cover pieces had lightmap errors. So, like everywhere else, I went to work. More than 60% of the Water Hall has been re-rendered, and now all cover pieces have smooth and accurate lighting without visible lightmap seams.
|
While waiting for some of the Water Hall renders to finish, I revisited the exterior plate floor texture. Unlike the interior version, which Dafi and I recreated for 1.3, the exterior plate floor was still the same as in stock Halo: half-resolution and paler than most of our other textures.
Using the interior remake as a starting point, I remade the exterior texture from scratch. I traced the salt lines and Forerunner shapes, hand-painted in grime and wear, and mostly tried to keep it as close to the original as possible. Like our other alternate Forerunner textures, the new plate floor ext is a little bit higher contrast, but overall it should ring true when compared to the original. |
|
Near the end of the lighting pass, Teh Lag pointed out that we could recompress the actual lightmap bitmaps. Up until now, all the lightmaps had gone through the same old imprecise tool.exe import method that caused the compression artifacts discussed earlier in this article. Now, with some additional arcane coding magic, she got them all imported at nearly the same quality as my original renders. In the best cases, this brought back soft gradients that had been hardened into abrupt edges, and helped bring out fine detail in plant shadows. The only downside is that this did shift the shading and brightness slightly across the map, with no way for me to compensate or account for it. After all, I’d used the compressed versions as in-game references for years. I decided it was better to have it as high quality as possible and not to worry about newly introduced lighting artifacts. Too much.
Part IX: Full Circle
At the time of this writing, it has been six months since we released. During that time we’ve gone through a period of inevitable burnout and recovery, followed by a minor set of bug fixes meant to ship alongside these postmortem articles.
Even as I assembled these articles, Dafi flagged some really awful errors in the standard lightmaps. While normally the standard maps are only used for scenery lighting, we wanted anyone who happens to have DLMs turned off to still have a decent experience. I admit the standard lightmaps are far less refined than the DLMs, but at least now they more closely resemble their polished counterparts.
I did poke some final DLMs too. Additional purple glow accents the player’s crossing from Shaft A into the spooky hallway, and the L-Room Flood vents glow in a more consistent fashion.
But my final fix was an adjustment to the hidden NB + KK heart easter egg that serves as a gift to my now-wife.
Even as I assembled these articles, Dafi flagged some really awful errors in the standard lightmaps. While normally the standard maps are only used for scenery lighting, we wanted anyone who happens to have DLMs turned off to still have a decent experience. I admit the standard lightmaps are far less refined than the DLMs, but at least now they more closely resemble their polished counterparts.
I did poke some final DLMs too. Additional purple glow accents the player’s crossing from Shaft A into the spooky hallway, and the L-Room Flood vents glow in a more consistent fashion.
But my final fix was an adjustment to the hidden NB + KK heart easter egg that serves as a gift to my now-wife.
When we created an MCC-compatible tagset as part of 1.4, we figured it would be fun to include a multiplayer demo using those tags, in the same vein as a50_revisited. The clear option was Precipice, a multiplayer map I dabbled with in high school and finally released in 2011. It was the world in which I learned to 3D model and eventually to render lightmaps through Firescythe’s tool, Aether (the predecessor of the DLM rendering tech used in TSC:E). It had been protected at the time of release (an unfortunate relic of the community mindset of the era) and thus its source material had never truly been released in the same way we had everything else.
Precipice, my first and only other substantial Halo modding project
I was quite happy at the concept of returning, both to showcase our better-looking (but stock-gameplay-identical) weapons and vehicles and for a full source release. We did a palette swap for the TSC:E tags in Guerilla, shipped all the pieces with 1.4, and congratulated ourselves on a job well-done.
Months later, during this final bug fix pass, I decided we should take the extra few steps to actually polish the map for the 1.4.1 release. Dafi made some new fire visuals, considerable upgrades from the 2001-era stock effects originally in the map. I did some manual blurring on a particularly compressed section of lightmaps in the control room, and added a few extra weapons and grenades.
As a final touch, we brought over our TSC:E versions of the doors and holograms. Lag kindly refurbished those holos that hadn’t already been updated with the others, including some from The Library.
Months later, during this final bug fix pass, I decided we should take the extra few steps to actually polish the map for the 1.4.1 release. Dafi made some new fire visuals, considerable upgrades from the 2001-era stock effects originally in the map. I did some manual blurring on a particularly compressed section of lightmaps in the control room, and added a few extra weapons and grenades.
As a final touch, we brought over our TSC:E versions of the doors and holograms. Lag kindly refurbished those holos that hadn’t already been updated with the others, including some from The Library.
Upgrading an old project felt like a perfect way to wrap up, a microcosm of the whole
It really is nice to be able to walk around the map with the additional level of detail. To showcase our new work using my old art just feels so natural, and a wonderful way to bring things back to where they started.
Part X: Was It Worth It?
That’s the question I do find myself asking, now that we’re through the mad crush of release preparation and well into post-release contemplation. On the one hand, it did eat a lot of my free time for much of a year, and re-introduced many of the stresses that our original development caused in all our lives. These articles have extended the process by yet another few months.
But there’s something to be said about returning to a beloved project, knowing you have the knowledge and ideas and solutions to those lingering problems that always hovered at the back of your mind. To fully, truly realize the original potential without compromising the original art style.
That this brief adventure turned into nearly a year long endeavor is a testament to how deeply all of us care about this project, and how much we care about the people playing it.
We hope that everyone returning to check out the new version, as well as those now discovering it for the first time, set off on their own adventures knowing that we tried to make TSC:E the absolute best we could. After all, that’s as much as anyone could ask.
But there’s something to be said about returning to a beloved project, knowing you have the knowledge and ideas and solutions to those lingering problems that always hovered at the back of your mind. To fully, truly realize the original potential without compromising the original art style.
That this brief adventure turned into nearly a year long endeavor is a testament to how deeply all of us care about this project, and how much we care about the people playing it.
We hope that everyone returning to check out the new version, as well as those now discovering it for the first time, set off on their own adventures knowing that we tried to make TSC:E the absolute best we could. After all, that’s as much as anyone could ask.