BlamLib: The .NET counterpart in OpenSauce

BlamLib has finally been added to the OpenSauce source-depot!

(Note: Not a programmer? You should probably just move along now)

BlamLib is composed of a series of .NET assemblies (one C++/clr, the rest C#) used for interoping with Blam Engine (the stuff that powers the Halo series and a couple other things) data. Data interop ranges from tag files (the engine's asset storage) to cache files (the engine's storage for release/shipping level builds) and all the little odds and ends that compose both facets.

BlamLib provides the building blocks needed to deal with tag files so you can create\edit\read\etc information without having to worry about various underlying details of the tag system. It allows you to open a cache file (ie, a level's monolithic data storage file) and perform various operations, eg, gathering build details, listing string ids, listing tags which compose the cache file, extracting tags, etc.

BlamLib also powers OpenSauceIDE. This IDE provides users with a somewhat automated tool interface for various OpenSauce operations. The IDE includes the CheApe tool which you use to create an optimized data storage (built from source XML files) which the OS HEK will use to interop with your new tag/scripting definitions that you've created in your game extension. Without building this data storage, the OS HEK would have no knowledge of your modification's new additions to the game.

The IDE also includes a tool for "applying" the patches to HEK EXEs which are needed to run the CheApe modifications. This makes it so that you can apply it to an already customized (albeit, unpacked) HEK program. It also makes it so that you don't have to download even more EXEs. You get to see what changes are being applied to each HEK program's EXE as they're maintained in OpenSauceIDE's project code.

Future tools could eventually make it into OpenSauceIDE. Like say, it's powered by BlamLib so it's really meant to act as the UI for all the functionality found in it.

Just like the current status of OpenSauce, this is in-development (read: beta) code. Nothing is final. I've only put it up now for programmers to get a feel for the library which I've been [growing tired of] developing in some way since 2004/2005. Some parts really need to be rewritten to make use of advances in .NET tech and also just because they're not very good (lived/learned over the past 6+ years). However, as each new game came out I had to adjust some of the library to include functionality (that wasn't there to begin with!) to interop, and interop in a standard way, with the new engine. BlamLib began development with only Halo 1 and Halo 2 (Xbox) interop in mind. Now it has support for the Halo series and even Stubbs. Albeit, I didn't always fully flesh out each engine's support as much as others.

So buyer beware. While I have no intention of doing anymore major changes (ie, code interface changes) to this specific codebase, things could change down the road. Right now, the road I'm on detours from BlamLib's for quite a few miles so I didn't want to be sitting on this code for any longer than I've already been.

Wow, it must feel good to dump all that off finally! You don't get nearly enough credit for what you for this community. I can't wait to see what people make with this.

Blimey, Korn can really pull stuff out his sleeve cant he :D/

I've been browsing through the BlamLib tree and kind of confused. I am not a C# or .NET fan to say the least but could you make a simple vidoc like you did with OpenSauce just explaining the tree and how programmers would begin to use this code? Does BlamLib build a shared library for other applications to import from or am I missing something? Please explain in layman's terms -- if I hear the word 'interop' one more time I think I'm gonna puke lol! I'd like to check this out and make use of it in some way. Wondering also if this would be some how useful in my forge project. hmm.

Yeah, I would have to agree, some sort of vidoc for BlamLib would be required before I leave it be for a while. I hope to have one soon but I can't promise anything.

.NET produces "assemblies" from code which other assemblies can then reference. So yes, BlamLib builds as a shared assembly which other .NET programs can use. Two examples are BlamLib.Test (the project used to actually validate the code behind BlamLib) and OpenSauceIDE.

@ Con. Yeah, it feels nice, but it would feel ever better to try and get people into it so that they can start learning some of the awesome sauce behind Blam. Basically I now offer two avenues people can take: native and managed. The core OpenSauce's game modifications let people understand the runtime of the engine. The BlamLib codebase let people understand some of the data interaction of the engine (plus, isn't limited to a specific engine build) while also keeping things simple since it's done in managed land (no worrying about memory management techniques, or sneaky preprocessor macros, etc).

@ Limited. Just wait until you see what I have in store for Halo 2/Xbox :)

So I just fired up VS08 and started talking about the codebase. Completely unscripted and it shows. Really badly. But, hopefully it sparks some connections or at least some questions which I can then consider answering in a later, more polished, vidoc.

It's about 22mins long and I exported to a really low-res encoding. The original capture was 339mb; this comes out to 24mb. The low res part shouldn't be an issue since the recording tool zooms in automatically and also, the code is posted so you can easily follow my mouse movements and see the code yourself.

http://kornnersoftware.com/open_sauc...Lib.Intro.html

Good introduction to BlamLib! Even unscripted it was very informational and helped me understand the basics. I really like how you explained what you were thinking when you wrote certain code and what it's purpose is in life. That and I really got a laugh at some parts.

So in layman's terms: BlamLib interacts with files that Halo and other Blam engine games use (mainly maps & tags, including the stock editors) while the game is not running ( I hate the word "cache" ). OpenSauce on the other hand interacts directly with the Blam engine while the game is running.

I don't really have any REAL questions as of yet since I have yet to tinker with the code but you said at the beginning it's C++ based but they code files are .cs extensions which isn't that C# files? I am not familiar with .NET or XNA or any of that jazz. I am a low level code kind of guy and this managed crap isn't in my cup of tea, yet. I guess I should read up on it more if I want to dive in.

Thanks,

Basically. BlamLib provides a managed bridge to game data while OpenSauce provides direct access to the game itself (so, the runtime). It's called a "cache" because the tools take a scenario plus all of its dependents and build a memory optimized (as a majority of it is memory addressable) monolithic file. Pretty much everything related to that scenario that is needed to run it in-game will be found in the cache/map file.

Not everyone is a gameplay or a C++ programmer. Not everyone is into tool development or a .NET developer. But chances are you're one or the other so you can find yourself starting in at least one of the codebases.

The only C++ based part of the code is the LowLevel project. LowLevel is where I put all "unsafe" code (that is, code that I must specify with the "unsafe" keyword in C#. See SwapFloat in ByteSwap.cs) and is also where I put any native code wrappers. XNA Math is a native C++ set of code, so I have to use something to bridge it into the .NET world. SlimDX does the same thing. The only time you'll find non-.cs code files is in the LowLevel project. All other projects are C#.

I created a youtube account specially for Kornner Studios and will eventually be using that for all video hosting (including vidocs) where applicable. Then I can let youtube manage and host various quality levels.

Attempted compiling of Blamlib and then running of Open Sauce idea via clean install:

Code:

---

An unhandled exception of type 'System.BadImageFormatException' occurred in BlamLib.Forms.dll

Additional information: Could not load file or assembly 'BlamLib, Version=1.0.3971.23573, Culture=neutral, PublicKeyToken=null' or one of its dependencies. An attempt was made to load a program with an incorrect format.

---

MenuItemParseName(value, ref name);

line 86 of Utils.cs

Edit:

Think you could do a quick intro to adding our plugins to OS IDE? And commenting out the line allows the program to start up however it still crashes on that menu option.

I have BlamLib setup to run under x86 or x64, however, you need to make sure you select those configurations expliciltly under the build configuration in VS.

You see the dropdown box at the top of VS which says "Debug" or "Release"? Next to it, it probably has "Any CPU" or "Mixed Platform". That's bad. Select "x86" (if you're wanting to run and debug the projects with VS at least).

When compiling for x64, LowLevel builds with x64 versions of DX and BlamLib (the actual library) uses 64-bit versions of API calls in Util.Windows.cs (if there is no 64-bit version then the function is undefined)


The reason why it's crashing is because that function is in BlamLib.Forms. If you're on Win64 then when it tries to load a 32-bit assembly from a 64-bit .NET assembly, then you'll get a BIF Exception. Hence why it's always crashing in that function.

Awesome, thanks for clearing that up! And yeah, I have mixed platforms selected, should have changed that pronto.

I think I meant to cover that (32 and 64 bit compat) in the intro vidoc, but you know, unscripted and all, easy to slip the mind ;o