HaloCE - OpenSauce - V3.1 Released

[Kornman00]

It only works for the machine that is running that code.

Those numbers are in game ticks IIRC. Changing them to 5 would make them update every 5 game ticks.

[Kornman00]

Today, we here at the (virtual) offices of Kornner Studios are happy to announce that version 3.1 of OpenSauce is coming soon! How soon? Why, none other than next month: January!

Unlike previous releases of the v3.0.x brand, this release introduces a lot of new features and improvements instead of -just- addressing bugs/issues. Hence why that '0' after the period ranked up to a '1'.

For OS 3.1 both FireScythe and I have some really juicy goodies in store for you. But we're not the only ones contributing to this next release of OS. ChokingVictim is a special guest this time around with some more gameplay enhancing features for you map makers to play with! He has contibuted work to OS before (e.g., he helped us implement the unit-infections system) but has become more active on the project for the past few months. We're hoping to keep him with us for the foreseeable future.

However, it being near the holidays and all he didn't really have the time to get all his and video written up and made before this annoucement was posted. So we're going to follow up later on with details directly from him on what you can expect in 3.1 and (hopefully) beyond. For today, you'll have to make do with what FireScythe and I. Deal with it.

Speaking of FireScythe, let's start with him first. He took the time to write up this longpost about this small feature he made, so why not. Wait, did I say small?

In-game Map Downloads

I’m sure you’ll be happy to hear that I have written an in-game map download system that is built in to OpenSauce v3.1.0 which will be coming to a Halo near you mid-January!

The system has 3 parts to it, the OS client, the OS Dedi and the web server.

You are probably a client which means you have the easiest job , simply try to join a server running a map you do not have and the following will happen:

* You will join, then disconnect from the server as you don’t have the current map
* OpenSauce will sense your distress and try to find and download the map for you
* It requests the map from the dedicated server first, which may or may not be serving it
* If the dedi isn’t serving the map OS will then try to download it from one of a list of master servers (described below)
* If it finds the map on the dedi or master server it will automatically download it, and if successful, extract it into your maps folder
* The map is then added to Halo’s map list and with the map now available, OS will attempt to reconnect to the dedi

All this happens without the user having to interact, restart Halo, or any other nonsense!

This takes place whenever you don’t have a map the server is running, so if a game ends and a different map is loaded that you don’t have, the process starts again to download that map and reconnect you to the server.

Of course, a map download system is pointless without somewhere to download the map from. If you are a cool guy you may well be running one or more dedicated servers, in which case you can serve the maps yourself.

The OS dedi has a HTTP server embedded, which is what makes the magic happen. The server is setup using functions you can pop in to your init.txt to keep things nice and simple. Full details on the OS dedi map server setup can be found here but the main jist of it is:

* Compress and split your maps using OpenSauceIDE (included in the OS installer)
* Upload the map parts to a 3rd party file host that supports direct linking
* Setup your OS dedi to enable map downloads, pointing to your 3rd party file host as the data source
* Start the game server

When the client requests a map from the OS dedi, it responds with some information about the map and its parts, after which the client requests each part from the dedi. The dedi redirects those requests to the 3rd party file host you are using so that the client can download from them. This means the dedi’s bandwidth is free to use for the important things in life such as pretending to kill people.

Since the OS dedi has a HTTP server embedded it can also serve the map downloads itself, and as long as the HTTP servers bandwidth is throttled (again, set in the init.txt) this may be a viable option for some people.

If you are running a passworded server you can even encrypt your map using the server password at the map compression stage, so that people who shouldn’t have your map don’t get your map.

But wait! There’s more!

“What if the dedi is bad and isn’t serving the maps?” I hear you say. Well, thats where master servers come in. The master server is a set of php scripts that can be used on a normal web server running PHP and MySQL and is used to serve maps to the entire OS user base.

And this is where OS needs YOU!

The map download system won’t take off without a decent amount of master servers running, as that not only provides more choice to the players, but also distributes the bandwidth cost amongst many kind volunteers which is better for everyone. Halomods is currently the only website running as a master server.

So, we need people who run websites with a good deal of spare bandwidth to donate some of their bandwidth to help make Halo CE more accessible, and hopefully make custom map servers the norm.

The details on how to set up a master server can be found here

FS, if the community had a voice, I think it would be saying "music to our collective-ears". But it doesn't. It's a voiceless being. Well, until you make something that doesn't play well with Xfire. Then it grows a mouth and starts jibber jabbering and raising a bunch of stink.

But guess what?

OS 3.1 no longer fights with Xfire. In fact, they're seeing other people now. OS now infiltrates the game code via pretending to be dinput8.dll, instead of d3d9.dll. That means all you Xfire lovers who have never heard of the superior software known as Steam will be able to play Halo, use OS, and chat via Xfire's ingame overlay all at the same time. Of course, the Steam lovers have been able to do this for some time now. Neener-neener.


On top of various other fixes, we've also got some more user-oriented features coming in OS 3.1:

* You know that 'checking for updates...' message that comes up every time you try to do something MP related? Well now there's an xml setting to make the game think it already ran this check, saving you an additional second or two.
* We now have a command line parameter for specifying what version of the game you want to display a server list for. By default, if a value isn't given via this parameter, we get the ENTIRE server list from all the versions of Custom Edition. Or you can specify "disable" and the server list will behave as normal, populating the server list with 1.09 only servers.
* We've added new xml setting for forcing OS to use .yelo files first when searching for .map files (off by default). So with this feature on, OS will pick a10.yelo first instead of a10.map, assuming both files are present.
* There's now a command line parameter for turning off all OS-based gfx additions/upgrades.
* The installer has been improved. E.g. it installs the VC++9 runtime, allows the user to -not- install OpenSauceIDE, etc
* Mini-dump support has been added to haloce/dedi/tool/sapien (guerilla is the oddman out) to aid in any future debugging efforts when the EXEs crash

Of course, average joe users aren't our only target demographic. Map makers are in for some treats too:

* We've increased the max grenade types to 4. This works in multiplayer as well and there are fields in the scenario's starting profiles to specify the grenade counts for the 2 new types. In order to keep this feature working in MP without having to wait on new networking features, we couldn't go any higher than 4. Some people were already catching on to this new feature
* We've added a script function for playing .bik videos (need to be in the same location as the game's existing .bik videos)
* We've added a system for displaying UI widgets via script code. You execute a function and then it displays a ui widget definition specified in the project_yellow/globals tag.
* We've added a 'runtime vectors' system in a similar fashion as the 'runtime integers' system. This system allows you to perform vector operations in script code. CMT was able to use this setup to essentially perfect vehicle boosting. You can check out teh_lag's prototype video of this here
* We've added script functions to perform bit operations (AND, OR, etc) and converting hex string to an integer
* We've increased the maximum number of particles that can exist at any given time. We're still playing with these settings, so the exact upgrade amount isn't set in stone yet.
* We've increased the maximum buffer size for model vertex and index data that Tool uses when building a cache file.
* The project_yellow_globals tag features a new flag for forcing the game to use 'stun jumping penalty' instead of 'turning penalty' in the engine's biped jump code. We're considering this a "patch" as I'm pretty sure this wasn't the intended behavior in the stock engine.
* We patched the weapon tag's magazine's magazine-objects block so that it will no longer crash when you try to add more than 2 elements. Uses the equipment field for the block name now too. This is a "patch" for a bug in the stock HEK. If you try to open the weapon tag in the stock Guerilla with more than 2 elements in the magazine-objects block, it will crash.

I want to make two special notes here that are aimed at map makers:
* Since ChokingVictim has become a lot more active and has presented quite a number of potential new features that we'll be adding later on, we're giving him his own tag to use to implement these new features: project_yellow_globals_cv. Why this matters to you is because we will be moving the unit-infection definitions *to* this tag -from- the project_yellow_globals tag. We're not aware of any maps that have been released that make use of the unit-infections system yet, so in order to Keep It Clean, we're just going to add this system's tag fields to his new tag group. If there have been any final maps that have been released which use the unit-infections system, speak now or forever hold your peace (and not have working unit-infections in 3.1)
* The team index offset for MP teams for the GBuffer has been changed. MP teams now start at 10, not 9 (game_teams come first, tho technically the last few are 'unused' by Halo proper). This is a breaking fix, but I'm not aware of any major MP map releases that used this specific part of postprocessing.


Developers have things to look forward to as well (well, there's no looking forward as most of this is already in the code already):

* The codebase has been reworked to match that of the actual game engine's.
* There are now only two shared source folders (aside from 3rd party libs), "blamlib" and "YeloLib". These help to draw cleaner lines where there is data/code added/injected by OS, or where there is code that is specific to OS.
* The CheApe compiler has been modified to support faux tag-structs. So if you have tag definitions that have repeated fields (eg, in the case of the engine's shader and hud interface tags), you can use these to essentially have the compiler cut&paste the field definitions for that tag-struct.
* We've been working on a string_id system that is very much like what is used in the games following Halo1. A first-class citizen StringId field type has been added to the CheApe parser so it will handle the underlying hacks that we made to get such a system working in Halo1. When a cache is being built, the system will also take tag_string fields and create ids for their values, then mapping the address of those fields to those generated ids. So existing tag_string fields can be used in code that works with string_ids.

That's all we're wanting to talk about today. Again, stay tuned for future updates involving what ChokingVictim has been working on. There may be even more features that make it into 3.1 from now until release, but we don't want to comment on them until we know their fate for sure.

[Disaster]

Epic work guys. Each new release is always impressive.

[DarkHalo003]

I'm so glad you guys are still trying to keep HCE alive for those who still love it. Maybe one day (even though the engine is still really old) we can bring a lot of people back with these advances you guys have done for us.

[Amit]

Hell, with all the features going into the new release, it's almost like OS v4.0 for us simple people.

[Con]

Does the map download system need master servers because you can't lookup maps easily on halomaps.org and/or Dennis doesn't want the site to be used that way (without ad revenue)?

[Cortexian]

It's a Dennis said no deal I believe.

Plus the halomaps.org servers are slow as hell.

[FireScythe]

Pretty much, from a technical standpoint I wouldn't think that halomaps' structure is able to serve data fast enough for map downloads since it copies data between servers and such before the user can download it. Saying that there's no reason why Dennis couldn't set up a master server on halomaps, but it's not fair to put extra burdon on him to do so when hes already done so much to provide halomaps to the community. If he wants to run one in future then great, but he doesn't have to and we shouldn't have such a reliance on halomaps for this kind of system.

The main issue with the idea of map downloads is the cost of hosting them. If you have a single monolithic database of every map that everyone downloads from, the cost of that would make it unfeasible to run (Which i'm sure Dennis's wallet would agree with!). But with the master server setup in OS, when a substancial number of smaller hosts are serving a subset of maps the cost is distributed and should be more viable.

[Kornman00]

Don't forget, maps can be hosted on the cloud as long as you can grab them via HTTP

[Con]

A feature you might want to add to the client is pushing new maps you've downloaded to your own master server.