User Content Library

[DEElekgolo]

This C# library will allow you to load any halo reach user content file into a neat object to access its data. It will only read, and not write. From the looks of things, Bungie has made a lot of consideration into user exploitation. The Tag object stores an array of nested tags that can be accessed like so:

Tag[0]To return the tag in the first index.
Or the much nicer:

Tag["chdr"]Which would return the tag named "chdr".

A tag object contains 3 fields.
Name, a 4 character ASCII string which contains the tag name
Size, an Int32 which stores the size of the entire tag(Including the name and size bytes)
Data, a byte array containing the tag's information.

The Data field can be accessed via byte array or BinaryReader.

Tag files can be initialized either directly from a STFS file(using DJ SkunkieButt's X360 Library), or from an already extracted file, and even directly from a stream.
Here is an example of me loading a Tag from a STFS(con) file.

Derp = new Tag();
Derp.LoadFromSTFS(openFileDialog1.FileName);Includ ed in the download is the Library its self, along with the source of an application that shows how to use it properly.


I only ask that you give credit to me and DJ SkunkieButt for its use.
Have fun.

http://deelekgolo.codebrainshideout.net/Pics/2010-09-04_0412.png

Download (http://www.mediafire.com/?l1b6oh6c0bth3e6)

Edit:
One of its uses. (http://yfrog.com/mwglutz)


public struct scnd
{
public string _scnd; //4 bytes
public int size; //Int32 Size of the entire tag file
public byte[] Unknown1; // 4 bytes of unknown
public int ImageSize; //Int32 Size of the image file
public byte[] Image; //The image file. Size of byte array is the ImageSize
}

[Kornman00]

You realize 'struct's are by-value right?

[Skarma]

A 4 byte ascii string doesn't seem to make sense to me, maybe an arbitrary byte array instead. Usually there is a null-terminating character because library functions use it to denote the end of the string and it's length. I would assign it as a 32 bit integer instead, but whatever floats your boat :p

good research, +rep

[CrAsHOvErRide]

Nice research but from the looks of it, the code looks ugly and you don't know how to use OOP right.

+rep

[Kornman00]

They're 32-bit identifiers, or group tags. Just like with their tag system they use an unsigned long to represent 'tagged' data. The C compiler has the ability to parse a character string into a number:

const unsigned long k_scenario_tag = 'scnr';You can't get that with the C# compiler. So they can use k_scenario_tag to 'tag' data as being a 'struct scenario' instance. Just like how you can 'tag' a thread to signify its relations (ie, "halo, leaks, hacks"). The practice stems from their days of working on the Mac platforms as Apple uses a similar process.

BLF files are composed of these tagged entries which allow any specific system to add it's related data but not worry about managing the header or signatures, etc. It can just tell the blffile system "add this data with this entry group tag" (ie, screenshot internal data to 'scnd'). Later on, the same system can go back and say "give me the data which is tagged with this group tag" (ie, 'sncd'). The blffile system manages the main header and footer entries, the user content handles the author and content entries, then if it's a film, the saved films system would handle the film header, film data, etc entries.

All the systems can contribute to the BLF on their own without worry about anything but themselves. They do this with standardized entry headers which store the data's group tag, plus what seems to be a couple versioning ids.

C doesn't restrict you on how you format character strings. However, the standard is to use null-terminated strings (c-strings). But you can also do length prefixed strings (iirc, this is how pascal does it) or what have you. In C, the null-terminated method is just the easiest as it's implicitly done when using string quotes, ie "!". Plus like Skarma said, the standard library works on this storage method so there isn't much reason to use any other kinds of storage method unless you were doing some kind of interop with something written in Delphi or VB, etc.

[Skarma]

Thanks for clearing that up, I wasn't too sure how that worked. I haven't been able to download the source, mediafire is down currently. I wanted to ask about your unsized byte arrays - I thought this was illegal and won't compile? I know your implementation is fine for them since you are just reading in data, but if you added to this project for writing/modifying data, I foresee a problem. Does C# have something like STL has with C++? vectors / lists or the like might be something to look into.

[DEElekgolo]

Thanks for the info, I only recently got back into the modding scene and I'm understanding the engine better and better each day.

Here is the source of the tag class. I am sure I'll get a lot of whippings about my programming habits. Good to fix them from the start.



using System;
using System.IO;
using System.Collections;
using System.Collections.Generic;
using System.Net;
using System.Linq;
using System.Text;
using X360.STFS;
using X360.Other;

namespace Tagger
{
public class Tag
{
/// <summary>
/// Byte Array of the tag data, including the header and size bytes
/// </summary>
private byte[] _Data;
/// <summary>
/// 4 Char name of the tag
/// </summary>
private string _Name;
/// <summary>
/// The size of the entire tag including the _Name and _Size Bytes
/// </summary>
private int _Size;
/// <summary>
/// List containing the Tags
/// </summary>
internal List<Tag> list;

/// <summary>
/// Tag Constructor
/// </summary>
public Tag(string filename)
{
Load(filename);
}

public Tag(string name,int size,byte[] data)
{
_Name = name;
_Size = size;
_Data = data;
}

public Tag()
{
_Name = "";
_Size = 0;
_Data = new byte[0];
}

public Tag(BinaryReader reader)
{
LoadFromStream(reader);
}

/// <summary>
/// Gets the Tag's Name
/// </summary>
public string Name
{
get
{
return _Name;
}
}

/// <summary>
/// Gets the Tag's Size
/// </summary>
public int Size
{
get
{
return _Size;
}
}

/// <summary>
/// Gets the Tag's Data byte Array
/// </summary>
public byte[] Data
{
get
{
return _Data;
}
}

/// <summary>
/// Loads a Tag by inputting the file name
/// </summary>
private void Load(string filename)
{
list = new List<Tag>();
using (FileStream input = File.OpenRead(filename))
{
BinaryReader reader = new BinaryReader(input, Encoding.BigEndianUnicode);
while (reader.BaseStream.Position < reader.BaseStream.Length)
{
list.Add(ReadTag(reader));
}
reader.Close();
reader.Dispose();
}
}

/// <summary>
/// Loads a Tag by inputting a stream
/// </summary>
private void LoadFromStream(BinaryReader reader)
{
list = new List<Tag>();
while (reader.BaseStream.Position < reader.BaseStream.Length)
{
list.Add(ReadTag(reader));
}
reader.Close();
reader.Dispose();
}

/// <summary>
/// Loads a Tag by inputting the Binary reader
/// </summary>
private void Load(BinaryReader reader)
{
list = new List<Tag>();
while (reader.BaseStream.Position < reader.BaseStream.Length)
{
list.Add(ReadTag(reader));
}
reader.Close();
reader.Dispose();
}

/// <summary>
/// Reads a single Tag
/// </summary>
private Tag ReadTag(BinaryReader reader)
{
try
{
_Name = Encoding.ASCII.GetString(reader.ReadBytes(4));
_Size = IPAddress.HostToNetworkOrder(reader.ReadInt32());
if (_Size <= 0)
{
reader.BaseStream.Position = reader.BaseStream.Length;
return new Tag();
}
reader.BaseStream.Seek(reader.BaseStream.Position - 8, SeekOrigin.Begin);
_Data = reader.ReadBytes(_Size);
return new Tag(_Name, _Size, _Data);
}
catch(Exception e)
{
reader.BaseStream.Position = reader.BaseStream.Length;
return new Tag();
}
}

/// <summary>
/// Gets the Tag using an index
/// </summary>
public virtual Tag this[int index]
{
get
{
return list[index];
}
}

/// <summary>
/// Gets the Tag using a string
/// </summary>
public virtual Tag this[String name]
{
get
{
if (list.Exists(Tag => Tag.Name == name))
{
return list.Find(Tag => Tag.Name == name);
}
else
{
return new Tag();
}
}
}

/// <summary>
/// Gets the Number of nested tags
/// </summary>
public int Count
{
get
{
return list.Count;
}
}


/// <summary>
/// Loads a Tag by inputting the file name of a STFS file(Container file)
/// </summary>
public void LoadFromSTFS(string filename)
{
LogRecord x = new LogRecord();
STFSPackage Package = new STFSPackage(filename, x);
if (!Package.ParseSuccess)
{
return;
}
FolderEntry root = Package.RootDirectory;
FileEntry file = root.GetSubFiles()[0];
if (file == null)
{
Package.CloseIO();
}
string tempFile = X360.Other.VariousFunctions.GetTempFileLocale();
if (!file.Extract(tempFile))
{
Package.CloseIO();
return;
}
BinaryReader reader = new BinaryReader(new FileStream(tempFile, FileMode.Open));
LoadFromStream(reader);
Package.CloseIO();
}

/// <summary>
/// Returns a BinaryReader Stream from the tag's Data
/// </summary>
public BinaryReader GetStream()
{
return new BinaryReader(new MemoryStream(_Data));
}




}
}

[Kornman00]

Arrays in .NET are different from arrays in C. In C, they're just a sequence of structure instances, but in .NET an array is an actual object who knows exactly what kind of types it's hold, how many elements it contains, etc. The only thing that really translates when coming from C to C# is some of the language's syntax. You have to check all of the explicit memory management stuff at the door when you enter the house of .NET, as with any managed framework. You're no longer dealing with pointers, you're dealing with references (not the C++ kind). You're no longer dealing with type-less data, you're dealing with typed objects (there's no casting from one object to another different object). Layout of fields in a object are determined by the framework, not by the order in which they're found in source (unless you use an attribute to explicitly state the layout properties).

DEE, you'd benefit from using auto-implemented properties (http://msdn.microsoft.com/en-us/library/bb384054.aspx) for most of your fields. Also, you should freshen up on some object-oriented designs. A BLF entry should be it's own object. Then you'd have a BLF manager which handles loading from various streams (ie, file, memory, network, etc), iterating entry data, etc. Basically:


// Handles the actual streaming of BLF data
class BlfStream;
// Handles the actual entry process of tagged data in BLFs to include the entry's header and raw data.
class BlfEntry;
// Handles parsing/writing of raw data (ie, the data which follows a BLF entry's header).
// An object would implement this to parse an explicit group tag's data. IE, SavedFilmData would implement this to handle the 'flhd' entry.
// You could also set it up to where a single object can manage multiple entries by passing the group tag to the streaming functions. IE, SavedFilm could then handle 'flmh' and 'flmd'.
interface IBlfStreamable;