Halo PiP

[Skarma]

I quit working on Forge for the last week or so due to boredom -- I get distracted easily with side projects. I will get back to it soon enough. :ohdear:

While browsing through my favorite Direct3D resource website (http://www.toymaker.info/Games/html/direct3d_faq.html#pip), there was a small section about Picture-in-Picture(PiP) and a basic description of how it works. I instantly was intrigued to do something like it for Halo and sure enough in a few long days time I got something going with OpenSauce.

I was looking through the rasterizer system and saw Kornman had hooked the end of the window rendering function if DX was not defined. I am still curious to what he was planning to do with this, since there was some unfinished rendering code there. Anyway this window rendering function was what I needed to accomplish PiP. I thought it would be too hacky to just hook portions of the code that needed modified, so of course I went all the way and reversed the entire function into OS compatible C++ code and rerouted the original function to my function. This means that Halo is being rendered through code inside OS and it makes it a hell of a lot easier to tinker with.

To do PiP, the render code is run twice. The first time is normal and the second time with a modified viewport and vertex buffer(adjusted width & height). I did not really notice any performance drop with video options on max settings and my system being a laptop. The minimal choppiness you do see is normal for my specs without running OS. I was actually quite surprised, I really thought it would be a huge hit in performance.

Well here is my PiP example video, the map is an alpha version of Precipice (I show the phantom bsp errors in this):
LFOcWObEwlM

I am posting this to get ideas on things we could make this useful for. My main idea is to make a rear view mirror for the warthog. To accomplish this, I would have to update the view matrix to change the camera angle as well as not render the HUD elements. Other idea for reversing this function was to easily modify how Halo is rendered or easily add post-processing effects.

There were additions to multiple files so I don't really feel the need to post all of it. Here is just the window rendering code. If you want the full source just let me know and I'll send it right over.


bool SoftwareVertexProcessing() PTR_IMP_GET(software_vertex_processing);

IDirect3DSurface9* RenderTarget() DPTR_IMP_GET(render_target);
IDirect3DBaseTexture9* Texture() DPTR_IMP_GET(texture);
IDirect3DIndexBuffer9* IndexData() DPTR_IMP_GET(index_data);
IDirect3DVertexBuffer9* StreamData() DPTR_IMP_GET(stream_data);

bool render_pip = false;
bool adjust_pip = false;

void RenderWindow()
{
if(GET_PTR(unknown_boolean_3) == 1 && !GET_PTR(unknown_boolean_4))
{
DX9::Direct3DDevice()->SetRenderTarget(0, RenderTarget());

D3DSURFACE_DESC desc;
RenderTarget()->GetDesc(&desc);

D3DVIEWPORT9 viewport = {
0, // x
0, // y
adjust_pip ? 200 : desc.Width, // width
adjust_pip ? 150 : desc.Height, // height
0.0f, // minz
1.0f // maxz
};

DX9::Direct3DDevice()->SetViewport(&viewport);

if(DebugOptions()->wireframe == false)
DX9::Direct3DDevice()->SetRenderState(D3DRS_FILLMODE, D3DFILL_SOLID);

DX9::Direct3DDevice()->SetTexture(0, Texture());
DX9::Direct3DDevice()->SetSamplerState(0, D3DSAMP_ADDRESSU, D3DTADDRESS_CLAMP);
DX9::Direct3DDevice()->SetSamplerState(0, D3DSAMP_ADDRESSV, D3DTADDRESS_CLAMP);
DX9::Direct3DDevice()->SetSamplerState(0, D3DSAMP_MAGFILTER, D3DTEXF_POINT);
DX9::Direct3DDevice()->SetSamplerState(0, D3DSAMP_MINFILTER, D3DTEXF_POINT);
DX9::Direct3DDevice()->SetSamplerState(0, D3DSAMP_MIPFILTER, D3DTEXF_POINT);
DX9::Direct3DDevice()->SetRenderState(D3DRS_CULLMODE, D3DCULL_CCW);
DX9::Direct3DDevice()->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_RED|D3DCOLORWRITEENABLE_GREEN| D3DCOLORWRITEENABLE_BLUE);
DX9::Direct3DDevice()->SetRenderState(D3DRS_ALPHABLENDENABLE, false);
DX9::Direct3DDevice()->SetRenderState(D3DRS_ALPHATESTENABLE, false);
DX9::Direct3DDevice()->SetRenderState(D3DRS_ZENABLE, D3DZB_FALSE);
DX9::Direct3DDevice()->SetRenderState(D3DRS_FOGENABLE, false);
DX9::Direct3DDevice()->SetPixelShader(NULL);
DX9::Direct3DDevice()->SetTextureStageState(0, D3DTSS_COLOROP, D3DTOP_SELECTARG1);
DX9::Direct3DDevice()->SetTextureStageState(0, D3DTSS_COLORARG1, D3DTA_TEXTURE);
DX9::Direct3DDevice()->SetTextureStageState(0, D3DTSS_ALPHAOP, D3DTOP_SELECTARG1);
DX9::Direct3DDevice()->SetTextureStageState(0, D3DTSS_ALPHAARG1, D3DTA_TEXTURE);
DX9::Direct3DDevice()->SetTextureStageState(1, D3DTSS_COLOROP, D3DTOP_DISABLE);
DX9::Direct3DDevice()->SetTextureStageState(1, D3DTSS_ALPHAOP, D3DTOP_DISABLE);

if(StreamData())
{
#define D3DFVF_CUSTOMVERTEX (D3DFVF_XYZRHW | D3DFVF_DIFFUSE | D3DFVF_TEX1)
uint32 fvf_size = D3DXGetFVFVertexSize(D3DFVF_CUSTOMVERTEX);
byte* vertex_data;
StreamData()->Lock(0, fvf_size*4, CAST_PTR(void**, &vertex_data), D3DLOCK_DISCARD);

uint32 height = adjust_pip ? 150 : GET_PTR(unknown_uint16_2) - GET_PTR(unknown_uint16_1);
uint32 width = adjust_pip ? 200 : GET_PTR(unknown_uint16_4) - GET_PTR(unknown_uint16_3);

// black
//argb_color vertex_color = {
// 255, // a
// 255, // r
// 255, // g
// 255 // b
//};
// ^^^ operator override was not working, so I did it this way instead
// it was telling me "cannot convert "Yelo::uint32" to "Yelo::uint32""
DWORD vert_color = 0xFFFFFFFF;

struct CUSTOMVERTEX {
real x, y, z;
real rhw;
uint32 color;
real tu, tv;
} vertices[4] = {
{ -0.5f, -0.5f, 0.0f, 1.0f, vert_color, 0.0f, 0.0f },
{ CAST(float, width) - 0.5f, -0.5f, 0.0f, 1.0f, vert_color, 1.0f, 0.0f },
{ CAST(float, width) - 0.5f, CAST(float, height) - 0.5f, 0.0f, 1.0f, vert_color, 1.0f, 1.0f },
{ -0.5f, CAST(float, height) - 0.5f, 0.0f, 1.0f, vert_color, 0.0f, 1.0f }
};

if(GET_PTR(unknown_uint32_1))
{
vertices[1].tu *= CAST(float, height);
vertices[2].tu *= CAST(float, height);
vertices[2].tv *= CAST(float, width);
vertices[3].tv *= CAST(float, width);
}

memcpy(vertex_data, vertices, fvf_size*4);

DX9::Direct3DDevice()->SetVertexShader(NULL);
DX9::Direct3DDevice()->SetFVF(D3DFVF_CUSTOMVERTEX);
StreamData()->Unlock();
DX9::Direct3DDevice()->SetSoftwareVertexProcessing(SoftwareVertexProcessi ng());
// Stride has to be equivalent to FVF size
DX9::Direct3DDevice()->SetStreamSource(0, StreamData(), 0, fvf_size);
DX9::Direct3DDevice()->SetIndices(IndexData());
DX9::Direct3DDevice()->DrawIndexedPrimitive(D3DPT_TRIANGLEFAN, 0, 0, 4, 0, 2);
DX9::Direct3DDevice()->SetFVF(NULL);
}

if(DebugOptions()->wireframe == true)
DX9::Direct3DDevice()->SetRenderState(D3DRS_FILLMODE, D3DFILL_WIREFRAME);
}

if(GameUI::HudChatGlobals()->active)
{
// Set chatbox background color
argb_color box_color = {
176, // a
32, // r
32, // g
32 // b
};

// Set chatbox rect coordinates
rectangle2d box_rect = {
0, // left
460, // top
480, // bottom
640 // right
};

Engine::DX9::RenderChatbox(box_rect, box_color);
}

bool b = true;

if(FAILED(DX9::Direct3DDevice()->SetSoftwareVertexProcessing(SoftwareVertexProcessi ng())))
{
b = false;
}

DX9::Direct3DDevice()->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_SRCALPHA);
DX9::Direct3DDevice()->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_INVSRCALPHA);
DX9::Direct3DDevice()->SetRenderState(D3DRS_BLENDOP, D3DBLENDOP_ADD);

if(Keystone::MainWindow() && !GET_PTR(unknown_boolean_1))
{

if(FAILED(Keystone::Update(Keystone::MainWindow()) ))
{
GET_PTR(unknown_boolean_1) = true;
}
}

if(FAILED(DX9::Direct3DDevice()->SetSoftwareVertexProcessing(SoftwareVertexProcessi ng())))
{
b = false;
}

if(render_pip)
{
if(SUCCEEDED(DX9::Direct3DDevice()->EndScene()))
{
if(b)
{
GET_PTR(unknown_boolean_2) = false;
}
}
}

render_pip = !render_pip;

if(render_pip)
{
adjust_pip = true;
RenderWindow();
}
else
{
adjust_pip = false;
}
}

[Shock120]

oh my, imagine trying to get splitscreen to work xD
Halo 2 Sniper?

[Limited]

Just wait for people to ask for AA...Although it is DX and you have the device, so you could apply AA could you not?

[Skarma]

An idea I just thought of was to display movies or other video through the viewport.

Someone I talked to suggested the sniper view, I forgot about that! It would be sick.

To render split screen, you would do the same thing as you would for PiP. You would have to render the window separately for every player but with different viewports. So for 4 player split screen you would render 4 times.

[Dwood]

You just need to get multiple players on 1 computer working.... Could we use controllers since mice are out of the question?

[Shock120]

Could you enable the "player0" on the dedicated server, and create a rendering window for it?
or, just as a observing camera.
I had a try at this (just enabled 4 players), they couldn't move obviously, they had no controls, but it was funny enabling 4 forever AFK players.
I did it after someone told me that a "Dedicated server is a stripped version of the full game", and thus the same values were present but in different memory regions.

These are just ideas, I don't expect you to do it.

[Con]

If you modify the camera position between renderings then you can achieve stereoscopic 3d with this (either cross or parallel viewing).

http://upload.wikimedia.org/wikipedia/commons/thumb/a/a0/1080par.jpg/800px-1080par.jpg

[Skarma]

Anti-aliasing can be added to Halo very easily, but your graphics card must support it. I am running a laptop and it doesn't support it so I cannot do any testing but I can go ahead and write up the code.

Stereoscopic 3D would be crazy awesome if I had some glasses to go with it. I don't think I can play a game cross eyed for very long! I still need to figure out the view matrix to do some of these cool ideas.

As far as split screen goes, I don't even want to get into it. There is a possible easy hacky way of doing it, but I'm not even interested right now.

[Skarma]

I need testers. I added anti-aliasing to OpenSauce this morning, but since my device does not support it, I can't test it out. I will tell you how it works.

Before the Direct3D device is created, you MUST check if your device supports anti-aliasing. If it doesn't and you tell your device it does, it will crash your game when it creates the device. I added this check for every multisample type and set the presentation parameters for the best multisample type and quality that your device can support for best results. If your device does support it, then in the render loop it sets one of the render states to enable anti-aliasing and... that's it!

Now I have been reading around the internets and I came across some people who couldn't get it to work this way. I guess it has something to do with the render target not having multisampling enabled. I think this may be problem in Halo because I checked the render target data and it is not set. I was hoping that updating the presentation parameters before the device was created would change the back buffer multisampling type, but I cannot check because I can't enable it on my device. Anyway I added this check as well.

Here's my output log, no MSAA for me.

11:18:32 Your device does NOT support D3DMULTISAMPLE_16_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_15_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_14_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_13_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_12_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_11_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_10_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_9_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_8_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_7_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_6_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_5_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_4_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_3_SAMPLES. Quality level: 0
11:18:32 Your device does NOT support D3DMULTISAMPLE_2_SAMPLES. Quality level: 0
11:18:35 RenderTarget MSAA type: D3DMULTISAMPLE_NONE. Quality level: 0
:mech:


Here's a link to the test d3d9.dll. This is for Halo CE 1.08, so it will NOT work on 1.09. Throw it in your Halo CE folder and run the game. Once the main menu pops up, close Halo and go to Documents\My Games\Halo CE\Yelo and open up debug.txt. Post your results here or PM them to me.

If the output says your device supports it, go in game and see if you notice any difference. Take a screen shot with and without OpenSauce and compare. Is there a difference? Let me know.

http://www.mediafire.com/file/s1xjdt1s11rp8a7/MSAA_TEST.zip

[Limited]

Anyone have a link to the 1.08 patch? (From 1.00), Ill give this a test Skarma, just gotta install Halo :O

[Amit]

Would this work in Halo PC 1.08?

[Limited]

Mon Oct 04 17:29:06 2010
17:29:06 Your device does NOT support D3DMULTISAMPLE_16_SAMPLES. Quality level: 0
17:29:06 Your device does NOT support D3DMULTISAMPLE_15_SAMPLES. Quality level: 0
17:29:06 Your device does NOT support D3DMULTISAMPLE_14_SAMPLES. Quality level: 0
17:29:06 Your device does NOT support D3DMULTISAMPLE_13_SAMPLES. Quality level: 0
17:29:06 Your device does NOT support D3DMULTISAMPLE_12_SAMPLES. Quality level: 0
17:29:06 Your device does NOT support D3DMULTISAMPLE_11_SAMPLES. Quality level: 0
17:29:06 Your device does NOT support D3DMULTISAMPLE_10_SAMPLES. Quality level: 0
17:29:06 Your device does NOT support D3DMULTISAMPLE_9_SAMPLES. Quality level: 0
17:29:06 Your device supports D3DMULTISAMPLE_8_SAMPLES. Quality level: 2
17:29:08 RenderTarget MSAA type: D3DMULTISAMPLE_8_SAMPLES. Quality level: 2


Pretty surprised my card doesnt, considering it supports DX10. By the way, menu items don't show up..that suppose to happen?

[Skarma]

No, this is from the latest build of OpenSauce which only supports Halo CE 1.08 currently.

I also forgot to mention that even if your card supports anti-aliasing, it may be disabled. To check for this, go to your graphics driver settings. You can find these if you dig deep enough in the Control Panel, refer to your graphics card manual.

[FireScythe]

I'm not sure by what you've posted whether you know this already, but Halo doesn't render directly to the backbuffer. It renders to an off screen render target and then (probably) StretchRects it to the backbuffer before presenting, which is an issue for AA since simply setting the device to have multisampling will create no change on the target actually being rendered to (Rasterizer::GlobalRenderTargets(*primary buffer*)). I have read of an alternative AA postprocess that uses the scene normals to detect edges and blur them slightly to simulate the effect.

[Limited]

With
http://limited-development.net/AA/with.png

Without
http://limited-development.net/AA/without.png

Settings
http://limited-development.net/AA/settings.png

[Skarma]

Limited, your card does support MSAA, type 8. And I was right, it does update the render target multisample type & quality! The reason the menu doesn't show up is because I am probably setting the AA wrong or I am using the wrong render target or something. Doesn't look like there is any difference.

[Limited]

It didnt show up when I had the screen at default size (640 x 480?), I increased the resolution to 1024 x 768 and menu showed up.

Do you need to run the game in fullscreen?

[Skarma]

Limited, not sure what the problem is.

FireScythe, I just found an article on NVIDIA website that explains and samples exactly what you are talking about. I'm looking through the code sample now. Thanks!
http://developer.download.nvidia.com/SDK/9.5/Samples/samples.html (http://developer.download.nvidia.com/SDK/9.5/Samples/samples.html#AntiAliasingWithPostProcessing) -- Antialiasing with Post-Processing
(http://developer.download.nvidia.com/SDK/9.5/Samples/samples.html#AntiAliasingWithPostProcessing)

[Higuy]

So Halo with AA is possible?

[Skarma]

It's possible with any game with Direct3D or OpenGL as long as your graphics card supports it and as long as the developers put code in the engine that makes it work. In Halo there isn't any code in the engine for it so I'm adding it. We really could modify the rendering system to do anything we want. I am familiar with the Direct3D API, but I am still amateur when it comes to the graphics pipeline and how to piece things together but I'll get there. This is great practice!

Edit: So I think MSAA will be put on hold for a while. I am not knowledgeable enough about the pipeline yet to understand what needs to be done. Halo uses MRT and requires post processing Antialiasing, which I am not update to date on yet. Let's move on for now.