Shadows won't show up on baked lightmaps

[LlamaMaster]

No matter what settings I try I can't get MAX to bake any of my shadows. My settings are as follows:

http://img62.imageshack.us/img62/6320/bakingproblem.jpg

http://img541.imageshack.us/img541/5892/bakingproblem2.jpg

http://img30.imageshack.us/img30/4382/bakingproblem3.jpg

http://img703.imageshack.us/img703/2412/hurrlightmaps2.jpg

Does anybody see something obvious that's wrong with those settings? I can make some badly misshapen and overblown shadows appear if I turn up the intensity of the sun, but I shouldn't have to do that when everything looks just fine in regular rendering (shown above).

[Siliconmaster]

Hmm. I can't see what's wrong :/ Though I notice you're using vray for your lights and settings- I know nothing about vray, but I didn't use it for my lightmaps, I just used the default scanline renderer. Not sure if that's the problem though.

[LlamaMaster]

I fixed the issue by doing away with projection mapping altogether and simply baking directly onto the lightmap geometry. Now I have another problem: how do I change the location of the sun in the sky? I tried looking for something obvious (like a marker) in the sky model tag using Kornman00, but I can't find anything that looks like it is related.

[Siliconmaster]

Interesting- I guess that would work, you just wouldn't get the bounce from the bsp textures. In your case though it's all outside, so it wouldn't be that noticeable anyway.

And the sun. Oh god it's this topic again. Back in the day Con and I did over 9000 tests to determine what actually set the sun in the correct position. I'll see what I can remember. No guarantees this is perfect.

Open the .sky tag, and down under "lights" there should be a single block for the existing sun. Yes, you can have more than one. Under "Radiosity" the direction parameter is the one you want. I think y is angle from 0 to 360 of rotation on the horizontal plane, and p is vertical degrees from 0 to 90 from the horizon to straight up. Theoretically, you change those, the sun moves in the sky tag, regardless of where the marker is in the model itself. However, do note that we definitely found cases where the sun refused to move. I vaguely remember cheating by making a second duplicate sun, then deleting the first. Hope that helps.

[LlamaMaster]

Ugh. I did what you said, but even by creating new suns I could still only change the pitch. Still, I've got it close enough to where it needs to be that it isn't distracting. Thanks.

[Siliconmaster]

Damn, sorry it's proving to be a pain- I have no idea why it acts like that sometimes. But yeah, if you can get the sun in the right general angle, most players won't notice it's slightly off.

[LlamaMaster]

Actually, it seems yaw was working, but it looked like it was moving in random directions when I gave it new values. I have no idea what Halo uses as its base point for pitch and yaw, but I've tried it enough ways to just say "fuck it, good enough."

[Siliconmaster]

Haha- y=0, r=0 is flat on the horizon at (what I believe is) due West if I remember. Which direction is West? Couldn't tell you :P That just seems right. Regardless, if you move it a few numbers at a time it will definitely move gradually. It might not use a full 360 rotation, more like 180 spread in a full circle, not sure. I do know it is possible to get accurate sun positions with effort.

[LlamaMaster]

http://img10.imageshack.us/img10/3513/unledwvz.jpg (http://imageshack.us/photo/my-images/10/unledwvz.jpg/)

Why.

I can render shadows just fine with the lightmap mesh UV's generated by tool, but the moment I try to make my own this shit happens. I made sure not to alter the geometry in *any* way, and yet it still does this. Is my usertool corrupt or something?

[Higuy]

Are you using your own lighmap UV's?

[LlamaMaster]

Are you using your own lighmap UV's?

Yes.

[Higuy]

Yes.
Could be the problem then. Check out the UV's for that area of the map. (I haven't done this in a long time, so excuse me if I'm forgetting something or being stupid)

[LlamaMaster]

As far as I can tell there is *nothing* wrong with the UV's in my problem areas (there are several). My guess is that something is screwing up on importation, but I'm not a code monkey, so I don't know how to look at usertool and see if anything is going wrong.

Edit: Alright, I recreated the aether project and reexported the lightmap meshes (the ones with the screwed up UV's). It looks like this:

http://img695.imageshack.us/img695/9311/unledavx.jpg (http://imageshack.us/photo/my-images/695/unledavx.jpg/)

So yeah, something is messing up majorly on export/import.

Edit 2: I tried exporting my custom UV's from 3DS MAX 8 as well as 3DS MAX 2009, and it still gives me the except same errors on the same polygons. I think it's safe to say it's a usertool error.

[Higuy]

Yeah, possibly. Does this do it with regular tool lightmaps after you've ran them for a while?

[LlamaMaster]

I've never done a full radiosity with tool, and I don't plan on doing so in the future (because I like the way the lightmap meshes are seperated right now, and I want to keep them that way). However, I never had any lightmap errors with the debug tool lightmaps. I can also import custom lightmaps from vray just fine using the tool-generated UV's, but they suck, so it's not a solution.

I'll contact FireScythe soon if he doesn't find this thread on his own.

[FireScythe]

Have you been creating a new Aether project after each run of tools debug radiosity? Since it copies the BSP data to an internal format it you have to create a new project before exporting your lightmaps again.

[LlamaMaster]

Have you been creating a new Aether project after each run of tools debug radiosity? Since it copies the BSP data to an internal format it you have to create a new project before exporting your lightmaps again.

Yeah, I've been doing that. I've remade the aether project several times, but it doesn't help.

[FireScythe]

If your comfortable with doing so, could you send me the BSP tag and Max (2009) files? I'll see if I can reproduce the error.

EDIT: Had a thought, when exporting your lightmaps from Max, are you exporting selected (after selecting the lightmap meshes) or exporting all?

[LlamaMaster]

Exporting selected. I'll upload my files and send them to you.

[FireScythe]

Right, I *think* I know what the issue is. When creating new UV coordinates you can't break up the chunks that are already there. For instance if the original lightmap UVs had two seperate quads, you could join them together just fine, but you couldn't break them into 4 triangles. Reason being that breaking up the chunks would require 1 vertex to have two or more texture coordinates, and since usertool does not add or remove vertices it simple overwrites them, it first sets the value of a vertex to one UV coordinate, and then overwrites that again when the vertices other coordinates are processed. Hence the UV's in your mesh end up stretching to a different UV coordinate for the same vertex.

So basically, for any problem areas make sure vertices that were welded together previously are welded back together.

[Siliconmaster]

I remember I had an issue like this as well- http://wiki.modacity.net/wiki/Aether#Additional_Comments_by_Siliconmaster

Sounds like what I experienced does indeed line up with your theory, Firescythe. Though why it pops up at such odd times is beyond me- I split existing groups and welded them in new areas without any issues, only running into the problem when using cylinder uv maps :/

[LlamaMaster]

Right, I *think* I know what the issue is. When creating new UV coordinates you can't break up the chunks that are already there. For instance if the original lightmap UVs had two seperate quads, you could join them together just fine, but you couldn't break them into 4 triangles. Reason being that breaking up the chunks would require 1 vertex to have two or more texture coordinates, and since usertool does not add or remove vertices it simple overwrites them, it first sets the value of a vertex to one UV coordinate, and then overwrites that again when the vertices other coordinates are processed. Hence the UV's in your mesh end up stretching to a different UV coordinate for the same vertex.

So basically, for any problem areas make sure vertices that were welded together previously are welded back together.

Wow, I never would have thought of that. I'll test this theory and let you know if I can get it to work. Thanks!

[LlamaMaster]

Alright, now I can get custom UV's ingame just fine (well, as "custom" as it gets when you can't make new seams), but now sapien is crashing because I'm feeding it "too much" bitmap data. I know I should just decrease the resolution of my individual bitmaps, but it feels like the bitmap data limit decreased itself somehow. This happened after I ran a radiosity quality of 1 in tool to get better-separated lightmap meshes (whereas before I was using lightmap meshes generated from a radiosity quality of 0). Is there something to this, or does it just feel like I can do less with my lightmaps now in terms of resolution now that they are mainly square instead of rectangular (thus taking more space)?

Basically, is there a strict cutoff point for how much data can fit into a lightmap regardless of other factors (like tool lightmap structure)?

[FireScythe]

There is a limit to the amount of bitmap data you can have in a bitmap tag, I can't remember the value though. Generally speaking quadrupling a bitmaps resolution will work on small to medium sized maps so 512x512 goes to 2048x2048, but on larger maps with more lightmaps you would have to reduce your resolution to only double. Though since you are in control of the lightmap size, you can decide to reduce quality in darker areas to make more space elsewhere. Its a bit of trial and error.

I think you can import a rectangular bitmap to replace a square one (been a while since i've done the process myself :O).

[Siliconmaster]

You can import a rectangular one to replace a square one, just be aware that you then lose half the resolution. The game automatically resizes for aspect ratio I believe, so the only visible side effect would be that an obviously large diagonal pixel line in game would be more smooth vertically than horizontally (or vice versa). If the resolution is high enough, shouldn't matter that much.

[FireScythe]

That would only be if the lightmap UV's weren't adjusted to match the ratio of the bitmap. If you just replace a square bitmap with a rectangular one thats twice the width then yes, double the pixels would be packed into the same faces on one UV axis, but if the lightmap UVs are adjusted to match the bitmap ratio it shouldn't be a problem. For instance if you doubled the width of the bitmap on the X/U axis you would have to scale the lightmap UV's down on the X/U axis by half to match the ratio of the bitmap.

[Siliconmaster]

Ah, gotcha. I always exported my lightmaps out of max as squares, regardless of how they ended up in Halo. Figured it was easier that way.