To dither or not to dither (that is to dither)

[Holy_Diver]

For some reason I was thinking about dithering, and I remembered that Som seems to want to have dithering. I double checked this, and it's true.

Now I don't think dithering was ever supported by ddraw/d3d in 32bpp modes. But in the 16bpp mode presumably there is supposed to be dithering. You may also notice in the various Objedit like tools there is dithering going on. The thing is apparently graphics hardware manufacturers at some point stopped supporting (hardware) dithering. Apparently this was bad for a lot of older 16bpp games because without dithering the rainbow banding effect is more plain to see and these games were never intended to be displayed without dithering, so that's a shame. The reason there is dithering in Objedit and friends is presumably because they are in software rendering mode.

I've added a variable that overrides Som's dither setting to turn off dithering for starters. The reason why is just because I figure games should be consistent across hardware and dithering really changes the look of the game and by default authors probably want their games to be consistent (even though there probably isn't many contemporary chips that implement dithering if any)

That said apparently dithering is how Som was meant to be seen... in 16bpp mode at least anyway. I know KF2 (Melanat) has dithering. In fact you all playing KF2 on emulators are not getting the full experience very likely because there probably is no dithering happening. I checked epsxe and my plugin was not dithering KF2 anyway.

Probably I was thinking about KF2's visuals when my mind found it's way to dithering this morning... speaking of which I completely forgot about my coffee...

I figure if one of you lot ever get an urge to make a game with the KF2 graphics it might look a lot better with a dithering effect for that matter. So I reckon I will try to implement some form of dithering in the D3D9 shaders... probably right before the gamma correction stuff.

The dithering I see in Objedit is not the same I think as what I plan to do (https://en.wikipedia.org/wiki/Ordered_dithering) but I think the kind of dithering I will try to achieve will be better in some ways because the DirectX dithering can cause problems when there are transparent pixels, because they get dithered with the non transparent ones and end up creating a kind of unnatural fog where every other pixel is transparent. It may in fact be Objedit is just dithering the textures, because otherwise I think there would be that kind of fog around every open edge of a polygon.

This (https://en.wikipedia.org/wiki/Ordered_dithering) kind of dithering just dithers the colour of each pixel depending on where it falls on the screen, which I'm pretty is how PSOne dithering works. In fact the PSOne dithering can be somewhat annoying on a non-standard-def-tube-display because the dithering mask is fixed even while moving so it seems like you're wearing a kind of veil. I think if I get that far probably the pattern can be swapped out whenever the view changes (kind of like how the stars change in KF2) to alleviate that.

EDITED: The dithering in Objedit seems to be something about how colorkey is implemented. It only happens within textures anyway. I don't know if it's better or worse than just knocking out the whole unfiltered (square) texel. I wonder what makes it work that way though.

[Holy_Diver]

Well I believe I had everything setup properly to do this, but the problem is it requires the PS3.0 pixel shader model to get the basic inputs you need and that requires that a vertex shader also be defined which is just not practical at this point. So I like got the dithering working but lost the texture coordinates so there was really not a whole lot to look at.

The irony is as the graphics acceleration technology gets to be ever more sophisticated the ability to program flexible software becomes increasingly nonexistent.

[Holy_Diver]

I'm still trying to focus on the .mdl related stuff, but I think I've also begun working on a fairly involved vertex shader framework.

I think everything Som needs can just barely fit into the minimum number of Vertex/Pixel Shader 1.0 registers (96) which is guaranteed by the standard to be on any card that does programmable shaders at all.

extern int vsWorldViewProj[4];
extern int vsWorldView[4];
extern int vsInvWorldView[4];
extern int vsWorld[4];
extern int vsInvView[4];
extern int vsTexture0[4];
extern int vsFogColor;
extern int vsFogFactors; //start/end/density
extern int vsMaterialAmbient;
extern int vsMaterialDiffuse;
extern int vsLightsAmbient[16];
extern int vsLightsDiffuse[16];
extern int vsLightsVectors[16]; //point (w=1) / directional (w=0)
extern int vsLightsFactors[16]; //range/attenuation (dir=1,0,0)

I say barely but Som probably does not require a couple of these and I've allotted space for 8 additional lights in case the number of on lights can be increased via an extension.

So very likely even quite old cards can support a dither shader and other effects once the vertex shader is setup.

Basically the Direct3D code will just blindly track these values if they're assigned to registers, then there is a shader override framework so the game can swap in its own shaders at the right moments. I don't mix anything Som specific with the D3D code so that the code could be adapted to other games. If the D3D code had to manage it's own shaders it would be impossible to manage, but I think Som only has about three distinct rendering states (one for 3D stuff, one for transparent 3D stuff, and one for the overlay sprites) so that's not so bad.

Tracking all of those variables is still a lot of work. After this is working it should really open up some possibilities, however perhaps more interestingly it could make a more or less direct port to D3D10 conceivable.

[Holy_Diver]

I think I already have everything working... just need to program the shaders for Som.

I setup a simple headlight shader and realized for the first time that the map graphics don't have lighting normals... which should be obvious (if you think about it)

That kind of trips up any dynamic lighting extensions down the pike. You could either A) replace all the per vertex colors with normals. Or B) somehow detect which graphic Som is displaying / replace it or slip in the normals somehow. An easy effect like a lamp/torch that just added light to the Som calculated lighting is not really doable. You could of course do a torch that just lights things that are closer up. But it would not take into account the angles of the surfaces being lit.

EDITED: I'm pretty sure the map files have normals... at least prior to the outputted game, but they never get passed along to Direct3D.

[Holy_Diver]

The most interesting unified map lighting model I can come up with off hand would be to "repaint" the per vertex colours generated by the light calculation algorithm with 24bit lighting normals (not quite the usual 96bit but probably acceptable) then use the remaining 8bits as a lookup into a palette with 256 slots that store 4 light ids. Every light in Som has an id which is probably stored in the maps but basically I think whenever Som opens a map it reserves that many D3D lights. So each component of the texture is one of these light IDs (up to 256 lights) which basically defines up to four lights to knockout for per vertex shadows. With that capability it should be possible to replicate the classic lighting model while unifying it with the object lighting.

The lights you'd knockout that way would most likely be fixed in location relative to the map piece. You could move the lights around at that point but the shadow would not move. You could change the color/intensity of the light, turn it off and on for example without any trouble.

This idea came pretty quick but I had to really think about how to figure out if a light was knocked out in the vertex shader for a while... then I realized you could subtract the lights id from each id in the set of four ids... then if it matched one that component would end up to be zero, then multiply the four components by themselves and if one is zero they all become zero... then depending on what shader model you're in you can conditionally skip the light or do the calculation and multiply it by the zero to get no light contribution.


A program could be written to automate everything / spit out a new map file, and you could store both sets of map files (old and new) in your game and let the player decide which set to use. It might even be possible to tessellate the map so the vertex lighting would be more uniform/precise.


At that point you really do some interesting things with lights like make magic fireballs be a new "lamp" in the map or let the player cast a spell to create a light source around them (or equip a torch) or whatever.


EDITED: Also I think it might be way more efficient to replace the global map lighting with a cubemap because that's really all it's doing (versus 4 lighting calculations) and you could even have a bit more control with a cubemap.

[Holy_Diver]

I'm fairly sure given two components of a unit vector if you know the sign of the vector you can reconstruct the third component (though I can't find any examples online... it's kind of an old technique I think from back when storage was at a premium... surface normal compression)

Anyway assuming so, the third component of the colour could be freed up to store something with 7bits of precision... because the sign bit would be used to store the sign (+/-) of the lighting normal. Then I'm thinking a useful thing to store there would be the rotation angle of the tangent vector where the reflection of the normal is used as the "up" vector. So you end up with a tangent vector that is precise to about 128 slices of pizza (edited: or maybe the sign bit should be taken from the fourth component instead, cutting the number of possible shadow sets from 256 to 128 so the tangent vector would be as precise as the surface normal vector)

Armed with the normal/tangent you can do modern bump mapping. Like where the lights in the room reflect off the stones that make up the floor even though the stones are really just a flat polygon with a stone floor texture.

The same compression technique can be applied to the monster model files etc. So you can have bump mapped armor with intricate metal work, eg. etchings that reflect light naturally.

And the final touch of brilliance I think, is to build a database of the textures based on the final mipmap levels. So like you take a texture and down sample it until it's a 1 pixel image or maybe 2x2 ... whatever it takes to identify the texture uniquely. Because all textures when down sampled should create a kind of fingerprint. And doing a lookup through a table of 2x2 pixel ids is pretty trivial. So that way when the mimaps are built in game the final level (or so) is used to determine what texture you just loaded in, then you can grab the matching bumpmap texture or whatever (there are any number of different kinds of textures that can be combined to generate effects... you can even grab shaders individualized for the textures)

At that point you have all kinds of options. Maybe almost anything you see in a modern game visuals wise


EDITED: You can also of course ditch Som's textures and grab higher resolution/depth ones. And even redirect Som to alternative texture directories where it will find tumbnail versions of the textures so the mipmapping process is short circuited (for speedy load times)

[Holy_Diver]

Omg, wow... with the dither you can't even tell it's not in 32bpp mode

I can't believe IHVs have dropped hardware dithering / probably so hard screwed so many 16bpp games. I'm guessing there must be a number that don't do 32bpp as well. The difference is night and day. If the textures were 24bit you could probably tell but you gotta really put your nose to the screen to tell the diff.

I'm surprised my first crack at it went off so well. I don't think you could get a better dither. It's not even chunky in 640x480 mode in fullscreen mode on a 1920x1200 display. You can't even tell it's there with the upsampling. It took me 25 minutes to convince myself there wasn't some bug causing it to drop out of 16bpp mode.

In fact it may even look better than 32bpp mode because of the artifacts introduced from using 15bit textures with 24bit colour. That would be fitting because it's an extra effect / not for free.

I will add a feature if you want to make your dither fat like the PSOne's ‎  ...for a retro feel, KF2 style. And will even let you substitute your own dither texture.


I think I have almost everything working with the vertex shader setup. It really threw me for a few loops. Just gotta look into why the lighting code is not taking. Maybe tomorrow I could release something.


PS: The dither is how Som was intended to look in 16bpp mode, but I will make it a turn on option both because it comes at an expense and because it adds something you would not get with Som on a modern graphics setup, and therefore breaks the transparent replication of Som by default. I've also forced dither off if the option is not set so people who are actually playing on hardware with dither will get the same experience as everyone else regardless.

[Holy_Diver]

I was trying to think how a penumbra could be worked into the shadow sets, and I'd given up on the idea more or less, but the solution was so obvious. If the light indices are signed values. Essentially you can use the absolute value to knockout the lights then mask that vector in such a way to get the affected index, and if it's negative say the vertex is inside the penumbra, and if it's positive it's inside the umbra (https://en.wikipedia.org/wiki/Penumbra)

So in other words every vertex that is part of the map can be in the shadow of up to four lights and can either be in the umbra or penumbra of each of those lights. In the umbra no light gets thru, and the penumbra is about half light. Any smoothing of the shadow would be due to blending between vertices. Penumbras especially happen (around lightbulbs anyway) when something is close to the light source that casts a sharp shadow across a large space, but the penumbra could also be used as a tweaking factor for softer outdoor type shadows when desired, and artificially smoothing out the edge of the shadow across adjoining vertices.

This will be the basis of my "Dark Slayer" map editor I think. I can't come up with anything better / more appropriate. It's seems like a logical progression of Som's lighting and doesn't suggest anything messy like shadow mapping or require anything much more than Som already provides.

[Holy_Diver]

Hmmm, rereading all the stuff I'd posted yesterday it occurred to me it's very likely Som does not store the fourth (alpha) component of it's map lighting in the map files. Really there is no reason to. It's very likely there are 32bits allotted per coloured vertex in the file but probably best hope is the remaining bits are pre-filled so they can be directly passed along to Direct3D. In that case they could be used. Otherwise you'd have to choose between the vertex lighting with shadows or the tangent space option.

Anyway I really ran myself into the ground yesterday trying to replicate Som in vertex shader form. It's almost there. Just a couple minor discrepancies to sort out (that I'm aware of)

I really think the 16bpp dither mode might be the best filter for Som. Especially with the assets that come with it. I might try to make the in game filter options under Direct3D9, point / linear / dither or some point. It's funny, I never thought of a dither as a filter but it's actually a pretty damn good final pass filter. It doesn't seem to suffer from the stretchiness of interpolation based filters.

This weekend I feel as though I need a break in so far as I can pull myself away. I will try to make a point of looking into why those .x files aren't jibing with Assimp in the meantime.

[Holy_Diver]

At last I think I've successfully/faithfully recreated Som in shader form.

I will try to get a demo up over the next few days. Only super neat effect so far is dither (if you don't count the whole replication of Som thing)

Below can be found the shaders. They're not that much to speak of. If you wanted to customize the shaders for your game you could put them in a file or two and they could be used instead.

People mod games by swapping out their own shaders. The registers up top are kept track of by SomEx. They pretty much describe what Som is up to.

Code:

#define cDbgColor "float4 dbgColor : register(c0);"

#define cDimViewport "float2 dimViewport : register(c1);"
#define cFpsRegister "float2 fpsRegister : register(c1[2]);"

#define cFvfFactors "float4 fvfFactors : register(vs,c2);"
#define cColFactors "float4 colFactors : register(c3);"    Â 

#define cColFunction "float4 colFunction : register(ps,c4);"    Â 

#define cX4mWVP "float4x4 x4mWVP : register(vs,c4);"
#define cX4mWV  "float4x4 x4mWV  : register(vs,c8);"
#define cX4mIWV "float4x4 x4mIWV : register(vs,c12);"
#define cX4mW   "float4x4 x4mW   : register(vs,c16);"
#define cX4mIV  "float4x4 x4mIV  : register(vs,c20);"

#define cFogColor   "float3 fogColor   : register(ps,c23);"
#define cFogFactors "float4 fogFactors : register(vs,c24);"
#define cMatAmbient "float4 matAmbient : register(vs,c25);"
#define cMatDiffuse "float4 matDiffuse : register(vs,c26);"
#define cMatEmitted "float4 matEmitted : register(vs,c27);"
#define cEnvAmbient "float3 envAmbient : register(vs,c28);"

#define cLitAmbient(n) "float3 litAmbient["#n"] : register(vs,c32);"
#define cLitLookup1(n) "float  litLookups["#n"] : register(vs,c32[3]);"
#define cLitDiffuse(n) "float3 litDiffuse["#n"] : register(vs,c48);"
#define cLitLookup2(n) "float  litLookups["#n"] : register(vs,c48[3]);"
#define cLitVectors(n) "float4 litVectors["#n"] : register(vs,c64);"
#define cLitFactors(n) "float4 litFactors["#n"] : register(vs,c80);"

#define iNumLights "int numLights : register(vs,i0);"

static const char *som_shader_classic_vs =

    cDbgColor
    cDimViewport
    cFvfFactors
    cColFactors
    cX4mWVP
    cX4mWV
    cX4mW
    cFogFactors
    cMatAmbient
    cMatDiffuse
    cMatEmitted
    cEnvAmbient
    cLitAmbient(8)
    cLitDiffuse(8)
    cLitVectors(8)
    cLitFactors(8)
    iNumLights

    "struct BLIT_INPUT"
    "{"
    "    float4 pos : POSITION; "
    "    float4 col : COLOR;    "
    "    float2 uvs : TEXCOORD; "
    "};"
    "struct UNLIT_INPUT"
    "{"
    "    float4 pos : POSITION; "
    "    float4 col : COLOR;    "
    "    float2 uvs : TEXCOORD; "
    "};"
    "struct BLENDED_INPUT"
    "{"
    "    float4 pos : POSITION; "
    "    float3 lit : NORMAL;   "
    "    float2 uvs : TEXCOORD; "
    "};"

    "struct CLASSIC_OUTPUT"
    "{"
    "    float4 pos : POSITION; "
    "    float4 col : COLOR;    "
    "    float2 uvs : TEXCOORD; "
    "    float  fog : FOG;    Â   "
    "};"

    "CLASSIC_OUTPUT blit(BLIT_INPUT In)"
    "{"
    "    CLASSIC_OUTPUT Out; "

    //screen space calculation
    "    Out.pos.x = In.pos.x/dimViewport.x*2.0f-1.0f; "
    "    Out.pos.y = 1.0f-In.pos.y/dimViewport.y*2.0f; "
    
    "    Out.pos.z = 0.0f; Out.pos.w = 1.0f; "

    //projection space if fvfFactors.w is 0, screen space if 1
    "    Out.pos    = Out.pos*fvfFactors.w+ "
    "        mul(x4mWVP,In.pos)*(1.0f-fvfFactors.w); "    Â 

    "    Out.col = In.col; "
    "    Out.uvs = In.uvs; "
    "    Out.fog = 0.0f;    Â  "

    "    Out.col+=colFactors; " //select texture

    "    Out.col = saturate(Out.col); "

    "    return Out; "
    "}"

    "CLASSIC_OUTPUT unlit(UNLIT_INPUT In)"
    "{"
    "    CLASSIC_OUTPUT Out; "

    "    Out.pos    = mul(x4mWVP,In.pos); "

    "    Out.col = In.col; "
    "    Out.uvs = In.uvs; "        

//    "    float Z = mul(x4mWV,In.pos).z; "
    "    float Z = length(mul(x4mWV,In.pos)); " //rangefog

    "    Out.fog = fogFactors.w-fogFactors.w* "
    "        saturate((fogFactors.y-Z)/  "
    "        (fogFactors.y-fogFactors.x)); "

    "    return Out; "
    "}"

    "struct LIGHT{ float3 ambient; float3 diffuse; };"
    
    "LIGHT Light(int i, float3 P, float3 N)"
    "{"    
    "    float3 D = litVectors[i].xyz-P; float M = length(D); "
        
    "    if((litFactors[i].x-M)*litVectors[i].w<0.0f) return (LIGHT)0; " //Model 2.0+

//    "    float vis = saturate(ceil(litFactors[i].x-M*litFactors[i].w)); " //Model 1.0

    "    float att = 1.0f; " //"    float att = 1,0f*vis; " //Model 1.0

    "    att/=litFactors[i].y+litFactors[i].z*M+litFactors[i].w*M*M; "
            
    "    float3 L = normalize(D*litVectors[i].w +      " //point light
    "        litVectors[i].xyz*(1.0f-litVectors[i].w)); " //directional (assuming normalized)

    "    LIGHT Out = {litAmbient[i].rgb*att, "
    "                 litDiffuse[i].rgb*att*max(0.0f,dot(N,L))}; " 
    "    return Out; "
    "}"

    "CLASSIC_OUTPUT blended(BLENDED_INPUT In)"
    "{"
    "    CLASSIC_OUTPUT Out; "

    "    Out.pos    = mul(x4mWVP,In.pos); "

    "    Out.uvs = In.uvs; "        

//    "    float Z = mul(x4mWV,In.pos).z; "
    "    float Z = length(mul(x4mWV,In.pos)); " //rangefog

    "    Out.fog = fogFactors.w-fogFactors.w* "
    "        saturate((fogFactors.y-Z)/  "
    "        (fogFactors.y-fogFactors.x)); "

    "    float3 P = mul(x4mW,In.pos).xyz; "
    "    float3 N = normalize(mul((float3x3)x4mW,In.lit)); "
    
    "    float4 ambient = {0,0,0,0}, diffuse = {0,0,0,1}; "

    "    [loop] " //silly compiler, tricks are for kids
    "    for(int i=0;i<numLights;i++) " //TODO: Shader Model 1.0
    "    {"
    "        LIGHT sum = Light(i,P,N); "
    
    "        ambient.rgb+=sum.ambient; "
    "        diffuse.rgb+=sum.diffuse; "
    "    }"

    "    ambient.rgb+=envAmbient.rgb; "

    "    Out.col = matAmbient*ambient+matDiffuse*diffuse; "

    "    Out.col+=matEmitted; "
    "    Out.col+=colFactors; " //select texture

    "    Out.col = saturate(Out.col); "

    "    return Out; "
    "}";

static const char *som_shader_classic_ps =
                                    Â   
    cFogColor
    cColFactors
    cColFunction

#define classic_dither(sign) \
    "   float2 f88 = float2(8.0f,8.0f); "\
    "    Out.col.rgb"#sign"=tex2D(sam1,fmod(In.pos,f88)/f88).r/8.0f; "\

#define classic_colorkey(lin,exp) \
    "    clip(Out.col.a-0.3f); "\
    "    Out.col.rgb*=-pow(Out.col.a-"#lin"f,"#exp"f);"\
    "    Out.col.a = 1.0f; "    

    "texture2D tex0 : TEXTURE0;"
    "sampler2D sam0 = sampler_state"
    "{"
    "    Texture = <tex0>; "
    "};"

    "texture2D tex1 : TEXTURE1;" //dither
    "sampler2D sam1 = sampler_state"
    "{"
    "    Texture = <tex1>; "
    "    MinFilter = POINT; "
    "    MagFilter = POINT; "
    "    MipFilter = POINT; "
    "};"

    "struct CLASSIC_INPUT"
    "{"
    "    float4 col : COLOR;    "
    "    float2 uvs : TEXCOORD; "
    "    float  fog : FOG;      "
    "    float2 pos : VPOS;    Â   "
    "};"

    "struct CLASSIC_OUTPUT{ float4 col:COLOR0; };"

    "CLASSIC_OUTPUT blit(CLASSIC_INPUT In)"
    "{"
    "    CLASSIC_OUTPUT Out; "
    
    "    Out.col = tex2D(sam0,In.uvs); "

        classic_colorkey(2.0,1.0)
    
    "    Out.col*=In.col; "

        classic_dither(+)

    "    return Out; "
    "}"    

    "CLASSIC_OUTPUT unlit(CLASSIC_INPUT In)"
    "{"
    "    CLASSIC_OUTPUT Out; "
    
    "    Out.col = tex2D(sam0,In.uvs)*In.col; "    

    "    Out.col.rgb = lerp(Out.col.rgb,fogColor,In.fog); "

        classic_dither(-)

    "    return Out; "
    "}"    

    "CLASSIC_OUTPUT blended(CLASSIC_INPUT In)"
    "{"
    "    CLASSIC_OUTPUT Out; "
    
    "    Out.col = tex2D(sam0,In.uvs); "
    
        classic_colorkey(2.0,1.0)
    
    "    float4 one = {1.0f,1.0f,1.0f,1.0f}; "
    
    "    Out.col = Out.col*In.col*(one-colFunction)+ " //modulate
    "            Â  Out.col*colFunction+In.col*colFunction; " //add

    "    Out.col.rgb = lerp(Out.col.rgb,fogColor,In.fog); "

        classic_dither(-)

    "    return Out; "
    "}";