• Unity

  • Sprite Shaders for Unity

Related Discussions
...

Cool. Ok I totally missed the lastest big Spine update, which is why my last shader package wasn't working for peops.
I've now grabbed it and updated the code to work with it. Saw you left some nice comments for where to hook in the tangent stuff 🙂
I'm not 100% of the new mesh generation flow so its worth grabbing it and having a look at what I've changed to SkeletonRenderer.

Also see you dropped in the skeleton utility component for flipping in editor, nice!

Anyways yep, grab the latest package in the first post in order to have normal maps work with the latest spine!

Huh? The SkeletonUtility thing was always there.
Not much changed with SkeletonRenderer. I'd say it was 90% cleanup and tiny fixes.

Oops my bad I thought you could now use the flip flags on the utility to properly flip spine animations in scenes but its not serialised so it revert back when you enter play mode - its cool I've got a component that does it anyways.

Plus yeah must've been a while since I grabbed it - all the SubmeshInstruction stuff is new. But yeah looks a lot tidier!

3 months later

@ToddRivers thank you very much for sharing this and putting the work into it.

Just to let you know. I just downloaded the shaders from the first post and importeted it and got some skeletonrendere errors. As it seems this has been mentioned before so I don't know if this is helpful.

But as I said. Many thanks for sharing this.

Hey! Sorry I fixed the shaders locally to work with the latest spine-unity runtimes but forgot to post them!
If you grab latest from the first post they should all be working (I also removed some out of date info/intructions from that post).

@Pharan been a while dude! is there any chance the tangent solver could become part of the spine unity runtimes? Otherwise I and others have to always redo the changes to spine code after they update thier runtimes.

The tangent solving function is in its own class which you're free to take and I simply made the following change to SkeletonRenderer:

Inside the check to see if triangles should be updated (around line 652), just bellow

for (int i = 0; i < submeshCount; ++i)
currentMesh.SetTriangles(submeshes.Items.triangles, i);

I stuck...

#if SPINE_OPTIONAL_NORMALS
if (calculateTangents)
this.tangents = currentMesh.tangents = SpineTangentSolver.Solve(currentMesh.vertices, currentMesh.triangles, currentMesh.uv, new Vector3(0, 0, -1));
else
this.tangents = currentMesh.tangents = null;
#endif

This seems to work and I think means its only happening when it needs to, not every frame.

Thanks @ToddRivers!

Working on spine-unity 3.3 right now. Will have a look at it after!
Thanks for the code! I may put the tangent solver itself in one of the MeshGeneration classes.

17 days later

Hey Pharan! I've updated the shaders and tangent code to work with Spine 3.4 (looking forward to properly playing around with the new editor stuff for 3.4 btw!)

I'm now using the render separator stuff to render a Unity Cloth mesh in between a sprite attachments and realised the tangent code isn't working with that stuff so I've had to expose the tangents to some of those classes.

Anyways all the tangent stuff is now working with latest. It would be sweet if it could be added to the spine-unity runtimes so the calculate tangents flag works (meaning me and others using bump maps don't have to edit the files everytime we grab the latest spine runtimes).
These are the files I had to edit:

SkeletonRenderer.cs
SkeletonRenderSeperator.cs
ArraysMeshGenerator.cs
ArraysSubmeshedGenerator.cs
ArraysSubmeshSetGenerator.cs
ISubmeshedMeshGenerator.cs

All the changes are in the zip file in my first post, most of them are one line changes! Cheers!

Great! You're still in the forums. I was a bit worried! 😃

I was wondering if I could ask you some things about it 'cause I just crossed a few things out in my list, and I think making the changes to incorporate the optional tangent solver is a good thing to do next.

Thanks for the info!

Heh yeah forgot about shader stuff for a while but still here!

Yeah ask away! - The one bit I'm still a bit fuzzy on is the changes to SkeletonRenderer. I'm not sure if the tangents could be calculated and stored in each mesh attachment somehow, but maybe that wouldn't actually be more efficient since you can't set them per submesh (I think).

Okay, so I've tested the solver that's been incorporated. Seems to work fine.
All this stuff is still going to be disabled by default but the code will be there and it'll be a matter of uncommenting some #defines for people who want to use them.

I was playing around with your shader too.
You mentioned the shader not working properly if the Transform is rotated.
One of the points I had in mind with the fixed-normal idea was that it could also (I guess optionally) not care about the Transform rotation.

So I removed the part of the shader that transforms the fixed normal value according to the Transform matrix. ie, UnityObjectToWorldNormal(normal); and just used the raw normal value and it kinda works except the light looks like it's coming from the bottom instead of the top.
So I also removed the part that rotated the tangents and used the raw tangent values. ie, UnityObjectToWorldDir(tangent);. And as far as I can tell, that did the trick: Lighting with normals works with rotation.

Both of these changes were only made in ShaderShared.cginc

It's definitely a different paradigm though. If you did it that way, you can't expect the 2D mesh to be lit like an embossed 3D sheet of paper. Instead, partial rotations are lit as if the mesh is just being 2D squashed.
Haven't thoroughly tested it but I did rotate and translate the lights and the skeletons around and it seemed to act as expected under this paradigm.

Anyway, I think this way of doing it is useful for a lot of people.

16 Jul 2016 11:23 pm

I wonder if the solver can't be simplified to assume the mesh is two-dimensional for the sake of speed.
Apart from skipping RegionAttachments by getting tangents from region uvs transformed by bone world rotation, this may be a good way to have performance gains.

Hey all sounds great!

So like you suggest defining the fixed normal in world space only works if your sprite is always facing along the same world axis (ie the negative Z axis).
That might be the case for most Spine games but for my own game sprites can potentially face along any axis (the gameplay happens in 3d rather than a fixed plane).
This means to work correctly they do have to be defined in local/model space and then calculated into world using UnityObjectToWorldNormal and UnityObjectToWorldDir.

It might be nice to be able to specify a fixed normal in World Space in the shaders as an option though, as like you say for something like a 2d platformer where sprites are always going to be facing along the Z axis, it does works nicely even when you rotate the sprite by 180 instead of flipping.

For me though flipping the sprites X instead of rotating and using model space fixed normals works great.
Also if you write to the depth buffer (for proper per pixel lighting or effects like Depth of Field etc) then you can't rotate by 180, you have to flip X instead to ensure the draw order is correct.

I totally agree the tangent solver must be able to be optimised though! Doing that and the RegionAttachment stuff makes a lot of sense.

btw is there any reason this stuff is defined out rather than turned off by default and toggled with the calculate tangents flag on the SkeletonAnimation component?

I guess it doesn't define the fixed normal in "world space" as much as it does in "camera space".
If you do this, and rotate the camera around, it still works. Still might be not what you're looking for but it's part of the paradigm.

Regarding defining out: it's mostly just not wanting people to "pay for stuff they don't use". Though at this point, I guess I've simplified and isolated it enough that just the bools will do.

There's still work to be done before I add this to the thing. I've converted the solver to 2D. I now need it to support submeshes.

17 Jul 2016 10:35 pm

Something's wrong tangents in submeshes.
I've checked the generated Vector4[] tangents with 1 submesh or with 3. They come out with the exact same values, but their behavior seems different.

Image removed due to the lack of support for HTTPS. | Show Anyway


single submesh - correct
The single submesh mesh works fine with rotation.

Image removed due to the lack of support for HTTPS. | Show Anyway


multiple submesh - lit upside down when you rotate on y??
The multi-submesh one breaks when you rotate.

There also seems to be some other problems, from some combination of the solver and modifications to the shader (whether or not we do the transformation of the tangent according to the GameObject).

Hmm I'm confused with this rotate around Y stuff - is this happening with fixed normals and using your shader changes? It works fine with my animations that all use multiple submeshes... I think! I'll double check - do you mean multiple mesh attachments?

Anyhow Unitys lighting data in shaders is all defined in world space so to use a fixed normal in 'camera space' like you're suggesting then you would also need to convert the light directions into camera space too.
I did try making those changes you mentioned to SharedShader.cginc meaning the tangent and normal are defined in world space.
This looks all good intially for the test scene as the character is facing along the world Z axis. However if you rotate the entire scene (inc the character and the camera / lights / level etc), then the lighting isn't consitant as the character no longer faces down the world Z axis as your telling the shader.

I think defining a fixed normal in model space like it is at the moment and calling flip X to rotate the character 180 around the Y axis is the way to go personally.
Plus you can't rotate the character around Y if you are writing to depth as the draw order becomes backwards. I think in order to use lighting and bump mapping properly you need to treat the spine meshes as bascially being one sided, with a fixed normal that points out from this one side.

Thanks again for looking at this stuff btw!

Okay, at the very least we can properly define limitations for users so a certain set of things simply means "no, you can't do that. It logically doesn't work."

In that context, my current problem is that using flipX makes the sprite lit from the wrong side.

19 Jul 2016 6:37 pm

Okay, I think I've hammered out the bugs on the C# side.

On the shader side, it's been helpful to optionally remove the normals transformation but not the tangent transformation.
Removing the normals transformation causes the shader to pretend that the normals are always pointed at the camera. The paradigm is explained above.

Image removed due to the lack of support for HTTPS. | Show Anyway


vertexlit with normal map

This setup works correctly with:

  • Unity Transform negative scale
  • Unity Transform Z rotation (aka 2D rotation)
  • skeleton FlipX and FlipY
  • any bone rotation
  • bone or attachment negative scale (in other words, flipping, and backwards triangles)
  • bent weighted meshes (as far as I can tell)
  • meshes with submeshes.

It doesn't work with Unity Transform 180º Y rotation, even with with fixed camera-space normals and transformed tangents.
The result is that X is correctly rotated, but Y is not. But I think this is handled on the shader side; it has something to do with how the shader treats the calculated binormal.

Optimizations are likely to be done in the future on RegionAttachments.

Normal shader limitations will apply. For example: Multipass forward (pixel) lighting requires depth buffer writes to prevent overlapping light passes. But writing to the depth buffer, which also causes hard alpha blending behavior, requires z spacing, and will cause 180º rotation to not work (the spacing would be backwards).

And for the love of all good things, disable backface culling on your shaders if you're using them on sprites. (ToddRiver's SpriteShaders is fine.)

Image removed due to the lack of support for HTTPS. | Show Anyway

This looks pretty good! You guys are awesome scientists 🙂

Yep awesome work Pharan, looking forward to it being in the runtimes 🙂

I've tidied up the shaders to make the fixed normal defined in view space instead of model space and it looks like its working nicely. Instead of calling calculateWorldNormal I now rotate the fixed normal by the view matrix to convert from a view space normal to world space for the lighting calculations.

As you say its a bit of a head-f*ck to get your brain around - instead of treating the sprite like a flat piece of paper we're telling the renderer the sprite is always facing towards the camera even when its angled away from it. It does make sense to do this though I think and the lighting should look correct even when the sprites facing away from the camera.

The 180 degree thing is a bit tricky to deal with. In calculateWorldBinormal the binormal gets flipped when there's a negative scale on the model or the tangents.w value is negative. This would have to be done when the camera is rendering the 'back' of the sprite too.
There's probs a way of doing this by using the dot product of the model forwards and the camera forwards.
Will have a quick try!

19 Jul 2016, 15:49

Ok managed to get it working for 180 degree rotations too!
I'll put the changes in a new package in the first post. I might make a flag for allowing sprites to rotate away from the camera or not just as its a bit of extra calculations in the shader that aren't needed if you use flipX instead.
Still its nice to have the option 🙂

It's merged.
It's #defined by default but bool calculateTangents is false by default (as it's always been).
Also tested the new checkboxes on the shader. Works like a champ!

Awesome! Will grab the runtimes in a bit and then post a fresh package with just the shaders and none my own hacks. Thanks again!

21 Jul 2016, 17:24

Ok the new shaders are up in the first post and updated the info for getting them to work. Hopefully that will be the last time they have to be updated for a while now!

Btw I'm getting some errors in Unity using the latest runtimes - I think you need to check 'target' is null in your custom inspectors on enable (I've had to do this in my own inspectors, don't know why Unity can't null check beforehand grrr..)

There hasn't been any notable changes to the inspectors, specially regarding UnityEditor.Editor.target. I also haven't been able to repro it but it seems to be happening to people. So I tried to google it and I found this: http://answers.unity3d.com/questions/1216599/custom-editor-gets-created-multiple-times-and-rece.html

So it sounds like a recent Unity bug. (I haven't updated to patch releases recently.)
What version of Unity are you using?

10 days later

Yeah think its a recent Unity thing, I'm on latest (5.3.5.f1) and now I think about it I've been getting a bunch of weird inspector based errors and yep looks like other people are getting them to.

I keep getting another werid one saying "MissingReferenceException: The object of type 'Object' has been destroyed but you are still trying to access it. Your script should either check if it is null or you should not destroy the object inspector window."

Anyways its definitely Unity side so don't worry about changing the Spine runtimes. Lets hope Unity get on it in their next build (they've got all the moneys, so they've got no excuses!)

31 Jul 2016, 16:55

Haha knew it wouldnt be the last time I changed the shaders - I've fixed a couple of bugs to do with pixel lighting and premultiplied alpha, plus I've added support for fog which can be enabled using a toggle box in the material inspector 🙂