[beta] Please, please change UI for animating sequences

spineappletree

I'm trying out the beta version now to get to know the sequence feature. Which I really like btw! Great addition to the software, very useful and yet another 'small' feature with huge amount of possibilities! 8)

The UI to animate sequences now is a completely different thing though. I find it unintuitive and I don't understand why the beta has a workflow to animate sequences that is completely different in comparisson to animate every other property in the graph editor. The way it is build now doesn't only feel unintuitive to me, it's also causing things to not be possible.

Workflow for other properties:

values over the y-axis
timing over the x-axis

Workflow for sequences:

no values visible in the y-axis, everything on the same y-level
x-axis doesn't show timing the same way, now only shows a representation of the framerate

This is how it looks now:
Everything is on one line in the graph and no handles or other known features:

Because of this workflow we can't do now:

Speed up or down a sequence with easing by moving handles, like how we can animate other properties
Hold keys /switch to linear curves / switch to bezier curves. Which is such a great interface with many possibilities
Play a sequence in reverse (which for some sequences literally makes us use half the resources in runtimes)
Immediately see in the graph at what frame the sequence currently is at
Set the sequence to an exact frame-number on a position on the timeline

Expected alternative
Why not use the proven workflow instead to animate sequences? Like so:

That way we could:

Speed up or down easily with visual representation in the graph editor
Use easing exactly how we want to by moving the handles of a bezier curve
Switch to linear 'curve' if we don't want easing
Switch to hold keyframes if we want to jump to keyframes
Easily and exactly play the sequence in reverse
Use exact frame numbers and have a visual representation of exactly on what keyframe the sequence is currently at
Jump around in the sequence with no problem as we have full freedom in the graph editor
Use the graph editor for what it is: a 2d graphical editor, and use both axis and all features we are familiar with!
Only need a single timeline; the one for framenumber (instead of an additional fps timeline as it looks now)
Use drag'n'drop and selections as expected
So basically work like with the rotation timeline; when a frame number exceeds the max frame number it just goes round, just like the rotation passes 360 degrees and rotates around from 0 degrees again. So if we put a keyframe at 30 frames in the timeline to go to frame 200, while the sequence only has 50 frames, it loops a few times

Please reconsider the interface to animate sequences. IMO it gots too many limitations now, is lacking ways to do advanced stuff and is counterintuitive IMO, because it's so different to the rest of Spine's graph editors workflow. It also doesn't look very scalable for future features.

Thanks for considering! 🙂

Erika

I think what you are suggesting is already covered well by the seqeunce properties that are keyable in Animate mode.

Attaching images can't really have a timeline, because it's more of a boolean option (e.g. an image can be turned on or off)
there are keyable settings for the frame by frame behavior that can be changed during the animation for better control (the frame rate, loop, ping pong, hold, etc), but imho this looks incompatible with the graph.

But perhaps the others have a different opinion.

I would however suggest you to look better at the properties cause playing in reverse (just to quote an example) is already possible and very easy to do without having to meddle with the graph.

spineappletree

Erika wrote
I think what you are suggesting is already covered well by the seqeunce properties that are keyable in Animate mode.

Sorry, I dissagree. That's why I took the time to write this feature request and explained why in details. Please re-read the post, it's all in there.

Attaching images can't really have a timeline, because it's more of a boolean option (e.g. an image can be turned on or off) there are keyable settings for the frame by frame behavior that can be changed during the animation for better control (the frame rate, loop, ping pong, hold, etc), but imho this looks incompatible with the graph.
But perhaps the others have a different opinion.

I'm surprised by your strict tone.
You are talking to a developer here. The timeline values handles doubles (values like 6.9). But frame numbers need to be integer values (so values like 7). To show the right frame number is just a matter of getting the integer value from a float value (for example 6,9 will be fr 7, but 6.4 will be fr 6), that's really the only 'challenge' as far as I could think of. How to do that could be discussed about, but nothing too crazy about that, pretty basic stuff on itself. If only there is a will to understand why I am asking this, which is all in the post above. And a little openess to take a minute to see the perspective of a user who doesn't open a feature request and takes the time to explain himself because he has nothing else to do.

I would however suggest you to look better at the properties cause playing in reverse (just to quote an example) is already possible and very easy to do without having to meddle with the graph.

Hey, why being so offensive and pointing with your finger? Like this on me you come across as if you think you have all the wisdom in the world and you see the only truth. Relax. I would appreciate it if you were just a little more open to other professional opinions, don't underestimate who you have in front of you, be a little more customer friendly and if my post isn't clear, just ask.

If the door to give feedback while the software is even in beta is already closed the only thing you might accomplish is that people don't feel appreciated and heared and stop giving their opinions. If that's what you want than just tell me, than I understand it's 'take it, or leave it' and we as users have nothing to say about that and are not allowed to give input.

Nate

spineappletree wrote
I'm surprised by your strict tone.

There should not be much read into "tone" during textual communication. We should all assume others have good intentions.

spineappletree wrote
that's really the only 'challenge' as far as I could think of. How to do that could be discussed about, but nothing too crazy about that, pretty basic stuff on itself. If only there is a will to understand why I am asking this, which is all in the post above. And a little openess to take a minute to see the perspective of a user

It would be possible to represent sequences completely differently. Instead of using FPS, keys in the dopesheet would control which frame is shown, as you showed above. This is interesting and I agree it would be powerful, but it would take away some of the simplicity given by specifying the FPS.

Our goal for this feature was to make an easy way to bring in image sequences, such as those for VFX. Typically those are created with the intention to play them at a particular FPS. Previously, bringing them into Spine meant setting a key to show each frame. While this gives complete control over when frames are shown, it is quite tedious. With our sequence implementation in the 4.1-beta, it is now very easy. We've wanted this feature since we released Spine via Kickstarter, 9 years ago! It looked like a good feature to get into 4.1, as it wouldn't take too long (spoiler: it was harder than expected).

If we had only your proposal, it wouldn't meet the simplicity goals for the most common use case, which again is to play a sequence of images at a specific FPS. To do so with the graph would require the user to calculate how many frames their sequence will play (say 53 frames), convert that to seconds (53 / 30 = 1.7667s), multiply by the FPS they want (say 12 FPS * 1.7667s = 21.2 image sequence frames), and then set keys to play from frame 1 to frame 21. I expect many users would find this a difficult concept to grasp. For those that understand it, it would be tedious to setup every time a sequence needs to play. Also, if your animation timing changes, you would need to recalculate and update your sequence keys.

You proposal excels at giving fine control over the image sequence. I could see a campfire or other image sequence could have some cool variability by controlling the frame using a curve. I don't think the most common use of image sequences need such control. If we could have both for reasonable effort, I'd be all for it.

As Erika mentioned, if you need more control, the old way is still available. It's definitely not as good or easy to use as your suggestion for some use cases, but at least having a workaround is much better than an impossibility. Note that some of the items you listed are possible without going all the way back to the old way. You can play a sequence in reverse. You can key the FPS a few times to adjust the speed smoothly over time (smooth enough, since frame-by-frame animation is be definition not very smooth). You can set keys for exact frame numbers and hold a frame. You can see the current frame number, though you have to select the attachment. Of course it is still lacking a visual representation in the graph and some of the other benefits you listed. I'm not dismissing those, only pointing out what parts we do have.

Unfortunately I don't want us to spend more time on sequences for 4.1. They were already more work to implement than we expected and we have many other features to work on, mainly graph improvements (which you requested and we are nearing completion on) and physics (which we made good progress on but tabled as the graph is a more immediate need). However, in the future we could have a mode for sequence attachments to allow specifying the FPS (as it is now) or the frame with interpolation (as you propose).

spineappletree wrote
Hey, why being so offensive and pointing with your finger?

I don't get this from Erika's post at all and I don't feel such defensiveness is warranted. While I don't think she fully understood your proposal for how curves in the graph would control the frame number, I can guarantee that her intent was not to be a jerk. She wanted to point out that some of what you propose is already possible. If you missed that you can play a sequence backward, maybe you missed other things. She is not a coder and can't always provide details about why we chose a particular implementation. She expected me or another developer to chime in, hence her mention of the "others", not for her response to stand alone. I've been busy lately and haven't had a chance the past couple days. I know full well your thought out suggestions take a good amount of effort to compile. They are worthy of a proper response, but doing that can take a fair amount of effort.

BTW, for some loosely related if unrequested bonus information, and to anticipate a possible next question of, "if we stick with FPS, why don't we allow it to be interpolated":

Sequences are controlled by a framerate, eg 12 frames per second (FPS). Currently the interpolation between keys for the sequence FPS is stepped. With a constant FPS between keys, we know at 12 FPS after 2 seconds we are showing frame 24.

How would linear or Bezier work? Eg if I'm going from 1 to 10 FPS over 4 seconds, with linear I know that at 2 seconds the FPS would be 5.5. That's OK, but what frames do we show between the keys? If we use the interpolated FPS to show the frames, the frames will appear to change faster than the intended FPS. I've made an animation to show what this looks like:

Loading Image

The sequence key on frame 0 is 1 FPS, the one on frame 120 is 10 FPS. I have hacked the editor to use linear interpolation for the FPS between these keys (ignore the tick lines, they are no longer matching the frame changes with the hack). When the animation hits the 2nd sequence key, it shows the mouth, so you have a visual to know when that time is reached. The animation continues to play at 10 FPS for 2 seconds (to frame 180) after the second sequence key. Here it is:

Loading Image

The numbers appear to change very fast as the linear interpolation gets closer to 10 FPS. Once we reach the 2nd sequence key (the mouth shows), the numbers are changing at a constant 10 FPS, which appears slower. The reason is that at a constant FPS, we can pick the frame to show with simple math. If we use the same math and interpolate the FPS, you get two levels of interpolation and the results are not what the user expected, as shown in the GIF above. Possibly there is other math we could use for linear, but I wasn't able to come up with anything in the time I had building the sequence feature. I don't expect there is reasonable math for Bezier.

spineappletree

@Nate Thanks for your extensive reply. I wanted to react sooner, but couldn't find the time, needed to let it sink in, testing and taking the time to form a response that is hopefully helpful to you.

Nate wrote
It would be possible to represent sequences completely differently. Instead of using FPS, keys in the dopesheet would control which frame is shown, as you showed above. This is interesting and I agree it would be powerful, but it would take away some of the simplicity given by specifying the FPS.

I hear you and you bring up valid points and giving me great inside in why it is build this way in the beta. Another point against my proposal I can give is that with many frames the graph can get very high quickly. This would probably never be an issue on projects made for runtimes I would expect, as these are probably always build to load quick, especially online. But projects with animations for video might use sequences with a lot of frames though, once people see the benefits.

I think both ways have easy parts and tedius parts. One isn't easier than the other per se, it's just a matter of the use case. The way it works now is indeed easy when we treat the sequence as a player and count on all sequences to be completely prebaked and not used (much) differently; we have play/stop/loop/pingpong/reverse etc and can change the framerate. But when we want to do more advanced stuff and want to treat the sequence as a creative resource to creatively play around with and manually control, it can get complicated fast.

I think it's safe to say lots of Spine users got pretty advanced creative results from Spine and could do that because tools can be manually controlled. I don't believe many people would have thought people would come up with even 3d-looking animations this good for example.

So to assume people would only use sequences for simple playback of prebaked frames is a oversimplification IMO. Not saying that you do simplify this though, as you made that clear in your post and I understand where you're coming from. I'm just a little afraid that the chosen way is too short lived, too simplified for what people really gonna use it for, too limiting and that we soon more users will find out that it's not convenient to do the stuff we'd like to do with sequences.

Than we are building Spine files on a system that soon will be seen as too limited and than it's hard to rebuild it because of compatibility issues etc. That's why I post this while still in beta.

Spine users come up with the craziest creative results, people would not even think would be possible before, just because they have the tools to do so. I think it would be an oversimplification to think users would only use sequences to play a pre-baked animation. The real power is in the control. And only when the control is there great things can happen.

To illustrate this watch this simple real life example; scaling the animation doesn't scale the sequence:

https://youtu.be/CAVzc3ohPVk

I think we agree there are two ways to look at this. As you rightfully pointed out. I understand your view on it better now and agree with you that both ways have pros and cons and in an ideal situation we could either find the right balance or switch to one of the methods, depending on the use case.

Nate wrote
Our goal for this feature was to make an easy way to bring in image sequences, such as those for VFX. Typically those are created with the intention to play them at a particular FPS.

What is 'constant speed'? What will a runtime do with animations in general when the framerate drops? And what will it do with the framerate of sequences when that happens?
- Will Spine animations slow down? Or does the animation keeps its speed, but skips frames to stay on time?

How about the frame sequences? Does the sequence keep its speed, or skips frames to stay on time?
Do both keep their sync to each other?[/b]

If a sequence slows the animation down on slow computers, but the sequence is skipping frames instead, or visa versa, the animation goes out of sync and that could cause problems when the sequences are intended to be synced with the 'bone animations'.

Nate wrote
Previously, bringing them into Spine meant setting a key to show each frame. While this gives complete control over when frames are shown, it is quite tedious. With our sequence implementation in the 4.1-beta, it is now very easy. We've wanted this feature since we released Spine via Kickstarter, 9 years ago! It looked like a good feature to get into 4.1, as it wouldn't take too long (spoiler: it was harder than expected).

Yes, we can do it the old way. To recreate the example above like this:

https://youtu.be/ZgGXCcbj8bg

But than it could be even easier to program everything ourselves in the host of the runtime and the whole point of the Spine editor is that we can animate it in the editor right?

It's definitely a way forward in the beta. I am only a little scared that we will start building projects with this new system, which IMO is difficult to use for real life use cases when wanting to be a little more creative besides just playing of a completely pre-baked animation. And yes, I understand that there might be difficulties in framerates when scaling it down. But that's up to the animator to judge IMO and I'm pretty sure people will come up with the craziest new creations just by having the tools, we cannot even predict yet.

I see Spine as an animation editor and think it's safe to assume some people like to animate in Spine. Also sequences. Especially because that might help us keeping the amount of frames needed small and be creative with it. To treat sequences as a player with a framerate is too limited and assumes the use case too much IMO.

Nate wrote
If we had only your proposal, it wouldn't meet the simplicity goals for the most common use case, which again is to play a sequence of images at a specific FPS.

I fully agree that when just wanting to just play a sequence (start / stop and think in frame rates) the chosen path is the easiest. But that's pretty limited. If we even want the slightest do more advanced stuff it gets complicated fast. So I think the simplicity is depending on the use case.

Nate wrote
BTW, for some loosely related if unrequested bonus information, and to anticipate a possible next question of, "if we stick with FPS, why don't we allow it to be interpolated":

Sequences are controlled by a framerate, eg 12 frames per second (FPS). Currently the interpolation between keys for the sequence FPS is stepped. With a constant FPS between keys, we know at 12 FPS after 2 seconds we are showing frame 24.

How would linear or Bezier work? Eg if I'm going from 1 to 10 FPS over 4 seconds, with linear I know that at 2 seconds the FPS would be 5.5. That's OK, but what frames do we show between the keys? If we use the interpolated FPS to show the frames, the frames will appear to change faster than the intended FPS. I've made an animation to show what this looks like:

Sorry, I don't get the animation you posted with the numbers in it, but I do understand what you mean and of course you are right that easing isn't as smooth (when not done well). But IMO you're assuming too much now for the user:

What if a user is having a sequence with a higher framerate than the actual animation and the runtime skips frames to match the actual framerate of the animation? Of coarse, if not an integer factor of the actual framerate things don't line up, but that might also be used as an effect for instance and there are ways to make it work too. IMO not even giving the tool to the creator is an oversimplification and thinking for creative people. People come up with the craziest ideas and effects we cannot predict. And even on tv and films a lot of VFX even were invented by 'issues' in technical systems (like the neverending loop of picture in picture because of feedback loops in a video mixer for example).
It's never sure at what framerate animations plays in the runtime and framerates might drop on slow computers/phones etc.. As stated above; How does the runtime treat sequences vs bone animations when this happens?

BTW, your example gif of the numbers running is an excelent example of a great use case for easing. I can think of ways that are really interesting in animations when animating counters for instance that need to slow down with an ease in the end.

So this long post in short:

I fully get where you're coming from and I agree my proposal isn't the only way and both ways have pros and cons.
I understand having a way to play a sequence is better than nothing
I agree that it's simple to use like this when treated as a player of pre-baked animations. But when needing more control it's getting complicated and frustrating IMO soon; preventing innovation and creative use cases to happen IMO.
Please don't underestimate Spine users and assume users want simplicity over full control
Please keep in mind people come up with the craziest ideas and use cases we cannot even think of, once they have the tools
Hope you could consider the above before launching the system as is. Because once it's out there it could cause compatibility issues when wanting to change later.

I think the best way would be something in between or having both worlds, like you also seem to think. And some things you see as challenge can be solved, like things that auto calculate for you.

But I think the real questions are:

do we mostly focus to create tools for simplification and limit based on assumptions, or create tools that are easy to use for more advanced users to give users full control? Or else: how could we have the best for both worlds.
do we launch sequences quickly because now we don't have anything else for that yet and it's nice to mention in the changelog and blogs, or do we rethink the system further to be sure we don't have compatibility issues when needed later?

These are questions only you can answer. I can only hope. Hope this makes sense and you understand I get your points though.

13 jan 2022:

Hi @Nate, it's been two weeks ago since I responded directly to you on the forum but so far no reaction from you here. I see you react daily here and also see you react to forum posts with the message that you missed posts.

Just to be clear you didn't miss this one, as I took the time to test things out, upload a video and respond: could you let me know if you read this post and what you think of it?

Thanks in advance!

Nate

spineappletree wrote
Hi @Nate, it's been two weeks ago since I responded directly to you on the forum but so far no reaction from you here. I see you react daily here and also see you react to forum posts with the message that you missed posts.

Sorry for the delay! I didn't forget, just needed to make progress on things so I don't get too far behind. When a post is time consuming to respond to sometimes I won't have time to handle it, but I still need to use my time here and there to help in other forum threads.

I do appreciate the time you spend on conveying your ideas and I'm glad I can finally respond! We got 4.1.10-beta released today so that feels good, even though there's still lots to do. It was really bothering me to have so much time pass between beta releases, but it's nature of the beast to underestimate the effort required. At any rate, check out the new Curves view!

spineappletree wrote
I don't believe many people would have thought people would come up with even 3d-looking animations this good for example.

Indeed, sometimes even I'm surprised by the crazy stuff people have come up!

spineappletree wrote
To illustrate this watch this simple real life example; scaling the animation doesn't scale the sequence:

https://youtu.be/CAVzc3ohPVk

I think we agree there are two ways to look at this.

True. Either scaling or not scaling the sequence could be what the user wants. If they are using the keys so the sequence plays (probably repeatedly) for some amount of time (like a torch burning), they may not want the sequence speed to change when the keys are moved/scaled. If they want the sequence to be on a specific frame when the next key is reached, then they may want the speed of the sequence to change so it still finishes on the same frame when the key is moved/scaled. As you showed, they can't get that to happen automatically.

spineappletree wrote
What will a runtime do with animations in general when the framerate drops?

The skeleton is posed by an animation for a particular time, then drawn to the screen. This happens repeatedly, usually with the time increasing a small amount between drawings. If a longer time passes between drawings, the skeleton is still just posed for the new time and then drawn.

The effect of this is that if you can draw very fast, you will see smoother motion where Spine is interpolating between keyed values. When you have lots of time between drawing, the poses between drawings are not shown. This means Spine essentially "drops frames". What that means for the quality of the animation varies.

Both sequences and using attachment keys to make frame-by-frame animation behave the same way. Your delay between drawings would need to be larger than the time between frames (frames is quite an overloaded term) in the frame-by-frame animation for you to completely missing seeing a frame. If that is happening, nothing we can do can give a smooth animation experience. Even then, frame-by-frame animation is inherently not very smooth, often using a framerate like 12 FPS, so if an app can't draw fast enough it's probably the interpolated values of the skeletal animation that are more noticeably not smooth.

There can never be an "out of sync" issue. For any given time on the timeline, we know exactly what pose will be shown and you will always get that pose. The only time that won't be true is once we release physics, which are nondeterministic. In that case the pose of the physics elements depends on the previous poses. How often you draw, if you interrupt an animation with another, and other things can cause the physics to be in a different state rather than the exact pose we have for everything besides physics. That's generally OK because you design your physics to behave like you want, then you get cool, dynamic behavior.

spineappletree wrote
But than it could be even easier to program everything ourselves in the host of the runtime and the whole point of the Spine editor is that we can animate it in the editor right?

It's possible to generate frame-by-frame at runtime, but it's probably not easier than even the somewhat tedious attachment keys. Doing it in the editor allows you to see it there, scale the keys, etc.

spineappletree wrote
To treat sequences as a player with a framerate is too limited and assumes the use case too much IMO.

With the current beta sequences, we've traded simplicity for limited functionality. The only reason I'm not too worried about that is because we have a workaround via attachment keys, so people aren't completely stuck. We used to have only attachment keys, it was a bit annoying for a common task, so we added a simpler way that is great if it meets your needs. Your proposal is much more powerful for some use cases and it's possible we could add that in the future.

spineappletree wrote
If we even want the slightest do more advanced stuff it gets complicated fast. So I think the simplicity is depending on the use case.

Yep, absolutely. Generally it's most efficient to target the most common uses cases, but we still consider others and try to enable as much flexibility as possible. When you sit down to build something from nothing, with so many possible directions you can go, it can be difficult to decide on a reasonable path.

spineappletree wrote
Please don't underestimate Spine users and assume users want simplicity over full control

Agreed! It's not often we do a feature with simplicity as the primary goal.

spineappletree wrote
Hope you could consider the above before launching the system as is. Because once it's out there it could cause compatibility issues when wanting to change later.

I've thought about it and I think it can be done. Some sort of "mode" that is either framerate-based, as we have now, or "graph-based".

FWIW, another approach could be to generate the attachment keys for the user. That's not great for a few reasons (we'd probably not do it), but it's sort of between the sequence and attachment keys.

spineappletree wrote
do we mostly focus to create tools for simplification and limit based on assumptions, or create tools that are easy to use for more advanced users to give users full control? Or else: how could we have the best for both worlds.

We typically aim to give the most functionality to cover the most use cases. For 4.1 we want to do more "quality of life" improvements, ie to make some common use cases less tedious. So far this isn't going very well at all, as we've been bogged down by graph improvements and physics. 🙁

spineappletree wrote
do we launch sequences quickly because now we don't have anything else for that yet and it's nice to mention in the changelog and blogs, or do we rethink the system further to be sure we don't have compatibility issues when needed later?

I think we'll launch it as is because it is useful for some common use cases that are otherwise tedious. There's still a path for more elaborate solutions in the future. Doing those now I think is not the time, as we have other important things we want to get done. Still, your ideas for a more powerful way of doing sequences were interesting to see and we'll revisit them when the timing is better.

Cheers! :beer:

spineappletree

Nate wrote
Sorry for the delay! I didn't forget, just needed to make progress on things so I don't get too far behind. When a post is time consuming to respond to sometimes I won't have time to handle it, but I still need to use my time here and there to help in other forum threads.

Absolutely no problem and completely understandable. I get it that you are busy and my post was also quite large. I was just wondering if you read it because we can't see that on this forum. Thanks for letting me know now and taking the time to explain where you are at and what your thoughts on this are in this great detail now. Also great to read that you thought about probable future plans for sequences too in the back of your head, so that's comforting to know for future compatibilty if things will be added or changed. Looks like we're on the same page and the way sequences are built in the beta definitely have a good place for usage too.

I'm very looking forward to these physics too btw. Seems like a great new feature! And will checkout the changes to the curves.

Keep up the great work and have a nice day!