I think I fixed it. This is our internal description that might be useful for someone else:
In Unity, when using Spine with UI Toolkit, it needs to know the default size of the animations to scale them correctly to the Visual Element.
To set this up, you use something called the StartingAnimation parameter in C# code. This parameter will set the animation for the system to pick the first frame of the animation, which Spine uses to determine the initial size or "artboard" size.
For example, if you choose an animation that starts at 50% of its full size, Spine will use this as the reference size. When the animation scales up to 100%, it will appear twice as big compared to its starting size, as shown in the picture.
We have therefor chosen to have a “default” animation with keys on scale 100% to use as our StartingAnimation and override the animation immediately with AnimationState.SetAnimation.