Dynamic Texture Loading and Mixing in Unity

itsatony

Hi.

We (team behind bobmob.gg) use spine extensively with a 'skinning' system and many character-slots that users in a multiplayer onlinegame can (almost) freely mix and match, allowing for tens of thousands of unique characters all based on a few spine skeletons.
Loading Image
Loading Image

As the number of assets constantly increases (we have a open marketplace system), we lazy-load the sprite images for the slot-attachments on demand from our servers and create textures on the fly. We established the system using PIXI and PIXI-spine (example code for that is at the end of the post).

Now, we have to re-create that system using Unity-Spine to create what we call mini-games.
Loading Image
We got animations and basic spine usage down, but I am too stupid to create attachments from runtime-www-loaded pngs.
E.g. I am missing Spine.core.TextureRegion and the unity-quivalent for PIXI.BaseTexture.fromImage

So, basically, my question is: how the hell can i create textures and subsequently attachments from images I load at runtime from a url?

This is a rough (shortened, contextless) example of what I do in JS/PIXI to achieve dynamic creation of textures and from them attachments, which I later assemble into runtime-created skins:

Character.prototype.setAssets = function(assets, show) { // show meaning: if false delete the item
  var skeleton = this.spine.skeleton;
  var skin = this.spine.skeleton.skin;
  var asset = null;
  for (var i = 0; i < assets.length; i += 1) {
    asset = assets[i];
    var slotName = asset.slot;
    var slotIndex = skeleton.findSlotIndex(slotName);
    var placeholder = asset.placeholder;
    if (show) {
      var textureRegion = this.getTextureRegionOfAsset(asset);
      var name = asset.name;
      var attachment = new pixi_spine.core.RegionAttachment(name);
      attachment.name = name;
      attachment.path = name;
      attachment.region = textureRegion;
      attachment.region.name = name;
      attachment.x = asset.attachmentData.x;
      attachment.y = asset.attachmentData.y;
      attachment.rotation = asset.attachmentData.rotation;
      attachment.scaleX = asset.attachmentData.scaleX;
      attachment.scaleY = asset.attachmentData.scaleY;
      attachment.width = asset.attachmentData.w;
      attachment.height = asset.attachmentData.h;
      skin.addAttachment(slotIndex, placeholder, attachment);
    } else {
      var placeHolderAttachments = skin.attachments[slotIndex];
      delete placeHolderAttachments[placeholder];
    }
  }
};

Character.prototype.getTextureRegionOfAsset = function(asset) {
  var textureRegion = new pixi_spine.core.TextureRegion();
  textureRegion.texture = this.getTexture(asset.sourceFile, asset.x, asset.y, asset.w, asset.h);
  return textureRegion;
}

Character.prototype.getTexture = function(sourceFile, x, y, w, h) {
  var baseTexture = PIXI.BaseTexture;
  if (this.baseTextureMap[sourceFile]) {
    baseTexture = this.baseTextureMap[sourceFile];
  } else {
    baseTexture = PIXI.BaseTexture.fromImage(hookd.server.staticSpineAssetUrl + sourceFile, undefined, PIXI.SCALE_MODES.NEAREST);
    this.baseTextureMap[sourceFile] = baseTexture;
  }
  baseTexture.mipmap = true;
  var frame = new PIXI.Rectangle(x, y, w, h);
  return new PIXI.Texture(baseTexture, frame);
}

itsatony

Pharan

If you have the latest spine-unity.unitypackage, you should see sample code for this in Mix and Match.scene, and particularly the sample code, MixAndMatch.cs.
We are in the process of cleaning up some of docs and example code and scenes so it will be a bit different in 3.6.

But that sample will show creating or remapping existing attachments with Sprites. Not Texture2D.
We haven't implemented a texture-to-attachment convenience method but most of the API is already in place.

There are a few gotchas with what you're trying to do regarding batching (the runtime repacking feature solves that but you need to flag all your textures readable) and Premultiply Alpha (you would need to clone and apply PMA, if you use the default Spine/Skeleton shader). I'm not sure what limitations you have if you load assets from a URL.

RyanRothweiler

Hi Pharan,
I'm a unity dev brought in to help with this. I know Unity, but I don't have any experience with Spine.

I think I have the hard part done. I can load in sprites from a web server, create attachments with them, and add that into an existing slot. During runtime (with unity paused), I can manually enable the new attachment in the inspector and everything works just great. When unity is unpaused, the game then selects the original attachment. I assume the animation data is referencing the original attachment, not the new ones. The animations are done by cycling through the attachments like frames.
Is there a way to update existing attachments? Maybe instead it's best to remove the old attachments? Do I need to update the animation references?

The MixAndMatch example doesn't help here because the attachments in it aren't being controlled by animation data.

Thanks for the help. Feel free to correct me if my Spine knowledge is wrong.

Pharan

Existing attachments are stored in the "stateless"/shared part of the skeleton (SkeletonData.Skins), so any changes you make to it will also be applied to other skeletons using that attachment. This is the default behavior of the base runtime so you can spawn as many of that skeleton as you like and it doesn't need to create a bunch of new attachments.

So we recommend duplicating both the Skin with it whenever you want to do customization, and then make clones any attachments that you want to change.

There should never be a need to update the animations. Attachment animations work based on skin keys (strings), and each Skin object defines what attachments are used based on what keys you give it. So working with Skins (rather than directly setting slot attachments) will allow it to work with animations you have.

Currently in 3.5 your code would look something like this (this is similar to MixAndMatch.cs):

var skeleton = skeletonAnimation.Skeleton;
         Skin clonedSkin = skeleton.UnshareSkin(true, false, skeletonAnimation.AnimationState); // UnshareSkin duplicates and merges the active and default skins.
         Shader shader = Shader.Find("Spine/Skeleton"); // Default shader used in Spine-Unity.

     // For every attachment, do the following:
     {
        string originalKey = "original key"; // the name of the attachment or skin placeholder in Spine Editor
        int slotIndex = skeleton.FindSlotIndex("my slot name");
        var originalAttachment = clonedSkin.GetAttachment(slotIndex, originalKey);

        // Use this if you are using the Spine/Skeleton shader or any PMA shader.
        // NOTE: ToAtlasRegionPMAClone requires that your texture is read/write enabled or else PMA cannot be applied.
        var regionFromSprite = spriteSource.ToAtlasRegionPMAClone(shader);
        //AtlasRegion atlasRegion = spriteSource.ToAtlasRegion(new Material(Shader.Find("Sprites/Default")) { mainTexture = spriteSource.texture } ); // use this if you are using a straight alpha shader on your main skeleton. Appropriate PMA settings should be used.

        var clonedAttachment = originalAttachment.GetClone(true); // clone the attachment
        clonedAttachment.SetRegion(regionFromSprite); // change the region to the one taken from the sprite
        clonedSkin.Attachments[originalKey] = clonedAttachment; // put the cloned attachment into the skin it will be used during animations.
     }

     // Then, optionally, repack the skin so all the used images are in one texture, to minimize render overhead.

     Material outputMaterial;
     Texture2D outputAtlas;
     clonedSkin.GetRepackedSkin("repacked", shader, out outputMaterial, out outputAtlas);

RyanRothweiler

Great, thanks for the help!
I was able to get the attachments working. I had the wrong attachment name, I was using the file name not the attachment name.

Next question. Does there need to be any unit conversion between unity and the web? I'm setting the attachment transform info like so, but they aren't aligning correctly. The newAsset.attachmentData comes straight from the web app.

newWeapon.ScaleX = newAsset.attachmentData.scaleX;
newWeapon.ScaleY = newAsset.attachmentData.scaleY;
newWeapon.Rotation = newAsset.attachmentData.rotation;
newWeapon.X = newAsset.attachmentData.x;
newWeapon.Y = newAsset.attachmentData.y;
newWeapon.UpdateOffset();

Thanks for the help!

Pharan

If they're not aligning correctly, you may need to copy values from a source attachment which was pre-aligned in Spine.
Otherwise, you'll have to play around with the values until it looks right, and store those values somewhere yourself.

RyanRothweiler

The scale / width / height / position values are the same as what is initially in the atlas.
Changing the values when creating a new attachment, even to the same values as what is in the atlas, breaks the alignment.

This must mean I'm doing something wrong, possibly in the wrong order? Any thoughts on something that might point me in the right direction?

Here is the code.

Sprite spriteFromServer = Sprite.Create(newAsset.texture,
                                        new Rect(newAsset.x, newAsset.texture.height - newAsset.h - newAsset.y, newAsset.w, newAsset.h),
                                        new Vector2(0.5f, 1.0f),
                                        135f);
spriteFromServer.name = newAsset.name + "_" + newAsset.placeholder;

RegionAttachment newWeapon = spriteFromServer.ToRegionAttachmentPMAClone(Shader.Find("Spine/Skeleton"));

newWeapon.X = newAsset.attachmentData.x;
newWeapon.Y = newAsset.attachmentData.y;
newWeapon.ScaleX = newAsset.attachmentData.scaleX;
newWeapon.ScaleY = newAsset.attachmentData.scaleY;
newWeapon.Rotation = newAsset.attachmentData.rotation;
newWeapon.Width = newAsset.attachmentData.w;
newWeapon.Height = newAsset.attachmentData.h;
newWeapon.UpdateOffset();

int weaponSlotIndex = skeleton.FindSlotIndex(slotName);
clonedSkin.AddAttachment(weaponSlotIndex, attachmentName, newWeapon);

skeleton.SetSkin(clonedSkin);
skeleton.SetToSetupPose();
skeleton.SetAttachment(slotName, attachmentName);

I'm not repacking the skin, because I can't find how to mark a texture loaded from the web as readable. That shouldn't be a problem, but still.

Thanks again for all the help.

Pharan

You may not need that last SetAttachment call if it's appropriately keyed in the animation or part of the setup pose.
If it's neither of those things, it may not function properly when the skeleton is animated.

To make sure that works. Make sure this line:

clonedSkin.AddAttachment(weaponSlotIndex, attachmentName, newWeapon);

that "attachmentName" is the same as the name of the attachment that was keyed in Spine.

kafk3d

Hello! Can you please explaine how to remove the attachment? I have the script (big thanks to you) what adding new attachments to slots by click and wanted to remove (revert) this attachments to old by second click. My code:

using UnityEngine;
using Spine.Unity.Modules.AttachmentTools;

namespace Spine.Unity.Examples
{
    public class Mix : MonoBehaviour
    {
        private GameObject player;
        bool check = true;
        #region Inspector

    [Header("MeshAttachment.SetRegion")]
    public bool applyHeadRegion = true;
    public AtlasAsset atlasSource;

    [System.Serializable]
    public class SkinPair
    {
        [SpineAtlasRegion("atlasSource")] public string region;
        public string itemSlot;
        public string attachmentName;
    }
    public SkinPair[] skinItems;
    #endregion
    void Start()
    {
        player = GameObject.FindGameObjectWithTag("Player");
    }
    void OnMouseDown()
    {
        var skeletonAnimation = player.GetComponentInChildren<SkeletonAnimation>();
        var skeleton = skeletonAnimation.Skeleton;
        var newSkin = skeleton.UnshareSkin(true, false, skeletonAnimation.AnimationState);
        var oldSkin = skeleton.Skin;
        
        foreach (var pair in skinItems)
        {
            if (applyHeadRegion)
            {
                AtlasRegion atlas = atlasSource.GetAtlas().FindRegion(pair.region);
                int headSlotIndex = skeleton.FindSlotIndex(pair.itemSlot);
                var newItem = newSkin.GetAttachment(headSlotIndex, pair.attachmentName).GetClone(true);
                newItem.SetRegion(atlas);
                newSkin.AddAttachment(headSlotIndex, pair.attachmentName, newItem);
                
                //Debug.Log("slot: " + headSlotIndex + "; att:" + pair.attachmentName + "; new: " + newItem + "; check: " + check);
            }
        }
        skeleton.SetSkin(newSkin);
        skeleton.SetToSetupPose();
        Resources.UnloadUnusedAssets();
    }
}
}

Thanks

18 июн 2017, 13:45

lol :o its not fanny, but accidentally I found that combining two of my half-worked scripts gives me result with removing attachment and I dont know why it works. This is my shitcode:

public class PickClothes : MonoBehaviour {

private GameObject player;
[System.Serializable]
public class SkinPair
{
    [SpineAttachment(currentSkinOnly: true, returnAttachmentPath: true, dataField: "skinSource")]
    [UnityEngine.Serialization.FormerlySerializedAs("sourceAttachment")]
    public string sourceAttachmentPath;
    public string targetSlot;
}
SkeletonRenderer skeletonRenderer;
SkeletonAnimation skeletonAnimation;
public Skin customSkin;
public Skin skeletonSkin;
public SkinPair[] skinItems;
public SkeletonDataAsset skinSource;

void Start () {
    player = GameObject.FindGameObjectWithTag("Player");
    skeletonRenderer = player.GetComponentInChildren<SkeletonRenderer>();
    var skeletonAnimation = player.GetComponentInChildren<SkeletonAnimation>();
    var skeleton = skeletonAnimation.Skeleton;
    var newSkin = skeleton.UnshareSkin(true, false, skeletonAnimation.AnimationState);
    var oldSkin = skeleton.Skin;
    skeleton.SetSkin(newSkin);
    skeleton.SetToSetupPose();

    Resources.UnloadUnusedAssets();
}
void OnMouseDown()
{
    Skeleton skeleton = skeletonRenderer.skeleton;

    customSkin = skeleton.Skin;
    skeletonSkin = customSkin;
    foreach (var pair in skinItems)
    {
        string path = pair.sourceAttachmentPath.ToString().Split('/')[0];
        var attachment = SpineAttachment.GetAttachment(path + "/" + pair.targetSlot + "/" + pair.targetSlot, skinSource);
        var attachmentNaked = SpineAttachment.GetAttachment("naked/" + pair.targetSlot + "/" + pair.targetSlot, skinSource);
        var check = customSkin.GetAttachment(skeleton.FindSlotIndex(pair.targetSlot), pair.targetSlot);
        if (attachment != check)
        {
            customSkin.AddAttachment(skeleton.FindSlotIndex(pair.targetSlot), pair.targetSlot, attachment);
        }
        else
        {
            customSkin.AddAttachment(skeleton.FindSlotIndex(pair.targetSlot), pair.targetSlot, attachmentNaked);
        }
        skeleton.SetSkin(customSkin);
        skeleton.SetSlotsToSetupPose();
    }
}
}