Sprite Puzzle Thread

[round void]

Here is the Sprite Puzzle thread now

[gloomy olive]

I wonder if the puzzle pieces really need to be "sprites"?

[fervent shore]

Alright, that works I suppose; heh. Thank you.

My push for 'sprites' is just the assumption that it will be the most efficient (ie: "best") approach.
I'm not 100% sold on it, just that it - to my understanding - would be highly efficient.

The next-closest thing I can think of, is generating meshes - as I've mentioned a couple times. (Or can re-explain, no worries)

[round void]

I thought about suggesting maybe a type of data file with just the data needing to be extracted from it.

Basically you can create a JSOn or binary file with the data and pull it in there to create your pieces.

[gloomy olive]

The point of sprites is for the sprite renderer to be able to use their meta data to give itself the shape of any sprite, and to also atlas the sprites in memory
This stuff seems more like an obstacle than a benefit in this case

[fervent shore]

The ideal perfect-world scenario, is the image and puzzle is saved a file - like a sprite-sheet. sprite-sheet. Saved to a file.

Then, when loading... there's no need to iterate the points or create geometry. It's in the sprite sheet already, and Unity just... knows how to handle that.

However, you may be correct - I don't know if, perhaps, just using Mesh and MeshRenderer - would be much different. I think it'd be less efficient, but I don't know.

[gloomy olive]

I would generate the puzzle pieces as meshes (using whichever method to slice a mesh into arrangeable bits), they could even be pre-generated
If they share UV coordinate space you can slap any texture on them without doing anything to the texture itself

[fervent shore]

If you're refering to a material with a texture, that they all use the same material - and just individual use the 'offset'.
That was one of the approaches I was considering, yes.

Then just the point data is saved to a file, along with the image.

I just have a suspicion that approach is less efficient than an actual sprite sheet. -- But, maybe it's my best (if not only) option.

It still requires manually iterating all of the points, and generating 100's of meshes, when loading.... instead of loading 100's of sprites from a sprite sheet.
But, again... I don't know if I even can use a sprite sheet, so. Might be my only option.

[round void]

welp got working code. Just need to figure out why my folder is protected and not letting me export files during run time to it.

Seems a nice security feature in my file exploerer is not letting the game ap save to the folder.

[fervent shore]

Heh, not sure on that one. I don't remember doing anything wonky to get my file saving working... unless you're not using the standard save path, maybe.

[round void]

Oh figured it out.

[fervent shore]

Yay, heh.

[gloomy olive]

Sprite sheets are procedural meshes as well, with extra data for the sprite renderer and sprite atlas to use

[gloomy olive]

What points did you have to iterate in this scenario?

[fervent shore]

Yeah, but the mesh-generation is built in - arguably going to be faster than me manually building meshes.

[fervent shore]

The points are outlining the shape of the piece. That's done with some other logic elsewhere (using bezier curves) - I already have that part built... giving me a point list of the outline.

[fervent shore]

Previously, I was using the points to use GetPixel/SetPixel and generating individual sprites. But that's slow and inefficient.
I'm trying to improve, by not having hundreds of textures - instead using one singular texture.

[round void]

Start of the example. The video just shows how I am taking in a bigger texture and cutting out a piece of it to make a sprite and saving the sprite as an actual file that can be loaded back anytime.

[gloomy olive]

In my view the generation of the physical pieces could happen entirely separately from the the texture data itself
Even offline if the pieces are in predictable configurations
Or generated using tilemap techniques if they consist of a predictable grid with random connective bits each

[fervent shore]

I think what you're exampling, is saving images of each piece. I've done that before, as well; but it has the same problem of 100's of images loaded, where really there should only be 1 image loaded. (The problem I'm trying to solve).
[If I am understanding what you are doing correctly]

[gloomy olive]

Whichever method you use, if you can allow the pieces to share UV space then they can all use the same texture and you don't need to texture data at all

[round void]

The method can accept any amount of sprite to cut out.

I can make example where it cuts out more than one sprite if you want.

[gloomy olive]

Whatever the shape of the puzzle, it'd adapt to any texture with no modifications

[fervent shore]

I'm not sure if I'm understanding UV what you're meaning on 'share UV space'.
It might be what I'm already thinking of? In the shared material/texture with 'offset'.

[gloomy olive]

I suppose so
Offsets could be included in the mesh itself

[round void]

Yeah, I think at this point maybe a mesh point cut system to inject into it would be better.

Where the shared UV space is just cut out.
I was actually loaded the sprite that were cut out by a shared UV space in my example.

[gloomy olive]

If you want classical grid type puzzle piece shape, you could generate them randomly using wang tiles with a unique mesh shape only for each unique quadrant
And project the main texture UVs when it's ready

[fervent shore]

I was just assuming using the 'Offset' there?

Is there something else I'm missing, that I should use instead?

You're mentioning shared UV space, has me thinking I'm missing something.

[fervent shore]

Also no idea what 'wang tiles' is.
I already have my own shape generating written, though - creating random/dynamic shapes is the easy part... rendering them, is proving the challenging part.

[round void]

Wang tiles is a system relating to math.
It actual is used to do stuff like this.

[gloomy olive]

It is a tiling technique for creating grids with tiles that can connect to their adjacent tiles

[fervent shore]

Though if going the mesh-generation route... that leads me to another problem. I tried solving it earlier, and failed.
Given I already have the outline points, I can't figure out how to make a mesh from that.
Since meshes need triangles... I couldn't find anything that easily converts an outline into a mesh.

Ahhh, gotcha. I may not stick with a grid in the long term - it is what I am doing currently, though.

Ideally I'd like future puzzle shapes to not be strictly gridded... but, one step at a time; heh.

[gloomy olive]

Offsetting at shader level will not be very helpful
Mesh vertices have UV coordinates that determine how a texture maps onto them

Overwriting these UV coordinates will let you drape the texture over all the puzzle pieces collectively before they're scrambled, which should be visible in my image earlier

[fervent shore]

Notably, I know how to make a mesh programmatically; I just don't know how to build the triangles for a mesh, given an outline. That's some math I don't follow.

[gloomy olive]

That's a challenge I'd skip entirely in this situation

Unless you specifically needed to make meshes from outlines at runtime?

Just to build the puzzle pieces you can use premade parts, or any method of mesh slicing

[fervent shore]

I don't think I need a mesh. -- Either a mesh-collider, if I have one... or just a box-collider, that checks if the point/cursor is over a pixel of the piece, should suffice.

(To move/select the piece)

[gloomy olive]

That seems like a separate concern entirely

Are you familiar with UV mapping?

[fervent shore]

Not in this context. I've done UV texturing on models before, so am familiar with it in that context.

But I'm not sure how it quite applies here. Sort of. Maybe. I just need to see a working example to 'click', looking into it right now.

(ie: Watching this atm https://www.youtube.com/watch?v=8rejPA8q8-Q)

So I can have square meshes - simple, low memory. But the UV's make up the puzzle piece.

[gloomy olive]

UV coordinates are data on each vertex that tells where each vertex is on the UV map (which is then interpolated over the polygon), a texture's pixels is samples from those UV coordinates on each pixel to make the texturing happen

[fervent shore]

Yawp. So, basically, the 'points' I already have - they're coordinates on the image, already. Pixel points.
I should be able to just use them as-is.

(If I'm seeing this correctly)

[gloomy olive]

Not on the image, they're arbitrary numbers on the mesh vertices

More like addresses where to sample the image when it's being renderer as a texture

[fervent shore]

But the uv points equate to pixel coordinates, do they not?

ie: Given an image of 100x200 pixels.
What would the UV coordinates be for, say, a roughly center-point pixel?

ie: (50,100) ? ... Or are they normalized 0-1? Or something else I'm not following?

[gloomy olive]

They do equate, when sampling the texture for rendering

They are normalized as well

[fervent shore]

I'm looking into it a bit further, to better understand.

[gloomy olive]

These could be our puzzle pieces that we generated

This is them "sharing" the UV space

[fervent shore]

Yawp, that's basically what I was expecting; from what I'm learning about now.
I'm just looking into figuring out how to apply the points I currently have.

[gloomy olive]

What happened here in the modeling program is that the vertex UV coordinates were assigned by "projecting"

Since you generate the order and shape of the pieces at runtime, this UV projection has to happen at runtime as well

Otherwise an individual tile will not know where they should be in UV space to display their respective region of the image

[fervent shore]

Oh... I think I follow. Not sure that's what I need, though?
Since I'd want the UV's to be the piece shapes?

An individual tile knowing where it belongs, is part of another component; easy. Already done.

My assumption is that I'd programmatically assign the UV points, using the shape points I already have?

[gloomy olive]

Once you have all the meshes, you write to their UV data

[fervent shore]

So I am creating meshes? Shaped? Not just square meshes, with UV's set?

[gloomy olive]

The shape does not matter for this operation

As long as you have vertices in some form, you can do the UV projection

[fervent shore]

Hm. Yeah, I think I'm still confused. I need to learn more about this to better understand it, there's a piece missing keeing me from it 'clicking'.

[gloomy olive]

It's important to know the theory of the mesh data in this case

I think the UV projection itself should be simple to do

If your whole puzzle after generation fits within 0 to 1 world coordinates, you can assign each vertice's world position as its individual UV coordinate

[fervent shore]

It looks like that sample video is just building a mesh. Not assigning UV's or anything like that...
So, I need to better understand UV's... I think. Maybe.

[gloomy olive]

If it doesn't fit, such as with differently proportioned puzzles, you'd remap the range during the operation

[fervent shore]

Presently the puzzle is in real-world pixel coordinates. (ie: A 100x200 pixel puzzle, would be massively 100x200 unity units)

[gloomy olive]

Then dividing by 100 and 200 respectively when assigning the data should bring the vertices into normalized range

As long as the corner of the puzzle is at 0 coordinate
If not, you need to add an offset as well

[fervent shore]

Yeah, easy enough. I just have to find out how to deal with UV's (or projections, in your example) - looking into it, I need to find some references.... and some RL interruption is slowing me down, too.

[gloomy olive]

https://youtu.be/Sf8pk2q1vQ8 this seems to be the most relevant tutorial, going over both UV theory and code in unity
Can't check how good it is in this timespan though

[fervent shore]

The part I'm still trying to find an answer to:
Does the mesh need to be shaped - or do I just use a square mesh, and the puzzle-shape is defined by UV's... and anything outside the "UV points" will be transparent on the square.

[gloomy olive]

None of that quite makes sense to me

[fervent shore]

Hm. How to better explain...

If I have to actually make the mesh, shaped like the puzzle piece.... then that's solution (B) I listed at the very beginning.
(And I'm back to struggling with trying to generate a mesh from an outline shape)

[gloomy olive]

Remind me what's the "outline" from

[fervent shore]

The 'outline' is a series of points that I am dynamically generating, that defines the shape of the piece.

[gloomy olive]

Hm, why

[fervent shore]

The 'why' is to define the shape? How else do I get the puzzle piece shape?

[gloomy olive]

Could be modeled offline to avoid having to do that whole thing procedurally

There's a very limited number of those tile varieties after all

[fervent shore]

That defeats the purpose, heh; needs to be done at run-time.

That puzzle piece is just one 'standard' piece example... I plan to have irregular shaped pieces.

[gloomy olive]

You can do it at runtime even with predefined tile types

Unless you mean like really irregular

[fervent shore]

Yes, they'll be very irregular. (Also, notably; actual puzzle pieces are typically varied, and don't fit together except where they're supposed to, heh) -- Regardless, very irregular pieces will be used, yes.

Just one example of an irregular piece.

[gloomy olive]

That makes sense

In that challenge I'm not that much help
You'd have to look into algorithms that can fill a face from edges, maybe some type of tessellation or triangulation algorithm
And if you want 3D depth, extrusion on those edges as well

[fervent shore]

Yeah, so that's the 'outline'. I need to render that, as an individual piece. Well, hundreds of pieces. That's the end goal.

As a bit of a re-summarization...

I currently have written it using individual image files for each piece (horrifically inefficient), and as dynamically creating textures Get/Set Pixel which also is inefficient.

I also have written an Editor tool to generate sprite-sheets, using-
dataProvider.GetDataProvider<ISpriteOutlineDataProvider>().SetOutlines(spriteGUID, outlines);
However, that is editor-only.

Given that I want a user to be able to upload their own images, and for puzzle pieces to be dynamic (logic already written to generate the 'outline') - I cannot use that Editor-only tool.

[fervent shore]

Yeah. It sounds like I need to use the outline I have, to create a Mesh. Which was option (B) I listed earlier (outside of thread) - which I think is similar to your UV suggestion. (I think)

[gloomy olive]

Another method is to start with simple puzzle pieces compiled from pre-created meshes (using wang tile technique here cuts down the unique number of mesh shapes you need to 4), then subdivide and deform, which will be likely much easier than filling and extruding

Regardless of how you end up with the generated puzzle pieces, then you project their UVs to avoid doing any hacky stuff with the textures

[fervent shore]

That's the part I'm still a little confused on. I'll look into it,though.

It sounds like I do need to actually have shaped meshes, though. Right?
I can't just use square meshes, and rely on UV mapping to make squares look like pieces.
Correct?

[gloomy olive]

There is a way to make that work but it won't be 3D at all

[fervent shore]

The pieces are 2d. No extrusion or anything. Just flat.

[gloomy olive]

A mesh can have multiple UV channels, so in that scenario you'd have one set of UV coordinates for the whole board and another for each individual tile

The individual UV coordinates would have a black&white texture of their own puzzle piece shape, determined by your piece generator

[fervent shore]

Oh, you're talking about masking. That kinda defeats the purpose, though... since I'd still have to generate additional textures.

[gloomy olive]

But that makes kinda difficult to have weirdly shaped pieces

[fervent shore]

(ie: May as well just generate the piece texture itself)

[gloomy olive]

Not impossible though, the quad can be bigger than the "piece" itself to give you leeway to warp the shape in the shader using the first UV channel

[fervent shore]

Yeah, that's where I'm confused. I'm still trying to find an example of that being done, so I can see it and understand it.

Every example I'm finding, is just making a custom Mesh shape.

[gloomy olive]

The method requires quite a bit of familiarity with shaders

And I doubt there even is an example of that specific thing anywhere

It'd seem productive to make you unlearn that texture generation stuff for a while 😅
It's really not a great hammer and these really ain't nails

[fervent shore]

Ah, yeah; I'm only novice there. Using the Shader Graph, I've made a couple basic-ish ones... and even then, had tutorial help that I tweaked for my own use.

Also, yeah, I do want to get away from the texture generation. It's why I'm trying to do something else. I wanted to use Sprite Sheets, but... noooooo.... Editor Only. Huff. > : P

[gloomy olive]

Not an easy challenge to pick either

[fervent shore]

It's just, the alternative, as far as I understand so far, is to use mesh generation.

Or, shaders perhaps, yeah; but I think that's outside of my scope. (ie: Pass in an outline of coordinates, and fill them with the source texture?) - That sounds like a fun challenge, but I don't know if it's a better solution than mesh generation.

[fervent shore]

So, I finally got a mesh built from the outline/perimeter points to make a piece shape.
I've tried several shader graph approaches (and watched several videos), but can't seem to figure out how to just... get the texture onto the piece. The result I keep getting, almost always paints the whole piece one color. Prefab plane/sphere showing the full texture (instead of one color), but that's not what I need.
Any suggestions on video, tutorial, or somesuch to look for?

Also just my current oddity - still not working right (one color on the piece), but... my lack of understanding of shader graphs, I think - resulting in some fun oddities.
The first image makes it look like the order/layering of the objects, is oposite of it's actual ordering. Heh. No objects moved... I just rotated the camera to show the actual order.

Better example of the images, heh -

[gloomy olive]

Transparent gameobjects are sorted by their origin's distance from camera, sprites use 2D sorting priorities
https://docs.unity3d.com/Manual/2DSorting.html

[fervent shore]

Ah, so I can probably just use sorting layer - when it comes to it. I'll look into that.