#How to detect squares in a array of coordinates (Using python)

24 messages · Page 1 of 1 (latest)

fathom timber
#

I want to detect squares on my screen and return a list of coordinates for each square.
I use color pixel detection(RGB) to detect the squares

def find_pixel_coordinates(target_color, image):
    screenshot_array = np.array(image)
    indices = np.argwhere(np.all(screenshot_array == target_color, axis=-1))
    if len(indices) == 0:
        return []
    coordinates = [(idx[1], idx[0]) for idx in indices] 
    return coordinates

There are 2 green squares on the image the output looks like this

[(317, 127), (318, 127), (319, 127), (320, 127), (307, 128), (308, 128), (309, 128), (310, 128), (311, 128), (312, 128), (313, 128), (314, 128), (298, 129), (299, 129), (300, 129), (301, 129), (302, 129), (303, 129), (304, 129), (305, 129), (292, 130), (293, 130), (294, 130), (295, 130), (321, 130), (292, 131), (321, 131), (293, 133), (322, 133), (293, 134), (322, 134), (294, 136), (323, 136),  (362, 263), (331, 264), (331, 265), (363, 265), (363, 266), (332, 267), (332, 268), (364, 268), (332, 269), (363, 269), (364, 269), (358, 270), (359, 270), (360, 270), (333, 271), (353, 271), (354, 271), (355, 271), (333, 272), (348, 272), (349, 272), (350, 272), (343, 273), (344, 273), (345, 273), (334, 274), (338, 274), (339, 274), (340, 274), (334, 275), (335, 275)...]

How do I detect which x,y coordinates belong to which square?

vocal flaxBOT
fallen whale
fathom timber
fallen whale
fathom timber
#

They cannot. They can however in very rare case be attached to eachother like this with no white space in between. But those cases could be ignored if we can't find a solution

image.png
fallen whale
#

Alright. What I would do is group the points by x-coordinate. The resulting groups could consist of 4 groups with the same number of elements (these will be the vertical sides of the squares), and a whole bunch of groups of 2.

#

However, if the squares are on top of each other vertically, this won't be the case.

#

There will be groups with 4 elements.

#

You can detect this by counting the number of groups with 4 elements. If there are none, or 4 and the largest group has 4 elements, you are good.

#

If not, then redo the steps but with the y-coordinate, and this should succeed.

#

The size of the largest groups indicate the size of the squares. You can then find the maximum and minimum in each group.

fathom timber
#

Thx ill try it right now

fallen whale
#

This might not be the best algorithm, but it is jank that I thought up of.

#

Are the 2 squares always the same size?

#

If not, then some extra work will be needed to be done.

fathom timber
#

For now yes, but in the future they might change of size

fallen whale
#

@fathom timber I think I found a better method. Group by x and y coordinates separately. If there are more x-coord groups, consider those. Otherwise consider the y-coord groups.

#

The two squares will cover the maximal amount of x/y coords of they are not stacked/beside each other.

#

The four largest groupings will indicate where the vertical edges are.
Of course this will fail if one square is a 2x2 block and the other isn't. This is an edge case.
Another is if both are 2x2 blocks, but this one should be easier to deal with.

#

Hmm, @fathom timber A better idea might be to just loop over every 2x2 block on the image and determine if it looks like a corner of a square.
Basically find all these:

##
#.

.#
##

that don't look like this:

##
##
#

You'll end up with 2 of each. Then you just have to match them.

#

And those give you the square coordinates.