#🔒 Numpy Filtering 9600 single color objects from pixel rgb value

14 messages · Page 1 of 1 (latest)

sweet galleon
#

Hi, I have a 3D-array from an image in Numpy (4096, 8192, 3), with 4096 rows, 8192 colmuns with each cell being a list [RED, GREEN, BLUE]
When I want to extract all cells with the same RGB value I use mask = numpy.all(image == (R,G,B), axis=-1) then I can use image[mask] to modify the pixels I filtered for.

My issue is I need to do this around 9600 times and each time takes ~2 seconds.
I'm looking for ways to speed up the process.
I've tried multi-processing but that only reduced the time from 5 hours to around 3.
Currently I'm thinking of reusing the mask for the next filter so it skips pixels I've already checked before, but I'm having trouble implementing as its a 2-D array instead of a 3-D one.
new_mask = numpy.all(~mask & (province_map == filter_array), axis=-1) returns me the error operands could not be broadcast together with shapes (4096,8192) (4096,8192,3)

storm brambleBOT
#

@sweet galleon

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
sweet galleon
#

My end goal is to get the border pixels of each color in this image ( Ireland ) and turn them into coordinates I can put into a Javascript Canvas. Using OpenCV my process is get the pixels by color, turn everything else black, apply the treshold and findContours. If the process above can be skipped to quickly give me the contours I'm fine with it too.

image.png
quasi hedge
#

Hey, I've got 2 ideas

  1. Batch the arrays
import numpy as np

images = [
    np.random.randint(255, size=(4096, 8192, 3), dtype=np.uint8)
    for _ in range(10)
]

# concatenate all images
images = np.concatenate(images, axis=0)
print("shape:", images.shape)


target_color = np.array([254, 3, 0], dtype=np.uint8)

# Compute global boolean mask for more than 1 image at a time
mask = np.all(images == target_color, axis=-1)
print("mask shape:", mask.shape)

output

shape: (40960, 8192, 3)
mask shape: (40960, 8192)

2.. Lower the precision dtype of the array. This allows to increase the batch size in the first example, and the comparison is probably faster because you compare numbers with less bits.
(I bit like in my example above where I use np.uint8 instead of np.float64)

import numpy as np

image = np.random.random((4096, 8192, 3))
print(image.nbytes / 1024 / 1024, "MB")
small_image = image.astype(np.float16)
print(small_image.nbytes / 1024 / 1024, "MB")

output:

768.0 MB
192.0 MB
#

Btw, I think what you want to do might be a connected component analysis, I've never done it though so I can't help straightforwardly

sweet galleon
#

There are no multiple images, the one I showed above is just a small crop of the imagine I'm filtering in, which is 4096x8192

quasi hedge
#

(Sorry, I always read too fast and end up not fully understanding everything.)

livid tulip
#

I also wonder if it would speed up the process if you preprocess the image once at the beginning. Instead of having 1 colour represented as RGB, represent it as a single int f.e. this should make the comparison with a mask faster

quasi hedge
sweet galleon
livid tulip
storm brambleBOT
#
Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.