foggy notch
Title. I'm attempting to automate button clicking by looking for a specific string of text on screen, and then moving the mouse to it. However, even though I've used OpenCV to get a pretty clear image (atleast for an OCR), it still can't find the text I'm looking for
Attached is the manipulated image from OpenCV (everytime it fails to find the text, I have it write the image to disk). The word I'm looking for is "SHAKE", but as can be seen from the attached image, the word shake is clear without any visual noise surrounding it apart from some white pixels which aren't obscuring the text.
Here's the detection part of the code:
def find_button():
now = time.time()
img = cv2.cvtColor(np.array(sct.grab(rect)), cv2.COLOR_BGR2GRAY) # rect defined elsewhere, it's just a bounding box the window I'm capturing
otsu_threshold, image_result = cv2.threshold(
img, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU,
)
img_dilation = cv2.dilate(image_result, kernel, iterations=1)
#Get all text from screen
d = pytesseract.image_to_data(img_dilation, output_type=pytesseract.Output.DICT,lang='eng')
index = -1
for i in range(len(d['text'])):
if d['text'][i] == 'SHAKE':
index = i
break
if index == -1:
#Image was not foumd, write the image to disk and return None
cv2.imwrite('screenshot.png', img_dilation)
return None
nbox = index
if (int(d['conf'][nbox]) > 0):
elapsed = time.time() - now
print(elapsed)
#Return the bounding box of the text
return (d['left'][nbox], d['top'][nbox], d['width'][nbox], d['height'][nbox])
return None
Is there some sort of configuration I'm doing incorrectly? I've tried a bunch of different PSMs for tesseract, but none ever worked. Would appreciate some guidance.
neon coral
