🔒 Finding a fast OCR capture/process loop for game automation

[bleak lantern]

I want to make an autopilot for a plane game called Aeronautica on Roblox, and I need to capture in game values like speed, altitude and heading at a pretty fast framerate so it would actually work. So I'm making this post to ask for recommendations on what packages/modules I could use for this.

Since some might be interested I did ask a game mod if its allowed and they said that as long as I don't break the game's limits or automate gaining money it should be fine.

[wind hamlet]

@bleak lantern

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[bleak lantern]

for context: I've tried using tesseract and pyautogui and its taking about 0.33 seconds for capturing a screenshot and 6 seconds for processing it, which is terribly slow and basically unusable

code:

    frame_start = time.time()
    
    screenshot_start = time.time()
    pyautogui.screenshot("image.png")
    screenshot_done = time.time()

    processing_start = time.time()
    print(pytesseract.image_to_string(Image.open(r"image.png"), lang="eng"))
    processing_done = time.time()
    
    frame_end = time.time() 
    
    frame_time = frame_end-frame_start
    screenshot_time = screenshot_done-screenshot_start
    processing_time = processing_done-processing_start
    framerate = 1/(frame_end-frame_start)

    print(f"FPS: {round(framerate)}, Screenshot: {round(screenshot_time, 3)}, Processing: {round(processing_time, 3)}")```

[torn cargo]

crop the image before trying to run it through tesseract
ideally capture only the region of the screen with the text in first place

[bleak lantern]

is it possible to capture a specific part of a specific window with pyautogui?

ex. the left quarter of the window

and if its a non rectangle shape im trying to capture?

i guess just split it up into 2 rectangles in my case

Another thing is, can I transfer the image straight to ocr without saving it in storage, straight from memory

[torn cargo]

idk check the documentation of pyautogui

[bleak lantern]

Update:
I've used this library and i was able to get 60+ fps https://github.com/NiiightmareXD/windows-capture/tree/main/windows-capture-python
images are numpy arrays so they can be worked with in opencv

still have no idea what i might use for ocr

any help appreciated

[wind hamlet]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.