#🔒 Refreshing pages with Selenium causing memory leaks

9 messages · Page 1 of 1 (latest)

slender wind
#

Hey guys, i'm currently building a bot that refreshes a webpage every 30 seconds, but as i'm running it off my own machine I don't really want it opening and closing a browser every minute so I opted to use driver.refresh() over driver.quit(). The issue i'm having is that after every refresh the memory usage on the page increases by 100~ mb and after about an hour it crashes due to lack of memory. Does anyone have a solution they can think of to solve this. Below is some of the code used for reference and a snapshot of the memory usage after a minute:

Tried:

switching browser
running in a separate thread
setting max script run time to 10s

reset = True
def check_site():
    global reset
    page_source = driver.page_source
    try:

        element_detected = WebDriverWait(driver, 3).until(
            EC.presence_of_element_located((By.XPATH, ""))
        )

        send_discord_notification(f"@skyrift3r Message")
        reset = False
        return reset
    
    except TimeoutException as e:
        print("TimeoutException: Element not found within the specified time.")
        time.sleep(3)
        driver.refresh()
        return reset
    except Exception as e:
        print("Error:", str(e))
        time.sleep(3)
        driver.refresh()
        return reset

while reset:
  check_site()```
image.png
unkempt cosmosBOT
#

@slender wind

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
thick aspen
#

u prob. face a hard time to request scraping advice here as its mostly against ToS and against this servers rules

#

!rule 5

unkempt cosmosBOT
#
Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

slender wind
#

also in case this is brought up again, i'm not self-botting, send_discord_notification is connected to a webhook

unkempt cosmosBOT
#
Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.