🔒 Python/Selenium Problem

[sour hound]

I am trying to get the program to do a basic scraping function. I run the code below, and get no errors. I also do not get the expected action of the search field being selected and then the search terms being typed. Please help. Thank you in advance.

import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.webdriver import WebDriver

Set headless mode

options = Options()

Add headless argument to ChromeOptions object

options.add_argument('--headless')

Set the path to your ChromeDriver executable

service = Service("C:\Users\mabri\OneDrive\Desktop\Programming\chrome-win\chrome.exe")

Create a Chrome driver with the options

driver = WebDriver(service=service, options=options)

Load a website

driver.get('https://www.google.com')

Wait for the page to load

time.sleep(10)

Locate the search bar and input the search query

CSS_SELECTOR = '#input'

search_bar = driver.find_element_by_css_selector(CSS_SELECTOR)

search_bar.send_keys('Python programming language')

[mental ibex]

@sour hound

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[hardy idol]

!code

[mental ibex]

Formatting code on Discord

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

For long code samples, you can use our pastebin.

[sour hound]

Thanks. I got in a hurry.

[craggy obsidian]

“Superuser” website: Can find all the problems that this server has had before?

[hardy idol]

what?

[craggy obsidian]

Can all the questions asked in this group be found on this website "superuser"

[stoic halo]

Is it fine if I just drop the code that fixes this?

[stoic halo]

Well, to start off, you don't need "from selenium.webdriver.chrome.options import Options" and "from selenium.webdriver.chrome.webdriver import WebDriver", since it already comes with "from selenium import webdriver"

You can set options to webdriver.ChromeOptions()

I'm not sure if using service is necessary, since I don't need it for code

driver should be set to webdriver.Chrome(service=service, options=options)

Actually, I'm just gonna drop the code instead

import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium import webdriver
from selenium.webdriver.common.by import By

options = webdriver.ChromeOptions()
service = Service()


driver = webdriver.Chrome(service=service, options=options)

driver.get('https://www.google.com')

time.sleep(10)

CSS_SELECTOR = '#APjFqb'

search_bar = driver.find_element(By.CSS_SELECTOR, CSS_SELECTOR)
search_bar.send_keys('Python programming language')

driver.quit()

I noticed your original search_bar statement didn't work (maybe it was deprecated or something)

It's possible you're using an older selenium version and need to update it

Also, you had the wrong CSS Selector, so I used Chrome Dev Tools to inspect the search bar and find a selector that would work

Oh, and you can also add the headless option if you wish

[sour hound]

Thank you @stoic halo , I will give this code a try. I have updated Selenium, but i could still be getting an older verision i know it is at least 4.0. I did the dev tools also on the chormium browser that came up should I have done it on the normal browser?

[stoic halo]

I think it should be fine either way

[sour hound]

Ok, I ran it and it runs a little better in that it actually opens and closes like it should, but still no send keys action

[stoic halo]

Maybe the CSS Selector I gave you isnt the same on your end? Try opening https://www.google.com in your chromium browser, then use chrome dev tools to get the CSS Selector from the search bar

then add it to the code

[sour hound]

I'll give it a try.

It is the same selector

[stoic halo]

Try adding time.sleep(5) under the search_bar.send_keys

Maybe it's doing it so fast that you dont see it?

[sour hound]

ok

That worked.. Thank you.

[stoic halo]

Your welcome

One last thing

[sour hound]

Ok

[stoic halo]

If you add options.add_experimental_option("detach", True) to the chrome options and remove driver.quit() it lets you stop the driver from automatically closing the chrome window

I like to do that so I can inspect the elements in the browser and change it

and then I don't need to add the time.sleep()

[sour hound]

Ok. Then just remove that when going to production

[stoic halo]

Yeah

[sour hound]

Ok cool

[stoic halo]

And dont forget to add the headless option afterwards

[sour hound]

Thanks! I may hit you up for more help in the future if you're up for it.

[stoic halo]

Yeah sure ping me any time

Always happy to help (except when I'm sleeping)

[sour hound]

LOL, I hear you there

[mental ibex]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.