🔒 unable to get the list of usernames from the codeforces website

[runic moon]

https://codeforces.com/contest/1945/standings/page/31
this is the url im testing it on
i used the requests library to get the contents of the page but when i run a search on the results with the one of the username on the page i get nothing
i basically want all the text youll get if you do ctrl a and ctrl c on the webpage and its okay if its embedded in html i just want it to be there
help?

[limpid goblet]

@runic moon

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[runic moon]

my code was

well it wasnt much

r = requests.get(url)
print(r.content)

[strange blade]

Data probably loaded in via JavaScript, so requests won't help you.

[crystal night]

import requests
from bs4 import BeautifulSoup

def get_text_from_webpage(url):
    # Fetch webpage content
    response = requests.get(url)
    
    # Check if request was successful
    if response.status_code == 200:
        # Parse HTML content
        soup = BeautifulSoup(response.text, 'html.parser')
        
        # Extract all text
        text = soup.get_text()
        
        return text
    else:
        print("Failed to fetch webpage:", response.status_code)
        return None

# Example usage
url = 'https://example.com'
webpage_text = get_text_from_webpage(url)
if webpage_text:
    print(webpage_text)```

That should help

Didn't test it tho cus I am on mobile

[strange blade]

Looking further, the usernames are in the source and I can find them in r.text, so... should be fine.

[runic moon]

huh

[strange blade]

A lot pages don't work for the reason I gave. But I looked into this further, and it isn't the case here.

[runic moon]

yeah ill try that

[runic moon]

if i may ask how are you saying this

[strange blade]

If you go to the webpage and use 'View Source' you'll see the HTML that is served up that requests can reach. The DOM contains more than just the source, though. It contains tags and data that has changed as a result of other network requests and JavaScript executions.

requests gets you the source, and nothing more.

[runic moon]

oh

[runic moon]

when i initally did this

i found the usernames

inside the table headers

[strange blade]

Yeah, I confirmed that they're there and requests will work for you in this case.

Unfortunately, this method won't work on many sites these days. But here it will.

[runic moon]

i need to do get and then text right

[strange blade]

.text is the source, yeah.
Or try the other method that was given to you to just get the readable text.

[runic moon]

they are present as ids and not the username or the name of the problem

[runic moon]

yeah one minute ill try the code given

[strange blade]

I checked if username in r.text and got True, so the usernames are there which is what you were asking about. Or at least that's what I thought you wanted.

[runic moon]

this is what i get

i stored the text from the code in a test file

i cant find those usernames

[strange blade]

!code

[runic moon]

and then 5998

[limpid goblet]

Formatting code on Discord

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

For long code samples, you can use our pastebin.

[runic moon]

this is the code

i can send the results i got

for the url

https://codeforces.com/contest/1944/standings/participant/177215361/page/31

[strange blade]

I took all the names from the page (202) and they all appear in r.text, apparently

import requests

names = names = [
    "uttamkumar00865",
    "dmrmra",
    "dheetCoder",
    "ABTurjo121",
    "DevanshKesan",
    ...
]

url = "https://codeforces.com/contest/1944/standings/participant/177215361/page/31"
source = requests.get(url).text

for name in names:
    if name not in source:
        print(name)

ie, that printed nothing. Because they all appear.

here's the JS I used to get all the names:

cells = document.querySelectorAll(".contestant-cell")
names = []
for (cell of cells) {
    names.push(cell.innerText.trim())
}

[runic moon]

thank you

that works

[limpid goblet]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.