#🔒 How to Open an HTMl file on python?

77 messages · Page 1 of 1 (latest)

wary nest
#

I noticed sth about python while using the beautifulsoup module, local html files using with open don't open correctly

Using the requests module, if I want to work on anything using the bs4 module and write a code like the one below, it works just fine.

billboard_page_html = requests.get("https://www.billboard.com/charts/hot-100/2024-05-04/")
soup = BeautifulSoup(billboard_page_html.text, 'html.parser')

For the webpage above, I saved the html of the webpage as a file, when I view the saved html file through an IDE/browser, everything works fine, but when I try write code by accessing the saved file, I get issues.

Below is an example where I tried to print out the saved html file.

with open("Billboard_Data.html") as file:
    contents = file.readlines()
print(contents)

When I printed contents, the output I got isn't what I expected, I got this - 1st image. What you see in Image1 is the beginning of the file not the whole thing, when I try to run it through bs4 through a code like this

with open("Billboard_Data.html") as file:
    contents = file.readlines()
soup = BeautifulSoup(contents, 'html.parser')

I get the error code TypeError: expected string or bytes-like object, got 'list'
Don't know if its useful, but if anyone wants the full error code - https://pastebin.com/eMrXks3m

The second image is a cutout of the beginning of the html file I saved on my pc. Looking at it, it looks like a normal html file, it starts with <!DOCTYPE html> and the likes. The 1st image is the output I get when I try to open it locally on python which is giving me an issue and doesn't come out in the right format.

Is there a way for me to open files on python and have it open in the right format? There are times when I want to use a local html file instead of getting it directly from the web.
This is the code I'm writing - https://pastebin.com/f1fDnAeF
It's a WIP, I'm trying to make a program that gets the top 100 charting songs on Billboard.

image.png image.png
clever girderBOT
#

@wary nest

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
quiet wraith
#

Try specifying the mode you're opening the file in, it's probably using rb instead of r or something.

#

Oh no nvm

wary nest
#

what do you mean by rb? What does it do?

#

For now, i only know about w, r, and a

quiet wraith
#

I read the error, the BeautifulSouo class wants everything in a single string, but readlines() returns a list of strings. So you can either join them or figure a way to read the whole file to a single string

quiet wraith
wary nest
#

Oh. I'll try it.

pine silo
clever girderBOT
#
Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

pine silo
#

what you are doing violates the ToS of the site , you cant ask for help with that here

wary nest
#

UnicodeEncodeError: 'charmap' codec can't encode character '\u0104' in position 6085: character maps to <undefined>

wary nest
#

lemme check. Didnt think it would be violate tos. Lemme reread it

pine silo
remote sphinx
#

name checks out

pine silo
# remote sphinx name checks out

only if people utilized the F3 button in their browser and search for something like scraping in it and check if its allowed which would take like 10 sec max , i wouldnt have to do this shit 💀

remote sphinx
#

ok saul

pine silo
#

i didnt make the rules man 🤷‍♂️

wary nest
#

So how am i supposed to open an html file though?

#

Should i make a new post with an html file i wrote and use it as an example?

pine silo
#

we know what you are trying to do

remote sphinx
#

i'm not disagreeing with you here yert

wary nest
main thicket
pine silo
pine silo
wary nest
pine silo
wary nest
#

The only reason why im using the site is because its an activity in the course im doing on python

main thicket
pine silo
wary nest
wary nest
pine silo
#

which would be you manually making the HTML file using the data from that website

wary nest
#

do you know how long the html file for billboard is

It's like wanting to manually write the html code from google

pine silo
pine silo
wary nest
pine silo
# wary nest Oh, but I do?😂

if you have the time , no one is stopping you from doing what you want. its your PC afterall.
but , since what you do violates the ToS of the site , you cant ask for help with it here

wary nest
pine silo
#

if you want python to do it, use the open function

wary nest
#

i'll post it in a while

pine silo
wary nest
#

who has time for that?

pine silo
#

then what is your end goal ? what do you want to do here ?? why is the site getting involved here ??

wary nest
#

the title of the thread is how to open an html file. That's my goal. how to correctly open an html file using python

I'm doing a course on python and I'll be travelling soon, and there wont be network so I'm saving the file as an html file so i can work on it on the trip

#

if i have to connect to the internet everytime for it, i wouldn't be able to do anything because there's no network. I dont really care about the site, i've already seen a few that allow scrapping and ive used them already, the issue is that i wont have network reception on the road and I noticed that while working on it and hence started the thread

pine silo
wary nest
# 
<!DOCTYPE html>
<html>

<head>
    <meta charset="utf-8">
    <title>Testrun</title> 
</head>

<body>
    <h1 id="heading">LAYOUT FOR A WEBISTE</h1>
    <h3 id="grocery">GROCERY LIST</h3>
    <ul>
        <li>Cabbages</li>
        <li>Carrots</li>
        <li>Spinach</li>
        <li>Lettuce</li>
        <li>Onions</li>
    </ul>
    <hr>
    <h3 id="books">BOOKS TO READ</h3>
    <ul>
        <li>The Stormlight Archives</li>
        <li>MistBorn</li>
        <li>Reverend Insanity</li>
        <li>Lord of the Mysteries</li>
        <li>Harry Potter</li>
    <h3 class="heading">Other
</body>

</html>

how about this

proper seal
#

hi

pine silo
pine silo
# proper seal hi

hello

OP is trying to save a html file of a website that doesnt allow scraping to parse with python later on.

but from what it looked like , they arent really interested in the scraping part yet ig 🤷‍♂️

#

might have been a bad ping on my part , sorry for that

proper seal
#

np

wary nest
#

Lemme write a code for the html code i wrote. and see if it works

pine silo
#

maybe you are more interested in the .read() function instead of .readlines() ?

wary nest
#

anyone would work

#

I dont really know the difference that much. i used read originally

#

But i had an issue with it reading the file so i just went with readlines.

#

When i was tinkering with it, I'll change it back

pine silo
#

there is also this way of directly passing the HTML page to bs4

# 
from bs4 import BeautifulSoup

with open("C:\\example.html") as fp:
    soup = BeautifulSoup(fp, 'html.parser')
wary nest
#

It worked.

#

Thank you

clever girderBOT
#
Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.