🔒 How to scrape images from a manga site

[misty token]

I'm trying to figure out how to extract the pages from this manga chapter in mangapark.io. In my code, I tried to call it using the find_all for every div data-name with all the other specific attrs i put in there, but every time nothing comes out in the list. I don't know how to access it, especially since its nested so deep.

the code:
import requests, os
from bs4 import BeautifulSoup

url = "https://mangapark.io/title/49542-en-grand-blue-dreaming/1227239-ch-001"
r = requests.get(url)
layout = BeautifulSoup(r.text, "html.parser")
images = layout.find_all("div", attrs={"data-name":"image-item", "class":"flex flex-coljustify-start items-center w-full", "q:key":"8X_2"})
print(images)

[vernal hill]

@misty token

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[random owl]

This does sound like it would be a copyright minefield, leaving aside what I imagine to be probably against this website's terms of service.

This doesn't sound like something we can help you with on this server, absent a sufficient rebuttal of these issues.

[misty token]

Ohh damn. I'm kinda new to this web scraping thing and I didn't realize that it's not really allowed.

I just thought it would be a nice beginner project.

[random owl]

Webscraping in and of itself is permissible, but a number of factors may render a particular project impermissible.

[misty token]

Oh, I see. Forgive me but I'm still curious though about Beautiful soup. Like in my example, how do you access a specific div when it's nested in so deep, where the div data name is under div class and other stuff? I'm not really familiar with HTML stuff. You don't have to answer if you're uncomfortable .

[random owl]

I'm not proficient in webscraping. Sorry. I only stopped by because of the above-mentioned issues. Sorry. 🙂

[misty token]

Oh, ok. Thanks for your time anyways.

[harsh wharf]

you can check robots.txt from the website. as long as u don't distribute your scraped images or violate their robot regulation, you are fine

As for your element selecting issue, you can copy css selector from chrome's dev tool tab and use soup.select(selector) to select element

[random owl]

robots.txt is unstandardised and lack of prohibition is robots.txt should not be taken as permission.

[harsh wharf]

Search engines also scrape pages according to robots.txt. And there are often request freq and allowed domain marked in it

besides, The point is not to DISTRIBUTE the content

Sending low freq requests for learning purpose do not cause legal problems

[misty token]

Ah I see now, didn't know there were so many measures put in place. I am definitely not gonna be distributing content anytime soon.

Thanks guys.

[vernal hill]

Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.