🔒 Web scraping problem.

[topaz mountain]

The product price repeats itself in the first function and I don't understand how to solve it.
the link I use for testing https://www.dr.com.tr/kategori/Kitap/Edebiyat/grupno=00055?Page=1&ShowNotForSale=true

[thorn vortex]

@topaz mountain

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[topaz mountain]

import requests
from bs4 import BeautifulSoup
import csv

def get_books(raw_url, pages):
    for page in range(1, pages + 1):
        url = f"{raw_url}{page}&ShowNotForSale=true"
        response = requests.get(url)
        if response.status_code == 200:
            print(f"Scraping page {page}")
            soup = BeautifulSoup(response.text, 'html.parser')
            elements = soup.find_all('h3', class_="seo-heading")
            price_list = soup.find_all('div', {"class": "prd-price"})
            with open('books.csv', mode='a', newline='', encoding='utf-8') as file:
                writer = csv.writer(file)
                for element in elements:
                    siblings = element.find_next_siblings(limit=1)

                    for sibling, price in zip(siblings, price_list):

                        psibling = sibling.find_previous_sibling()

                        title = psibling.get_text(strip=True)
                        author = sibling.get_text(strip=True)
                        price_text = price.get_text(strip=True)
                        writer.writerow([title, author, price_text])

            print(f"Page {page} successfully scraped and data written to CSV file.")
        else:
            print(f"Failed to scrape page {page}. Status code:", response.status_code)


def main():
    print("What type of data do you want to scrape?")
    print("1. Books")
    choice = input("Enter your choice (1): ")

    while choice != '1':
        choice = input("Invalid choice! Enter 1: ")

    pages = input("How many sections do you want to scrape? ")
    print("as example url: https://www.dr.com.tr/kategori/Kitap/Edebiyat/grupno=00055?Page=")
    raw_url = input("Enter the URL without the page number at the end: ")

    if choice == '1':
        get_books(raw_url, int(pages))

if __name__ == "__main__":
    main()

this is the function gives me same price

I got it that problem is something about loops( I think) but idk how to solve

[thorn vortex]

@topaz mountain

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.