🔒 Data Scraping (Beginner )

[polar bay]

Hi everyone, i was making a code so that i have a text file of if a film is red to make it 'fresh' and if its green to make it 'rotten'
I feel as if im almost there but dont know what im doing wrong (Very new )

[shrewd swift]

@polar bay

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[polar bay]

from bs4 import BeautifulSoup
import re
import time
import requests
import csv

I imported these

[mental kestrel]

can you give a sample of your input and what you expect?

[polar bay]

def run(url):

fw=open('reviews.txt','w',encoding='utf8') # output file

writer=csv.writer(fw,lineterminator='\n')#create a csv writer for this file

for i in range(5): # try 5 times
        #send a request to access the url
        response=requests.get(url,headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36', })
        if response: # explanation on response codes: https://realpython.com/python-requests/#status-codes
            break # we got the file, break the loop
        else:
            print('fail',i)
            time.sleep(2) # wait 2 secs
        

# all five attempts failed, return  None
if not response: return None

html=response.text# read in the text from the file

soup = BeautifulSoup(html,'html') # parse the html 

reviews=soup.findAll('div', {'class':'review-row'}) # get all the review divs

for review in reviews:
   
    critic,text,date,plat,rate='NA','NA','NA','NA','NA' # initialize critic and text
    
    criticChunk=review.find('a',{'href':re.compile('/critics/',)})
    if criticChunk: critic=criticChunk.text.strip()
    
    textChunk=review.find('p',{'class':'review-text'})
    if textChunk: text=textChunk.text.strip()
        
    dateChunk=review.find('span',{'data-qa':'review-date'})
    if dateChunk: date=dateChunk.text.strip()
        
    platChunk=review.find('a',{'href':re.compile('/critics/source/')})
    if platChunk: plat=platChunk.text.strip()
    
    rateChunk=review.find('span',{'data-qa':'tomatometer'})
    if rateChunk: 
        icon = rateChunk.find('span', {'class': 'icon'}) 
        if icon and 'red' in icon.get('class',[]):
            rate = 'fresh'
        if icon and 'green' in icon.get('class',[]):
            rate = 'rotten'
            
      
    
    writer.writerow([critic,text,date,plat,rate]) # write to file 
    
fw.close()

uh nvm i dont think i understand how to post code on here lol

[shrewd swift]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.