#🔒 Question: Issue with Scraping Data from Cars.com - Data Inconsistencies in Excel Output

4 messages · Page 1 of 1 (latest)

halcyon axle
#

Hi everyone,

I’m encountering an issue with my web scraping script for Cars.com. I'm using Selenium and BeautifulSoup to scrape car listings and save the data to an Excel file. While the script prints all the expected data correctly to the console, the resulting Excel file only shows the car names and links with correct data. The other columns (price, mileage, distance) contain zeros.

Interestingly, the same script works perfectly for Autotrader, and the data for Autotrader is saved correctly in the Excel file. The issue seems to be specific to the Cars.com data.

Here’s a summary of my setup:

Code Highlights:

Scraping Function (FindCarsFromCars):
Uses Selenium to navigate and scroll through pages.
Extracts data including name, price, mileage, distance, and link from each listing.
Data Processing (clean_and_process_data):
Cleans and converts price, mileage, and distance.
Adds calculated fields and filters based on criteria.
Excel Output (output_data_to_excel):
Saves processed data to an Excel file using Pandas.
Observed Issue:

The console output shows correct data for all fields from Cars.com.
The Excel file only shows the name and link columns correctly; the price, mileage, and distance columns contain zeros.
Code Snippets:

Scraping and Processing Code
Excel Output Code
What I’ve Tried:

Verified that data is printed correctly to the console.
Checked the HTML structure to ensure correct element selection.
Added debugging outputs to clean_and_process_data to monitor data changes.
Questions:

Has anyone experienced similar issues with data scraping and saving to Excel?
Are there common pitfalls or overlooked details in the data processing or Excel output stages that might cause this problem?
Any tips on additional debugging steps or tools that might help identify where the data is going wrong?
Thanks in advance for any insights or suggestions!

tall treeBOT
#

@halcyon axle

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
tall treeBOT
#

@halcyon axle

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.