Web Scraper: Auto Trader

BeautifulSoup4SeleniumData PreprocessingWeb ScrapingPythonData Collection

Friday, May 17, 2024

View Project

The Auto Trader Web Scraper project automates the extraction of vehicle listings from AutoTrader.com. Powered by Python and leveraging BeautifulSoup for parsing, this scraper retrieves detailed information about vehicles based on predefined search queries.

Key Features

  • Configuration File Integration: Utilizes a config.txt file containing search queries to dynamically update URLs and fetch relevant page data.
  • Data Extraction: Parses vehicle details such as make, model, year, price, mileage, and location from the HTML content.
  • CSV Output: Outputs scraped data into a structured CSV format using Pandas, facilitating easy analysis and further processing.
  • Automated Scraping: Automates the scraping process to gather comprehensive data across multiple searches without manual intervention.

Requirements

  • Python 3.x
  • BeautifulSoup (bs4)
  • Requests
  • Pandas

Getting Started

  • Clone the Repository:
https://github.com/Muneeb1030/Autotrader_Web_Scapper.git
cd auto-trader-scraper
  • Install Dependencies:
pip install beautifulsoup4 requests pandas
  • Run the Scraper:
python auto_trader.py