web scraping using beautiful soup saving in csv

I am learning web-scraping.
I got succeed scraping top youtubers ranking with this as reference.
I am using same logic to scrape the PL ranking, but having two issues.

  1. it is only collecting up to 5th place.
  2. some reason, it is getting only the first place for the result
  3. and then, getting attribute error.

error

    from bs4 import BeautifulSoup
    import requests
    import csv
    
    
    url = 'https://www.premierleague.com/tables'
    page = requests.get(url)
    soup = BeautifulSoup(page.content, 'html.parser')
    
    standings = soup.find('div', attrs={'data-ui-tab': 'First Team'}).find_all('tr')[1:]
    print(standings)
    
    file = open("pl_standings.csv", 'w')
    writer = csv.writer(file)
    
    writer.writerow(['position', 'club_name', 'points'])
    
    for standing in standings:
        position = standing.find('span', attrs={'class': 'value'}).text.strip()
        club_name = standing.find('span', {'class': 'long'}).text
        points = standing.find('td', {'class': 'points'}).text
    
        print(position, club_name, points)
    
        writer.writerow([position, club_name, points])
    
    file.close()

Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *