web-scraper

Technical information about websites along with a short summary. Mostly things you can't see when browsing a website.

how to use

web_scrape: enter in a url and it will print out results. It will also append you results into the your-websites.csv (please download libraries, also uncommenting lines 13-14 might be nessesary first time running).
scrape_url_list: this goes through all the companies on the helpers/websites.csv. If you want it to go through helpers/url_list.csv uncomment the two functions that are commented and comment the two that are active.
scrape_from_top_500: this function finds all urls beggining with www. and then end with </td> . Originally it was meant to scrape all with a certain html class for the links but first website didn't allow scraping.

Name		Name	Last commit message	Last commit date
Latest commit History 6 Commits
__pycache__		__pycache__
helpers		helpers
history		history
README.md		README.md
scape_url_list.py		scape_url_list.py
scrape_from_top_500_list.py		scrape_from_top_500_list.py
web_scrape.py		web_scrape.py