Building Scraper Service

By Venu Gudavalli (venugudavalli@hotmail.com)

Development Platform

Anaconda 3

OS: Ubuntu Linux 16.04 LTS (xenial)(Windows 10 Subsystem Linux)

Python: 3.6

Special Packages:

• beautifulsoup4
• scrapy
• pip:
  • pymongo

Database : Mongodb

Steps

1. Install Mongodb
2. Set up the environment
3. Setup the scraper
4. Run scraper
5. Verify the Data)

Next: Install Mongodb