Companion repository of the article "How to scrape websites in 5 minutes ?" in Theodo's blog
- If you don't have
virtualenvinstalled on your computer, install it using the official documentation - Setup a virtualenv with
virtualenv venv - Activate your virtualenv with
source venv/bin/activate - Install the required libraries with
pip install -r requirements.txt
Within your virtual environment with Scrapy installed, you can run each Spider with scrapy runspider selected_spider.py --output output_file.json