##Setup
$ sudo apt-get install python-pip python-virtualenv
$ git clone git://github.com/techism/crawler.git
$ cd crawler
$ virtualenv --no-site-packages venv
$ source venv/bin/activate
(venv)$ pip install -r requirements.txt
Create aggregator/config.yaml