Skip to content

Latest commit

 

History

History
84 lines (66 loc) · 2.67 KB

README.adoc

File metadata and controls

84 lines (66 loc) · 2.67 KB

Community Graph

This is the start of a project that will allow you to create a community graph that collects data from Twitter, GitHub, StackOverflow, and Meetup and stores it in a Neo4j graph. You’ll then have a database that you can query as well as a dashboard which summarises the community activity over the last week

If you have any questions please send us an email devrel@neo4j.com

Pre requisites

You’ll need to have the following installed:

Node.js 6.7.0
Python  3.6.0

How do I use it?

Make sure you have your AWS credentials in ~/.aws/credentials.

At the moment you’ll need to manually specify your Neo4j server URL. You’ll also need to have your API keys for Meetup, GitHub, Twitter, and StackOverflow ready.

Once you’ve done that you’re ready to go.

npm install -g .
community-graph create

On completion the create command populates communitygraph.json with the values you passed in.

{
  "communityName": "Community Name Here",
  "tag": "Tag for searching on GitHub",
  "s3Bucket": "Name of S3 bucket",
  "logo": "Path to an image of your community",
  "serverUrl": "IP Address/Hostname of Neo4j Instance (e.g. 127.0.0.1:7687)",
  "twitterSearch": "Twitter search term",
  "credentials": {
    "keyArn": "KMS Key ARN",
    "meetupApiKey": "Meetup API key (encrypted by KMS key)",
    "githubToken": "GitHub Token (encrypted by KMS key)",
    "twitterBearer": "Twitter Bearer (encrypted by KMS key)",
    "readonly": {
      "user": "Neo4j user name",
      "password": "Neo4j password (encrypted by KMS key)"
    },
    "write": {
      "user": "Neo4j user name",
      "password": "Neo4j password (encrypted by KMS key)"
    }
  }
}

To deploy the lambdas to AWS and have them populate the graph you’ll need to run the following command:

community-graph deploy

We’ll be automating more of this so it’s a more friendly user experience. Check back soon and hopefully it’ll be simpler.

Things to do

  • ✓ Add lambdas to import data from StackOverflow

  • ❏ Add lambdas to import data from Meetup

  • ✓ Add lambdas to import data from Twitter

  • ✓ Add lambdas to import data from GitHub

  • ✓ Interactive community-graph so that users can set parameters

  • ✓ Generate communitygraph.json file based on parameters passed into the community-graph command

  • ✓ Create S3 bucket to put summary page

  • ✓ Permissions for S3 bucket <→ lambda user

  • ✓ Generate KMS key

  • ✓ Encrypt passwords using KMS key

  • ✓ Delete pycache directory at the end of a failing run

  • ❏ Spin up Neo4j server using AWS AMI

  • ❏ Install APOC on the Neo4j server

  • ❏ Run all the lambdas once on creation

  • ❏ Run the twitter lambda with catch-up: false