Skip to content

Redwood

Jump to bottom
Benjamin Ran edited this page Apr 13, 2017 · 36 revisions

Components

Note: while dcc-metadata-indexer plays a role in the function of redwood, it is not part of dcc-ops/redwood compose setup.

Administration Guide

The following should work on dev or prod redwood (as long as you use the right credentials--see dcc-ops/redwood/.env).

Create an accessToken for a user (granting the requested scopes):

dcc-ops/redwood/admin/bin/redwood token create

List a user's (e.g. user@ucsc.edu's) access tokens:

dcc-ops/redwood/admin/bin/redwood token list -u user@ucsc.edu

Revoke an accessToken (e.g. ae335...):

dcc-ops/redwood/admin/bin/redwood token revoke ae335944-37a6-43bb-b71c-7163eb9c2976

Revoke a user's (e.g. beni's) access to all scopes:

curl -XDELETE http://localhost:8444/admin/scope/beni -u admin:secret

Developer Guide

For developing dcc-storage, dcc-metadata, dcc-auth, or the redwood infrastructure

Build Process

Each constituent redwood server is built into a docker image from one of the above git repositories:

To build one of the redwood servers, check out the corresponding git repository. Then, from the project root:

./mvnw

That builds a tar archive and docker image of the server. For example, in dcc-storage server dcc-storage-server/target/dcc-storage-server-*-dist.tar.gz and a quay.io/ucsc_cgl/redwood-storage-server:VERSION docker image will be built.

The docker image and tar archive are tagged with the the project version as defined in pom.xml.

Debugging

You can remote debug the redwood servers as they run in the docker container.

To do this, exec into the container, cd $(which dcc-storage-server)/../conf, edit the wrapper.conf file to specify java remote debugging options (-agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=n), and restart the java process with dcc-storage-server restart. Similar steps work for dcc-auth and dcc-metadata.

Then check dcc-ops/redwood/dev.yml to see which host port maps to the container's port 8000 and start a remote debugging session in your ide pointing to that port.

Inspect the databases

All redwood state is stored in the configured S3 bucket, the redwood-auth-db, and the redwood-metadata-db

redwood-metadata-db

MongoDB instance that tracks bundle id, file id, filename, etc.

Simple connection:

$ docker exec -it redwood-metadata-db mongo dcc-metadata

Find all records for a particular bundle_id (e.g. efa...):

> db.Entity.find({gnosId:{$eq: 'efac875b-faf5-5e0d-b778-0ef411b81cad'}}).limit(10)

redwood-auth-db

Access tokens and scope storage

Simple connection:

docker exec -it dcc-auth-db psql -Upostgres -d dcc

User scopes are stored in the authorities table.

select * from authorities;

OAuth Client information (id, password, scopes, etc) is stored in the oauth_client_details table.

select * from oauth_client_details;

A note on credentials

This guide assumes the default development credentials for the different principals at play in the auth service. These prinicipals, credentials, and their definition locations are listed here.

  • Postgres (dcc-auth-db)
    • Database Admin User
      • Username: postgres (default for postgres image)
      • Password: password (POSTGRES_PASSWORD environment variable of dcc-auth-db container as defined in docker-compose.yml)
      • Used for connecting to postgres
  • Auth Service
    • Auth Service Admin
      • Username: admin (security.user.name property in dcc-auth-server/src/main/resources/application.yml)
      • Password: secret (security.user.password property in dcc-auth-server/src/main/resources/application.yml)
      • Used for making calls to all admin endpoints of auth-server (/admin/* endpoints)
    • Auth OAuth Management Client
      • Username: mgmt (defined in postgres://dcc/users:username and postgres://dcc/oauth_client_details:client_id and
      • initialized via dcc-auth-db/auth-schema-postgresql.sql)
      • Password: pass (defined in postgres://dcc/users:password and postgres://dcc/oauth_client_details:client_secret and
      • initialized via dcc-auth-db/auth-schema-postgresql.sql)
      • Used as credentials of OAuth client for making oauth calls to auth-server (/oauth/* endpoints)

Release Process

You should follow the hubflow release process.

Once all tests pass and the code is ready for release, update the project version by running ./mvnw versions:set -DnewVersion=1.2.3 (replace '1.2.3' as appropriate). This should be committed on the release branch just before finishing the release.

Then finish the release with hubflow and update the project version to the next SNAPSHOT (e.g. ./mvnw versions:set -DnewVersion=1.2.4-SNAPSHOT with '1.2.4' replaced as appropriate).

Clone this wiki locally