diff --git a/README.md b/README.md index f035973..4b170b5 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ To run it quickly: 1. Clone the repo. 2. [Create a GitHub App](https://github.com/settings/apps/new) using the config guide below. * Your callback URL will be https://localhost:8001/_/api/v1/account/signin-github -3. Add the GitHub ClientId and ClientSecret to the appsettings.json file. +3. Add the GitHub ClientId and ClientSecret to the appsettings.json file, it should look something like the configuration example below 4. Delete the "Google" and "Okta" sections out of the appsettings.json file. 5. From the NetGoLynx directory: @@ -58,179 +58,47 @@ docker run --rm -it -p 8001:443 -e ASPNETCORE_URLS="https://+" -e ASPNETCORE_HTT Open https://localhost:8001 in your browser and you should be good to go. Log in, add a link, then use `https://localhost:8001/your-link` to resolve it. If you're feeling really fancy you can add a hosts file redirect for `go` on your machine to really give it a spin. -Keep in mind that as soon as you close that window or stop the process you will lose all your links. +Keep in mind that as soon as you close that powershell terminal or stop the process you will lose all your links, you should only use this for testing! -### Configuration - -This app is configured via appsettings.json. You can find an example (with details you should fill out) provided in the repo that you can either edit directly, or provide an override file in the same directory called appsettings.release.json. - -The basic structure looks like this and is explained section by section below. +#### Configuration Example ```json { "ConnectionStrings": { - ... + "Sqlite": "Data Source=redirects.db" }, + "AllowedHosts": "*", "Logging": { "LogLevel": { - "Default": "Warning" + "Default": "Information", + "System": "Information", + "Microsoft": "Information" } }, - "AllowedHosts": "*", - "Authentication": { - ... + "AccountSettings": { + "AdminList": "your_github_email_address" }, - "ProxyNetworks": { - ... - } -} -``` - -#### ConnectionStrings - -This section contains the connection string to connect to the appropriate SQL data store. There should only be one connection string at a time, the app doesn't know what to do with more than one. - -The database provider is chosen based on the name of the connection string key provided. You should provide a connection string in the format appropriate for the provide you select. The key must match with one of the implemented providers: - -* DefaultConnection: Alias for Sqlite. -* Sqlite: For a local (or volume-mapped) SQLite database. Good for home installations. - * Example: `"Sqlite": "Data Source=redirects.db"` -* SqlServer: For an MS SQL Server connection. - * Example: `"SQLServer": "server=127.0.0.1,1433;Database=netgolynx;User Id=SA;Password=yourStrong(!)Password"` -* Postgresql: For a PostgreSQL server connection. - * Example: `"PostgreSQL": "Host=localhost;Database=netgolynx;Username=postgres;Password=password"` - -NetGoLynx is configured to fire off database migrations automatically when the app starts to make sure the database is present and the schema is up to date. If there are any breaking schema changes that requires any kind of manual migration the release notes will walk you through the operations necessary. - -There is no mechanism for migrating between database providers at this time, taking a backup of the existing provider and loading it into a new provider after running NetGoLynx once may just work. Alternatively pester me to implement full import/export or something. - -#### Logging - -Control how noisy the logging system is to the log file. - -#### AllowedHosts - -A semicolon separated list of hosts the app will run as. Using `*` allows anything, which is probably too broad for production. - -If you wanted to host NetGoLynx on your domain at go.contoso.com you'd want to use: - -```json - "AllowedHosts": "go.contoso.local;go" -``` - -The list of hosts is semicolon separated and should not include port numbers. You'll probably want to have "go" in that list specifically if you want the go/link format to work. - -If you're running this behind a TLS-terminating load balancer on a secure network setting it to "*" may be fine. - -Note: This is seprate from the ProxyNetworks configuration, in case your load balancer configuration needs it. Modern load balancer proxy systems such as Azure Front Door or AWS ALBs should be able to use the same values for both. - -#### Authentication - -Configure the available authentication providers. You must have at least one configured. You can configure as many as you want to support. - -##### GitHub - -Authenticate using a GitHub app. You can configure this to be either GitHub.com or a GitHub Enterprise instance. Follow the [instructions for creating a GitHub App](https://docs.github.com/en/developers/apps/creating-a-github-app) and get a ClientId and ClientSecret. You'll need to update the URLs for the various endpoints if you're using GitHub Enterprise. - -The only permission the app requires is Read-Only access to Email addresses. **You should not grant the app any other permissions**. - -The GitHub App's redirect URL will be `/_/api/v1/account/signin-github`. - -Example with the appropriate URLs for github.com: - -```json + "Authentication": { "GitHub": { - "ClientId": "Paste your client ID here", - "ClientSecret": "Paste your client secret here", + "ClientId": "Your_ClientID_from_step_2", + "ClientSecret": "Your_Client_Secret_from_step_2", "AuthorizationEndpoint": "https://github.com/login/oauth/authorize", "TokenEndpoint": "https://github.com/login/oauth/access_token", "UserInformationEndpoint": "https://api.github.com/user", "UserEmailsEndpoint": "https://api.github.com/user/emails" } + } +} ``` -##### Google - -Authenticate using Google authentication. You'll need to follow the process to create a [Google OAuth App](https://developers.google.com/identity/sign-in/web/sign-in) and you _absolutely_ must use an HTTPS endpoint for this app. This can make supporting the go/ dns serch domain tricky. Good luck. - -The Google App's redirect URL will be `/_/api/v1/account/signin-google`. - -```json - "Google": { - "ClientId": "Paste your client ID here", - "ClientSecret": "Paste your client secret here" - } -``` - -##### Okta - -Authenticate using an Okta account. You'll need to create an OAuth2 app in your Okta developer console (not the Admin console) following [the guide](https://developer.okta.com/docs/guides/implement-oauth-for-okta/create-oauth-app/). The only allowed grant type that is required is Authorization Code, the rest can be unchecked. - -The Okta App's redirect URL will be `/_/api/v1/account/signin-okta`. - -The OktaDomain needs to be your Okta account's Organization URL. You can find it by visiting your developer dashboard and looking in the top-right. Don't include a trailing slash! - -If you are using Custom Authorization Servers this may not work out of the box for you. Please file an issue or otherwise let me know that this is a feature you need and I can easily add in support for it. [The API URLs change slightly when using a Custom Auth Server.](https://developer.okta.com/docs/reference/api/oidc/) - -```json - "Okta": { - "ClientId": "Paste your client ID here", - "ClientSecret": "Paste your client secret here", - "OktaDomain": "https://some-okta-account.okta.com" - } -``` - -#### ProxyNetworks - -You'll probably be running NetGoLynx behind a TLS-terminating load balancer, such as in a Docker container in k8s or Nomad. When you do this you'll want to configure the app to listen to X-Forwarded headers so that it will properly handle TLS termination at your load balancer. - -You will also want to limit the allowed hosts from X-Forwarded-Host, usually set to the domain that you're running it on. So if you plan to run NetGoLynx at go.contoso.com you'll want to limit the AllowedHosts to just that domain like so: - -```json - "ProxyNetworks": { - "AllowedHosts": ["go.contoso.local", "go"] - }, -``` - -You'll probably want to make sure `go` is in the list. +## Configuration -Note: This list is seprate from the AllowedHosts above to be more flexible for your proxy configuration. Modern proxy systems such as Azure Front Door or AWS ALBs should be able to use the same values for both. +For more details on configuration see [the configuration docs](docs/configuration.md). -#### Complete Example +## Running on Nomad -An example configuration all together: +For an example jobspec for Nomad see [the configuration example](docs/nomad_cluster.md). -```json -{ - "ConnectionStrings": { - "DefaultConnection": "Data Source=redirects.db" - }, - "Logging": { - "LogLevel": { - "Default": "Warning" - } - }, - "AllowedHosts": "go.contoso.local;go", - "Authentication": { - "GitHub": { - "ClientId": "This is very not real", - "ClientSecret": "Also not real!", - "AuthorizationEndpoint": "https://github.com/login/oauth/authorize", - "TokenEndpoint": "https://github.com/login/oauth/access_token", - "UserInformationEndpoint": "https://api.github.com/user", - "UserEmailsEndpoint": "https://api.github.com/user/emails" - }, - "Okta": { - "ClientId": "This is not very real!", - "ClientSecret": "Still not real at all!", - "OktaDomain": "URL of your okta account with no trailing slash!" - } - }, - "ProxyNetworks": { - "AllowedHosts": ["go.contoso.local", "go"], - } -} -``` ### Health checks diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..b497254 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,194 @@ + +# Configuration + +This app is configured via appsettings.json. You can find an example (with details you should fill out) provided in the repo that you can either edit directly, or provide an override file in the same directory called appsettings.release.json. + +The basic structure looks like this and is explained section by section below. A complete example is at the end of the page. + +```json +{ + "ConnectionStrings": { + ... + }, + "Logging": { + "LogLevel": { + "Default": "Warning" + } + }, + "AllowedHosts": "*", + "Authentication": { + ... + }, + "ProxyNetworks": { + ... + } +} +``` + +## ConnectionStrings + +This section contains the connection string to connect to the appropriate SQL data store. There should only be one connection string at a time, the app doesn't know what to do with more than one. + +The database provider is chosen based on the name of the connection string key provided. You should provide a connection string in the format appropriate for the provide you select. The key must match with one of the implemented providers: + +* DefaultConnection: Alias for Sqlite. +* Sqlite: For a local (or volume-mapped) SQLite database. Good for home installations. + * Example: `"Sqlite": "Data Source=redirects.db"` +* SqlServer: For an MS SQL Server connection. + * Example: `"SQLServer": "server=127.0.0.1,1433;Database=netgolynx;User Id=SA;Password=yourStrong(!)Password"` +* Postgresql: For a PostgreSQL server connection. + * Example: `"PostgreSQL": "Host=localhost;Database=netgolynx;Username=postgres;Password=password"` + +NetGoLynx is configured to fire off database migrations automatically when the app starts to make sure the database is present and the schema is up to date. If there are any breaking schema changes that requires any kind of manual migration the release notes will walk you through the operations necessary. + +There is no mechanism for migrating between database providers at this time, taking a backup of the existing provider and loading it into a new provider after running NetGoLynx once may just work. Alternatively pester me to implement full import/export or something. + +## Logging + +Control how noisy the logging system is to the log file. + +## AllowedHosts + +A semicolon separated list of hosts the app will run as. Using `*` allows anything, which is probably too broad for production. + +If you wanted to host NetGoLynx on your domain at go.contoso.com you'd want to use: + +```json + "AllowedHosts": "go.contoso.local;go" +``` + +The list of hosts is semicolon separated and should not include port numbers. You'll probably want to have "go" in that list specifically if you want the go/link format to work. + +If you're running this behind a TLS-terminating load balancer on a secure network setting it to "*" may be fine. + +Note: This is seprate from the ProxyNetworks configuration, in case your load balancer configuration needs it. Modern load balancer proxy systems such as Azure Front Door or AWS ALBs should be able to use the same values for both. + +## Authentication + +Configure the available authentication providers. You must have at least one configured. You can configure as many as you want to support. + +### GitHub + +Authenticate using a GitHub app. You can configure this to be either GitHub.com or a GitHub Enterprise instance. Follow the [instructions for creating a GitHub App](https://docs.github.com/en/developers/apps/creating-a-github-app) and get a ClientId and ClientSecret. You'll need to update the URLs for the various endpoints if you're using GitHub Enterprise. + +The only permission the app requires is Read-Only access to Email addresses. **You should not grant the app any other permissions**. + +The GitHub App's redirect URL will be `/_/api/v1/account/signin-github`. + +Example with the appropriate URLs for github.com: + +```json + "GitHub": { + "ClientId": "Paste your client ID here", + "ClientSecret": "Paste your client secret here", + "AuthorizationEndpoint": "https://github.com/login/oauth/authorize", + "TokenEndpoint": "https://github.com/login/oauth/access_token", + "UserInformationEndpoint": "https://api.github.com/user", + "UserEmailsEndpoint": "https://api.github.com/user/emails" + } +``` + +### Google + +Authenticate using Google authentication. You'll need to follow the process to create a [Google OAuth App](https://developers.google.com/identity/sign-in/web/sign-in) and you _absolutely_ must use an HTTPS endpoint for this app. This can make supporting the go/ dns serch domain tricky. Good luck. + +The Google App's redirect URL will be `/_/api/v1/account/signin-google`. + +```json + "Google": { + "ClientId": "Paste your client ID here", + "ClientSecret": "Paste your client secret here" + } +``` + +### Okta + +Authenticate using an Okta account. You'll need to create an OAuth2 app in your Okta developer console (not the Admin console) following [the guide](https://developer.okta.com/docs/guides/implement-oauth-for-okta/create-oauth-app/). The only allowed grant type that is required is Authorization Code, the rest can be unchecked. + +The Okta App's redirect URL will be `/_/api/v1/account/signin-okta`. + +The OktaDomain needs to be your Okta account's Organization URL. You can find it by visiting your developer dashboard and looking in the top-right. Don't include a trailing slash! + +If you are using Custom Authorization Servers this may not work out of the box for you. Please file an issue or otherwise let me know that this is a feature you need and I can easily add in support for it. [The API URLs change slightly when using a Custom Auth Server.](https://developer.okta.com/docs/reference/api/oidc/) + +```json + "Okta": { + "ClientId": "Paste your client ID here", + "ClientSecret": "Paste your client secret here", + "OktaDomain": "https://some-okta-account.okta.com" + } +``` + +## ProxyNetworks + +You'll probably be running NetGoLynx behind a TLS-terminating load balancer, such as in a Docker container in k8s or Nomad. When you do this you'll want to configure the app to listen to X-Forwarded headers so that it will properly handle TLS termination at your load balancer. + +You will also want to limit the allowed hosts from X-Forwarded-Host, usually set to the domain that you're running it on. So if you plan to run NetGoLynx at go.contoso.com you'll want to limit the AllowedHosts to just that domain like so: + +```json + "ProxyNetworks": { + "AllowedHosts": ["go.contoso.local", "go"] + }, +``` + +You'll probably want to make sure `go` is in the list. + +Note: This list is seprate from the AllowedHosts above to be more flexible for your proxy configuration. Modern proxy systems such as Azure Front Door or AWS ALBs should be able to use the same values for both. + +### WebInterfaceHost + +When running NetGoLynx under multiple DNS names you will generally want to redirect users to a single HTTPS web interface that you've configured for your various OAuth providers. This includes redirecting http://go to a proper HTTPS address. + +NetGoLynx provides an internal mechanism for doing this. When you provide a WebInterfaceHost in the ProxyNetworks configuration it will redirect any UI or API requests that don't match the host and scheme you provide. + +For example, + +```json + "ProxyNetworks": { + "AllowedHosts": ["go.contoso.local", "go"], + "WebInterfaceHost": "https://go.contoso.local" + }, +``` + +will redirect `http://go/_/Account/Login` to `https://go.contoso.local/_Account/Login` automatically. This helps ensure users are using a secure connection when attempting to create links, or sign in. + +It _will not_ redirect `http://go/some_redirect` to the HTTPS url, it will simply return the redirect to the ultimate URL instead. If you'd like to ensure go/links always route through HTTPS you might want to look into the [companion browser extension](https://github.com/Cellivar/NetGoLynx-Extension) which will internally handle the redirect from `http://go/` to `https://`. + +Note that you can provide a port here if you're, for some reason, running HTTPS over something other than 443. + +## Complete Example + +An example configuration all together: + +```json +{ + "ConnectionStrings": { + "Sqlite": "Data Source=redirects.db" + }, + "Logging": { + "LogLevel": { + "Default": "Warning" + } + }, + "AllowedHosts": "go.contoso.local;go", + "Authentication": { + "GitHub": { + "ClientId": "Put in your ClientID here", + "ClientSecret": "And put in your ClientSecret here", + "AuthorizationEndpoint": "https://github.com/login/oauth/authorize", + "TokenEndpoint": "https://github.com/login/oauth/access_token", + "UserInformationEndpoint": "https://api.github.com/user", + "UserEmailsEndpoint": "https://api.github.com/user/emails" + }, + "Okta": { + "ClientId": "Put in your Client ID here", + "ClientSecret": "And your client secret here", + "OktaDomain": "https://okta.contoso.com" + } + }, + "ProxyNetworks": { + "AllowedHosts": ["go.contoso.local", "go"], + "WebInterfaceHost": "https://go.contoso.local" + } +} +``` diff --git a/docs/nomad_cluster.md b/docs/nomad_cluster.md new file mode 100644 index 0000000..a1d193b --- /dev/null +++ b/docs/nomad_cluster.md @@ -0,0 +1,138 @@ +# Running on Nomad + +At home I have a Nomad cluster running on a few machines, this is where I run (and test!) NetGoLynx. Nomad is [a cluster management system](https://www.nomadproject.io/) built by HashiCorp that is generally lighter and easier to deal with than Kubernetes. I also use Consul and Vault for storage of configuration values and service discovery. + +## General overview + +NetGoLynx only needs a database of some sort, it doesn't need to mount a volume unless you're using that volume to store the database (like with sqlite). Using an external database is the preferred hosting method as it gives you better control over the backup of the database. For this I use Postgres running elsewhere on my cluster. + +Outside of the database NetGoLynx can just be run somewhere. It's designed to have a load balancer do TLS termination so you can handle TLS certificates via your favorite service instead of having to import them into the running service. For this I use Fabio with a Let's Encrypt certificate stored in Vault. Fabio makes use of a specific set of tags in the Service stanza, which is why you see + +Obviously NetGoLynx will require DNS entries. I have dnsmasq running as a service in my cluster that is managed automatically with Consul. You'll see the service stanza has some odd tags, this is why. + +To update the job file I make use of Terraform. Terraform lets me store the configuration of various jobs in my cluster as templates in Git and as state in Vault. This example doesn't make use of Terraform to simplify things. You could simplify a number of the parameters if you used Terraform instead of this raw jobpsec. + +### Nomad Job Spec + +netgolnyx.nomad: + +```hcl +job "netgolynx" { + datacenters = ["your_datacenter_name"] + + group "main" { + count = 1 + + network { + port "http" { to = 80 } + } + + task "netgolynx-ui"{ + driver = "docker" + config { + image = "netgolynx" + ports = ["http"] + volumes = [ + "secrets/appsettings.json:/app/appsettings.json" + ] + } + + # The vault policy you use should have read access to the secret path + # used in the template below. + vault { + policies = [ "vault_policy_for_secrets" ] + change_mode = "restart" + } + + constraint { + attribute = "${attr.cpu.arch}" + value = "amd64" + } + + # This is the appsettings file, templatized to use Consul-Template sytanx + # The values will be populated from Vault, where I store the credentials + # for various things, including this app. + template { + data = <