diff --git a/canary-checker/docs/concepts/artifacts.md b/canary-checker/docs/concepts/artifacts.md index f5eb8bcb..c15a429f 100644 --- a/canary-checker/docs/concepts/artifacts.md +++ b/canary-checker/docs/concepts/artifacts.md @@ -29,7 +29,7 @@ The following checks support archiving artifacts The only configuration required is to provide the path(s) of the artifacts generated by the check. -### Ex1. Archiving `/tmp/results/` directory +### Archiving `/tmp/results/` directory For the following script in an exec check diff --git a/canary-checker/docs/concepts/health-evaluation2.md b/canary-checker/docs/concepts/health-evaluation2.md deleted file mode 100644 index eabc588d..00000000 --- a/canary-checker/docs/concepts/health-evaluation2.md +++ /dev/null @@ -1,34 +0,0 @@ -# Health Evaluation - -All checks expose a health stats of passing (Green) or failing (Red), for example the HTTP check fails if the status is `200 >= code < 299` -these default behaviors can be changed using the `test` field: - -In the example below the http check will fail if the SSL certificate expiry age is below 7 days - -```yaml title=http-check-expr.yaml -apiVersion: canaries.flanksource.com/v1 -kind: Canary -metadata: - name: http-check-expr -spec: - interval: 30 - http: - - name: http pass response 200 status code - endpoint: https://httpbin.demo.aws.flanksource.com/status/200 - test: - expr: "code in [200,201,301] and sslAge < Duration('7d')" -``` - -Expressions must return a boolean type object and can use all 3 expression languages: -| Expression Type | Example | Reference | -| --------------- | ----------------------------------------------------- | ------------------------------------------ | -| `expr` | `code in [200]` | [CEL](/scripting/go-template) | -| `javascript` | `code == 200 ` | [Javascript](/scripting/javascript) | -| `template` | `{{if eq .result.code 200 }}true{{else}}false{{end}}` | [Go Text Template](/scripting/go-template) | - - - -## Variables - -Each check exposes different variables to use in the `test` expression, See the **Result Variables** section for each check. - diff --git a/docs/incidents/concepts/responders.md b/docs/incidents/concepts/responders.md index 2cdb221e..00ba6886 100644 --- a/docs/incidents/concepts/responders.md +++ b/docs/incidents/concepts/responders.md @@ -41,8 +41,8 @@ responder_clients: | Field | Description | Scheme | Required | | ---------- | ------------- | ----------------------------------------------------------------------------- | -------- | | `url` | Jira url | `string` | `true` | -| `username` | Jira username | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | `true` | -| `password` | Jira password | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | `true` | +| `username` | Jira username | *EnvVar* | `true` | +| `password` | Jira password | *EnvVar* | `true` | | `defaults` | Jira defaults | `string` | `option` | | `values` | Jira values | `string` | `option` | @@ -53,7 +53,7 @@ responder_clients: | `tenant_id` | MS Planner tenant id | `string` | `true` | | `client_id` | MS Planner client id | `string` | `true` | | `group_id` | MS Planner group id | `string` | `true` | -| `username` | MS Planner username | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | `true` | -| `password` | MS Planner password | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | `true` | +| `username` | MS Planner username | *EnvVar* | `true` | +| `password` | MS Planner password | *EnvVar* | `true` | | `defaults` | MS Planner defaults | `map[string]string` | `option` | | `values` | MS Planner values | `map[string]string` | `option` | diff --git a/mission-control/docs/architecture.md b/mission-control/docs/architecture.md index 0c1486fe..04c2cdcd 100644 --- a/mission-control/docs/architecture.md +++ b/mission-control/docs/architecture.md @@ -9,11 +9,11 @@ -Mission Control has a micro-service architecture with a shared data source with multiple deployment models. +Mission Control has a micro-service architecture with a shared data source with multiple deployment models. -1. CLI +1. CLI 2. Kubernetes (Helm Chart) 3. SaaS @@ -21,46 +21,27 @@ Mission Control has a micro-service architecture with a shared data source with -Communication Model +## Communication Model Communication between services happen in 3 ways: -1. **Database** - A shared database with interface library enables services to query the data owned by other services by directly hitting the database using an interface provided by a shared library +1. **Database** - A shared database with interface library enables services to query the data owned by other services by directly hitting the database using an interface provided by a shared library 2. **Messaging** - A postgres based message bus is used, database triggers insert events into queues which are then consumed by various services 3. **HTTP/REST** - This model is primarily used when the service need to interact with services outside the DB (e.g. the APM hub needs to connect to log stores to retrieve logs) +## Postgres +Postgres is the only data store used by Mission Control and is also used as a JSON document database and message queue. This limits the dependencies and complexity especially when self-hosting. -Shared Data Source - - - -All microservices use a shared database and model via the https://github.com/flanksource/duty project, this provides the following benefits: - - - -* Each microservice can update the database directly with full referential integrity between tables owned by other services -* - - +All services use a shared database and model via the [duty](https://github.com/flanksource/duty) project, this provides the following benefits: +* Limit RPC calls improving latency and performance +* Services can run with slightly different versions of the library, limiting the need for coordinated migrations +* Library updates happen automatically using dependabot ## Kubernetes & Gitops - - Mission control is kubernetes-native with all configuration being possible by Custom Resource Definition (CRD's) The single source of truth is still the database, the operators only function is to synchronize CRD's into the database and update the CRD status periodically. - - - - - - - - - - - diff --git a/mission-control/docs/config-db/concepts/transform.md b/mission-control/docs/config-db/concepts/transform.md index c519a63a..cd86607d 100644 --- a/mission-control/docs/config-db/concepts/transform.md +++ b/mission-control/docs/config-db/concepts/transform.md @@ -3,7 +3,7 @@ Transformation can be performed after the configs have been scraped from the target. This can be useful when you want to - hide/remove sensitive data from the scraped configuration (e.g. passwords, tokens, etc.) -- transform the scraped configuration using Javascript and [Go templates](https://pkg.go.dev/text/template). +- transform the scraped configuration using CEL - remove certain fields from the scraped configuration | Field | Description | Scheme | Required | @@ -12,7 +12,7 @@ Transformation can be performed after the configs have been scraped from the tar | `javascript` | Specify javascript syntax for script | `string` | | | `jsonpath` | Specify JSONPath | `string` | | | `expr` | Specify Cel expression | `string` | | -| `change` | Apply transformaion on the scraped changes | [`[]TransformChange`](#transformchange) | | +| `change` | Apply transformaion on the scraped changes | [`[]Changes`](#changes) | | | `exclude` | Fields to remove from the config, useful for removing sensitive data and fields | [`[]Exclude`](#exclude) | | | | that change often without a material impact i.e. Last Scraped Time | | | | [`mask`](#mask) | Specify configurations to replace sensitive fields with hash functions or static string. | [`[]Mask`](./mask.md) | | @@ -154,7 +154,7 @@ file: This configuration specifies 2 different masks. The first one will replace the value of the field `password` with the md5sum of the value. The second one will replace the value of the field `secret` with `***`. :::info -All the masks will be applied in the order they are specified in the configuration file. +Masks are applied in the order they are specified in the configuration file. ::: ### Supported hash functions @@ -171,25 +171,23 @@ At the moment, only `md5sum` is supported. More hash functions will be added in `Config DB` allows selectively applying masks to certain types of configs. So you could apply a mask to all the `Config` types and another mask to all the `Secret` types. -### TransformChange +## Changes | Field | Description | Scheme | Required | | --------- | ----------------------------------------------------------------- | ---------- | -------- | -| `exclude` | Exclude is a list of CEL expressions that excludes a given change | `[]string` | | +| `exclude` | A list of CEL expressions that excludes a given change | `[]string` | | The scraped changes can be accessed using the `details` field. -#### Examples: -1. Excluding canary pass/fail events - -```yaml title="" +```yaml title="exclude-canary-pass-fail.yaml" spec: kubernetes: - clusterName: local-kind-cluster transform: change: exclude: + # Canary checker events are handled natively, no need to import the K8S events - 'details.source.component == "canary-checker" && details.reason == "Failed"' - 'details.source.component == "canary-checker" && details.reason == "Succeeded"' ``` @@ -200,7 +198,6 @@ This feature allows you to specify custom creation and deletion times for config You'll be making use of the `createFields` and `deleteFields` fields that are supported by all the scrapers. They are both a list of [JSONPath expression](../concepts/templating.md#jsonpath) and are used to extract the created/deleted time of the config item from the scraped configuration. If multiple fields are specified, the first non-empty value will be used. -### Example Consider the following configuration file @@ -233,13 +230,12 @@ aws: When the scraped configuration is saved in the database, the created date will be `2017-03-06T21:04:11Z` instead of being the current time and the deleted date will be `2017-04-04T15:04:05Z` instead of being empty. -### Custom timestamp format +### Timestamp Format By default, the timestamp format is RFC3339 (`2006-01-02T15:04:05Z07:00`). If the scraped configuration follows a different timestamp format, then you can specify it in the `timestampFormat` field. The format is specified using the [Go time format](https://golang.org/pkg/time/#Time.Format). In the above example if the value of `made_at` was `2017/03/06 21:04:11Z`, then the `timestampFormat` file would look like this ```yaml ---- timestampFormat: '2006/01/02 15:04:05Z' ``` diff --git a/mission-control/docs/config-db/scrapers/kubernetes-file.md b/mission-control/docs/config-db/scrapers/kubernetes-file.md index 4e09f438..9d191bac 100644 --- a/mission-control/docs/config-db/scrapers/kubernetes-file.md +++ b/mission-control/docs/config-db/scrapers/kubernetes-file.md @@ -39,9 +39,9 @@ The KubernetesFile config type is used to scrape configurations contained in you | `deleteFields` | DeleteFields is a JSONPath expression used to identify the deleted time of the config. If multiple fields are specified, the first non-empty value will be used | `[]string` | `false` | | `selector` | Specify Kubernetes resource for configuration based on `namespace`, `kind`, `name` and more. | [`ResourceSelector`](./resourceselector) | yes | | `container` | Set container name | `string` | | -| `files` | Specify path to file contained in Pod | `[]PodFile` | | +| `files` | Specify path to file contained in Pod | `[]File` | | -### PodFile +### File | Field | Description | Scheme | Required | | -------- | --------------------------------- | ---------- | -------- | diff --git a/mission-control/docs/config-db/scrapers/kubernetes.md b/mission-control/docs/config-db/scrapers/kubernetes.md index deafa6f8..51968166 100644 --- a/mission-control/docs/config-db/scrapers/kubernetes.md +++ b/mission-control/docs/config-db/scrapers/kubernetes.md @@ -44,32 +44,30 @@ kubernetes: ### Kubernetes -| Field | Description | Scheme | Required | -| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | -------- | -| `id` | A static value or JSONPath expression to use as the ID for the resource. | `string` | `true` | -| `name` | A static value or JSONPath expression to use as the Name for the resource. Default value is the `id`. | `string` | `false` | -| `items` | A JSONPath expression to use to extract individual items from the resource | `string` | `false` | -| `type` | A static value or JSONPath expression to use as the type for the resource. | `string` | `true` | -| `transform` | Specify field to transform result | [`Transform`](../concepts/transform.md) | `false` | -| `format` | Format of config item, defaults to JSON, available options are JSON | `string` | `false` | -| `timestampFormat` | TimestampFormat is a Go time format string used to parse timestamps in createFields and DeletedFields. If not specified, the default is `RFC3339`. | `string` | `false` | -| `createFields` | CreateFields is a list of JSONPath expression used to identify the created time of the config. If multiple fields are specified, the first non-empty value will be used | `[]string` | `false` | -| `deleteFields` | DeleteFields is a JSONPath expression used to identify the deleted time of the config. If multiple fields are specified, the first non-empty value will be used | `[]string` | `false` | -| `clusterName` | Specify cluster name | `string` | | -| `namespace` | Specify namespace for scraping of Kubernetes resources | `string` | | -| `useCache` | Specify boolean value to toggle fetching results from Kube-apiserver or fetch response from etcd | `bool` | | -| `allowIncomplete` | | `bool` | | -| `scope` | Specify scope for scrape. e.g `cluster` for scraping at Cluster level | `string` | | -| `since` | Set time constraint for scraping resources within the set period | `string` | | -| `selector` | Specify Kubernetes resource to scrape based on selector. e.g `matchLabels` | `string` | | -| `fieldSelector` | Specify Kubernetes resource based on value of resource fields. e.g `status.Phase=Running` | `string` | | -| `maxInflight` | Set value for maximum inflight requests | `int64` | | -| `exclusions` | Specify Kubernetes resources to be excluded from scraping | `[]string` | | -| **`kubeconfig`** | Specify kubeconfig for access to your Kubernetes Cluster | [`kommons.EnvVar`](https://pkg.go.dev/github.com/flanksource/kommons#EnvVar) | yes | -| `event` | Specify configuration to handle Kubernetes events. See [**KubernetesEvent**](#kubernetesevent) | [`KubernetesEvent`](#kubernetesevent) | yes | -| `relationships` | Create relationships between kubernetes objects. See [**KubernetesRelationships**](#kubernetesrelationships) | [`[]KubernetesRelationships`](#kubernetesrelationships) | `false` | - -### KubernetesEvent +| Field | Description | Scheme | Required | +| ----------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | -------- | +| `id` | A static value or JSONPath expression to use as the ID for the resource. | `string` | `true` | +| `name` | A static value or JSONPath expression to use as the Name for the resource. Default value is the `id`. | `string` | `false` | +| `items` | A JSONPath expression to use to extract individual items from the resource | `string` | `false` | +| `type` | A static value or JSONPath expression to use as the type for the resource. | `string` | `true` | +| `transform` | Specify field to transform result | [`Transform`](../concepts/transform.md) | `false` | +| `format` | Format of config item, defaults to JSON, available options are JSON | `string` | `false` | +| `timestampFormat` | TimestampFormat is a Go time format string used to parse timestamps in createFields and DeletedFields. If not specified, the default is `RFC3339`. | `string` | `false` | +| `createFields` | CreateFields is a list of JSONPath expression used to identify the created time of the config. If multiple fields are specified, the first non-empty value will be used | `[]string` | `false` | +| `deleteFields` | DeleteFields is a JSONPath expression used to identify the deleted time of the config. If multiple fields are specified, the first non-empty value will be used | `[]string` | `false` | +| `clusterName` | Specify cluster name | `string` | | +| `namespace` | Specify namespace for scraping of Kubernetes resources | `string` | | +| `useCache` | Specify boolean value to toggle fetching results from Kube-apiserver or fetch response from etcd | `bool` | | +| `scope` | Specify scope for scrape. e.g `cluster` for scraping at Cluster level | `string` | | +| `since` | Set time constraint for scraping resources within the set period | `string` | | +| `selector` | Specify Kubernetes resource to scrape based on selector. e.g `matchLabels` | `string` | | +| `fieldSelector` | Specify Kubernetes resource based on value of resource fields. e.g `status.Phase=Running` | `string` | | +| `exclusions` | Specify Kubernetes resources to be excluded from scraping | `[]string` | | +| **`kubeconfig`** | Specify kubeconfig for access to your Kubernetes Cluster | [`kommons.EnvVar`](https://pkg.go.dev/github.com/flanksource/kommons#EnvVar) | yes | +| `event` | Specify configuration to handle Kubernetes events. | [`Event`](#sevent) | yes | +| `relationships` | Create relationships between kubernetes objects. | [`[]Relationships`](#srelationships) | `false` | + +### Events `Config DB` maps Kubernetes Event objects to config changes unlike other objects that are mapped to config items. This configuration allows you to exclude or include the Kubernetes Event objects based on the reason. @@ -96,11 +94,11 @@ We support static values, expressions and label lookups to find the parent | Field | Description | Scheme | Required | | ----------- | -------------------------------- | --------------------------------------------------------------- | -------- | -| `kind` | `kind` of Kubernetes Object | [`KubernetesRelationshipLookup`](#kubernetesrelationshiplookup) | `true` | -| `name` | `name` of Kubernetes Object | [`KubernetesRelationshipLookup`](#kubernetesrelationshiplookup) | `true` | -| `namespace` | `namespace` of Kubernetes Object | [`KubernetesRelationshipLookup`](#kubernetesrelationshiplookup) | `true` | +| `kind` | `kind` of Kubernetes Object | [`RelationshipLookup`](#relationshiplookup) | `true` | +| `name` | `name` of Kubernetes Object | [`RelationshipLookup`](#relationshiplookup) | `true` | +| `namespace` | `namespace` of Kubernetes Object | [`RelationshipLookup`](#relationshiplookup) | `true` | -#### KubernetesRelationshipLookup +#### RelationshipLookup | Field | Description | Scheme | Required | | ------- | -------------------------------------------------- | -------- | -------- | @@ -108,9 +106,9 @@ We support static values, expressions and label lookups to find the parent | `expr` | CEL Expression to evaluate | `string` | | | `label` | Label key containing the value of the the resource | `string` | | -#### Kubernetes Relationship Example +#### -```yaml +```yaml title="relationship-example.yaml" kubernetes: - clusterName: 'eks' ... diff --git a/mission-control/docs/images/how-it-works.svg b/mission-control/docs/images/how-it-works.svg new file mode 100644 index 00000000..23521874 --- /dev/null +++ b/mission-control/docs/images/how-it-works.svg @@ -0,0 +1,3 @@ + + + 1x 1x 1x 1x Icon-devops-261
Catalog
Playbooks
Health
GitOps
Topology
 \ No newline at end of file diff --git a/mission-control/docs/index.md b/mission-control/docs/index.md index c976c8ca..7de1a790 100644 --- a/mission-control/docs/index.md +++ b/mission-control/docs/index.md @@ -1,47 +1,21 @@ --- slug: / -title: Overview +title: Flanksource Mission Control hide_title: true # hide_table_of_contents: true # pagination_next: null # pagination_prev: null --- -![](./images/flanksource.svg) -### Flanksource Mission Control +Flanksource Mission Control is an Internal Developer Platform focused on GitOps platforms. -Flanksource Mission Control is an internal developer platform that helps teams to understand and operate complex systems easier. Mission Control includes the following features: +![](./images/how-it-works.svg) -- A real-time map or dashboard showing the location and status of resources, including personnel, equipment and facilities. -- A messaging system for communication between the incident commander and other members of the response team. -- Tools for tracking and managing incident-related tasks, including assigning and prioritizing tasks, tracking progress and recording outcomes. -- Tools for tracking and monitoring health checks. -- Configuration tool that enables you to view and search the change history of your configuration across multiple dimensions (node, zone, environment, application, technology, etc). -- Integration with other systems such as; scrape data from network device in order to help incident commanders make informed decisions. -## Config DB +* [Catalog](./config-db/overview) - Catalog all your infrastructure, applications, pipelines and configuration into a schema-less JSON database, with automatic change tracking. +* [Playbooks](./playbooks/overview) - Self-Service portal for day 0-2 operations like provisioning a new namespace, restarting a deployment, or updating files in git repositories. Playbooks also be triggered via webhooks and events. +* [Health Checks](./canary-checker/overview) - RAG (red, amber, green) statuses across infrastructure, applications and commercial off the shelf software, With alert aggregation, synthethic application and infrastructure checks. +* [Topology](./topology/overview) - Visualize complex systems using a multi-dimensional hierarchical cards. +* [Notifications](./notifications/overview) - Send notifcations during playbook execution or based on any event fired from catalog, health or topology changes. -Config DB is an open source tool that allows developers to easily configure, scrape and manage data within their application. It provides a user-friendly interface for setting up and managing database connections, as well as configuring tables and fields within the database. This tool can be particularly useful for developers who need to quickly set up a database for their application, without having to manually write complex SQL queries or code. It also allows for easy updates and changes to the database as the application evolves, making it a useful tool for maintaining a healthy and efficient database. - - - -## Health Checks - -Understand the health of complex services at a glance with red, amber, green statuses which leverage active/passive health checks and consolidated alerts from Prometheus, AWS, Dynatrace, etc. - -Canary checker is an open source tool that allows users to monitor the status of their canaries (server monitoring tools) in real-time. Canary checker allows users to set up alerts for when their canaries go down or encounter any issues, giving them the ability to quickly respond and fix any potential problems before they become more significant. It also provides users with detailed logs and analytics of their canary activity, giving them valuable insights into the performance and reliability of their systems. Canary checker is an essential tool for anyone who relies on canaries for server monitoring and maintenance, helping them to ensure the stability and uptime of their systems. - -One potential use of Canary Checker is, if you want to get the cert expiry dates for your URLs and get warn when we are X number of days from the expiry date. - -With Mission Control up and running, one can have a better understanding of their infrastructure, which helps in planning of disaster recovery and reducing downtime to the minimum. - -## - -Playbooks - -Empower developers to be more self-sufficient without the need to become experts in the Cloud and Kubernetes. - -Run playbooks automatically on failing health checks/ alerts -Implement security best practices of least privilege and just in time (JIT) access -Use the built-In library of actions including HTTP, SQL, kubectl, AWS CLI or run any custom code from Git diff --git a/mission-control/docs/playbooks/actions/exec.md b/mission-control/docs/playbooks/actions/exec.md index b7e3327c..0df4e955 100644 --- a/mission-control/docs/playbooks/actions/exec.md +++ b/mission-control/docs/playbooks/actions/exec.md @@ -41,7 +41,7 @@ spec: | `script` | Script can be an inline script or a path to a script that needs to be executed. Executed via Powershell on windows and via bash on Darwin and Linux. | _string_ | `true` | `true` | | `connections` | Connections for some CLIs | [`ExecConnection`](#exec-connection) | | | | `artifacts` | Specify what artifacts generated by the exec action needs to be saved | [`[]Artifact`](#artifact) | | | -| `env` | Specify environment variables that are available to exec processes | [`[]types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | | +| `env` | Specify environment variables that are available to exec processes | []*EnvVar* | | | | `checkout` | Checkout details the git repository that should be mounted to the process | [`[]GitCheckout`](#git-checkout) | | | ### Exec Connection @@ -70,10 +70,10 @@ For authentication, either provide the connection name or the basic auth or the | Field | Description | Type | Required | | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | -------- | | `url` | Git repository URL. | `string` | `true` | -| `connection` | Specify the connection name to use for git authentication (if required) | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | -| `username` | Git auth username. | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | -| `password` | Git auth password. | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | -| `certificate` | Git auth certificate. | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | +| `connection` | Specify the connection name to use for git authentication (if required) | *EnvVar* | | +| `username` | Git auth username. | *EnvVar* | | +| `password` | Git auth password. | *EnvVar* | | +| `certificate` | Git auth certificate. | *EnvVar* | | | `destination` | Destination is the full path to where the contents of the URL should be downloaded to. If left empty, the sha256 hash of the URL will be used as the dir name. | `string` | | ## Templating diff --git a/mission-control/docs/playbooks/actions/http.md b/mission-control/docs/playbooks/actions/http.md index 2ab05042..99da9b3d 100644 --- a/mission-control/docs/playbooks/actions/http.md +++ b/mission-control/docs/playbooks/actions/http.md @@ -40,12 +40,12 @@ spec: | -------------- | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | -------- | ----------- | | `connection` | Connection name. e.g. connection://http/google | `string` | | | `url` | Url to make the request to | `string` | | -| `username` | Username to authenticate with | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | -| `password` | Password to authenticate with | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | +| `username` | Username to authenticate with | *EnvVar* | | +| `password` | Password to authenticate with | *EnvVar* | | | `method` | HTTP method to use _(default: GET)_ | `string` | | | `ntlm` | NTLM when set to true will perform authentication using NTLM v1 protocol | `bool` | | | `ntlmv2` | NTLM when set to true will perform authentication using NTLM v2 protocol | `bool` | | -| `headers` | Header fields to be used in the request | [`[]types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | +| `headers` | Header fields to be used in the request | []*EnvVar* | | | `body` | Request Body Contents | `string` | | `true` | | `templateBody` | When set to true the request body is templated _(default: `false`)_. [Read more ...](../concepts/templating.md) | `bool` | | diff --git a/mission-control/docs/playbooks/references/connections.md b/mission-control/docs/playbooks/references/connections.md index 0d2acdf1..a484d69b 100644 --- a/mission-control/docs/playbooks/references/connections.md +++ b/mission-control/docs/playbooks/references/connections.md @@ -6,8 +6,8 @@ | Field | Description | Type | Required | | ---------------- | --------------- | ----------------------------------------------------------------------------- | -------- | | `connectionName` | Connection name | `string` | | -| `accessKey` | Access key | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | -| `secretKey` | Secret key | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | +| `accessKey` | Access key | *EnvVar* | | +| `secretKey` | Secret key | *EnvVar* | | | `region` | Region | `string` | | | `endpoint` | Endpoint | `string` | | | `skipTLSVerify` | Skip TLS verify | `bool` | | @@ -20,13 +20,13 @@ | ---------------- | --------------- | ----------------------------------------------------------------------------- | -------- | | `connectionName` | Connection name | `string` | | | `endpoint` | Endpoint | `string` | | -| `credentials` | Credentials | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | +| `credentials` | Credentials | *EnvVar* | | ### Azure Connection | Field | Description | Type | Required | | ---------------- | --------------- | ----------------------------------------------------------------------------- | -------- | | `connectionName` | Connection name | `string` | | -| `clientID` | Client ID | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | -| `clientSecret` | Client Secret | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | | +| `clientID` | Client ID | *EnvVar* | | +| `clientSecret` | Client Secret | *EnvVar* | | | `tenantID` | Tenant ID | `string` | | diff --git a/mission-control/docs/playbooks/triggers/webhook.md b/mission-control/docs/playbooks/triggers/webhook.md index a2d770c1..c0f84518 100644 --- a/mission-control/docs/playbooks/triggers/webhook.md +++ b/mission-control/docs/playbooks/triggers/webhook.md @@ -60,8 +60,8 @@ If multiple authentication methods are specified, all of them will be used. | Field | Description | Scheme | Required | | ---------- | ----------- | ----------------------------------------------------------------------------- | -------- | -| `username` | Username | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | `true` | -| `password` | Path | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | `true` | +| `username` | Username | *EnvVar* | `true` | +| `password` | Path | *EnvVar* | `true` | #### Github Auth @@ -69,13 +69,13 @@ If the webhook is being called by Github, you can use [GitHub's webhook verifica | Field | Description | Scheme | Required | | ------- | ---------------------------- | ----------------------------------------------------------------------------- | -------- | -| `token` | Secret token for the webhook | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | `true` | +| `token` | Secret token for the webhook | *EnvVar* | `true` | #### Svix Auth | Field | Description | Scheme | Required | | ----------------- | ---------------------------------------------------- | ----------------------------------------------------------------------------- | -------- | -| `secret` | Signing secret | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | `true` | +| `secret` | Signing secret | *EnvVar* | `true` | | `verifyTimestamp` | Specify the tolerance for the timestamp verification | `string` | | #### JWT Auth diff --git a/mission-control/docs/security.md b/mission-control/docs/security.md index f76df344..a7fb8d1c 100644 --- a/mission-control/docs/security.md +++ b/mission-control/docs/security.md @@ -1 +1,60 @@ -# Security \ No newline at end of file +# Security + +At Flanksource, security isn't just a feature; it's the foundation of the design that went into the Mission Control platform. We've meticulously designed every aspect of our internal developer platform to meet the stringent requirements of security teams. + +## Secure SDLC + +Flanksource follows a secure SDLC + +1. Code scanning using Github [CodeQL](https://codeql.github.com/) +2. Merge blocking unit and integration tests using Github Actions +3. Branch protection to prevent history rewrite +4. Automatic dependency scanning and updates with Github [Dependabot](https://github.com/features/security/software-supply-chain) +5. Project CI/CD compliance scanning using [OpenSSF Scorecards](https://securityscorecards.dev/) +6. Automated build and publishing of artifacts +7. CI Supply Chain Runtime Scanner using [Step Harden Runner](https://github.com/step-security/harden-runner) + +## Secret Management + +All flanksource projects are built with secure secret management in mind, where possible secrets are automatically generated at install time and saved to Kubernetes Secrets, Pre-existing secrets are read from environment variables/files supplied by end users using Kubernetes Secrets or Helm Values + +Role based IAM identity is possible and preferred for Kubernetes, AWS, GKE and Azure. + +## Source Open + +All the code for Mission Control self-hosted is publicly available and free to use for non-prod purposes, Enabling any security researcher to review the source code and perform white-box testing. + + The security scan results for all projects are available in the links below. + +## Security Dashboard + +| Project | Description | License | Scorecard | CII Best Practises | +| -------------------- | ------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| Mission Control | Primary microservice and orchestrator | Static Badge | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/flanksource/mission-control/badge)](https://securityscorecards.dev/viewer/?uri=github.com/flanksource/mission-control) | | +| Canary Checker | Health checks and topology scanning | GitHub License | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/flanksource/canary-checker/badge)](https://securityscorecards.dev/viewer/?uri=github.com/flanksource/canary-checker) | CII Best Practices | +| Config DB | Catalog Scraper | Static Badge | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/flanksource/config-db/badge)](https://securityscorecards.dev/viewer/?uri=github.com/flanksource/config-db) | | +| Duty | Data Access Library | GitHub License | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/flanksource/duty/badge)](https://securityscorecards.dev/viewer/?uri=github.com/flanksource/duty) | | +| Is-Healthy | Library for get health status of Kubernetes objects | GitHub License | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/flanksource/is-healthy/badge)](https://securityscorecards.dev/viewer/?uri=github.com/flanksource/is-healthy) | | +| Gomplate | Go and CEL templating library | GitHub License | ![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/flanksource/gomplate/badge) | | +| Flanksource UI | Dashboard | Static Badge | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/flanksource/flanksource-ui/badge)](https://securityscorecards.dev/viewer/?uri=github.com/flanksource/flanksource-ui) | | +| **External Dependencies** | | | | | +| PostgREST | REST API for Daytabase | GitHub License | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/PostgREST/postgrest/badge)](https://securityscorecards.dev/viewer/?uri=github.com/PostgREST/postgrest) | | +| Kratos (Self-Hosted) | 3rd Party Application for Authentication | GitHub License | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/ory/kratos/badge)](https://securityscorecards.dev/viewer/?uri=github.com/ory/kratos) | | +| Clerk (SaaS) | 3rd Party Service for Authentication | | [Docs](https://clerk.com/docs/security/overview) | | + +## Reporting a Vulnerability + +If you discover any security vulnerabilities within this project, please report them to our team immediately. We appreciate your help in making this project more secure for everyone. + +To report a vulnerability, please follow these steps: + +1. **Email**: Send an email to our security team at [security@flanksource.com](mailto:security@flanksource.com) with a detailed description of the vulnerability. +2. **Subject Line**: Use the subject line "Security Vulnerability Report" to ensure prompt attention. +3. **Information**: Provide as much information as possible about the vulnerability, including steps to reproduce it and any supporting documentation or code snippets. +4. **Confidentiality**: We prioritize the confidentiality of vulnerability reports. Please avoid publicly disclosing the issue until we have had an opportunity to address it. + +Our team will respond to your report as soon as possible and work towards a solution. We appreciate your responsible disclosure and cooperation in maintaining the security of this project. + +Thank you for your contribution to the security of this project! + +**Note:** This project follows responsible disclosure practices.