(ASP-3778) Review and update current Jobbergate's documentation (#398)

* docs: review the documentation Fix spelling Fix grammar Apply linting Check links Remove outdated content
omnivector-solutions · Oct 31, 2023 · 52c0446 · 52c0446
1 parent a59ade1
commit 52c0446
Show file tree

Hide file tree

Showing 16 changed files with 232 additions and 499 deletions.
diff --git a/jobbergate-agent/README.md b/jobbergate-agent/README.md
@@ -4,44 +4,43 @@
 
 To install the package from Pypi simply run `pip install jobbergate-agent`.
 
-
 ## Setup parameters
 
 1. Setup dependencies
-  You can use whenever dependency manager you want to. Just run the command below (and the ones following) on behalf of the manager you prefer.
 
-  ```bash
-  make dependencies
-  ```
+    Dependencies and environment are managed in the project by [Poetry](https://python-poetry.org/). To initiate the development environment run:
 
-2. Setup `.env` parameters
+    ```bash
+    make install
+    ```
 
-  ```bash
-  JOBBERGATE_AGENT_BASE_API_URL="<base-api-url>"
-  JOBBERGATE_AGENT_BASE_SLURMRESTD_URL="<slurmrestd-endpoint>"
-  JOBBERGATE_AGENT_X_SLURM_USER_NAME="<slurmrestd-user-name>"
-  JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_PATH="/path/to/the/jwt/secret/key"
-  JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_STRING="jwt-secret-key-in-plain-text"
-  JOBBERGATE_AGENT_SENTRY_DSN="<sentry-dsn-key>"
-  JOBBERGATE_AGENT_OIDC_DOMAIN="<OIDC-domain>"
-  JOBBERGATE_AGENT_OIDC_AUDIENCE="<OIDC-audience>"
-  JOBBERGATE_AGENT_OIDC_CLIENT_ID="<OIDC-app-client-id>"
-  JOBBERGATE_AGENT_OIDC_CLIENT_SECRET="<OIDC-app-client-secret>"
-  ```
+2. Setup `.env` parameters
 
-  NOTE: `JOBBERGATE_AGENT_SENTRY_DSN` is optional. If you do not pass it the agent understands Sentry will not be used.
+    ```bash
+    JOBBERGATE_AGENT_BASE_API_URL="<base-api-url>"
+    JOBBERGATE_AGENT_BASE_SLURMRESTD_URL="<slurmrestd-endpoint>"
+    JOBBERGATE_AGENT_X_SLURM_USER_NAME="<slurmrestd-user-name>"
+    JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_PATH="/path/to/the/jwt/secret/key"
+    JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_STRING="jwt-secret-key-in-plain-text"
+    JOBBERGATE_AGENT_SENTRY_DSN="<sentry-dsn-key>"
+    JOBBERGATE_AGENT_OIDC_DOMAIN="<OIDC-domain>"
+    JOBBERGATE_AGENT_OIDC_AUDIENCE="<OIDC-audience>"
+    JOBBERGATE_AGENT_OIDC_CLIENT_ID="<OIDC-app-client-id>"
+    JOBBERGATE_AGENT_OIDC_CLIENT_SECRET="<OIDC-app-client-secret>"
+    ```
 
-  NOTE: When both `JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_PATH` and `JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_STRING` are passed, the agent will completely ignore the `JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_PATH` and will prioritize the `JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_STRING`. Beware this behaviour.
+    **Note**: `JOBBERGATE_AGENT_SENTRY_DSN` is optional. If you do not pass it the agent understands Sentry will not be used.
 
+    **Note**: When both `JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_PATH` and `JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_STRING` are passed, the agent will completely ignore the `JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_PATH` and will prioritize the `JOBBERGATE_AGENT_SLURMRESTD_JWT_KEY_STRING`. Beware this behaviour.
 
 ## Local usage example
 
 1. Run app
 
-  ```bash
-  jg-run
-  ```
+    ```bash
+    jg-run
+    ```
 
-**Note**: this command assumes you're inside a virtual environment in which the package is installed.
+    **Note**: this command assumes you're inside a virtual environment in which the package is installed.
 
-**NOTE**: beware you should care about having the same user name you're using to run the code in the slurmctld node. For example, if `cluster_agent` will run the `make run` command then the slurmctld node also must have a user called `cluster_agent`.
+    **Note**: beware you should care about having the same user name you're using to run the code in the slurmctld node. For example, if `cluster_agent` will run the `make run` command then the slurmctld node also must have a user called `cluster_agent`.
diff --git a/jobbergate-api/README.md b/jobbergate-api/README.md
@@ -1,6 +1,5 @@
 # Jobbergate API
 
-
 The Jobbergate API provides a RESTful interface over the Jobbergate data and is used
 by both the `jobbergate-agent` and the `jobbergate-cli` to view and manage the
 Jobbergate resources.
@@ -15,12 +14,10 @@ See also:
 
 * [jobbergate-cli](https://github.com/omnivector-solutions/jobbergate/jobbergate-cli)
 
+## License
 
-# License
-
-* `MIT <LICENSE>`_
-
+* [MIT](./LICENSE)
 
-# Copyright
+## Copyright
 
 * Copyright (c) 2020 OmniVector Solutions <info@omnivector.solutions>
diff --git a/jobbergate-cli/README.md b/jobbergate-cli/README.md
@@ -15,7 +15,6 @@ the main command:
 jobbergate job-scripts --help
 ```
 
-
 There is also help and parameter guides for each of the subcommands that can be accessed
 by passing them the `--help` flag:
 
@@ -27,10 +26,10 @@ See also:
 
 * [jobbergate-api](https://github.com/omnivector-solutions/jobbergate/jobbergate-api)
 
-
 ## License
-* [MIT](./LICENSE)
 
+* [MIT](./LICENSE)
 
 ## Copyright
+
 * Copyright (c) 2020 OmniVector Solutions <info@omnivector.solutions>
diff --git a/jobbergate-core/README.md b/jobbergate-core/README.md
@@ -3,10 +3,10 @@
 Jobbergate-core is a sub-project that contains the key components and logic that is shared among all other sub-projects
 (CLI, API, and Agent). Additionally, jobbergate-core exists to support custom automation built on top of Jobbergate.
 
+## License
 
-# License
 * [MIT](LICENSE)
 
+## Copyright
 
-# Copyright
 * Copyright (c) 2023 OmniVector Solutions <info@omnivector.solutions>
diff --git a/jobbergate-docs/Makefile b/jobbergate-docs/Makefile
@@ -10,18 +10,18 @@ install:
 
 .PHONY: lint
 lint: install
-	poetry run black --check src
-	poetry run isort --check src
-	poetry run flake8 --max-line-length=120 --max-complexity=40 src
+	poetry run black --check source
+	poetry run isort --check source
+	poetry run flake8 --max-line-length=120 --max-complexity=40 source
 
 .PHONY: qa
 qa: lint
 	echo "All tests pass! Ready for deployment"
 
 .PHONY: format
 format: install
-	poetry run black src
-	poetry run isort src
+	poetry run black source
+	poetry run isort source
 
 .PHONY: docs
 docs: install

diff --git a/jobbergate-docs/docs/mkdocs.yaml b/jobbergate-docs/docs/mkdocs.yaml
@@ -2,6 +2,7 @@
 
 site_name: Jobbergate Documentation
 repo_url: https://github.com/omnivector-solutions/jobbergate
+repo_name: omnivector-solutions/jobbergate
 docs_dir: source
 theme:
   name: material
@@ -10,21 +11,27 @@ theme:
     code: Fira Code
   features:
     - navigation.expand
+    - navigation.footer
     - navigation.tracking
     - content.code.copy
-  pallete:
-    # Dark/Light mode toggle is not working and can't figure out why
-    - scheme: default
+  icon:
+    repo: fontawesome/brands/github
+  palette:
+    - media: "(prefers-color-scheme: light)"
+      scheme: default
       toggle:
         icon: material/brightness-4
         name: Switch to dark mode
-    - scheme: slate
+    - media: "(prefers-color-scheme: dark)"
+      scheme: slate
       toggle:
         icon: material/brightness-7
         name: Switch to light mode
 watch:
+  - ../../jobbergate-agent/
   - ../../jobbergate-api/
   - ../../jobbergate-cli/
+  - ../../jobbergate-core/
 plugins:
   search:
     lang: en
@@ -57,28 +64,28 @@ extra_css:
 nav:
   - Home: index.md
   - Elements:
-    - Resources:
-      - Overview: elements/resources/index.md
-      - Job Scripts: elements/resources/job_scripts.md
-      - Job Script Templates: elements/resources/job_script_templates.md
-      - Job Submissions: elements/resources/job_submissions.md
-    - Apps:
-      - Overview: elements/apps/index.md
-      - Jobbergate Agent: elements/apps/agent.md
-      - Jobbergate API: elements/apps/api.md
-      - Jobbergate CLI: elements/apps/cli.md
-      - Jobbergate Core: elements/apps/core.md
+      - Resources:
+          - Overview: elements/resources/index.md
+          - Job Scripts: elements/resources/job_scripts.md
+          - Job Script Templates: elements/resources/job_script_templates.md
+          - Job Submissions: elements/resources/job_submissions.md
+      - Apps:
+          - Overview: elements/apps/index.md
+          - Jobbergate Agent: elements/apps/agent.md
+          - Jobbergate API: elements/apps/api.md
+          - Jobbergate CLI: elements/apps/cli.md
+          - Jobbergate Core: elements/apps/core.md
   - Tutorial: tutorial.md
   - Authors: authors.md
   - Developer Guide:
-    - Developer Tools: developer_guide/dev_tools.md
-    - Quality Assurance Tools: developer_guide/qa_tools.md
-    - Integration Testing: developer_guide/integration_testing.md
-    - Continuous Integration: developer_guide/ci.md
-    - Keycloak Setup: developer_guide/keycloak_setup.md
-    - Template Workflows: developer_guide/template_workflows.md
+      - Developer Tools: developer_guide/dev_tools.md
+      - Quality Assurance Tools: developer_guide/qa_tools.md
+      - Integration Testing: developer_guide/integration_testing.md
+      - Continuous Integration: developer_guide/ci.md
+      - Keycloak Setup: developer_guide/keycloak_setup.md
+      - Template Workflows: developer_guide/template_workflows.md
   - Reference:
-    - Jobbergate Agent: reference/agent.md
-    - Jobbergate API: reference/api.md
-    - Jobbergate CLI: reference/cli.md
-    - Jobbergate Core: reference/core.md
+      - Jobbergate Agent: reference/agent.md
+      - Jobbergate API: reference/api.md
+      - Jobbergate CLI: reference/cli.md
+      - Jobbergate Core: reference/core.md
diff --git a/jobbergate-docs/docs/source/authors.md b/jobbergate-docs/docs/source/authors.md
@@ -3,7 +3,6 @@
 Jobbergate is written and maintained by [Omnivector, LLC](https://omnivector.io){:target="\_blank"}. It is an [open
 source](https://github.com/omnivector-solutions/jobbergate) project developed in collaboration with Scania, AB.
 
-
 ## Attribution
 
 This project began as a rewrite of the original
@@ -15,8 +14,7 @@ Building upon Jimmy's great idea, Jobbergate has grown into an entire
 system for managing and submitting reusable Slurm jobs, but the core idea of reusing templates combined with user input
 for the template values has remained the core of the project throughout its evolution.
 
-
-## Jobbergate Development Team:
+## Jobbergate Development Team
 
 The Jobbergate project's main contributors are as follows:
 
@@ -26,7 +24,6 @@ The Jobbergate project's main contributors are as follows:
 * [Matheus Tosta](https://github.com/matheushent) [📧](mailto:matheus@omnivector.solutions)
 * [Tucker Beck](https://github.com/dusktreader) [📧](mailto:tucker@omnivector.solutions)
 
-
-## Get in touch:
+## Get in touch
 
 Contact [Omnivector by email](mailto:info@omnivector.solutions)
diff --git a/jobbergate-docs/docs/source/developer_guide/ci.md b/jobbergate-docs/docs/source/developer_guide/ci.md
@@ -3,7 +3,6 @@
 Jobbergate employs [GitHub actions](https://github.com/omnivector-solutions/jobbergate/actions) for its continuous
 integration processes. Detailed descriptions of these actions are provided on this page.
 
-
 ## Automated Quality Assurance
 
 Jobbergate's git repository incorporates a GitHub Action, specified in
@@ -16,58 +15,54 @@ The suite of quality assurance tools encompasses unit tests, code coverage, lint
 code formatters, and static type checkers. Comprehensive documentation about each tool
 is available in the [Quality Assurance Tools](./qa_tools.md) section.
 
-
 ## Automated Publication to PyPI
 
 The major components of Jobbergate are published on PyPI, the Python Package Index.
 They are available at:
 
- - [jobbergate-api](https://pypi.org/project/jobbergate-api/)
- - [jobbergate-cli](https://pypi.org/project/jobbergate-cli/)
- - [jobbergate-agent](https://pypi.org/project/jobbergate-agent/)
- - [jobbergate-core](https://pypi.org/project/jobbergate-core/)
+- [jobbergate-api](https://pypi.org/project/jobbergate-api/)
+- [jobbergate-cli](https://pypi.org/project/jobbergate-cli/)
+- [jobbergate-agent](https://pypi.org/project/jobbergate-agent/)
+- [jobbergate-core](https://pypi.org/project/jobbergate-core/)
 
 These packages are automatically published to PyPI by three linked GitHub Actions that
 are detailed below.
 
-
 ### Prepare for release
 
 The first action involved in publication is the
 [prepare_release.yaml](https://github.com/omnivector-solutions/jobbergate/blob/main/.github/workflows/prepare_release.yaml))
 action. It is triggered manually on github through a "workflow dispatch event" whenever
 new features or fixes need to be published.
 
-
 The action takes two arguments that must be supplied by the user. They are:
 
- - **Use workflow from:**
+- **Use workflow from:**
    The branch from which the release will be created. The default is `main`, and it's
    highly recommended that releases are cut from this branch in order to keep a linear
    commit history between releases and pre-releases.
- - **Release Type:**
+- **Release Type:**
    This will describe the release type that will be created. Because Jobbergate uses
    semantic versioning, it's important to carefully select the correct type of release.
    For mor information on release types, please see the
    [Poetry documentation](https://python-poetry.org/docs/cli/#version) to learn more.
 
 Once activated, this action:
 
- - Uses Poetry to bump the version number of all the Jobbergate sub-packages according to
+- Uses Poetry to bump the version number of all the Jobbergate sub-packages according to
    the release type selected.
- - Checks if the new version number is synchronized between the sub-packages, and fails if they are not.
- - Creates a new dated entry for the new release on each of the sub-packages' changelog
+- Checks if the new version number is synchronized between the sub-packages, and fails if they are not.
+- Creates a new dated entry for the new release on each of the sub-packages' changelog
    files from the contents of the "Unreleased" section.
- - Creates a new branch named `prepare-release/<version>`.
- - Opens a draft pull request titled `Release <version>`.
+- Creates a new branch named `prepare-release/<version>`.
+- Opens a draft pull request titled `Release <version>`.
 
 In this way, all the changes above can be reviewed before the release is published,
 and all quality assurance tests are executed for the pull request.
 
 The remaining steps of the workflow are chained automatically once the PR is
 accepted and merged into main.
 
-
 ### Create a new tag
 
 The next action in the sequence is the
@@ -76,7 +71,6 @@ action. Once the automatically created release PR is merged into the `main` bran
 action is triggered. It creates and pushes a new git tag to GitHub. The tag is based on
 the new version number for the release.
 
-
 ### Publish on Tag
 
 The final action is