Update to v5 of tech docs publisher and adopt lychee for the link che…

…cker (#30)

* update to v4 of tech docs publisher

The old link checker is no longer included, so use lychee instead.

* fix: remove dead links from the readme

The tech docs template's own documentation has been missing since
GOV.UK decommissioned GOV.UK PaaS in 2023.

See alphagov/tdt-documentation#210

* fix: simplify directory structure

For the link checker to work, the directory structure must mirror the
directory path structure of the final site.

Since we had two parallel directory structures, `source`,
containing erb, and `source/documentation`, containing markdown,
and those structures were slightly inconsistent, we ended up in the case
where a relative link in the markdown could not be resolved
by the link checker, even though the link worked when the documentation
was published.

There is really no need to have a parallel file structure just to
include markdown files. Middleman (which the tech docs template is based
on) will happily mix erb files with the
extension `.html.erb` or `.html.md.erb` and markdown files with the
extension `.html.md`. In fact, we don't rely on erb at all, so we can
just name our files `.html.md`.

* Update to v5 and add phase banner

Update the tech docs publisher to v5.

This upgrades to govuk frontend v5.7.1.

.github/workflows/check-links.yml

@@ -1,17 +1,39 @@
-name: Check for broken links
-
+name: Build and Test
 on:
-  workflow_dispatch:
   pull_request:
-    paths:
-      - "source/**"
+    branches:
+      - main
 
+permissions: {}
 jobs:
-  check_links:
+  build:
+    name: Build
     runs-on: ubuntu-latest
     container:
-      image: ministryofjustice/tech-docs-github-pages-publisher:v3
+      image: ghcr.io/ministryofjustice/tech-docs-github-pages-publisher@sha256:35699473dbeefeeb8b597de024125a241277ee03587d5fe8e72545e4b27b33f8 # v5.0.0
+    permissions:
+      contents: read
+    steps:
+      - name: Checkout
+        id: checkout
+        uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0
+      - name: Build
+        id: build
+        run: |
+          /usr/local/bin/package
+
+  test:
+    name: Test
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
     steps:
-      - uses: actions/checkout@v4
-      - name: htmlproofer
-        run: /scripts/check-url-links.sh
+      - name: Checkout
+        id: checkout
+        uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0
+      - name: Lychee
+        id: lychee
+        uses: lycheeverse/lychee-action@7cd0af4c74a61395d455af97419279d86aafaede # v2.0.2
+        with:
+          args: --verbose --no-progress './**/*.md' './**/*.html' './**/*.erb' --exclude-loopback --accept 401,403,200,429 --exclude '^https://github.com/moj-analytical-services/create-a-derived-table' --fallback-extensions 'md,html.md.erb'
+          fail: true

.github/workflows/publish.yml

@@ -23,48 +23,44 @@ jobs:
   build:
     runs-on: ubuntu-latest
     container:
-      image: ministryofjustice/tech-docs-github-pages-publisher:v3
+      image: ghcr.io/ministryofjustice/tech-docs-github-pages-publisher@sha256:35699473dbeefeeb8b597de024125a241277ee03587d5fe8e72545e4b27b33f8 # v5.0.0
+    permissions:
+      contents: read
     steps:
       - name: Checkout
-        uses: actions/checkout@v4
-      - name: Compile Markdown to HTML and create artifact
+        id: checkout
+        uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0
+
+      - name: Build
+        id: build
         run: |
-          /scripts/deploy.sh
-      - name: Upload artifact to be published
-        uses: actions/upload-artifact@v4
+          /usr/local/bin/package
+
+      - name: Upload Artifact
+        id: upload_artifact
+        uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
         with:
           name: github-pages
           path: artifact.tar
           retention-days: 1
+          overwrite: true
 
-  deploy:
+  publish:
     needs: build
+    name: Publish
+    runs-on: ubuntu-latest
     environment:
       name: github-pages
-      url: ${{ steps.deployment.outputs.page_url }}
-    runs-on: ubuntu-latest
+      url: ${{ steps.configure_pages.outputs.base_url }}
+    permissions:
+      contents: read
+      id-token: write
+      pages: write
     steps:
-      - name: Setup Pages
-        uses: actions/configure-pages@v5
-      - name: Deploy to GitHub Pages
-        id: deployment
-        uses: actions/deploy-pages@v4
+      - name: Configure Pages
+        id: configure_pages
+        uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
 
-  url-check:
-    needs: deploy
-    runs-on: ubuntu-latest
-    steps:
-      - name: Download a Build Artifact from build
-        uses: actions/download-artifact@v4.1.7
-        with:
-          name: github-pages
-          path: github-pages
-      - name: Unpack files and check URL links
-        run: |
-          cd github-pages
-          tar -xvf artifact.tar
-          npm install linkinator
-          npx linkinator . --recurse --markdown \
-            --skip https://ministryofjustice.github.io/find-moj-data-user-guide/images/govuk-large.png \
-            --skip https://github.com/moj-analytical-services/create-a-derived-table \
-          # "URL Check will fail on private and internal GitHub repositories"
+      - name: Deploy to GitHub Pages
+        id: deploy_pages
+        uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5

README.md

@@ -24,9 +24,7 @@ You can then browse to http://localhost:4567 to view the website.
 You can update the documentation by editing any of the `*.html.md.erb` files in
 the [source](source) directory.
 
-The syntax used in `*.html.md.erb` is Markdown, though it also supports some
-GOV.UK Design System specifics, as listed on [Tech Docs Template - Write your
-content](https://tdt-documentation.london.cloudapps.digital/write_docs/content/).
+The syntax used in `*.html.md` is Markdown, although you can insert arbitrary HTML from the GOV.UK design system.
 
 ## Publishing changes
 
@@ -37,13 +35,5 @@ This website is hosted on [GitHub Pages](https://pages.github.com/).
 
 ## Configuring the website
 
-### Global configuration
-
-The [GOV.UK Tech Docs Template global configuration options](https://tdt-documentation.london.cloudapps.digital/configure_project/global_configuration/)
+The [GOV.UK Tech Docs Template global configuration options](https://github.com/alphagov/tech-docs-template/blob/main/optional/config/tech-docs.yml.tt)
 can be used in this repository to configure the Cloud Platform user guide.
-
-### Structuring documentation and page configuration
-
-The [GOV.UK Tech Docs Template "Configure your documentation project"](https://tdt-documentation.london.cloudapps.digital/configure_project/)
-offers a range of guidance regarding configuration options to help structure
-documentation and configure pages separately.

config/tech-docs.yml

@@ -7,6 +7,7 @@ host: https://ministryofjustice.github.io/find-moj-data-user-guide
 show_govuk_logo: false
 service_name: Find MoJ data user guide
 service_link: /
+phase: INTERNAL
 
 # Links to show on right-hand-side of header
 header_links:

makefile

@@ -1,23 +1,17 @@
-IMAGE := ministryofjustice/tech-docs-github-pages-publisher:v3
+IMAGE := ghcr.io/ministryofjustice/tech-docs-github-pages-publisher@sha256:35699473dbeefeeb8b597de024125a241277ee03587d5fe8e72545e4b27b33f8 # v5.0.0
 
 # Use this to run a local instance of the documentation site, while editing
-.PHONY: preview check
+.PHONY: preview package
 
 preview:
-	docker run --rm \
-		-v $$(pwd)/config:/app/config \
-		-v $$(pwd)/source:/app/source \
+	docker run --rm --platform=linux/amd64 \
+		-v $$(pwd)/config:/tech-docs-github-pages-publisher/config \
+		-v $$(pwd)/source:/tech-docs-github-pages-publisher/source \
 		-p 4567:4567 \
-		-it $(IMAGE) /scripts/preview.sh
+		-it $(IMAGE) /usr/local/bin/preview
 
-deploy:
-	docker run --rm \
-		-v $$(pwd)/config:/app/config \
-		-v $$(pwd)/source:/app/source \
-		-it $(IMAGE) /scripts/deploy.sh
-
-check:
-	docker run --rm \
-		-v $$(pwd)/config:/app/config \
-		-v $$(pwd)/source:/app/source \
-		-it $(IMAGE) /scripts/check-url-links.sh
+package:
+	docker run --rm --platform=linux/amd64 \
+		-v $$(pwd)/config:/tech-docs-github-pages-publisher/config \
+		-v $$(pwd)/source:/tech-docs-github-pages-publisher/source \
+		-it $(IMAGE) /usr/local/bin/package

source/documentation/glossary.md → source/glossary/index.html.md

@@ -1,3 +1,11 @@
+---
+title: Glossary
+weight: 20
+last_reviewed_on: 2024-10-24
+review_in: 6 months
+owner_slack: "#data-catalogue"
+---
+
 # Glossary
 
 ## Data governance

source/documentation/index.md → source/index.html.md

@@ -26,12 +26,12 @@ All MoJ users can sign in to [Find MoJ data](https://find-moj-data.service.justi
 
 ## Glossary
 
-Find MoJ data uses organisational terms. [Our glossary](data/glossary/index.html) provides clear definitions of key terms and concepts related to data management, analytics, and governance within our catalogue.
+Find MoJ data uses organisational terms. [Our glossary](glossary) provides clear definitions of key terms and concepts related to data management, analytics, and governance within our catalogue.
 
 ## Adding to Find MoJ data
 
-- [How to register a model/table from Create a Derived Table](data/cadet-registration/index.html)
-- [How to add database metadata from Create a Derived Table](data/cadet-db-registration/index.html)
+- [How to register a model/table from Create a Derived Table](ingestion/cadet-registration/)
+- [How to add database metadata from Create a Derived Table](ingestion/cadet-db-registration/)
 
 ## Contact us
 

source/documentation/ingestion/cadet-db-registration.md → source/ingestion/cadet-db-registration/index.html.md

@@ -7,7 +7,7 @@ review_in: 2 months
 
 # Adding Database metadata to Find MoJ data from Create a Derived Table (CaDeT)
 
-If your model has been tagged with `dc_display_in_catalogue` then by default it will have its parent database created in Find MoJ data, see guidance for [registering a model/table](../../data/cadet-registration/index.html)
+If your model has been tagged with `dc_display_in_catalogue` then by default it will have its parent database created in Find MoJ data, see guidance for [registering a model/table](../../ingestion/cadet-registration/)
 
 However, this database will have no associated metadata available to make it more useful to someone browsing Find MoJ data. But don't worry you can create this metadata and this section will explain how.
 

source/javascripts/govuk_frontend.js

@@ -0,0 +1 @@
+//= require govuk_frontend_all