Improved handling of missing .env values

[reecehill]

Problem

During development, the .env file will naturally evolve. In the scenario a new key-value pair is added, a forgetful developer (🙋) may not modify their .env file...

The app continues to work in some instances. However, where a .env value is required (but not found) there are non-specific error messages.

Recommendation

It would be good to handle a missing .env value appropriately, or to perform a check on start-up that .env contains expected keys (e.g., from .env.example).

Example of error

In the absence of...
RCPCH_NHS_ORGANISATIONS_API_URL
RCPCH_NHS_ORGANISATIONS_API_KEY
RCPCH_DGC_API_URL
RCPCH_DGC_API_KEY

A visit to https://npda.localhost/patient/<id>/visits yields:

 [2024-11-28 16:50:50,195] ERROR [django.server:213] "GET /patient/370/visits HTTP/1.1" 500 199075
[2024-11-28 16:51:49,698] WARNING [project.npda.views.mixins:33] User SuperuserAda  has bypassed 2FA for PatientVisitsListView as settings.DEBUG is True and user has role RCPCH Audit Team and is superuser status: True
[2024-11-28 16:51:49,753] ERROR [django.request:248] Internal Server Error: /patient/370/visits
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/psycopg/cursor.py", line 97, in execute
    raise ex.with_traceback(None)
psycopg.errors.UndefinedColumn: column npda_visit.height_centile does not exist
LINE 1: ..."npda_visit"."visit_date", "npda_visit"."height", "npda_visi...
                                                             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/project/npda/views/mixins.py", line 41, in dispatch
    return super().dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/project/npda/views/mixins.py", line 95, in dispatch
    return super().dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/contrib/auth/mixins.py", line 109, in dispatch
    return super().dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/views/generic/list.py", line 174, in get
    context = self.get_context_data()
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/project/npda/views/visit.py", line 39, in get_context_data
    for visit in visits:
                 ^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/db/models/query.py", line 400, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.12/site-packages/django/db/models/query.py", line 1928, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1574, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 122, in execute
    return super().execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/psycopg/cursor.py", line 97, in execute
    raise ex.with_traceback(None)
django.db.utils.ProgrammingError: column npda_visit.height_centile does not exist
LINE 1: ..."npda_visit"."visit_date", "npda_visit"."height", "npda_visi...
                                                             ^

[mbarton]

I agree, this is my fault for not making them mandatory. The app should fail to start if mandatory settings aren’t present

[mbarton]

The error you reported looks like an unapplied migration though which is strange, I would have expected a crash calling the API because it didn’t know the URL.

We should apply migrations in our startup script

national-paediatric-diabetes-audit/s/start-dev

Line 5 in 6c38003

python manage.py migrate

[reecehill]

The error you reported looks like an unapplied migration though which is strange, I would have expected a crash calling the API because it didn’t know the URL.

We should apply migrations in our startup script

national-paediatric-diabetes-audit/s/start-dev

Line 5 in 6c38003

python manage.py migrate

Yeah strange one that. I did not notice once I added the .env values a migration occurred. I'm probably at fault too, something tells me I didn't restart the container on switching branches!!