Config setup docs - through tests

[stan-dot]

please validate the directional correctness of the test first. #tdd

[stan-dot]

it works until line 179 ( call to loader.use_values_from_yaml)

[stan-dot]

    config_data = {
        "env": {},
        "sources": [
            {"kind": "dodal", "module": "dodal.adsim"},
            {"kind": "planFunctions", "module": "dls_bluesky_core.plans"},
            {"kind": "planFunctions", "module": "dls_bluesky_core.stubs"},
        ],
        "data_writing": {
            "visit_directory": "/dls/p38/data/2023/cm33874-1",
            "group_name": "BL38P",
        },
    }

does not map directly to this:

#532 (comment)

perhaps config loading could be simplified, to mirror more closely the helm structure

[stan-dot]

all good

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.46%. Comparing base (d3a8d9f) to head (326d1b1).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #634   +/-   ##
=======================================
  Coverage   92.45%   92.46%           
=======================================
  Files          35       35           
  Lines        1657     1658    +1     
=======================================
+ Hits         1532     1533    +1     
  Misses        125      125           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[stan-dot]

class BasicAuthentication(BaseModel):
    """
    User credentials for basic authentication
    """

    username: str = Field(description="Unique identifier for user")
    password: Secret[str] = Field(description="Password to verify user's identity")

    @field_validator("username", "password", mode="before")
    @classmethod
    def get_from_env(cls, v: str):
        if v.startswith("${") and v.endswith("}"):
            return os.environ[v.removeprefix("${").removesuffix("}").upper()]
        return v

current error is coming from bluesky-stomp, where for some reason it uses the class method to get the password from env. therefore full validation from a file might require an adjustment

[stan-dot]

@ZohebShaikh any ideas?

[DiamondJoseph]

class BasicAuthentication(BaseModel):
    """
    User credentials for basic authentication
    """

    username: str = Field(description="Unique identifier for user")
    password: Secret[str] = Field(description="Password to verify user's identity")

    @field_validator("username", "password", mode="before")
    @classmethod
    def get_from_env(cls, v: str):
        if v.startswith("${") and v.endswith("}"):
            return os.environ[v.removeprefix("${").removesuffix("}").upper()]
        return v

current error is coming from bluesky-stomp, where for some reason it uses the class method to get the password from env. therefore full validation from a file might require an adjustment

AttributeError: 'Secret' object has no attribute 'startswith'

@stan-dot you need to unwrap the Secret[str] before using it.

It fetches it from the ENV if configured to do so for injection of the RMQ password. If you want to adjust how the handling of that env var fetching is done to your override levels handling then feel free. @keithralphs needs to mount a bunch of tracing values as env vars, so having that handling in one place is preferable. But the type of the password should remain Secret[str] to prevent accidental logging.

[stan-dot]

File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/bluesky_stomp/models.py", line 17, in get_from_env
if v.startswith("${") and v.endswith("}"):
AttributeError: 'Secret' object has no attribute 'startswith'

odd, digging deeper

[stan-dot]

FAILED tests/unit_tests/test_config.py::test_config_yaml_parsed_complete[temp_yaml_config_file0] - blueapi.utils.invalid_config_error.InvalidConfigError: Something is wrong with the configuration file: 
 {'type': 'model_type', 'loc': ('scratch', 'repositories', 0), 'msg': 'Input should be a valid dictionary or instance of ScratchRepository', 'input': 'dodal', 'ctx': {'class_name': 'ScratchRepository'}, 'url': 'https://errors.pydantic.dev/2.9/v/model_type'}
FAILED tests/unit_tests/test_config.py::test_config_yaml_parsed_complete[temp_yaml_config_file1] - blueapi.utils.invalid_config_error.InvalidConfigError: Something is wrong with the configuration file: 
 {'type': 'model_type', 'loc': ('scratch', 'repositories', 0), 'msg': 'Input should be a valid dictionary or instance of ScratchRepository', 'input': 'dodal', 'ctx': {'class_name': 'ScratchRepository'}, 'url': 'https://errors.pydantic.dev/2.9/v/model_type'}
FAILED tests/unit_tests/worker/test_task_worker.py::test_worker_and_data_events_produce_in_order - concurrent.futures._base.TimeoutError
============= 3 failed, 

got a more specific error

[callumforrester]

Should: Uncomment and use or remove

[callumforrester]

LGTM, thanks