WIP

.devcontainer/devcontainer.json

@@ -20,6 +20,7 @@
                 "editor.codeActionsOnSave": {
                     "source.organizeImports": "explicit"
                 },
+                "remote.autoForwardPorts": false,
                 "[python]": {
                     "editor.defaultFormatter": "charliermarsh.ruff"
                 }
@@ -38,7 +39,7 @@
     },
     "features": {
         // add in eternal history and other bash features
-        "ghcr.io/diamondlightsource/devcontainer-features/bash-config:1.0.0": {}
+        "ghcr.io/diamondlightsource/devcontainer-features/bash-config:1": {}
     },
     // Create the config folder for the bash-config feature
     "initializeCommand": "mkdir -p ${localEnv:HOME}/.config/bash-config",

docs/_templates/custom-module-template.rst

@@ -1,8 +1,3 @@
-.. note::
-
-    Ophyd async is considered experimental until the v1.0 release and
-    may change API on minor release numbers before then
-
 {{ ('``' + fullname + '``') | underline }}
 
 {%- set filtered_members = [] %}

docs/conf.py

@@ -98,7 +98,7 @@
 autodoc_inherit_docstrings = False
 
 # Add some more modules to the top level autosummary
-ophyd_async.__all__ += ["sim", "epics", "tango", "fastcs", "plan_stubs"]
+ophyd_async.__all__ += ["sim", "epics", "tango", "fastcs", "plan_stubs", "testing"]
 
 # Document only what is in __all__
 autosummary_ignore_module_all = False

docs/explanations/declarative-vs-procedural.md

@@ -0,0 +1,38 @@
+# Declarative vs Procedural Devices
+
+Ophyd async has two styles of creating Devices, Declarative and Procedural. This article describes why there are two mechanisms for building Devices, and looks at the pros and cons of each style.
+
+## Procedural style
+
+The procedural style mirrors how you would create a traditional python class, you define an `__init__` method, add some class members, then call the superclass `__init__` method. In the case of ophyd async those class members are likely to be Signals and other Devices. For example, in the `ophyd_async.sim.SimMotor` we create its soft signal children in an `__init__` method:
+```{literalinclude} ../../src/ophyd_async/sim/_sim_motor.py
+:start-after: class SimMotor
+:end-before: def set_name
+```
+It is explicit and obvious, but verbose. It also allows you to embed arbitrary python logic in the creation of signals, so is required for making soft signals and DeviceVectors with contents based on an argument passed to `__init__`. It also allows you to use the [](#StandardReadable.add_readable_children) context manager which can save some typing.
+
+## Declarative style
+
+The declarative style mirrors how you would create a pydantic `BaseModel`. You create type hints to tell the base class what type of object you create, add annotations to tell it some parameters on how to create it, then the base class `__init__` will introspect and create them. For example, in the `ophyd_async.fastcs.panda.PulseBlock` we define the members we expect, and the baseclass will introspect the selected FastCS transport (EPICS IOC or Tango Device Server) and connect them, adding any extras that are published:
+```{literalinclude} ../../src/ophyd_async/fastcs/panda/_panda.py
+:start-after: for docs: start PulseBlock
+:end-before: for docs: end PulseBlock
+```
+For a traditional EPICS IOC there is no such introspection mechanism, so we require a PV Suffix to be supplied via an annotation. For example, in `ophyd_async.epics.sim.Counter` we describe the PV Suffix and whether the signal appears in `read()` or `read_configuration()` using [](#typing.Annotated):
+```{literalinclude} ../../src/ophyd_async/epics/sim/_counter_.py
+:start-after: class Counter
+:end-before: class MultiChannelCounter
+```
+It is compact and has the minimum amount of boilerplate, but is limited in its scope to what sorts of Signals and Devices the base class can create. It also requires the usage of a [](#StandardReadableFormat) for each Signal if using [](#StandardReadable) which may be more verbose than the procedural approach. It is best suited for introspectable FastCS and Tango devices, and repetitive EPICS Devices that are wrapped into larger Devices like areaDetectors.
+
+## Grey area
+
+There is quite a large segment of Devices that could be written both ways, for instance `ophyd_async.epics.sim.Mover`. This could be written in either style with roughly the same legibility, so is a matter of taste:
+```{literalinclude} ../../src/ophyd_async/epics/sim/_mover.py
+:start-after: class Mover
+:end-before: baa
+```
+
+## Conclusion
+
+Ophyd async supports both the declarative and procedural style, and is not prescriptive about which is used. In the end the decision is likely to come down to personal taste, and the style of the surrounding code.

docs/tutorials/implementing-devices.md

@@ -0,0 +1 @@
+# Implementing Devices

docs/tutorials/installation.md

@@ -27,6 +27,19 @@ You can now use `pip` to install the library and its dependencies:
 $ python3 -m pip install ophyd-async
 ```
 
+If you need to talk to a given control system, you will need to install
+the specific extra:
+- `ca` for EPICS Channel Access
+- `pva` for EPICS PVAccess
+- `tango` for Tango
+- `demo` for tutorial requirements like h5py and ipython
+- `testing` for testing requirements like pytest
+
+E.g.:
+```
+$ python3 -m pip install ophyd-async[ca,demo]
+```
+
 If you require a feature that is not currently released you can also install
 from github:
 
@@ -38,5 +51,5 @@ The library should now be installed and the commandline interface on your path.
 You can check the version that has been installed by typing:
 
 ```
-$ ophyd-async --version
+$ python -m ophyd_async --version
 ```

docs/tutorials/using-existing-devices.md

@@ -0,0 +1,136 @@
+# Using Devices
+
+In this tutorial we will create a bluesky RunEngine, instantiate some existing ophyd-async Devices, and use them in some bluesky plans. It assumes you have already run through the Bluesky tutorial on [](inv:bluesky#tutorial_run_engine_setup).
+
+## Run the demo
+
+Ophyd-async ships with some simulated devices and a demo script that will create them along with a RunEngine. Let's take a look at it now:
+```{literalinclude} ../../src/ophyd_async/sim/demo/__main__.py
+:language: python
+```
+
+We will explain the contents in more detail later on, but for now let's run it in an interactive [ipython](https://ipython.org) shell:
+```
+$ ipython -i -m ophyd_async.sim
+Python 3.11.11 (main, Dec  4 2024, 20:38:25) [GCC 12.2.0]
+Type 'copyright', 'credits' or 'license' for more information
+IPython 8.30.0 -- An enhanced Interactive Python. Type '?' for help.
+
+In [1]: 
+```
+
+This has launched an ipython shell, told it to import and run the demo script packaged inside `ophyd_async.sim`, then return to an interactive prompt.
+
+## Investigate the Devices
+
+We will look at the `stage.x` and `y` motors first. If we examine them we can see that they have a name:
+```python
+In [1]: stage.x.name
+Out[1]: 'stage-x'
+```
+
+But if we try to call any of the other methods like `read()` we will see that it doesn't return the value, but a [coroutines](inv:python:std:label#coroutine):
+
+```python
+In [2]: stage.x.read()
+Out[2]: <coroutine object StandardReadable.read at 0x7f9c5c105220>
+```
+
+This is because ophyd-async devices implement async versions of the bluesky [verbs](inv:bluesky#hardware). To get the value we can `await` it:
+ ```python
+In [3]: await stage.x.read()
+Out[3]: 
+{'x-user_readback': {'value': 0.0,
+  'timestamp': 367727.615860209,
+  'alarm_severity': 0}}
+```
+
+## Run some plans
+
+Although it is useful to run the verbs using the `await` syntax for debugging, most of the time we will run them via plans executed by the [](#bluesky.run_engine.RunEngine). For instance we can read it using the [`bps.rd`](#bluesky.plan_stubs.rd) plan stub:
+ ```python
+In [4]: RE(bps.rd(stage.x))
+Out[4]: RunEngineResult(run_start_uids=(), plan_result=0.0, exit_status='success', interrupted=False, reason='', exception=None)
+```
+
+and move it using the [`bps.mv`](#bluesky.plan_stubs.mv) plan sub:
+ ```python
+In [5]: RE(bps.mv(stage.x, 1.5))
+Out[5]: RunEngineResult(run_start_uids=(), plan_result=(<WatchableAsyncStatus, device: x, task: <coroutine object WatchableAsyncStatus._notify_watchers_from at 0x7f9c71791940>, done>,), exit_status='success', interrupted=False, reason='', exception=None)
+
+In [6]: RE(bps.rd(stage.x))
+Out[6]: RunEngineResult(run_start_uids=(), plan_result=1.5, exit_status='success', interrupted=False, reason='', exception=None)
+```
+
+There is also a point detector that changes its 3 channels of output based on the positions of the `stage.x` and `stage.y` motors, so we can use it in a [`bp.grid_scan`](#bluesky.plans.grid_scan):
+
+```{eval-rst}
+.. ipython:: python
+  :suppress:
+
+  from ophyd_async.sim.__main__ import *
+
+.. ipython:: python
+  
+  @savefig sim_grid_scan.png width=4in
+  RE(bp.grid_scan([pdet], stage.x, 1, 2, 3, stage.y, 2, 3, 3))
+```
+
+This detector produces a single point of information for each channel at each motor value. This means that the [](inv:bluesky#best-effort-callback) is able to print a tabular form of the scan.
+
+There is also a blob detector that produces a gaussian blob with intensity based on the positions of the `stage.x` and `stage.y` motors, writing the data to an HDF file. You can also use this in a grid scan, but there will be no data displayed as the `BestEffortCallback` doesn't know how to read data from file:
+
+```{eval-rst}
+.. ipython:: python
+  :okwarning:
+  
+  @savefig sim_grid_scan2.png width=4in
+  RE(bp.grid_scan([bdet], stage.x, 1, 2, 3, stage.y, 2, 3, 3))
+```
+
+:::{seealso}
+A more interactive scanning tutorial including live plotting of the data from file is in the process of being written in [the bluesky cookbook](https://github.com/bluesky/bluesky-cookbook/pull/22)
+:::
+
+## Examine the script
+
+We will now walk through the script section by section and examine what each part does. First of all we import the bluesky and ophyd libraries:
+```{literalinclude} ../../src/ophyd_async/sim/demo/__main__.py
+:language: python
+:start-after: Import bluesky and ophyd
+:end-before: Create a run engine
+```
+
+After this we create a RunEngine:
+```{literalinclude} ../../src/ophyd_async/sim/demo/__main__.py
+:language: python
+:start-after: Create a run engine
+:end-before: Define where test data should be written
+```
+We pass `call_returns_result=True` to the RunEngine so that we can see the result of `bps.rd` above. We call `autoawait_in_bluesky_event_loop()` so that when we `await bps.rd(x)` it will happen in the same event loop that the RunEngine uses rather than an IPython specific one. This avoids some surprising behaviour that occurs when devices are accessed from multiple event loops.
+
+Next up is the path provider:
+```{literalinclude} ../../src/ophyd_async/sim/demo/__main__.py
+:language: python
+:start-after: Define where test data should be written
+:end-before: All Devices created within this block
+```
+This is how we specify in which location file-writing detectors store their data. In this example we choose to write to a static directory `/tmp` using the [](#StaticPathProvider), and to name each file within it with a unique UUID using the [](#UUIDFilenameProvider). [Other PathProviders](#PathProvider) allow this to be customized.
+
+Finally we create and connect the Devices:
+```{literalinclude} ../../src/ophyd_async/sim/demo/__main__.py
+:language: python
+:start-after: connected and named at the end of the with block
+```
+The first thing to note is the `with` statement. This uses a [](#init_devices) as a context manager to collect up the top level `Device` instances created in the context, and run the following:
+
+- If `set_name=True` (the default), then call [](#Device.set_name) passing the name of the variable within the context. For example, here we call
+  ``det.set_name("det")``
+- If ``connect=True`` (the default), then call [](#Device.connect) in parallel for all top level Devices, waiting for up to ``timeout`` seconds. For example, here we will connect `x`, `y` and `det` at the same time. This parallel connect speeds up connection to the underlying control system.
+- If ``mock=True`` is passed, then don't connect to the control system, but set Devices into mock mode for testing.
+
+Within it the device creation happens, in this case the `x` and `y` motors and a `det` detector that gives different data depending on the position of the motors.
+
+## Conclusion
+
+In this tutorial we have instantiated some existing ophyd-async devices, seen how they can be connected and named, and used them in some basic plans. Read on to see how to implement support for devices via a control system like EPICS or Tango.