Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

De-abstract Savable by making it a protocol #298

Draft
wants to merge 41 commits into
base: master
Choose a base branch
from

Conversation

unkcpz
Copy link
Member

@unkcpz unkcpz commented Dec 12, 2024

The outcome of this refactoring is that the Savable become a protocol which provide the contract for how things needed to be saved should implemented.

@runtime_checkable
class Savable(Protocol):
    @classmethod
    def recreate_from(cls, saved_state: SAVED_STATE_TYPE, load_context: LoadSaveContext | None = None) -> 'Savable':
        """
        Recreate a :class:`Savable` from a saved state using an optional load context.

        :param saved_state: The saved state
        :param load_context: An optional load context

        :return: The recreated instance

        """
        ...

    def save(self, save_context: LoadSaveContext | None = None) -> SAVED_STATE_TYPE: ...

Compare to the original implementation where save calls save_instance_state calls each other recursively until all savable things is saved, the new implementation force the save method independent and have symmetry behavior of recreate_from.

unkcpz#5

@unkcpz unkcpz marked this pull request as draft December 12, 2024 11:04
unkcpz and others added 21 commits January 17, 2025 23:35
…and asyncio.Future

hand write wrap to kiwipy future (concurrent.futures.Future)

kiwipy.Future -> concurrent.futures.Future
Move communication to rmq
The refactoring is targeting to decouple the dependencies of using kiwipy+rmq as the communicator for the process control. 
By forming a `Coordinator` protocol contract, the different type of rmq/kiwipy related codes are removed out from plumpy logic. The new contract also pave the way to make it clearly show how a new type coordinator can be implemented (future examples will be the `tatzelwurm` a task broker that has scheduler support and file based task broker require no background service). 
For the prototype of how a coordinator should look like, the `MockCoordinator` in `tests/utils` is the coordinator that store things in memory, and can serve as the lightweight ephemeral daemon without persistent functionality.

Another major change here is hand write the resolver of future by mimic how tho asyncio does for wrapping `concurrent.futures.Future` into `asyncio.Future`. I use the same way to convert `asyncio.Future` into `concurent.futures.Future` (which is the `kiwipy.Future` as alias). 

- move the `aio_pika` import lazily by moving the rmq exceptions to `rmq` module, this can increase the performance of `import aiida; aiida.orm`.
- ~~`CancellableAction` using composite to behave as a Future like object.~~
- use `asyncio.Future` in favor of alias `plumpy.Future` and 
- use `concurrent.futures.Future` instead of alias `kiwipy.Future`.
- Hand write `_chain` and `_copy_future` since we can not just rely on the API of asyncio that is not exposed.
- Forming the `coordinator/Communicator` protocol.
- Just forming the `coordinator/Coordinator` protocol and wrap rmq/communicator as a coordinator that not require changs in kiwipy.
- Mock the coordinator for unit test.
- test against aiida-core see what need to be changed there and improve here. (aiidateam/aiida-core#6675)
- The API for plumpy process can be more compact instead of using kiwipy/rmq "subscriber" concept. (how to replace rpc pattern??)
@unkcpz unkcpz force-pushed the deab-savable-inh branch 3 times, most recently from cf43458 to 3e6a2dd Compare January 18, 2025 01:06
@unkcpz unkcpz force-pushed the deab-savable-inh branch 2 times, most recently from 34d9b01 to d524e63 Compare January 18, 2025 01:59
@unkcpz unkcpz changed the base branch from dev to master January 18, 2025 02:04
@unkcpz unkcpz force-pushed the deab-savable-inh branch 2 times, most recently from e149bd2 to a90dcff Compare January 22, 2025 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant