Implement validator sessions pallets

[mattdean-digicatapult]

Pull Request

Checklist

• Have you read Digital Catapult's Code of Conduct?
• I have performed a self-review of my own code.
• I have commented my code, particularly in hard-to-understand areas.
• I have made corresponding changes to the documentation.
• My changes generate no new warnings.
• I have added tests that prove my fix is effective or that my feature works.
• New and existing unit tests pass locally with my changes.

PR Type

Please delete options that are irrelevant.

• Bug Fix
• Chore
• Feature
• Documentation Update
• Code style update (formatting, local variables)
• Breaking Change (fix or feature that would cause existing functionality to change)

Linked tickets

https://digicatapult.atlassian.net/browse/SQNC-95

High level description

Adds pallet_session and pallet_validator_set to runtime

Detailed description

We need to be able to modify the validator set of a live chain and rotate keys for existing validators. This change is the first step in that journey.

First this PR adds pallet_session to the runtime and pallet_validator_set (forked from https://github.com/gautamdhameja/substrate-validator-set) to the runtime. Note however that we cannot for the live chain configure this with a new genesis as the prod chain is already live. To work around this we need to add the pallets and then run transactions to ensure that the validator set and and keys are set correctly.

As we cannot guarentee that this happens before a session change we need to ensure that the existing validator set is maintained even though the session currently has no validators. We do this by hijacking the SessionHandler implementation to always use the existing keys as fetched from Babe and Grandpa. See the struct SessionHandlerBaitAndSwitch.

A future PR will remove SessionHandlerBaitAndSwitch and replace the SessionHandler with <opaque::SessionKeys as OpaqueKeys>::KeyTypeIdProviders which then correctly passes through the new validators to Babe and Grandpa.

To test this I've performed the following procedure in a local chain in a docker compose running Alice, Bob, Charlie and Eve:

1. Export local chain genesis from docker image version v11.3.0
2. Export raw local chain genesis from docker image version v11.3.0
3. Run testnet (alice, bob, charlie, eve) with all nodes on version v11.3.0
4. Wait for babe epochIndex to reach 2
5. Update image for all testnet nodes to 3e6bd7d3cec7139f5f2bc362ddddac5261326a4a
6. Perform doas upgrade code to 1132
7. wait for babe epoch and session index to increment to at least 4 and 2 respectively
8. execute session.set_keys transaction as Alice to d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d 88dc3417d5058ec4b4503e0c12ea1a0a89be200fe98922423d4334014fa6b0ee with proof 0x00
9. execute session.set_keys transaction as Bob to d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d d17c2d7823ebf260fd138f2d7e27d114c0145d968b5ff5006125f2414fadae69 with proof 0x00
10. execute session.set_keys transaction as Charlie to 90b5ab205c6974c9ea841be688864633dc9ca8a357843eeacf2314649965fe22 439660b36c6c03afafca027b910b4fecf99801834c62a5e6006f27d978de234f with proof 0x00
11. execute session.set_keys transaction as Eve to e659a7a1628cdd93febc04a4e0646ea20e9f5f0ce097d9a05290d4a9e054df4e 1dfe3e22cc0d45c70779c1095f7489a8ef3cf52d62fbd8c2fa38c9f1723502b5 with proof 0x00
12. Submit and approve technical committee transactions to add Alice Bob and Charlie as validators using validatorSet.addValidator
13. Wait until next session when the state of session.queuedKeys should update to include alice, bob and charlie
14. Wait an additional session
15. Build a new version of the runtime 1133 which removes SessionHandlerBaitAndSwitch and sets type SessionHandler = <opaque::SessionKeys as OpaqueKeys>::KeyTypeIdProviders;
16. Perform a doas setCode with this new runtime as the technical committee
17. wait for a couple of sessions to pass
18. Submit and approve technical committee transaction to add Eve as a validator using validatorSet.addValidator
19. Wait for 2 sessions to pass at which point verify that Eve is creating blocks

Describe alternatives you've considered

The SessionHandlerBaitAndSwitch is obviously a bit of a hack. I considered several alternatives but all would have required further changes down the line. The main issue is the way Babe interacts with session.

Operational impact

As with all node upgrades this needs to be applied incrementally. First we should upgrade the nodes. Once done we can then safely upgrade the runtime in prod to this runtime and finally add in the validator identities. This will require a procedure to be written.

Additional context

N/A

[mattdean-digicatapult]

Need to create a spike to investigate disaster recovery should we go multiple sessions without block creation (see https://digicatapult.atlassian.net/browse/SQNC-106)