Implement PeerDAS subnet decoupling (aka custody groups) #6736
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jimmygchen
changed the title
jimmygchen
added
das
Squashed commit of the following: commit 898d05e Merge: ffbd25e 7e0cdde Author: Jimmy Chen <jchen.tc@gmail.com> Date: Tue Dec 24 14:41:19 2024 +1100 Merge branch 'unstable' into refactor-ef-tests-features commit ffbd25e Author: Jimmy Chen <jchen.tc@gmail.com> Date: Tue Dec 24 14:40:38 2024 +1100 Fix `SszStatic` tests for PeerDAS: exclude eip7594 test vectors when testing Electra types. commit aa593cf Author: Jimmy Chen <jchen.tc@gmail.com> Date: Fri Dec 20 12:08:54 2024 +1100 Refactor spec testing for features and simplify usage.
jimmygchen
force-pushed
the
decouple-subnets
branch
from
b449b85 to
d1233f8
Compare
jimmygchen
force-pushed
the
decouple-subnets
branch
from
d1233f8 to
de7293f
Compare
jimmygchen
added
ready-for-review
|
@dapplion @pawanjay176 I think this PR is ready for review now. The failing spec tests are new electra tests, which Pawan and Michael are currently working on, and the DAS spec tests are all passing. Please review and let me know what you think 🙏 |
dapplion
reviewed
Jan 7, 2025
beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs
Outdated
Show resolved
Hide resolved
|
A few comments, else looks solid and good to go |
dapplion
reviewed
Jan 7, 2025
# Conflicts: # beacon_node/beacon_chain/src/kzg_utils.rs # beacon_node/beacon_chain/src/observed_data_sidecars.rs # beacon_node/lighthouse_network/src/discovery/subnet_predicate.rs # common/eth2_network_config/built_in_network_configs/chiado/config.yaml # common/eth2_network_config/built_in_network_configs/gnosis/config.yaml # common/eth2_network_config/built_in_network_configs/holesky/config.yaml # common/eth2_network_config/built_in_network_configs/mainnet/config.yaml # common/eth2_network_config/built_in_network_configs/sepolia/config.yaml # consensus/types/src/chain_spec.rs
Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
realbigsean
requested review from
realbigsean
and removed request for
realbigsean
jimmygchen
force-pushed
the
decouple-subnets
branch
from
6233f23 to
357e3e9
Compare
# Conflicts: # consensus/types/src/chain_spec.rs
|
@dapplion thanks for the thorough review! I've addressed all comments and resolved conflicts. |
dapplion
reviewed
Jan 14, 2025
| "Custody group not found in subnet mapping"; | ||
| "custody_index" => custody_index, | ||
| "peer_id" => %peer_id | ||
| ); |
There was a problem hiding this comment.
This should never happen as we compute subnets_by_custody_group ourselves. Is it necessary to have a warn here?
There was a problem hiding this comment.
Yes agree that this should not happen - we also validate peer's custody_group_count when computing custody groups.
What error handling are you suggesting? I don't think we should panic, or just swallow the error if it happens in case of a bug or invalid input that somehow slip through our validation.
dapplion
approved these changes
Jan 14, 2025
jimmygchen
force-pushed
the
decouple-subnets
branch
from
8e89570 to
283f940
Compare
# Conflicts: # beacon_node/lighthouse_network/src/peer_manager/mod.rs
|
@mergify queue |
jimmygchen
added
ready-for-merge
✅ The pull request has been merged automaticallyThe pull request has been merged automatically at e98209d |
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue Addressed
#6709
This spec change changes the custody unit from subnets to custody groups, in order to decouple networking and DAS core parameters - this allow us to tweak DAS core and network parameters in isolation, i.e. changing number of column subnets will not have a direct impact to sampling parameters & security.
Spec: ethereum/consensus-specs#3832
Proposed Changes:
On startup, we construct
NetworkGlobalsand compute sampling subnets and sampling columns fromNodeId- this remains the same but the computation logic has been updated to compute these from custody groups. (node_id=>custody_groups=>subnetsandcolumns)custody_subnet_count(csc) tocustody_group_count(cgc) inMetaDataandENR. (spec)sampling_sizecomputation logic is has been updated to be base on custody groups instead of custody subnets. code (spec update)sampling_sizeandcustody_group_countlogic intoChainSpecto keep them in one place.