Bridges - Add improved congestion control mechanism #6231

bkontur · 2024-10-25T08:51:14Z

Closes: #5551
Closes: #5550

Context

Before permissionless lanes, bridges only supported hard-coded, static lanes. The congestion mechanism was based on sending Transact(report_bridge_status(is_congested)) from pallet-xcm-bridge-hub to pallet-xcm-bridge-hub-router. Depending on is_congested, we adjusted the fee factor to increase or decrease fees. This congestion mechanism relied on monitoring XCMP queues, which could cause issues like suspending the entire XCMP queue rather than just the affected bridge.

Additionally, we are progressing with deploying bridge message pallets/routing directly on AssetHub, where we don’t interact with XCMP to perform ExportXcm locally.

Description

This PR re-introduces and improves congestion for bridges:

Enhanced Bridge Congestion Mechanism: The bridge queue mechanism has been restructured to operate independently of XCMP, with a refined protocol for congestion detection and suspension management.
Bridge-Specific Channel Suspension: pallet-xcm-bridge-hub and pallet-xcm-bridge-hub-router now use BridgeId to identify specific bridges, enabling selective suspension and resumption of individual bridge channels.
Dynamic Congestion Detection: pallet-xcm-bridge-hub now includes callbacks for fn suspend_bridge and fn resume_bridge based on congestion status:
- For sibling chains, the router sends xcm::Transact(report_bridge_status(bridge_id, is_congested)) using the stored callback information.
- For local chain deployments, the router manages state directly.
New Stop Threshold: A stop_threshold limit in pallet-xcm-bridge-hub enables or disables ExportXcm::validate, providing a fallback mechanism when the router does not adhere to the suspend signal.
Flexible Message Routing: pallet-xcm-bridge-hub-router has been refactored to support message routing for both sibling chains (ExportMessage) and local deployment (ExportXcm).

These updates improve modularity, allow more granular bridge congestion handling, and support diverse deployment scenarios.

Open questions

when the router receives is_congested = false with fn do_update_bridge_status(, can we directly remove it (its fee factor) from Bridges and the next message fee wont be affected by increased fee factor or slowly start decreasing fee factor on idle (this is how it is implemented now)? Original comment

bkontur · 2024-10-25T11:25:35Z

bot fmt
/cmd prdoc --audience runtime_dev --bump patch

prdoc/pr_6231.prdoc

bkontur · 2024-10-26T20:30:05Z

bot fmt

bkontur · 2024-10-28T15:04:20Z

bot fmt

bkontur · 2024-11-05T12:40:39Z

bot fmt

bkontur · 2024-11-07T15:57:44Z

/cmd bench --runtime asset-hub-westend asset-hub-rococo --pallet pallet_xcm_bridge_hub_router

bkontur · 2024-11-07T16:18:32Z

bot bench cumulus-assets --runtime=asset-hub-westend --pallet=pallet_xcm_bridge_hub_router
bot bench cumulus-assets --runtime=asset-hub-rococo --pallet=pallet_xcm_bridge_hub_router

bkontur · 2024-11-07T16:51:47Z

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-westend --pallet=pallet_xcm_bridge_hub_router
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-rococo --pallet=pallet_xcm_bridge_hub_router

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages

bkontur · 2024-11-07T21:51:26Z

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-rococo --pallet=pallet_xcm_benchmarks::generic
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-westend --pallet=pallet_xcm_benchmarks::generic

bkontur · 2024-11-08T23:55:04Z

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_xcm_bridge_hub
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_xcm_bridge_hub

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-rococo --pallet=pallet_xcm_benchmarks::generic
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-westend --pallet=pallet_xcm_benchmarks::generic

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-westend --pallet=pallet_xcm_bridge_hub_router
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-rococo --pallet=pallet_xcm_bridge_hub_router

github-actions · 2024-11-27T12:30:20Z

Command "fmt" has started 🚀 See logs here

github-actions · 2024-11-27T12:30:45Z

Command "fmt" has finished ✅ See logs here

acatangiu

All of the XCM transport (pallet-messages layer) will be deployed to Asset Hub instead of Bridge Hub, so would it be easier to not go through all of the effort of upgrading Bridge Hub with all this only to deprecate it on the next iteration?

What if instead of changing and renaming xcm-bridge-hub to support permissionless lanes, we just create a new pallet xcm-bridge which is what you basically have in this PR, and we revert xcm-bridge-hub to the same code actually deployed today on Bridge Hub?

That way we keep legacy code with legacy lane on BH without any migrations or risk, and all of this new code goes straight to AH, then at some point we switch the AH exporter from legacy to new?

I know this is late and maybe we should've had this idea earlier, so I'm leaving it up to you to decide which way is easiest to do.

acatangiu · 2024-11-29T13:46:30Z

bridges/modules/relayers/src/extension/mod.rs

@@ -1127,7 +1126,6 @@ mod tests {
 		Option<PreDispatchData<ThisChainAccountId, BridgedChainBlockNumber, TestLaneIdType>>,
 		TransactionValidityError,
 	> {
-		sp_tracing::try_init_simple();


nit: why remove this? it's useful in debugging, no?

bridges/modules/xcm-bridge-hub-router/src/impls.rs

acatangiu · 2024-11-29T14:14:37Z

bridges/modules/xcm-bridge-hub-router/src/impls.rs

+				bridged_dest.clone()
+			}
+		} else {
+			// if `bridged_dest` does not contain `GlobalConsensus`, let's prepend one


is there a case when ensure_is_remote(UniversalLocation::get(), dest.clone()) returns Ok, but dest does not contain GlobalConsensus?
should we even support such a case?

bridges/modules/xcm-bridge-hub-router/src/lib.rs

bridges/primitives/xcm-bridge-hub/src/lib.rs

bridges/modules/xcm-bridge-hub/src/exporter.rs

prdoc/pr_6231.prdoc

Closes: #5551 ## Description With [permissionless lanes PR#4949](#4949), the congestion mechanism based on sending `Transact(report_bridge_status(is_congested))` from `pallet-xcm-bridge-hub` to `pallet-xcm-bridge-hub-router` was replaced with a congestion mechanism that relied on monitoring XCMP queues. However, this approach could cause issues, such as suspending the entire XCMP queue instead of isolating the affected bridge. This PR reverts back to using `report_bridge_status` as before. ## TODO - [x] benchmarks - [x] prdoc ## Follow-up #6231 --------- Co-authored-by: GitHub Action <action@github.com> Co-authored-by: command-bot <> Co-authored-by: Adrian Catangiu <adrian@parity.io>

Closes: #5551 ## Description With [permissionless lanes PR#4949](#4949), the congestion mechanism based on sending `Transact(report_bridge_status(is_congested))` from `pallet-xcm-bridge-hub` to `pallet-xcm-bridge-hub-router` was replaced with a congestion mechanism that relied on monitoring XCMP queues. However, this approach could cause issues, such as suspending the entire XCMP queue instead of isolating the affected bridge. This PR reverts back to using `report_bridge_status` as before. ## TODO - [x] benchmarks - [x] prdoc ## Follow-up #6231 --------- Co-authored-by: GitHub Action <action@github.com> Co-authored-by: command-bot <> Co-authored-by: Adrian Catangiu <adrian@parity.io> (cherry picked from commit 8f4b99c) # Conflicts: # Cargo.lock # cumulus/parachains/runtimes/assets/asset-hub-rococo/tests/tests.rs # cumulus/parachains/runtimes/assets/asset-hub-westend/src/lib.rs

Closes: #5551 ## Description With [permissionless lanes PR#4949](#4949), the congestion mechanism based on sending `Transact(report_bridge_status(is_congested))` from `pallet-xcm-bridge-hub` to `pallet-xcm-bridge-hub-router` was replaced with a congestion mechanism that relied on monitoring XCMP queues. However, this approach could cause issues, such as suspending the entire XCMP queue instead of isolating the affected bridge. This PR reverts back to using `report_bridge_status` as before. ## TODO - [x] benchmarks - [x] prdoc ## Follow-up #6231 --------- Co-authored-by: GitHub Action <action@github.com> Co-authored-by: command-bot <> Co-authored-by: Adrian Catangiu <adrian@parity.io>

yrong · 2024-12-18T17:57:41Z

bridges/modules/xcm-bridge-hub-router/src/lib.rs

+			for (bridge_id, previous_value, bridge_state) in bridges_to_update.into_iter() {
+				let new_value = bridge_state.delivery_fee_factor;
+				log::info!(
+					target: LOG_TARGET,
+					"Bridge channel with id {:?} is uncongested. Decreasing fee factor from {} to {}!",
+					bridge_id,
+					previous_value,
+					new_value,
+				);
+				Bridges::<T, I>::insert(&bridge_id, bridge_state);
+				Self::deposit_event(Event::DeliveryFeeFactorDecreased {
+					previous_value,
+					new_value,
+					bridge_id,
+				});
 			}


Just curious why not guard the weight usage with meter in second loop, what if there is a lot of bridges to update/remove here while the weight is used up in first loop?

cumulus/parachains/runtimes/assets/asset-hub-westend/src/lib.rs

bkontur · 2024-12-22T21:40:49Z

/cmd fmt

github-actions · 2024-12-22T21:42:46Z

Command "fmt" has started 🚀 See logs here

github-actions · 2024-12-22T21:43:09Z

Command "fmt" has finished ✅ See logs here

Closes: paritytech#5551 ## Description With [permissionless lanes PR#4949](paritytech#4949), the congestion mechanism based on sending `Transact(report_bridge_status(is_congested))` from `pallet-xcm-bridge-hub` to `pallet-xcm-bridge-hub-router` was replaced with a congestion mechanism that relied on monitoring XCMP queues. However, this approach could cause issues, such as suspending the entire XCMP queue instead of isolating the affected bridge. This PR reverts back to using `report_bridge_status` as before. ## TODO - [x] benchmarks - [x] prdoc ## Follow-up paritytech#6231 --------- Co-authored-by: GitHub Action <action@github.com> Co-authored-by: command-bot <> Co-authored-by: Adrian Catangiu <adrian@parity.io>

acatangiu · 2025-01-06T14:16:25Z

@bkontur please ping when this is ready for review

Co-authored-by: Adrian Catangiu <adrian@parity.io>

paritytech-workflow-stopper · 2025-01-13T15:18:32Z

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/12750200852
Failed job name: fmt

bkontur added the T15-bridges This PR/Issue is related to bridges. label Oct 25, 2024

bkontur self-assigned this Oct 25, 2024

bkontur force-pushed the bko-bridges-congestion branch 2 times, most recently from 659be89 to b48b8a5 Compare October 25, 2024 21:14

bkontur commented Oct 26, 2024

View reviewed changes

prdoc/pr_6231.prdoc Outdated Show resolved Hide resolved

bkontur force-pushed the bko-bridges-congestion branch from a663bc2 to cbc6ae7 Compare October 26, 2024 20:29

bkontur force-pushed the bko-bridges-congestion branch from c78f9bc to 501a5c0 Compare October 28, 2024 15:01

bkontur force-pushed the bko-bridges-congestion branch 7 times, most recently from c78e707 to 152389a Compare November 5, 2024 12:33

bkontur force-pushed the bko-bridges-congestion branch 3 times, most recently from edd9c5c to 38f1bb3 Compare November 7, 2024 13:33

bkontur force-pushed the bko-bridges-congestion branch from f06433a to d329dec Compare November 7, 2024 16:44

bkontur force-pushed the bko-bridges-congestion branch from 21baa7f to c00ff6b Compare November 8, 2024 17:44

bkontur added the A4-needs-backport Pull request must be backported to all maintained releases. label Nov 11, 2024

Nits

ad0931c

Update from bkontur running command 'fmt'

981e3df

Merge branch 'master' into bko-bridges-congestion

91567e0

acatangiu reviewed Nov 29, 2024

View reviewed changes

bkontur mentioned this pull request Dec 7, 2024

Bridges - revert-back congestion mechanism #6781

Merged

2 tasks

acatangiu changed the title ~~Bridges - revert-back and improve congestion~~ Bridges - add new congestion control protocol for dedicated lanes Dec 9, 2024

yrong reviewed Dec 18, 2024

View reviewed changes

cumulus/parachains/runtimes/assets/asset-hub-westend/src/lib.rs Show resolved Hide resolved

bkontur changed the title ~~Bridges - add new congestion control protocol for dedicated lanes~~ Bridges - Add improved congestion control mechanism Dec 19, 2024

bkontur added 2 commits December 20, 2024 22:32

Merge remote-tracking branch 'origin/master' into bko-bridges-congestion

18f31da

Merge remote-tracking branch 'origin/master' into bko-bridges-congestion

16ae079

paritytech-review-bot bot requested a review from a team December 20, 2024 21:34

bkontur added 2 commits December 22, 2024 22:38

Merge branch 'master' into bko-bridges-congestion

4569510

zepter/taplo

8e681fb

Update from bkontur running command 'fmt'

69e85cc

bkontur marked this pull request as draft January 13, 2025 12:25

bkontur and others added 4 commits January 13, 2025 13:25

Merge branch 'master' into bko-bridges-congestion

b7e33f9

Apply suggestions from code review

511e501

Co-authored-by: Adrian Catangiu <adrian@parity.io>

Finish renaming ToBridgeHubSender->MessageExporter + TODOs

b94b9b9

Apply suggestions from code review

1783fa0

Co-authored-by: Adrian Catangiu <adrian@parity.io>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bridges - Add improved congestion control mechanism #6231

Bridges - Add improved congestion control mechanism #6231

bkontur commented Oct 25, 2024 •

edited

Loading

bkontur commented Oct 25, 2024

bkontur commented Oct 26, 2024

bkontur commented Oct 28, 2024

bkontur commented Nov 5, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 8, 2024

github-actions bot commented Nov 27, 2024

github-actions bot commented Nov 27, 2024

acatangiu left a comment

acatangiu Nov 29, 2024

acatangiu Nov 29, 2024

yrong Dec 18, 2024

bkontur commented Dec 22, 2024

github-actions bot commented Dec 22, 2024

github-actions bot commented Dec 22, 2024

acatangiu commented Jan 6, 2025

paritytech-workflow-stopper bot commented Jan 13, 2025

Bridges - Add improved congestion control mechanism #6231

Are you sure you want to change the base?

Bridges - Add improved congestion control mechanism #6231

Conversation

bkontur commented Oct 25, 2024 • edited Loading

Context

Description

Open questions

bkontur commented Oct 25, 2024

bkontur commented Oct 26, 2024

bkontur commented Oct 28, 2024

bkontur commented Nov 5, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 8, 2024

github-actions bot commented Nov 27, 2024

github-actions bot commented Nov 27, 2024

acatangiu left a comment

Choose a reason for hiding this comment

acatangiu Nov 29, 2024

Choose a reason for hiding this comment

acatangiu Nov 29, 2024

Choose a reason for hiding this comment

yrong Dec 18, 2024

Choose a reason for hiding this comment

bkontur commented Dec 22, 2024

github-actions bot commented Dec 22, 2024

github-actions bot commented Dec 22, 2024

acatangiu commented Jan 6, 2025

paritytech-workflow-stopper bot commented Jan 13, 2025

bkontur commented Oct 25, 2024 •

edited

Loading