svm: speed up filter_executable_program_accounts
#4553
Draft
+349
−290
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
presently,
filter_executable_program_accounts
makes an expensive accounts-db lookup for every account in every transaction batch. this used to be an unfortunate necessity because account loading could not happen until the local program cache was built. a recent feature gate activation has severed that dependency, giving us freedom to improve thisSummary of Changes
create a local program cache for every transaction, instead of every batch, and do this after the transaction accounts have been loaded. this allows us to obviate all accounts-db lookups in
filter_executable_program_accounts
for checking if an account is owned by a program loader because we already have that information. based on testing done by @alessandrod, this speeds up transaction execution by 30-40%this pr will be immediately followed by another which removes nearly all accounts-db lookups fromreplenish_program_cache
as well. this pr will remain a draft until then; they are spiritually similar but split into two for readability and ease of reviewedit: this pr will not be as easy as originally hoped. there are some challenges moving from a per-batch to a per-tx cache (our ultimate goal) due to how the current implementation handles tombstones, and some challenges using the results of account loading to incrementally update a per-batch cache (a plausible short-term goal until the local program cache can be partially redesigned to behave better)
edit2: its viable again. because the other approach is nonviable. lol. clean up the comment history later but rn im based on #4656 for test coverage and need to:
load_program_with_pubkey
that plays nice w bank