Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

svm: speed up filter_executable_program_accounts #4553

Draft
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

2501babe
Copy link
Member

@2501babe 2501babe commented Jan 20, 2025

Problem

presently, filter_executable_program_accounts makes an expensive accounts-db lookup for every account in every transaction batch. this used to be an unfortunate necessity because account loading could not happen until the local program cache was built. a recent feature gate activation has severed that dependency, giving us freedom to improve this

Summary of Changes

create a local program cache for every transaction, instead of every batch, and do this after the transaction accounts have been loaded. this allows us to obviate all accounts-db lookups in filter_executable_program_accounts for checking if an account is owned by a program loader because we already have that information. based on testing done by @alessandrod, this speeds up transaction execution by 30-40%

this pr will be immediately followed by another which removes nearly all accounts-db lookups from replenish_program_cache as well. this pr will remain a draft until then; they are spiritually similar but split into two for readability and ease of review

edit: this pr will not be as easy as originally hoped. there are some challenges moving from a per-batch to a per-tx cache (our ultimate goal) due to how the current implementation handles tombstones, and some challenges using the results of account loading to incrementally update a per-batch cache (a plausible short-term goal until the local program cache can be partially redesigned to behave better)

edit2: its viable again. because the other approach is nonviable. lol. clean up the comment history later but rn im based on #4656 for test coverage and need to:

  • figure out something for load_program_with_pubkey that plays nice w bank
  • check modification slot somehwere when replenishing to set up tombstones. v3 this is easy. v1/v2 im not sure the right behavior... actually mb it doesnt matter? we dont really care if it goes in the cache or not if it always fails since it doesnt affect loading anymore? we only need to handle v1/2 i think

@2501babe 2501babe self-assigned this Jan 20, 2025
@2501babe
Copy link
Member Author

this approach is non-viable without major changes to the program cache, which i want to do, but will need to loop in several people and possibly feature-gate. for now i have a relatively simple way to fix filter_executable_program_accounts. going to do it in a separate pr because this branch may be useful in the future

@2501babe
Copy link
Member Author

as noted in #4574 we will in fact need to rework the program cache because there is no safe way to do this except per-transaction, after account loading

@2501babe 2501babe force-pushed the 20250120_programcache branch from f998162 to 30367fb Compare January 28, 2025 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant