PDP Service Contract Performance

[ZenGround0]

This is a placeholder issue for all speculative performance optimizations of the PDP service contract. We'll only do things that address measured bottlenecks (likely proving but possibly also the SumTree index and contract history recording).

Here are a few we can predict may be useful. Please add more ideas to the comments as you come up with them.

• Batch the add and remove methods
• SumTree performance improvements
  • track treeStart for efficient searching when front of proof set is garbage collected
  • store multiple SumTree entries in one word for better loading performance
  • change branching factor for shorter tree depth and to match word size for better loading performance
• Change hot code paths to inline assembly, especially in proving code path
  • probably this is taken care of in merkle proof libraries that are very good
• Do perf comparisons of merkle tree libraries
• Consider changing PDP hash function for best perf
• Recording history
  • Make assumptions about proving frequency to dramatically compress recorded bytes
  • Try a record keeping contract with a limited lifetime of events recorded. This keeps updating costs bounded rather than growing with map size

[anorth]

I think we should batch add and remove from the start in any case. There are obvious overheads to be avoided by doing this.

[ZenGround0]

Its worth investigating CID representation for efficiency. We could explore have separate mappings for cid hashes vs cid prefixes.
" // TODO: there is more to think about here.
// existing libraries (https://github.com/filecoin-project/filecoin-solidity/blob/master/contracts/v0.8/types/CommonTypes.sol#L91)
// don't give us any significant functionality so it makes sense to redeclare.
// There will be performance benefits both for storage and memory ops to using bytes32 but we should wait
// on measurment before making that decision. We could potentially use bytes32 hash and bytes8 for the prefix.
"

[ZenGround0]

We can explore changing the shape of the key space to one flat space vs two hierarchical proofset => root id key spaces. Current hierarchical approach is speculatively better for isolation between different proofsets.
" // TODO: a single mapping with a pair key i.e. (uint256, uint256 )
// might be more efficient as it will half the hashing needed for access and set"