Skip to content

lib_context: use the minimal indexing strategy by default

Craig Ferguson requested to merge CraigFe/tezos:minimal-indexing-strategy into master

Context

This MR is a follow-up to !4477 (merged), taking advantage of the direct pointers between objects in the context in order to index far fewer objects.

The default indexing strategy during normal operation of the node has been changed from "always" to "minimal", meaning that the only objects to be indexed are commits (and not tree nodes or blobs, as before). This substantially reduces the mean number of bytes written to disk per block, improving the overall write performance of the node and reducing the space usage of the context.

Perhaps most importantly, this means that the context index virtually never needs to do a merge during normal operation, meaning that the I/O behaviour of the node is much more stable than before. There are no longer any "spikes" in I/O usage due to recreating the index during a merge, and so no risk of the node blocking when these merges can no longer be performed asynchronously. The overall UX of Octez should therefore significantly improve for users with large stores (who no longer risk missing endorsements due to stacked merges).

This change also unlocks our prototype implementation of context garbage collection, which was previously bottlenecked by index merge operations during the sweep phase.

The semantic consequence of this change is that the context no longer guarantees perfect hashconsing, meaning that certain workloads may cause the data file to contain duplicated objects. Roughly speaking, this duplication happens whenever the data sharing is "by value" rather than "by reference", such as when two identical blobs are generated by chance in different regions of the context tree. Our testing suggests that the space overhead of this change is negligible compared to the space recovered in the index, but this is something to bear in mind when reviewing this MR.

There are two additional subtleties with the feature:

  • this MR does not change the indexing strategy used by the context when importing a snapshot. This is due to technical limitations that we plan to resolve in the next release of Irmin, which will enable an opt-in for faster snapshot imports (at the cost of maintaining an index in memory while importing).

  • this MR provides an environment variable switch to re-enable the old indexing behaviour via TEZOS_CONTEXT="indexing-strategy=always". Since the exact impact of this change is dependent on the workload of the node, it's prudent to provide some sort of fallback to the old perfect hashconsing behaviour (even if only for debugging purposes in future).

Manually testing the MR

Any operations that make intensive or unusual use of the context are good candidates for manual testing. For instance:

  • bootstrapping a fresh store with minimal indexing and verifying that (a) the index data file is much smaller than before and (b) the node functions as expected;
  • importing/exporting a snapshot to/from a store with minimal indexing.

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR
Edited by Craig Ferguson

Merge request reports