Use Timeline::get_vectored
for doing timestamp lookups in SLRUs
#7755
Labels
c/storage/pageserver
Component: storage: pageserver
m/good_first_issue
Moment: when doing your first Neon contributions
t/bug
Issue Type: Bug
Issue
Mapping timestamps to LSN is slow for tenats with large SLRU counts.
The slowness comes from repeatedly querying SLRUs without using vectored get. On a tenant with 190 slru
blocks this took like 4 seconds for each round of the binary search.
Timeline::map_all_timestamps
should useTimeline::get_vectored
instead of fanning out oneTimeline::get
for each block. Make sure to batch the calls intoget_vectored
up to the limit (32 or 64 I don't recall).Steps to reproduce
Import tenant
751897498d4ed3c54a6a28fcbce888b8
on a throaway ec2 instance (i used i3en.3xlarge) and do:Now apply this diff in order to see how much time is being spent in looking up slrus:
In my repro i had:
Note: this used the non-vectored read path. If the
get_impl="vectored"
config is used, it comes out to about13s.
Related:
The text was updated successfully, but these errors were encountered: