Implement lazy loading in rerun/__init__.py
#4054
Labels
🧑💻 dev experience
developer experience (excluding CI)
rerun/__init__.py
#4054
Situation
Currently, loading our SDK takes approximately ~350ms (2021 M1 Max), incurred primarily by loading numpy, PIL, and PyArrow.
This is particular impactful when running
python -m rerun
to launch a viewer (since #4053, using thererun
script is much faster), but probably also impacts (simple) logging script.Currently, ~50% of the loading time is incurred by Numpy+PyArrow. This is probably unavoidable as no use-case realistically exists that doesn't require any of those, at least indirectly (it's pervasive in our implementation).
However, our using of a large number of files does incur a cost that is non negligible:
In the case of datatypes, about 5-6% are fixed cost (numpy stuff, base class, etc.), but the remaining 10-11% is the sum of all datatypes, lots of which might not be actually need by user code. The same story applies for archetypes (~4%) and components (~4%). Today, this is not much (<20%, ~60ms?), but with a grow in time as our API becomes more rich.
Possible solutions
Multiple project make use of
__init__.py
-level__getattr__()
to implement lazy loading:Something like this, combined with codegen, should be feasible without too much pain.
The text was updated successfully, but these errors were encountered: