Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement python arrow serialization for all our types #6151

Open
emilk opened this issue Apr 29, 2024 · 1 comment
Open

Implement python arrow serialization for all our types #6151

emilk opened this issue Apr 29, 2024 · 1 comment
Labels
馃徆 arrow concerning arrow 馃 bug Something isn't working 馃悕 Python API Python logging API

Comments

@emilk
Copy link
Member

emilk commented Apr 29, 2024

Incredibly, we don't 馃槶

In particular, we don't implement it for batches of a lot of things.

rerun_py/rerun_sdk/rerun/blueprint/datatypes/visible_time_range.py
126:        raise NotImplementedError  # You need to implement native_to_pa_array_override in visible_time_range_ext.py

rerun_py/rerun_sdk/rerun/blueprint/datatypes/visible_time_range_boundary.py
100:        raise NotImplementedError  # You need to implement native_to_pa_array_override in visible_time_range_boundary_ext.py

rerun_py/rerun_sdk/rerun/blueprint/components/entity_properties_component.py
67:        raise NotImplementedError  # You need to implement native_to_pa_array_override in entity_properties_component_ext.py

rerun_py/rerun_sdk/rerun/components/visualizer_overrides.py
58:        raise NotImplementedError  # You need to implement native_to_pa_array_override in visualizer_overrides_ext.py

rerun_py/rerun_sdk/rerun/datatypes/uvec4d.py
64:        raise NotImplementedError  # You need to implement native_to_pa_array_override in uvec4d_ext.py

rerun_py/rerun_sdk/rerun/datatypes/uvec2d.py
64:        raise NotImplementedError  # You need to implement native_to_pa_array_override in uvec2d_ext.py

rerun_py/rerun_sdk/rerun/datatypes/uvec3d.py
64:        raise NotImplementedError  # You need to implement native_to_pa_array_override in uvec3d_ext.py

rerun_py/rerun_sdk/rerun/datatypes/translation_and_mat3x3.py
118:        raise NotImplementedError  # You need to implement native_to_pa_array_override in translation_and_mat3x3_ext.py

rerun_py/rerun_sdk/rerun/datatypes/float32.py
59:        raise NotImplementedError  # You need to implement native_to_pa_array_override in float32_ext.py

rerun_py/rerun_sdk/rerun/datatypes/translation_rotation_scale3d.py
168:        raise NotImplementedError  # You need to implement native_to_pa_array_override in translation_rotation_scale3d_ext.py

rerun_py/rerun_sdk/rerun/datatypes/tensor_buffer.py
287:        raise NotImplementedError  # You need to implement native_to_pa_array_override in tensor_buffer_ext.py

rerun_py/rerun_sdk/rerun/datatypes/tensor_dimension.py
84:        raise NotImplementedError  # You need to implement native_to_pa_array_override in tensor_dimension_ext.py

rerun_py/rerun_sdk/rerun/datatypes/time_int.py
59:        raise NotImplementedError  # You need to implement native_to_pa_array_override in time_int_ext.py

rerun_py/rerun_sdk/rerun/datatypes/scale3d.py
92:        raise NotImplementedError  # You need to implement native_to_pa_array_override in scale3d_ext.py
``
@emilk emilk added 馃 bug Something isn't working 馃悕 Python API Python logging API 馃徆 arrow concerning arrow labels Apr 29, 2024
@jleibs
Copy link
Member

jleibs commented Apr 29, 2024

The majority of those appear to be datatypes used as sub-fields of other datatypes or components. Not to say they shouldn't implement native_to_pa_array, but the reason they don't is that the recursive dispatch story for python serializers is annoying.

The right direction here for many of these will be to move the serialization code from the parent types to the currently unimplemented children, and likely to build out some supporting dispatch helpers to make it less painful to do all the required arrow plumbing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
馃徆 arrow concerning arrow 馃 bug Something isn't working 馃悕 Python API Python logging API
Projects
None yet
Development

No branches or pull requests

2 participants