Add support for time_series_dimension flag in _elastic/{index}/_field_caps API method

[kuzaxak]

Opensearch Dashboards calling _field_caps to build an index pattern. To use quickwit natively as an external data source for it dashboards need to determine a timestamp field to render histogram, such fields listed on index pattern settings page. For that it relies on a time_series_dimension flag in <field>.<type>.time_series_dimension.

Link to the doc.

Describe the solution you'd like

Filed chosen as timestamp_field in index should be marked as time_series_dimension in _field_caps output.

Describe alternatives you've considered

I've made a little proxy to query /api/v1/indexes?index_id_patterns= and patch response of _field_caps method.

Additional context

I'm working on integrating quickwit as an Opensearch Dashboards backend to replace existing setup which is using regular Opensearch.

[fulmicoton]

@kuzaxak have you started experimenting with adding opendashboard support by any chance? I'd love to exchange if you have more info on other missing pieces.

[kuzaxak]

@kuzaxak have you started experimenting with adding opendashboard support by any chance? I'd love to exchange if you have more info on other missing pieces.

Yes, I was able to migrate our Opensearch dashboard completely to quickwit backend. But I'm using a proxy to monkey patch some responses of quickwit.

I can share a proxy code, it is a simple python flask app.

[kuzaxak]

@fulmicoton here you can find a proxy server code.

I'm using quickwit as an external source not to deal with auth, dashboards storage, etc.

1. Enable this feature for OSD:

data_source.enabled: true

2. Add proxy as an additional datasource:

3. Add Index pattern using external source:

This way I'm using security features and SAML from OSD and logs storage and querying from Quickwit.

[fulmicoton]

@kuzaxak That's absolutely awesome, thank you for sharing the script, the approach and the screenshot, The hack is really cool!