[Entities] Add spaceId to entities

[simianhacker]

Summary

This PR closes https://github.com/elastic/observed-asset-model/issues/67 by capturing the spaceId from the API request and storing the entity.spaceId via the ingest pipeline.

[apmmachine]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

Pinging @elastic/obs-knowledge-team (Team:obs-knowledge)

[afharo]

One small gotcha 😇

[tommyers-elastic]

i wonder if the definition itself is scoped to the space, or if (as here) the definition is space-agnostic, and the definition is installed into a space? i think the former makes more sense, based on how we think the space-scoped config might apply to the definition (different index patterns etc).

if this is the case, we would need to store the space id with the definition too; this would come with some baggage, since our internal entity definition schema would no longer match our API entity definition schema (since the space id is typically a request param, as here).

what do you think? i'm probably over-thinking it, but i feel like maybe either way we should store the space id in the definition saved object too.

[simianhacker]

i wonder if the definition itself is scoped to the space, or if (as here) the definition is space-agnostic, and the definition is installed into a space? i think the former makes more sense, based on how we think the space-scoped config might apply to the definition (different index patterns etc).

The definition is installed into the default space for the SavedObjects if you don't specify /s/{spaceId} in the URL request; the SavedObjects client is already space aware, regardless of this PR. If/when we allow users to create definitions via a UI we will need to support spaces so we might as well make this work from the start. This ensures we can query the data and only return results that are appropriate for the space. This will become more important when we expose the POST /api/entities/_find endpoint because we should also include { "match": { "entity.spaceId": {spaceId} } } as a default query filter.

if this is the case, we would need to store the space id with the definition too; this would come with some baggage, since our internal entity definition schema would no longer match our API entity definition schema (since the space id is typically a request param, as here).

I don't think we need to store this in the entity definition. We should keep the definition space agnostic and the only time it's associated with a space is if you create the definition using POST /s/{spaceId}/api/entities/definition. For the OTB definitions, it will depend on which space the user was in when they installed the "EEM Integration" (which doesn't exist yet but we need to keep that in mind).

[tommyers-elastic]

got it - so the space the definition was installed into is part of the saved object itself, not the definition.

i guess it means you could install the same definition into multiple spaces at once. should we include the space ID in the output index names too, to avoid this scenario causing duplicate entities in the same index?

[simianhacker]

I'll add the space id to the indices in this PR.

After discussing with @tommyers-elastic offline, we'll add the spaceId to the index name after this is merge due to the changes in the historical PR I'm currently working on.

[simianhacker]

@elasticmachine merge upstream

[simianhacker]

@elasticmachine merge upstream

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: d22d228
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-183943-d22d22875b44
• Observability Deployment

Metrics [docs]

Canvas Sharable Runtime

The Canvas "shareable runtime" is an bundle produced to enable running Canvas workpads outside of Kibana. This bundle is included in third-party webpages that embed canvas and therefor should be as slim as possible.

id	before	after	diff
module count	-	5414	+5414
total size	-	8.8MB	+8.8MB

History

• 💚 Build #212612 succeeded eb3ade3
• 💛 Build #211632 was flaky a0604a0
• 💚 Build #211340 succeeded f9efe64
• 💔 Build #211312 failed 0b7e124

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream