Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[awsfirehose] Add routing rules for metrics ingested from Firehose #9916

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented May 17, 2024

Proposed commit message

This PR is to add routing_rules.yml for metrics ingested from Firehose. Without this integration or user specific set the parameter in Firehose, metrics coming into Elasticsearch by default goes to metrics-aws.cloudwatch-default. Idea to reroute is by checking the value of aws.cloudwatch.namespace field in each document to decide which data stream to route the document to. For example if aws.cloudwatch.namespace equals to AWS/EC2, then this document should go to metrics-aws.ec2_metrics-default.

For s3 daily storage and s3 request, we check for the actual metric name. If metric name is either BucketSizeBytes or NumberOfObjects, this document goes to aws.s3_daily_storage. If it's other metric name but still with AWS/S3 namespace, documents go to aws.s3_request.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@elasticmachine
Copy link

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

Copy link

Quality Gate failed Quality Gate failed

Failed conditions
77.8% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

@elasticmachine
Copy link

💚 Build Succeeded

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants