Add Telegraf support for VictoriaLogs

[denisgolius]

Is your feature request related to a problem? Please describe

Telegraf supports parsing logs and sends them to Elasticsearch. VictoriaLogs do support some Elasticsearch API, so I was trying to write the data from Telegraf to the VictoriaLogs, but I've failed in issue because Telegraf wants to write to the specific path: _template/telegraf, see a log below:

vlogs-1  | 2024-04-30T15:55:47.510Z     warn    VictoriaMetrics/lib/httpserver/httpserver.go:419        remoteAddr: "192.168.1.5:38368"; requestURI: /insert/elasticsearch/_template/telegraf; unsupported path requested: "/insert/elasticsearch/_template/telegraf"

Describe the solution you'd like

It would be great to add support to write the logs data from Telegraf by adding a new route on the VictoriaLogs side.

Describe alternatives you've considered

No response

Additional information

I used:

• Ubuntu 22.04
• telegraf from Influxdata repository for ubuntu
• Telegraf configuration example for Elasticsearch Telegraf fails to write data to Elasticsearch influxdata/telegraf#5676
• VictoriaLogs in docker from https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/victorialogs/fluentbit-docker/docker-compose.yml#L13-L23

telegraf --version
Telegraf 1.30.2 (git: HEAD@10c15ce4)

Telegraf agent configuration example from /etc/telegraf/telegraf.conf:

[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 100000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  debug = false
  quiet = false
  logtarget = "file"
  logfile = "/dev/null"
  hostname = "pop-os"
  omit_hostname = false

[[outputs.elasticsearch]]
  urls = ["http://192.168.1.5:9428/insert/elasticsearch"]
  timeout = "1m"
  flush_interval = "30s"
  enable_sniffer = false
  health_check_interval = "0s"
  index_name = "device_log-%Y.%m.%d"
  manage_template = true
  template_name = "telegraf"
  overwrite_template = false
  namepass = ["tail"]

# # Parse the new lines appended to a file
[[inputs.tail]]
  files = ["/var/log/telegraf/telegraf.log"]
  from_beginning = false
  interval = "10s"
  pipe = false
  watch_method = "inotify"
  data_format = "value"
  data_type = "string"
  character_encoding = "utf-8"

  [inputs.tail.tags]
     metric_type = "logs"
     log_source = "telegraf"

[AndrewChubatiuk]

disabled manage_template to get rid of this error, telegraf is able to push data to victorialogs, but there's no way how to remap it's fields to _msg_fields, _stream_fields, _time_fields. It's possible to add PR to telegraf to add custom params on ingestion