certificate verify failed with Airflow 1.10.15

[rinzool]

Issue

In my company we are facing an issue with the plugin. We are running Airflow with docker (with a custom image), in version 1.10.15.
After installing the plugin, we see in scheduler logs:
urllib3.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

We settle only two environment variables:

• NEW_RELIC_INSERT_KEY=***
• NEW_RELIC_HOST=metric-api.eu.newrelic.com

How to reproduce

The issue can be reproduce locally by pulling public image:

docker run -e NEW_RELIC_HOST=metric-api.eu.newrelic.com -e NEW_RELIC_INSERT_KEY=***  -it apache/airflow:1.10.15 bash

And on the image:

pip install newrelic-airflow-plugin
airflow scheduler

Which directly throw the following error:
urllib3.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

Full stacktrace

[2022-04-22 14:59:23,560] {harvester.py:83} ERROR - New Relic send_batch failed with an exception.
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 488, in wrap_socket
    cnx.do_handshake()
  File "/home/airflow/.local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1825, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/home/airflow/.local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1563, in _raise_ssl_error
    _raise_current_error()
  File "/home/airflow/.local/lib/python3.6/site-packages/OpenSSL/_util.py", line 57, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connection.py", line 371, in connect
    ssl_context=context,
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 386, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 494, in wrap_socket
    raise ssl.SSLError("bad handshake: %r" % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/newrelic_telemetry_sdk/harvester.py", line 75, in _send
    response = self.client.send_batch(*flush_result)
  File "/home/airflow/.local/lib/python3.6/site-packages/newrelic_telemetry_sdk/client.py", line 236, in send_batch
    return self._pool.urlopen("POST", self.PATH, body=payload, headers=headers)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 386, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connection.py", line 371, in connect
    ssl_context=context,
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 386, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 494, in wrap_socket
    raise ssl.SSLError("bad handshake: %r" % e)
urllib3.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

[rinzool]

However sending metrics directly with newrelic_telemetry_sdk seems to work fine

from newrelic_telemetry_sdk import Harvester as _Harvester
from newrelic_telemetry_sdk import MetricBatch, MetricClient
import os

insert_key = os.environ["NEW_RELIC_INSERT_KEY"]
host = os.environ.get("NEW_RELIC_HOST", None)

client = MetricClient(insert_key, host=host)
service_name = os.environ.get("NEW_RELIC_SERVICE_NAME", "Airflow")
batch = MetricBatch({"service.name": service_name})
batch.record_count("test", 1)

response = client.send_batch(*batch.flush())
response.status # 202

[rinzool]

We managed to workaround the issue by overriding ca_certs argument, following this comment
I think it may be nice to add information about that in the README I think, or to update this repository to add the option to use the certify