Add support for AWS IAM

[pmlopes]

https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/auth-iam.html

Technically we can use this mechanism as of today as we support redis authentication, we could however try to simplify the process to generating the v4 signature in a blocking call outside the event loop.

[holomekc]

Maybe via a more general approach: #376 ?

[geniusit]

Could you provide a code sample how to connect to a AWS ElasticCache Redis server with the IAM Module?
Here is some documentation
This sample code from AWS uses the Lettuce driver.
I m trying to accomplish it with low level vertx client as explained in the quarkus documentation
I need to use it with the STSAssumeRoleSessionCredentialsProvider
Some aws documentation

How to do ?

Thank you

[geniusit]

@pmlopes I don't see where you support redis authentication?
How to pass an AwsCredentialsProvider ?
see RedisOptions

[holomekc]

Hi sorry for the late response. I am currently drowning in work, that is also the reason, why I could not finish the requested more generic approach.

To your question. You do not pass the Credentials Provider from AWS. When you check the example code they also do not do that. They implement the RedisCredentialsProvider, which is a lettuce interface. They implement this interface in RedisIAMAuthCredentialsProvider. What they do is they use the default credentials provider to get the credentials provided by whatever (Static, STSAssumeRoleSessionCredentialsProvider, Web...something :), etc. ) and then use it together with IAMAuthTokenRequest to create a username and password and this is then provided to lettuce. Check the resolveCredentials method there.

That is exactly the same I created here. I do not provide a solution specific for AWS but instead you are able to dynamically exchange the credentials at any time. Form where the credentials are provided do not matter. You can implement your own solution then for AWS, Azure, etc.

[geniusit]

Hi,

I still have some trouble to establish a secure connection.
According to the vertx documentation, the syntax is : redis://[:password@]host[:port][/db-number]

So I try with the following URI :

redis://:myreplicationgroupid/?Action=connect&User=myuser&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEK3%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCWV1LXdlc3QtMyJGMEQCIDWGEasnjkPcZDSQrHhBLMG70%2FzXwu3eSzh80QhgtvyfAiBE2jeRY%2BMSLW7h9fBIe6nCDRrlRKevzGMDUTNkT1gGCCq6Agj2%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAAaDDI3NDA5NTU1MzQ4NCIMysrNfYDUx%2B6mQpJtKo4CnN7nvTF92tLV8sas23VOCcCzIkHslIc5G6f48v9CTpyA6%2FACtjtAj9HtW8DE%2BMT7X%2BCtc1Ar7TKpi4dmhdCkM%2F1FCdd5ROA9rtlovn3C4Ubn438mtFKVtjoTm0JEkkgwsl%2FG3AYsUzDMiIVFAEF0NMMR7Ov3p02qmKJEasOGZsGqMQvkYmCMrcYaf8h%2FjLLEq5ZeR%2BHuSYAd6QebG3TlCPshWbFzr%2BG6dTMI2Y3yUX1IhQn3jGz%2FcRZgkf1uLwsxYkrwjlTO6oG857nHsG7c%2BQLMdP33AD9lNUi3qerIRBi95t%2FyYUu9RDf0X8Ozn1Z6zClZ5CugCETs%2Fehc7bwfTflp5E6YBv8%2F%2FYFHVJmMMOC3o6QGOp4BhwP9E0uTCjDQBFDwKAIzDdnXh1X6fQFPpdw3PPmzW20NBttDPN%2B%2F9Loy3R1AFSeW4ECNxnfWWdY82VqsD3kOgaxMEyojIo4FTol2S8NyieHEeouiiq9uatpTKlijZVZVlTqrTMSIDiWxMqITvZpRy5HLbp8FVaVf0%2Fy2aw46LWYoeJ%2B0lh4zUOqJLNkY79lP3XdpOH%2FmltO7Pn6Hm5s%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230613T211304Z&X-Amz-SignedHeaders=host&X-Amz-Expires=900&X-Amz-Credential=ASIAT7UK6HPGISMY3FWE%2F20230613%2Feu-west-3%2Felasticache%2Faws4_request&X-Amz-Signature=1a770f6e4ab2d7024078281defd2e3814c409fdc5c4b068edfbf9e19627f9056@master.myserver.euw3.cache.amazonaws.com:6379/1

As I understand, this is the same than what the lettuce driver does. I don't have any trouble to establish a connection with lettuce.
As you say, the resolveCredentials method uses this password.

But with vertx, I got the following error :

 {
            "name": "Redis connection health check",
            "status": "DOWN",
            "data": {
                "reason": "client [<default>]: Fail to parse connection string authority"
            }
        },

The password has been generated in the same way than this sample code

I use quarkus and I follow their documentation : https://quarkus.io/guides/redis-reference#programmatic-redis-hosts

I tried to see how lettuce works by cloning the repo but I don't see exactly which URI syntax is used to establish the connection.

Any help? thank you

[holomekc]

Hi @geniusit
Sorry I need to clarify something first. The things I mentioned above are not implemented yet. So at the moment vertx only allows static credentials.

Regarding the other part. We are using Quarkus as well. We do not set the password in the url. We use quarkus.redis.password and then set the value via environment variable from a kubernetes secret. You also need rediss:// and not redis:// as far as I know. But as said static credentials and we also use the default user at the moment.

[geniusit]

Hi @holomekc
Thank you for the answer. I succeed to establish a connection with static user/password.
I think the documentation should precise about rediss:// for TLS. Both vertx and quarkus documentation.
Because I only found that information here : #53

Btw If I don't set the username I got an authentication error. So I added it like that :

rediss://myusername:mypassword@myserver.com:6379/1

The documentation should precise it as well.

Now it's working but my manager doesn't like static credentials. Do you think there is a chance that you implement dynamic credentials as Lettuce does?

Thank you!

[holomekc]

See #391
I will try to apply the requested changes today. Nevertheless, it is a bigger change and I hope I did not destroy something. At least the test locally were happy. But it may take some time.

[holomekc]

Maybe an additional info. Even when the changes are done. I think you will not be able to use them directly. The requested approach are applied on level of the Redis client creation and are not part of the RedisOptions. So I think Quarkus will need to adjust things as well. Something like this: https://quarkus.io/guides/redis-reference#customize-the-redis-options-programmatically will not work in this case. I guess. Not sure if there would be a different approach to work around the quarkus config.

[pmlopes]

The way to handle this is that we need some sort of async supplier in RedisOptions what will be used here:

https://github.com/vert-x3/vertx-redis-client/blob/master/src/main/java/io/vertx/redis/client/impl/RedisConnectionManager.java#L188

At the hello moment we can compute the password (the iam token from AWS is used as a password) see the link:

https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/auth-iam.html

[holomekc]

I provided exactly that in #376, but an even more general approach was requested.