botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Signature does not match.

[jgourmelen]

I'm encountering this error when trying to configure S3 on Scaleway: "botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Signature does not match."

Is it possible to override the signature_version parameter in Boto3?

[acjohnson]

What are you setting AWS_ENDPOINT_URL to? Check out this link for how to format your endpoint URL and let us know if that helps or not: https://stackoverflow.com/a/70383653

[jgourmelen]

AWS_ENDPOINT_URL: "https://s3.fr-par.scw.cloud"
I think the value is OK

[acjohnson]

I do not currently have a Scaleway account... I'm not adverse to creating one at some point, but if you are able to test this out I would appreciate it. I have two examples below in python, both examples should work, but I'll be curious what your results are:

Example 1 (based on grafana-backup s3_upload.py)

#!/usr/bin/env python3

import boto3
from botocore.exceptions import NoCredentialsError


backup_dir = "/tmp"
s3_file_name = 'testing.tar.gz'
archive_file = '{0}/{1}'.format(backup_dir, s3_file_name)

aws_access_key_id = 'SCWXXXXXXXXXXXXXXXXX'
aws_secret_access_key = '110e8400-e29b-11d4-a716-446655440000'
aws_default_region = 'fr-par'
aws_endpoint_url = 'https://s3.fr-par.scw.cloud'
aws_s3_bucket_name = 'my-bucket'
aws_s3_bucket_key = 'my-object'

session = boto3.Session(
    aws_access_key_id=aws_access_key_id,
    aws_secret_access_key=aws_secret_access_key,
    region_name=aws_default_region
)

s3 = session.resource(
    service_name='s3',
    endpoint_url=aws_endpoint_url
)

s3_object = s3.Object(aws_s3_bucket_name,
                      '{0}/{1}'.format(aws_s3_bucket_key, s3_file_name))

try:
    s3_object.put(Body=open(archive_file, 'rb'))
    print("Upload to S3 was successful")
except FileNotFoundError:  # noqa: F821
    print("The file was not found")
except NoCredentialsError:
    print("Credentials not available")

Example 2 (based on Scaleway doc: https://www.scaleway.com/en/docs/storage/object/api-cli/post-object/#python)

#!/usr/bin/env python3

import logging
import boto3
import requests
from botocore.exceptions import ClientError


# Generate a presigned URL for the S3 object
session = boto3.session.Session()

s3_client = session.client(
    service_name='s3',
    region_name='fr-par',
    use_ssl=True,
    endpoint_url='http://s3.fr-par.scw.cloud',
    aws_access_key_id='SCWXXXXXXXXXXXXXXXXX',
    aws_secret_access_key='110e8400-e29b-11d4-a716-446655440000'
)

bucket_name = "my-bucket"
object_name = "my-object"

try:
    response = s3_client.generate_presigned_post(Bucket=bucket_name,
                                                 Key=object_name)
except ClientError as e:
    logging.error(e)
    exit()

# The response contains the presigned URL and required fields
print(response)


with open('/tmp/testing.tar.gz', 'rb') as f:
    files = {'file': (object_name, f)}
    http_response = requests.post(
        response['url'], data=response['fields'], files=files)
    print(http_response.content)