S3 Policies

You can manage access to your buckets and objects using policies. Policies are JSON files. In a policy you specify a set of targets (buckets or objects), users, and operations the users are alowed to perform on the targets. A list of supported bucket and object operations are listed in the Ceph docs.

Open/public access

S3 buckets can be opened to the public, allowing read and/or write functionality to any user.

In this article we will explore how this can be achieved using the s3cmd client.

Note: before we start it would be appropriate to warn that allowing public upload to an S3 bucket is something which should be used with utmost caution. Please be careful if you enable this. Equally buckets which allow public downloads can quickly be discovered and scanned, chosing bucket names prefixed with a project name can help avoid them being discovered easily.

Using S3cmd

The s3cmd can be used to set bucket policies but requires that the policy be provided as a json document (no canned policies are available with s3cmd).

Assuming that the json policy is stored in public-policy.json (which can be obtained via the minio client get-json command).

Firstly, to ensure that no policy is set, we can use the s3cmd info command.

s3cmd info s3://publictest

Create the policy file, public-policy.json, here for public access:

{
  "Statement": [
    {
      "Action": [
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "*"
        ]
      },
      "Resource": [
        "arn:aws:s3:::publictest"
      ],
      "Sid": ""
    },
    {
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "*"
        ]
      },
      "Resource": [
        "arn:aws:s3:::publictest/*"
      ],
      "Sid": ""
    }
  ],
  "Version": "2012-10-17"
}

Now set the policy for the bucket

s3cmd setpolicy public-policy.json s3://publictest

Check the policy

s3cmd info s3://publictest

For public (unauthenticated) users simple curl commands can be used to access the bucket.

curl https://objectstore.hpccloud.mpcdf.mpg.de/publictest/test.1mb \
    -o download.file

Information about the bucket itself, including the contents, can be found by accessing the bucket URL via a web browser or curl

curl https://objectstore.hpccloud.mpcdf.mpg.de/publictest/ | xmllint --format -

And finally to delete the policy

s3cmd delpolicy s3://publictest

Share with specific projects

You can manage acces of other HPC Cloud projects to your buckets using policies. Projects are are identified by the project UUID, you can look it up with the openstack client:

openstack project show <project> -c id -f value

For example address a policy to the project 2a949b0cb461482ea7671b5bfdadc4f1 use the following principal in the policy:

"Principal": {
  "AWS": [
    "arn:aws:iam::2a949b0cb461482ea7671b5bfdadc4f1:root"
  ]
}

A policy allowing read access to this principle in the test bucket would then look like this:

{
  "Statement": [
    {
      "Action": [
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::2a949b0cb461482ea7671b5bfdadc4f1:root"
        ]
      },
      "Resource": [
        "arn:aws:s3:::test"
      ],
      "Sid": ""
    },
    {
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::2a949b0cb461482ea7671b5bfdadc4f1:root"
        ]
      },
      "Resource": [
        "arn:aws:s3:::test/*"
      ],
      "Sid": ""
    }
  ],
  "Version": "2012-10-17"
}