S3 Usage Howto
There are several ways to view your storage utilization in S3.
Here we’ll summarize how to view usage on a global level and also per container/bucket.
See the note at the end of this article about storage quotas.
Firstly using the openstack command:
To view global storage statistics
$ openstack object store account show
+------------+---------+
| Field | Value |
+------------+---------+
| Account | v1 |
| Bytes | 1048604 |
| Containers | 2 |
| Objects | 2 |
+------------+---------+
To view container level statistics:
$ openstack container show mycontainer
+--------------+-------------+
| Field | Value |
+--------------+-------------+
| account | v1 |
| bytes_used | 28 |
| container | mycontainer |
| object_count | 1 |
+--------------+-------------+
$ openstack container show mycontainer2
+--------------+--------------+
| Field | Value |
+--------------+--------------+
| account | v1 |
| bytes_used | 1048576 |
| container | mycontainer2 |
| object_count | 1 |
+--------------+--------------+
Using the s3cmd command:
To gain a global view
$ s3cmd du
28 1 objects s3://mycontainer/
1048576 1 objects s3://mycontainer2/
------------
1048604 Total
$ s3cmd du s3://mycontainer/
28 1 objects s3://mycontainer/
To get human readable numbers
$ s3cmd du -H
28 1 objects s3://mycontainer/
1024K 1 objects s3://mycontainer2/
------------
1024K Total
Using the minio client mc:
$ mc du -h
To gain a global view.
$ mc du cephs3
1.0MiB
To view global usage (split into containers/buckets)
$ mc du -r cephs3
28B mycontainer
1.0MiB mycontainer2
1.0MiB
To view usage per container/bucket
$ mc du -r cephs3/mycontainer2
1.0MiB mycontainer2
Note: working with sub-containers/buckets
Although S3 containers/buckets are the real units of storage many clients allow you to address data within these buckets in a similar manner to a filesystem (with / being used as a separator for sub-containers/buckets similar to a directory).
To view the storage usage of sub-containers/buckets you can use the s3cmd and/or the minio command.
$ s3cmd ls s3://mycontainer2/phase2/
2021-08-31 09:37 0 s3://mycontainer2/phase2/
2021-08-31 09:40 1048576 s3://mycontainer2/phase2/1mb_testfile_1
2021-08-31 09:40 1048576 s3://mycontainer2/phase2/1mb_testfile_2
Using the s3cmd:
$ s3cmd du s3://mycontainer2/
3145728 4 objects s3://mycontainer2/
Viewing the sub-container/bucket
$ s3cmd du s3://mycontainer2/phase2/
2097152 3 objects s3://mycontainer2/phase2/
For human readable output
$ s3cmd -c s3cfg.dev2 du s3://mycontainer2/phase2/ -H
2048K 3 objects s3://mycontainer2/phase2/
Using the minio client:
$ mc du cephs3/mycontainer2/phase2/
2.0MiB mycontainer2/phase2
Note: Storage Quotas.
It is important to remember that the S3 quotas apply to a whole openstack project and not individual users which are granted access to that project.
Also note that Quotas are applied to:
Storage Volume
Number of Objects