S3-compatible storage

Instead of the server's own disk, the site can keep its files in two buckets on any storage service that speaks the S3 protocol. Use it when the server's disk does not last, when you run more than one copy of the site, or on Cloudflare Workers if you would rather not use R2 bindings. This page is for whoever sets that up. How files are stored explains the two stores first.

Settings

Variable	What it does	Default
`STORAGE_PROVIDER`	`s3` chooses this kind of storage.	`local` on Node, R2 bindings on Cloudflare
`S3_ENDPOINT`	The address of the storage service. Leave it empty for AWS S3. A slash at the end is removed.	empty
`S3_REGION`	The region requests are signed for.	`auto`
`S3_ACCESS_KEY_ID`	The access key.	none, required
`S3_SECRET_ACCESS_KEY`	The secret key.	none, required
`S3_PUBLIC_BUCKET`	The bucket for public files.	none, required
`S3_PRIVATE_BUCKET`	The bucket for members' files and rehearsal tracks.	none, required
`S3_FORCE_PATH_STYLE`	Whether the bucket's name goes in the path of the address or in front of the host name.	`true` when `S3_ENDPOINT` is set

The same settings work on Node, in Docker and on Cloudflare Workers. On Workers, put the two keys in with npx wrangler secret put and the rest under [vars]; the R2 bucket bindings are then not needed.

One set of keys is used for both buckets.

`S3_REGION`

The default, auto, is right for Cloudflare R2 and Google Cloud Storage. For everything else set the real region. In particular, for AWS S3 always set the bucket's region: with no endpoint, a region of auto or an empty one makes the site address the bucket in us-east-1, and auto is not a region AWS signs for.

`S3_FORCE_PATH_STYLE` and how addresses are formed

`S3_ENDPOINT`	`S3_FORCE_PATH_STYLE`	A file is fetched from
empty	ignored	`https://BUCKET.s3.REGION.amazonaws.com/KEY`
set	`true`, or unset	`ENDPOINT/BUCKET/KEY`
set	`false`	the endpoint with the bucket's name put in front of its host: `https://BUCKET.s3.us-west-004.backblazeb2.com/KEY`

Whenever an endpoint is set, the bucket's name goes in the path unless you say otherwise. Services that want it in front of the host name need S3_FORCE_PATH_STYLE=false written out.

Before you start

Create both buckets yourself. The site never creates a bucket. (The Docker Compose file does it for its own MinIO, as a separate step.)
Leave both buckets private. Neither needs a public-read policy. The site fetches public files with its keys and serves them at /files/. A public address for the public bucket is optional, and comes later: Public file addresses and upload limits. The private bucket must never have one.
Make keys that reach only these two buckets, and that can read, write and delete objects in them. They need nothing else: not creating buckets, not other buckets.

Examples

Each block is the complete set of storage settings for that service. Replace the bucket names and keys with your own. The MinIO example is the one in the product's own Compose file; the others follow each service's published endpoint format and the product's own table of examples.

AWS S3

STORAGE_PROVIDER=s3
S3_REGION=ca-central-1
S3_ACCESS_KEY_ID=AKIAEXAMPLE
S3_SECRET_ACCESS_KEY=example
S3_PUBLIC_BUCKET=example-choir-public
S3_PRIVATE_BUCKET=example-choir-private

No S3_ENDPOINT. Avoid dots in the bucket names, because the name becomes part of the host name.

A policy for the IAM user that covers what the site does:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": [
        "arn:aws:s3:::example-choir-public/*",
        "arn:aws:s3:::example-choir-private/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": [
        "arn:aws:s3:::example-choir-public",
        "arn:aws:s3:::example-choir-private"
      ]
    }
  ]
}

The site never lists a bucket. s3:ListBucket is there because, without it, AWS answers "forbidden" instead of "not found" for a file that does not exist, and the site then reports an error where it should report a missing file.

Cloudflare R2, from a Node server

STORAGE_PROVIDER=s3
S3_ENDPOINT=https://YOUR_ACCOUNT_ID.r2.cloudflarestorage.com
S3_REGION=auto
S3_ACCESS_KEY_ID=example
S3_SECRET_ACCESS_KEY=example
S3_PUBLIC_BUCKET=choir-public
S3_PRIVATE_BUCKET=choir-private

Create an R2 API token with "Object Read & Write" permission, limited to the two buckets; it gives you the access key and secret. A site running on Cloudflare Workers itself normally uses bucket bindings instead.

MinIO

STORAGE_PROVIDER=s3
S3_ENDPOINT=http://minio:9000
S3_REGION=us-east-1
S3_FORCE_PATH_STYLE=true
S3_ACCESS_KEY_ID=choir
S3_SECRET_ACCESS_KEY=example
S3_PUBLIC_BUCKET=choir-public
S3_PRIVATE_BUCKET=choir-private

This is what docker-compose.yml in the source sets, with MinIO's root user and password as the keys and a small minio-setup service that creates the two buckets at first start. See Docker Compose: PostgreSQL and MinIO.

Backblaze B2

STORAGE_PROVIDER=s3
S3_ENDPOINT=https://s3.us-west-004.backblazeb2.com
S3_REGION=us-west-004
S3_FORCE_PATH_STYLE=false
S3_ACCESS_KEY_ID=example
S3_SECRET_ACCESS_KEY=example
S3_PUBLIC_BUCKET=example-choir-public
S3_PRIVATE_BUCKET=example-choir-private

The region is the part of the endpoint between s3. and .backblazeb2.com; B2 shows the endpoint on the bucket's page.

DigitalOcean Spaces

STORAGE_PROVIDER=s3
S3_ENDPOINT=https://nyc3.digitaloceanspaces.com
S3_REGION=nyc3
S3_FORCE_PATH_STYLE=false
S3_ACCESS_KEY_ID=example
S3_SECRET_ACCESS_KEY=example
S3_PUBLIC_BUCKET=example-choir-public
S3_PRIVATE_BUCKET=example-choir-private

Wasabi

STORAGE_PROVIDER=s3
S3_ENDPOINT=https://s3.ca-central-1.wasabisys.com
S3_REGION=ca-central-1
S3_FORCE_PATH_STYLE=false
S3_ACCESS_KEY_ID=example
S3_SECRET_ACCESS_KEY=example
S3_PUBLIC_BUCKET=example-choir-public
S3_PRIVATE_BUCKET=example-choir-private

Google Cloud Storage

STORAGE_PROVIDER=s3
S3_ENDPOINT=https://storage.googleapis.com
S3_REGION=auto
S3_FORCE_PATH_STYLE=true
S3_ACCESS_KEY_ID=GOOGEXAMPLE
S3_SECRET_ACCESS_KEY=example
S3_PUBLIC_BUCKET=example-choir-public
S3_PRIVATE_BUCKET=example-choir-private

Google Cloud Storage speaks S3 through its "interoperability" interface, which needs an HMAC key, not a service account's JSON file. Make one under the bucket settings' Interoperability tab, for a service account that can manage objects in the two buckets.

Check it

Run the configuration check. Under storage it should show "provider": "s3", your endpoint (or (AWS)), both bucket names and (set) for the access key.
Start the site, log in to the admin panel and upload a picture to the gallery. That writes to the public bucket and reads it back.
Upload a document for members, then open it. That does the same for the private bucket.

The check only confirms the settings are present. It does not contact the storage service, so step 2 is the real test.

Memory

An upload to S3 storage is held whole in the server's memory while it is sent on, because the storage service must be told the size first. A 100 MB video needs at least that much memory free for the moment of the upload. Downloads are passed through as they arrive and are not held. If the server is small, lower the upload limits.

Backups

Nothing in the site copies the buckets. Use the storage service's own versioning or replication: Back up PostgreSQL, MySQL, S3 storage and Cloudflare.

If something goes wrong

A failed upload shows as Upload failed in the admin panel. The reason is in the server log, as S3 PUT failed: followed by the status code and the start of the service's answer.

What you see	Cause
`STORAGE_PROVIDER=s3 needs S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, S3_PUBLIC_BUCKET and S3_PRIVATE_BUCKET` among the `[config]` lines	One of the four is not set. The server starts anyway and every upload fails.
`S3 PUT failed: 403` with `SignatureDoesNotMatch`	A wrong secret key, or the wrong `S3_REGION`.
`S3 PUT failed: 403` with `AccessDenied`	The keys may not write to that bucket.
`S3 PUT failed: 404` with `NoSuchBucket`	The bucket does not exist, or `S3_FORCE_PATH_STYLE` is the wrong way round for this service.
`S3 PUT failed: 400` with `AuthorizationHeaderMalformed`, on AWS	`S3_REGION` is missing or is not the bucket's region.
`S3 GET failed: 403` when opening a file that was deleted	On AWS, the keys lack `s3:ListBucket`.
A certificate or host name error	The bucket's name in front of the host does not match the service's certificate: set `S3_FORCE_PATH_STYLE=true`, or use a bucket name without dots.

Settings​

S3_REGION​

S3_FORCE_PATH_STYLE and how addresses are formed​

Before you start​

Examples​

AWS S3​

Cloudflare R2, from a Node server​

MinIO​

Backblaze B2​

DigitalOcean Spaces​

Wasabi​

Google Cloud Storage​

Check it​

Memory​

Backups​

If something goes wrong​