> For the complete documentation index, see [llms.txt](https://docs.cedana.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.cedana.ai/daemon/storage/s3.md).

# Amazon S3

Checkpoint/restore to/from Amazon S3 is as seamless as it is to/from local storage.

{% hint style="info" %}
Both [general-purpose](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html) and [directory](https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html) S3 buckets are supported. Use directory buckets for better performance for the cost of lower availability. See [Amazon S3 Storage Classes](https://aws.amazon.com/s3/storage-classes/#topic-0) for more details.
{% endhint %}

## Prerequisites

1. Create an account with Cedana, to get access to the GPU plugin. See [authentication](/daemon/get-started/authentication.md).
2. Set the Cedana URL & authentication token in the [configuration](/daemon/get-started/configuration.md).
3. Install the **storage/s3** plugin with `sudo cedana plugin install storage/s3`.
4. Configure an AWS credential mode as described below.
5. Ensure the daemon is running, see [installation](/daemon/get-started/installation.md).
6. Do a health check to ensure the plugin is ready, see [health checks](/daemon/get-started/health.md).

## Authentication

Set `AWS.CredentialsMode` to one of the following values. It defaults to `static` for compatibility with existing installations.

* `static`: set both `AWS.AccessKeyID` and `AWS.SecretAccessKey` (or `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`). Cedana uses only those credentials.
* `eksPodIdentity`: use [EKS Pod Identity](https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html). The EKS Pod Identity Agent must inject `AWS_CONTAINER_CREDENTIALS_FULL_URI` and `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` into the Cedana helper pods. Cedana validates both variables and uses the AWS default credential chain.
* `ambient`: use the AWS default credential chain without Pod Identity-specific validation. This supports sources such as IRSA, EC2 instance roles, environment credentials, and shared AWS configuration.

For example, to use EKS Pod Identity:

```json
{
  "aws": {
    "credentials_mode": "eksPodIdentity",
    "region": "us-east-1"
  }
}
```

Custom S3-compatible endpoints remain supported through `AWS.Endpoint`; Cedana enables path-style access when an endpoint is configured.

## Checkpoint

To checkpoint to an S3 bucket, simply set the `--dir` to a path that starts with `s3://<bucket>`, for example:

```sh
cedana dump ... --dir s3://my-bucket/path/to/dir
```

For example, as explained in [managed checkpoint/restore](/daemon/checkpoint-restore/cr.md#managed-checkpoint-restore), to checkpoint a job to S3:

```sh
cedana dump job my-job-1 --dir s3://checkpoints-bucket
```

If you do `cedana job list`, you will see the latest checkpoint:

```
ID            TIME                 SIZE     PATH
my-job-1      2025-02-19 12:30:36  -        s3://checkpoints-bucket/dump-job.tar
```

## Restore

Similarly, to restore from an S3 bucket, simply set the `--path` to your checkpoint path in S3, for example:

```sh
cedana restore ... --path s3://my-bucket/path/to/dump.tar
```

For example, as explained in [managed checkpoint/restore](/daemon/checkpoint-restore/cr.md#managed-checkpoint-restore), to restore a job from S3:

```sh
cedana restore job --attach my-job-1
```

This will automatically restore from the latest checkpoint for `my-job-1`, which is stored in S3.

## Compression

All compression algorithms supported for basic checkpoint/restore are supported. See [compression](/daemon/checkpoint-restore/cr.md#compression) for more information.

{% hint style="info" %}
For better performance when remote checkpointing/restoring large processes/containers, especially when using [checkpoint/restore with GPUs](/daemon/checkpoint-restore/cr-1.md), always use compression. The `lz4` compression algorithm is a good compromise between speed and compression ratio.
{% endhint %}

## Streaming

High-performance low-overhead streaming of checkpoints is also supported by the `storage/s3` plugin. Follow instructions on [checkpoint/restore streamer](/daemon/checkpoint-restore/cr.md#checkpoint-restore-streamer) to use streaming with this plugin.

## Enable by default

To enable streaming by default, set the `Checkpoint.Dir` field in the [configuration](/daemon/get-started/configuration.md) to a path that starts with `s3://`.

## See also

* [Google Cloud Storage](/daemon/storage/gcs.md)
* [Cedana Storage](/daemon/storage/cedana.md)
* [Cedana Storage Express](/daemon/storage/csx.md)