# Amazon S3

Checkpoint/restore to/from Amazon S3 is as seamless as it is to/from local storage.

{% hint style="info" %}
Both [general-purpose](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html) and [directory](https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html) S3 buckets are supported. Use directory buckets for better performance for the cost of lower availability. See [Amazon S3 Storage Classes](https://aws.amazon.com/s3/storage-classes/#topic-0) for more details.
{% endhint %}

## Prerequisites

1. Create an account with Cedana, to get access to the GPU plugin. See [authentication](/daemon/get-started/authentication.md).
2. Set the Cedana URL & authentication token in the [configuration](/daemon/get-started/configuration.md).
3. Install the **storage/s3** plugin with `sudo cedana plugin install storage/s3`.
4. Set AWS credentials in the [configuration](/daemon/get-started/configuration.md).
5. Ensure the daemon is running, see [installation](/daemon/get-started/installation.md).
6. Do a health check to ensure the plugin is ready, see [health checks](/daemon/get-started/health.md).

## Checkpoint

To checkpoint to an S3 bucket, simply set the `--dir` to a path that starts with `s3://<bucket>`, for example:

```sh
cedana dump ... --dir s3://my-bucket/path/to/dir
```

For example, as explained in [managed checkpoint/restore](/daemon/checkpoint-restore/cr.md#managed-checkpoint-restore), to checkpoint a job to S3:

```sh
cedana dump job my-job-1 --dir s3://checkpoints-bucket
```

If you do `cedana job list`, you will see the latest checkpoint:

```
ID            TIME                 SIZE     PATH
my-job-1      2025-02-19 12:30:36  -        s3://checkpoints-bucket/dump-job.tar
```

## Restore

Similarly, to restore from an S3 bucket, simply set the `--path` to your checkpoint path in S3, for example:

```sh
cedana restore ... --path s3://my-bucket/path/to/dump.tar
```

For example, as explained in [managed checkpoint/restore](/daemon/checkpoint-restore/cr.md#managed-checkpoint-restore), to restore a job from S3:

```sh
cedana restore job --attach my-job-1
```

This will automatically restore from the latest checkpoint for `my-job-1`, which is stored in S3.

## Compression

All compression algorithms supported for basic checkpoint/restore are supported. See [compression](/daemon/checkpoint-restore/cr.md#compression) for more information.

{% hint style="info" %}
For better performance when remote checkpointing/restoring large processes/containers, especially when using [checkpoint/restore with GPUs](/daemon/checkpoint-restore/cr-1.md), always use compression. The `lz4` compression algorithm is a good compromise between speed and compression ratio.
{% endhint %}

## Streaming

High-performance low-overhead streaming of checkpoints is also supported by the `storage/s3` plugin. Follow instructions on [checkpoint/restore streamer](/daemon/checkpoint-restore/cr.md#checkpoint-restore-streamer) to use streaming with this plugin.

## Enable by default

To enable streaming by default, set the `Checkpoint.Dir` field in the [configuration](/daemon/get-started/configuration.md) to a path that starts with `s3://`.

## See also

* [Google Cloud Storage](/daemon/storage/gcs.md)
* [Cedana Storage](/daemon/storage/cedana.md)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.cedana.ai/daemon/storage/s3.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.