Cedana Storage Express

Cedana Storage Express (CSX) is Cedana's NFS-backed storage solution. It uses tmpfs, local disk and NFS to efficiently do checkpoint/restore. The storage/csx plugin lets the Cedana daemon checkpoint and restore through a local CSX daemon.

Prerequisites

Authenticate with Cedana to access released plugins. See authentication.
Install the storage/csx plugin with sudo cedana plugin install storage/csx.
Install the csx binary with sudo cedana plugin install csx.
Ensure the Cedana daemon is running. See installation.

Start the CSX daemon

Once the Cedana daemon is running, start the CSX daemon:

sudo ./csx

The CSX daemon requires root privileges because it creates tmpfs mounts. The storage/csx plugin connects to its Unix socket, configured by CSX.SockAddr in the daemon configuration, so the CSX daemon must remain running while checkpointing or restoring. Complete documentation for the CSX daemon is work in progress.

Checkpoint

To checkpoint to CSX, set --dir to csx://:

cedana dump ... --dir csx://

For example, from a Cedana source checkout, start the included test workload as a managed job:

cedana exec ./test/workloads/date-loop.sh -j test -a

Then checkpoint the job from another terminal:

cedana dump job test --dir csx://

Cedana records the generated checkpoint path in the job metadata with the csx:// prefix. This prefix ensures that subsequent operations automatically use the storage/csx plugin.

Restore

As explained in managed checkpoint/restore, a managed job can be restored without specifying its checkpoint path:

cedana restore job test

Cedana reads the latest checkpoint path from the job metadata and automatically restores it through CSX.

To restore an unmanaged checkpoint, pass its recorded CSX path explicitly:

cedana restore ... --path csx://<dump-path>

Compression

All compression algorithms supported for basic checkpoint/restore are supported. See compression for more information.

Streaming

Checkpoint/restore streaming is not currently supported by the storage/csx plugin. Leave --streams unset or set it to 0 when using CSX.

Enable by default

To use CSX without passing --dir csx:// for every checkpoint, set CEDANA_CHECKPOINT_DIR=csx:// in the Cedana daemon's environment.

Alternatively, set Checkpoint.Dir in the daemon configuration:

{
  "checkpoint": {
    "dir": "csx://"
  }
}

Restart the Cedana daemon after changing its environment or configuration.