Snapshotting

Restate workers can be configured to periodically publish snapshots of their partition state to a shared destination. Snapshots act as a form of backup and allow nodes that had not previously served a partition to bootstrap a copy of its state. Without snapshots, rebuilding a partition processor would require the full replay of the partition's log. Replaying the log might take a long time or even be impossible if the log was trimmed.

Restate clusters should always be configured with a snapshot repository to allow nodes to efficiently share partition state. Restate currently supports using Amazon S3 (or another API-compatible object store) as a shared snapshot repository. To set up a snapshot destination, update your server configuration as follows:

[worker.snapshots]
destination = "s3://snapshots-bucket/cluster-prefix"
snapshot-interval-num-records = 10000

This enables automated periodic snapshots to be written to the specified bucket. You can also trigger snapshot creation manually using the restatectl create-snapshot command. We recommend testing the snapshot configuration by requesting a snapshot and examining the contents of the bucket. You should see a new prefix with each partition's id, and a latest.json file pointing to the most recent snapshot.

No additional configuration is required to enable restoring snapshots. When partition processors first start up, and no local partition state is found, the processor will attempt to restore the latest snapshot from the repository. This allows for efficient bootstrapping of additional partition workers.

Experimenting with snapshots without an object store

For testing purposes, you can also use the file:// protocol to publish snapshots to a local directory. This is mostly useful when experimenting with multi-node configurations on a single machine. The file provider does not support conditional updates, which makes it unsuitable for potentially contended operation.

For S3 bucket destinations, Restate will use the AWS credentials available from the environment, or the configuration profile specified by AWS_PROFILE environment variable, falling back to the default AWS profile.

To learn more about how Restate uses snapshots, see Logs and Snapshots.