# Automation

Cedana offers two distinct types of automation to best take advantage of our checkpoint/restore.

### Policies

Through the dashboard, you can enable and set policies, that configure bits of automation like heartbeat checkpointing, which will act as a cron job and checkpoint every *n* minutes.

<figure><img src="/files/T2vTQLxcHkTmKuCQ7Wb2" alt=""><figcaption></figcaption></figure>

Policies in cedana operate through a Trigger -> Filter -> Action mechanism, allowing users to build complex automations. For example, you could build:

* a policy that automatically fires a checkpoint if a pod exceeds 80% memory utilization in a specific namespace or deployment,
* a policy that deletes checkpoints past a certain age,
* a policy that checkpoints on some webhook

We aim for the policy engine to be extensible, so value feedback for more features! Policies work best when integrated into Kubernetes however, as the below section illustrates.

### Integration into Kubernetes

The second is simply our deep integration into Kubernetes. As users, you are likely to use tools such as Armada, Kueue, KServe, Dynamo, etc, that are come with their own opinions about container lifecycle, management, preemption and more.

As we work seamlessly with the scheduler, cedana automatically inserts the checkpoint if a pod (that had a checkpoint previously taken) auto-scales. For example, if using a tool like Karpenter, cedana will automatically restore from a checkpoint when the underlying node is preempted and a new node is spun up, *instead of starting from scratch.*

Our goal is to be as pain-free to integrate into your existing tooling as possible, and as such simply work with schedulers like the ones listed above. The best demonstration of this can be found in [Jobs](/cedana-kubernetes/jobs.md), which can more or less be applied universally to kubernetes kinds.

At the node level, Cedana uses the `cedana.ai/not-ready` taint while a node is bootstrapping. That taint is removed only after both the `cedana-helper` and `cedana-health-check` pods are Ready, which prevents workloads from landing on a node before Cedana is ready to restore them.

If you need Cedana to preserve additional pod spec fields during checkpointing, or map an older field name into the saved spec, see [Pod Spec Whitelist and Remapping](/cedana-kubernetes/podspec-whitelist-remapping.md).

See our Managed Dynamo for an example with a more complex Kubernetes deployment.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.cedana.ai/cedana-kubernetes/automation.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
