# Persistent Volumes

A Rapids container starts from its image every time: what it writes to its own filesystem, which holds 1 GB at most, is gone when the instance stops. A **volume** is storage that keeps its files across deployments, restarts and scale to zero. It is mounted into every instance of the container at a path you choose.

Use a volume for a SQLite database, uploaded files, a cache or an index that should survive scale to zero, or a model you download once. For a database that needs replicas, or point-in-time recovery for PostgreSQL, use a [managed database](https://docs.danubedata.ro/databases-overview); for files that several containers share or that outgrow 50 GB use [object storage](https://docs.danubedata.ro/object-storage).

## At a glance

| | |
|---|---|
| **Volumes per container** | 1 |
| **Size** | 1 to 50 GB in whole gigabytes. Larger later, never smaller |
| **Price** | EUR 0.12 per GB a month, on the full size, from the moment the volume is provisioned until we start removing it. Also while the container is stopped or scaled to zero |
| **Backups** | Up to 10 per container, billed on the space they take in the backup store (EUR 0.073 per GB a month) |
| **Limit per project** | 100 GB of volumes across all your containers, unless it has been raised for you |
| **Free allowance** | None |
| **Where it lives** | One copy, on one server in Falkenstein |

## What to know before you use one

- **There is one copy of the files, on one server.** If that server's disk is lost, the volume is lost with it. Backups are kept apart from the volume but in the same data centre campus; they are not an off-site copy. Keep a copy of anything you cannot lose somewhere else.
- **The volume is not encrypted at rest, and its backups are not guaranteed to be.** Encrypt sensitive data in your application.
- **Every instance mounts the same volume, so they all run on the same server.** Scaling out is limited by what is free on that one server.
- **The container depends on that one server.** While it is down or being maintained, a container with a volume cannot start.
- **Deleting the container deletes its volume and all its backups.** Backups do not protect against that.
- **The first request after the container was scaled to zero takes about 10 to 15 seconds longer**, because the volume is attached first. See [Cold starts](#cold-starts-and-timeouts).
- **You pay for the volume while the container is stopped or scaled to zero.**
- **A volume can be made larger but not smaller.** A larger volume is billed from the moment you ask, but running instances see the extra space only the next time they mount it. See [Making the volume larger](#making-the-volume-larger).

## Adding a volume

### With a new container

On the create form, the **Storage** section sets the size and the mount path. The volume is created with the container and is mounted in its first deployment.

### To an existing container

1. Open the container and go to the **Storage** tab
2. Click **Add a volume**
3. Enter the **Size** in GB and the **Mount path**, for example `/data`
4. Click **Add volume**

The container is rolled out to mount the volume, like any other deployment, and the previous revision keeps serving until the new one is ready. So that there is something to roll out, the container has to have been deployed at least once, and it must not be in the middle of another change; if it is, the console says so and the API refuses with `serverless.volume_container_busy` or `serverless.volume_container_not_deployed`. Adding, moving and deleting a volume each roll the container out, so each waits until the container is not being changed. Making a volume larger does not.

A container on the 10 Gb network tier cannot have a volume yet, and a container with a volume cannot be moved to that tier.

### Mount path

The path must be absolute and name a directory: `/data` or `/var/lib/myapp`, not `/data/` or `/var/../data`. Each part may contain the letters A to Z (without accents), digits, `.`, `_` and `-`, and the whole path at most 255 characters.

Whatever your image has at that path is hidden while the volume is mounted, so a volume cannot be mounted over what the system and your image need:

- These paths themselves are refused, but a directory beneath them, such as `/var/lib/myapp`, is fine: `/`, `/etc`, `/lib`, `/run`, `/var`, `/var/run`, `/usr`, `/usr/lib`, `/usr/libexec`, `/usr/local` and `/usr/local/lib`.
- These are refused with everything beneath them: `/dev`, `/proc`, `/sys`, `/run/secrets`, `/var/run/secrets`, `/bin`, `/sbin`, `/boot`, `/usr/bin`, `/usr/sbin`, `/usr/local/bin`, `/usr/local/sbin`, `/lib64`, `/usr/lib64`, `/lib/x86_64-linux-gnu`, `/lib/aarch64-linux-gnu`, `/usr/lib/x86_64-linux-gnu` and `/usr/lib/aarch64-linux-gnu`, the C library files in `/lib` (`/lib/ld-linux-aarch64.so.1`, `/lib/ld-musl-x86_64.so.1`, `/lib/ld-musl-aarch64.so.1`, `/lib/libc.musl-x86_64.so.1` and `/lib/libc.musl-aarch64.so.1`), and the files the system reads at fixed places: `/etc/hosts`, `/etc/hostname`, `/etc/resolv.conf`, `/etc/passwd`, `/etc/group`, `/etc/nsswitch.conf`, `/etc/ld.so.cache` and `/etc/localtime`.

The console and the API say which part of a path is the problem.

### Status

| Status | Meaning |
|---|---|
| **Provisioning** | The storage is being created. It is created when the container is rolled out with the volume, so for a container that is still being built or deployed this lasts until that rollout lands. Once the storage exists it usually binds within a minute |
| **Not provisioned yet** | The storage was not ready after 10 minutes. It goes on waiting and comes up by itself if the problem passes. If it does not, delete the volume and add it again; if **Delete volume** is refused because the container is still deploying, use **Cancel deployment** first, or wait until the deployment fails. Nothing is charged until a volume has been provisioned |
| **Active** | The volume is mounted and usable |
| **Growing** | A larger size is being applied. The volume stays mounted and keeps working |
| **Restoring** | A backup is being restored. See [Restoring a backup](#restoring-a-backup) |
| **Being deleted** | The container is being rolled out without the volume, and the volume will be removed |
| **Not provisioned** | Reserved for a volume that could not be created at all. The platform does not set it today: a volume that does not come up shows as **Not provisioned yet**. Nothing is charged. Delete it and add it again |

The same values are `pending`, `active`, `resizing`, `restoring`, `deleting` and `failed` in the API. A volume that is not provisioned after 10 minutes stays `pending`, with `failure.operation` `provision`.

## How a volume behaves

### Sharing

All instances of the container, and every [run](https://docs.danubedata.ro/rapids-runs) of it, mount the same volume at the same path. They run on the same server, so they see the same files at once. If several instances write the same files your application has to coordinate them with file locks or a single writer; for SQLite, **Maximum replicas** of 1 is the simplest.

Each volume belongs to one container. No other container, and no other project, can mount it.

### Permissions

A container that runs as a user other than root can write to the volume: the platform gives the instance the group `1000`, which owns the volume's files. Files your application creates belong to your user and to that group. Mount the volume where your image expects its data and leave ownership to the platform.

Every time an instance starts, the platform makes sure the files in the volume belong to that group. The more files a volume holds, the longer a start takes: about 3 seconds for every 100,000 files.

Writing files costs memory: the system's page cache and the bookkeeping for many files count toward the container's memory limit. A container that writes many files quickly needs more memory than its code alone uses.

### Cold starts and timeouts

A container that scales to zero detaches its volume. The first request after that attaches it again, which takes about 10 to 15 seconds on top of the usual cold start (we measured 14 to 15 seconds in all, against about 3 for a container without a volume). An instance that starts while another instance of the container is running does not wait for an attach.

- If a first request that slow is not acceptable, set **Minimum replicas** to 1. You then pay for a running instance, as for any container that does not scale to zero.
- The **request timeout** can count the wait for a cold start. Set it to 30 seconds or more for a container with a volume that scales to zero, or the first request after a quiet period can time out.

### Deployments, rollbacks and revisions

Old and new revisions mount the volume at the same time during a rollout, on the same server. A revision that was created before a volume was added, removed or restored mounted a different volume, and cannot be given traffic again: deploy a new revision instead.

### Free space

Deleting files frees room for new files inside the volume. It does not make the volume smaller or cheaper, and the platform does not hand the freed space back to the storage layer. A backup taken later still holds the blocks of the files you deleted until new data overwrites them. A volume cannot be made smaller; to get a smaller one, copy the files out (to object storage, say) and check the copy, delete the volume, add a smaller one and copy the files back. Backups taken from the larger volume cannot be restored into the smaller one.

## Making the volume larger

1. On the **Storage** tab, click **Make larger**
2. Enter the new size, which has to be larger than the current one and at most 50 GB (and no more than your project may still hold)
3. Click **Make larger**

The new size is billed from the moment you ask. The storage grows within a minute, and the volume stays mounted and keeps working. **The file system grows only when the volume is next mounted.** An instance that is already running keeps seeing the old size. To use the space at once, stop the container and start it again. A container that scales to zero gets the new size on its next start.

## Changing the mount path

On the **Storage** tab, click **Change mount path**. The container is rolled out with the volume mounted at the new path. The files do not move: they are the same files, seen at another path.

## Deleting a volume

On the **Storage** tab, click **Delete volume** and type the container's name to confirm. The container is rolled out without the volume, and when nothing mounts it any more, its files are deleted. This cannot be undone.

You are billed for the volume until we start removing its files, not until you click. A volume that never came up was never charged. [Backups](#backups) are kept when you delete the volume; they are deleted with the container.

Deleting the container deletes its volume and its backups.

## Backups

A backup is a copy of the volume's files, stored on separate storage in the same data centre campus. It is not an off-site copy. Backups belong to the container, not to the volume: they stay when the volume is deleted, and you can restore one into a container that has no volume.

### Taking a backup

1. On the **Storage** tab, in the **Backups** card, click **Back up now**
2. Give it a name if you like. Left empty, the backup is named after the time it was taken
3. Click **Back up now**

The backup is taken while the container runs, from a snapshot of the disk at one moment. It is what you would find after a power cut: files that were being written at that moment may be incomplete. If your application keeps a database in the volume, take the backup when it is idle, or use the database's own export as well.

How long it takes depends on how much data the volume holds. We measured about 35 to 40 MB per second, so a full 50 GB volume takes about 25 minutes. The backup shows its progress and becomes **Ready** when it can be restored.

| Limit | Value |
|---|---|
| **Backups kept per container** | 10 |
| **At the same time** | 1 |
| **Between two backups** | at least 10 minutes |
| **Requested per day** | 24, failed ones included |
| **A failed backup stays in the list** | 7 days |

A day is any 24 hours, counted back from the request, and backups you deleted count too.

### What a backup costs

A backup is billed for the space it takes in the backup store, EUR 0.073 per GB a month (the rate of VPS snapshots), until you delete it, also after the volume has been deleted. The space is counted in blocks of 2 MiB, so a backup can be larger than the files on the volume: blocks of files you deleted are stored until they are overwritten, and many small changes take whole blocks. The Backups card shows what is stored now and what it costs a month.

### Deleting a backup

Open the backup and click **Delete backup**. It stays in the list as **Being deleted** and is billed until the backup store has removed its data. That takes about 5 minutes plus half a minute for every GB the backup held (an hour at most). While one is being removed, no backup can be taken and none can be restored: the store holds a lock. At the limit of 10 backups, delete one ahead of time. A backup that is still being made shows **Cancel this backup** instead, and cancelling it stops it.

## Restoring a backup

Open a **Ready** backup and click **Restore backup**. For a container that has a volume you type the container's name to confirm; for a container without one you choose the mount path instead.

**A restore replaces the files on the volume with the backup's. Anything written after the backup was taken is lost,** including what is written while the restore runs.

### What happens

1. A second copy of the volume is made from the backup, next to yours. The volume keeps working as it is meanwhile. This takes about 20 to 25 seconds for every GB of data in the backup, plus up to two minutes
2. When the copy is complete, the container is rolled out onto it, like a deployment
3. When the container serves the restored data, the previous copy is deleted

The restore is finished a few minutes after the copy is complete, once the container serves the restored data. A container that is stopped has no instance to use it: the restore then finishes when you start the container. The second copy is not billed. While the restore runs, the volume cannot be made larger, moved, deleted or backed up, and the status is **Restoring**. A container has one restore at a time and 12 a day, failed ones included.

### Before you restore

- The backup has to be **Ready**.
- The backup must not be of a larger volume than the container has now. Make the volume at least that large first; the console and the API tell you the size.
- The volume keeps its size and its mount path.
- A container **without a volume** gets one made from the backup, as large as the volume was when the backup was taken. Its mount path is the one the backup was taken at, unless you choose another. The new volume counts against your project's volume limit and is billed from then on.

### If a restore fails

If the restore does not complete, the volume stays as it was before the restore, and the restore shows why it failed. You can try again once the platform has removed what it made, which takes a few minutes (`serverless.volume_restore_cleaning_up` in the API).

## Limits

| Limit | Value |
|---|---|
| **Volumes per container** | 1 |
| **Size of a volume** | 1 to 50 GB |
| **Volumes per project** | 100 GB in total, across all containers. Ask support to raise it |
| **Platform capacity** | Volumes are sold up to what the platform has room for. A request can be refused (503, retryable) while your project is below its limit |
| **Backups** | 10 kept per container, 1 at a time, 10 minutes apart, 24 a day |
| **Restores** | 1 at a time, 12 a day per container |
| **Changes through the API** | 20 a minute for one person and one container, for all of them together |

## Billing

| What | Price | Billed |
|---|---|---|
| **Volume** | EUR 0.12 per GB a month | Hourly, on the full size, from the moment it is provisioned until we start removing it. Also while the container is stopped |
| **Backups** | EUR 0.073 per GB a month | Hourly, on what the container's backups take in the backup store, until they are deleted |

A 10 GB volume costs EUR 1.20 a month. Twenty GB of backups cost EUR 1.46 a month. There is no free allowance for volumes or backups.

On your invoice the charges read `Serverless volume — container (mount path)`, one line per volume, and `Serverless volume backups — container`, one line per container, each with its GB-hours. In **Billing** and on the container's **Usage and billing** tab they are listed as **Serverless Volumes** and **Serverless Volume Backups**.

## Troubleshooting

| What you see | Why | What to do |
|---|---|---|
| The volume stays **Provisioning**, then **Not provisioned yet** | The storage could not be created in 10 minutes | Wait a few minutes more. If it stays, delete the volume and add it again; nothing was charged. If **Delete volume** is refused because the container is still deploying, use **Cancel deployment** first, or wait until the deployment fails |
| The container does not start after a volume was added, and the failure is `serverless.volume_node_full` | All instances run on the server that holds the volume, and it has no room for this one | Try again later, or lower **Maximum replicas** or the profile. This is retryable |
| The failure is `serverless.volume_not_ready` | The volume's claim is missing, not bound yet, or being removed | Look at the volume on the **Storage** tab. Delete it and add it again only if it shows **Not provisioned yet** or **Not provisioned**: it never held your files. If the volume was working before, or you did not delete it yourself, delete nothing and contact support |
| The first request after a quiet period is slow, or times out | The volume is attached before the container starts | Set **Minimum replicas** to 1, or the request timeout to 30 seconds or more |
| The application cannot write to the volume | The path is not the one the volume is mounted at, or the application sets its own file permissions | Write under the mount path; the volume is group-writable for group `1000` |
| The volume is full after you made it larger | The file system grows only when the volume is next mounted | Stop the container and start it again |
| **Add volume** or **Make larger** is refused as not available, or because of an approval, your budget or your project's limit, or because there is no room | Volumes are not enabled for your account, your project is under review, your budget is reached, the 100 GB of volumes your project may hold are in use, or the platform has no room for it right now (`serverless.volume_capacity_unavailable`, which is retryable) | The message says which. Delete what you do not need or ask support to raise the limit; for no room, try a smaller size or try again later |
| **Back up now** is refused | Volumes are not enabled for your account, your budget or approval status blocks new resources, a backup is being made or removed, the last one was less than 10 minutes ago, the day's limit or the limit of 10 is reached, or the volume is not **Active** | The message says which. At the limit, delete a backup you do not need, ahead of time: while one is being removed, no backup can be taken |
| **Restore backup** is refused | The backup is not **Ready**, it is of a larger volume, a restore is under way, a backup is being removed, 12 restores were already requested in the last 24 hours, or, for a container without a volume, the platform, your budget, your approval status or your project's limit does not allow the new volume | The message says which |
| A revision cannot receive traffic | It was created with a different volume than the container has now | Deploy a new revision |

Every refusal has a stable code. See [Failure codes](https://docs.danubedata.ro/failure-codes#rapids-volumes-backups-and-restores).

## Automation

Volumes, backups and restores have their own endpoints under `/api/v1/serverless/{id}`: add, change and delete a volume, take, list and delete backups, restore one, and follow it. See [Rapids Volumes API](https://docs.danubedata.ro/api-serverless-volumes). Creating a container through `POST /api/v1/serverless` also accepts a `volume` with `size_gb` and `mount_path`. The CLI, Terraform and Pulumi do not manage volumes yet.

## FAQ

### Does the volume survive a redeploy, a rollback or scale to zero?

Yes. It is separate from the revisions and from the instances. It does not survive deleting the volume or the container.

### Can two containers share a volume?

No. A volume belongs to one container. For files that several containers use, use [object storage](https://docs.danubedata.ro/object-storage).

### Can I get a volume bigger than 50 GB, or more than one per container?

Not today, and not on request: a volume is at most 50 GB and a container has one. Support can raise the total of 100 GB that your project may hold.

### Are my files replicated?

No. There is one copy of the files on one server. Take backups, and keep a copy of what you cannot lose elsewhere.

### Does a restore cost extra?

A restore into a volume you have costs nothing extra: the copy it makes while it runs is not billed. A restore into a container without a volume makes one, billed like any volume from when it is ready.

### What happens to my backups if I delete my project?

They go with it. They are no longer available to you and cannot be restored.