Failure codes
Every failure the platform reports carries a stable code. Branch on the code; display the message. Messages are written for humans and may be reworded at any time, codes are an interface and change additively.
Codes appear in two places, with the same meaning in both:
status_details.error.code— why a resource isfailedordegradedfindings[].codefrom/diagnose— everything diagnose correlated
All codes are namespaced by product: cache., database., queue., vps., vps-volume., storage., static-site., app., uptime., metric-alert., serverless..
See Platform automation for the surrounding contract.
Universal codes
Emitted for every product. {product} is the prefix from the list above.
| Code | Severity | Retryable | Meaning | What to do |
|---|---|---|---|---|
{product}.healthy | informational | — | Diagnose found nothing wrong | Nothing |
{product}.operation_in_progress | informational | yes | An operation is still running; diagnose short-circuits rather than reporting a half-built resource as broken | Keep polling operation.terminal |
{product}.stale_observation | action_required | yes | An in-flight operation has gone unconfirmed for 15 minutes | Retry the operation; if it repeats, open a ticket |
{product}.warnings_recovered | transient_recovered | — | Warning events occurred and the platform already recovered from them | Nothing. Not a current problem |
{product}.unknown | fatal | no | A failure was recorded but could not be classified | Read message; open a ticket if unclear |
Event-derived codes
Where a product exposes cluster events, warnings become findings coded {product}.event_{reason}, with the Kubernetes reason lowercased — so a CrashLoopBackOff on a cache instance is cache.event_crashloopbackoff.
These reasons are classified fatal — they do not resolve on their own:
| Reason | Code suffix | Typical cause |
|---|---|---|
Failed | event_failed | The container or controller reported a failure |
FailedCreate | event_failedcreate | The workload object could not be created |
FailedScheduling | event_failedscheduling | No node had room, or a volume was not placeable |
ErrImagePull | event_errimagepull | The image could not be fetched |
ImagePullBackOff | event_imagepullbackoff | Repeated pull failures; usually a bad tag or credential |
CrashLoopBackOff | event_crashloopbackoff | The process keeps exiting after start |
BackOff | event_backoff | Restarts are being throttled |
FailedMount | event_failedmount | A volume could not be attached or mounted |
Evicted | event_evicted | The node reclaimed resources |
OOMKilling | event_oomkilling | The container exceeded its memory limit |
These are classified transient_recovered and are not incidents — Kubernetes emits them during ordinary reconciliation races:
InternalError · UpdateConflict · FailedUpdate, plus any warning whose message matches a write-conflict pattern (object has been modified, please apply your changes to the latest version, conflict, try again).
Automation that treats every Warning event as an outage will page you for healthy systems. This classification is why /diagnose exists rather than handing you the raw event stream.
Object storage
| Code | Severity | Retryable | Meaning | What to do |
|---|---|---|---|---|
storage.quota_exceeded | fatal | no | The bucket is at its size limit. Writes are rejected while this holds | Delete objects, or raise the limit |
storage.quota_nearly_exceeded | action_required | no | Approaching the limit | Free space before writes start failing |
storage.quota_not_applied | action_required | yes | The bucket exists but the backend did not accept its quota, so no limit is being enforced | Retry the update; open a ticket if it persists |
storage.usage_stale | informational | yes | The usage figure is older than 30 minutes | Treat the number as approximate |
Usage is sampled, not live. Quota findings state the age of the measurement they are based on, because acting on a stale number is how a "you have room" answer turns into a rejected write.
Static sites
| Code | Severity | Retryable | Meaning | What to do |
|---|---|---|---|---|
static-site.build_failed | fatal | no | The build command exited non-zero | Read the build logs |
static-site.domain_unverified | action_required | no | A custom domain has no valid DNS record | Add the record shown in the dashboard |
static-site.pending_review | action_required | no | The site is awaiting review before publishing | Wait, or contact support |
static-site.suspended | fatal | no | The site was suspended | Contact support |
Managed apps
| Code | Severity | Retryable | Meaning | What to do |
|---|---|---|---|---|
app.upgrade_failed | fatal | no | A version upgrade did not complete | Read the logs; the previous version may still be serving |
app.upgrade_auto_rolled_back | action_required | no | An upgrade failed and the platform restored the previous version | Your app is running. Investigate before retrying |
app.upgrade_auto_rolled_back is deliberately not fatal: the rollback worked and the app is serving. Reporting it as an outage would be false.
Serverless containers
These come from correlating the container's status with its live Knative Revision and Route — two sources that give the wrong answer when read alone.
| Code | Severity | Retryable | Meaning | What to do |
|---|---|---|---|---|
serverless.no_pod_scheduled | fatal | no | The latest revision settled at Ready=False with zero replicas: no pod ever started | Fix the image reference or its credential. The empty log is expected — nothing ran to write one |
serverless.ingress_not_ready | fatal | no | IngressReady=False: the revision is healthy but the public URL does not serve | Check the custom domain and its TLS certificate. Redeploying will not help |
serverless.ingress_not_ready | informational | yes | IngressReady=Unknown: the route is still converging | Wait and re-read |
serverless.degraded_but_serving | action_required | no | A new revision failed while an older one keeps serving. The site is UP | Do not roll back automatically — traffic is already on the last good revision |
serverless.replica_count_lagging | informational | no | current_replicas disagrees with the live revision count | Nothing. The container figure is a five-minute cache; the two legitimately differ after a scale-to-zero wake |
serverless.revision_data_unavailable | informational | yes | The Knative Service or Route could not be read, so some checks did not run | Retry. Do not read the missing findings as "nothing wrong" |
Two of these are the reason /diagnose exists rather than a status field:
no_pod_scheduledexplains an empty log. Callers otherwise spend a long time hunting for output that cannot exist, because the image was never pulled.ingress_not_readyseparates "deployed" from "reachable". Every other signal — status, revision, events — says the deployment succeeded.
degraded_but_serving arrives alongside the finding that explains why the new revision failed — usually no_pod_scheduled. Read the whole array: the first says your site is still up, the second says what to fix.
Revision checks are skipped, and no code is emitted, while an operation is still running (operation.terminal is false) and for a stopped container. A healthy deploy is briefly Ready=False with zero replicas, and a stopped container is meant to have no pods; reporting either as a fault would be a false alarm.
Rapids volumes, backups and restores
The Rapids Volumes API refuses a request with success: false and an error whose code is one of the codes below, with the HTTP status in the table. retryable says whether the same request can succeed later with nothing changed on your side. Branch on the code, never on the message.
Refused because of the account
| Code | HTTP | Retryable | Meaning | What to do |
|---|---|---|---|---|
serverless.volume_not_available | 403 | no | Volumes, or backups of volumes, are not offered to this account | Ask support to enable them. Reading and deleting still work |
serverless.volume_manual_approval_required | 403 | no | This project is under review and creates no new resources until support has cleared it | Contact support |
serverless.volume_budget_exceeded | 403 | no | This month's or today's spend has reached the project's budget, which is set to refuse new resources | Raise the budget under Billing > Budget, or wait for the next billing period |
serverless.volume_team_limit_reached | 403 | no | The project holds all the volume storage it may (100 GB unless raised) | Delete a volume, or ask support to raise the limit |
Not found
| Code | HTTP | Retryable | Meaning | What to do |
|---|---|---|---|---|
serverless.volume_not_found | 404 | no | The container has no such volume. A volume that is being deleted stays at its address, as deleting, until it is gone | Check the id |
serverless.volume_backup_not_found | 404 | no | The container has no such backup | Check the id |
serverless.volume_restore_not_found | 404 | no | The container has no such restore | Check the id |
serverless.volume_container_gone | 404 | no | The container was deleted while the request was handled | Nothing to do |
The request is wrong
| Code | HTTP | Retryable | Meaning | What to do |
|---|---|---|---|---|
serverless.volume_cannot_shrink | 422 | no | A volume can be made larger, never smaller. error.field is size_gb | Send a larger size |
serverless.volume_size_out_of_range | 422 | no | The size is outside what a volume may be (1 to 50 GB). A request with such a size is normally answered by the validation answer below, with errors.size_gb, before the platform's own check raises this code. error.field is size_gb | Send a size from 1 to 50 GB |
serverless.volume_invalid_mount_path | 422 | no | The container cannot mount a volume at that path. A mount_path in your request that is not valid is normally answered by the validation answer below, with errors.mount_path. This code comes from a restore into a container without a volume, when you send no mount_path and the path the backup was taken at can no longer be mounted. error.field is mount_path | Send a mount_path of your own. See Mount path |
A body that is not valid at all (a field missing, a size that is not a number) is answered 422 with Laravel's {"message": "...", "errors": {...}} instead.
The state does not allow it now
| Code | HTTP | Retryable | Meaning | What to do |
|---|---|---|---|---|
serverless.volume_already_exists | 409 | no | The container has a volume already (one per container) | Use the volume it has |
serverless.volume_container_not_deployed | 409 | yes | The container has not been deployed yet, so there is nothing to roll a volume out with | Wait for its first deployment |
serverless.volume_container_busy | 409 | yes | Another change to the container is under way or queued | Wait for it to finish, or cancel it |
serverless.volume_tier_unsupported | 409 | no | The container is on the 10 Gb network tier, which cannot have a volume yet | Use a container on the standard tier |
serverless.volume_not_mounted | 409 | no | The volume is being deleted, so it cannot be moved or made larger | Wait until it is gone, then add a new one |
serverless.volume_cannot_resize_now | 409 | yes | The volume is still being made or grown | Wait until it is active. A volume that is still pending after 10 minutes, with failure.operation provision, will not come up by waiting: delete it and add it again |
serverless.volume_being_restored | 409 | yes | A backup is being restored into the volume | Wait for the restore to finish |
serverless.volume_changed_meanwhile | 409 | yes | The volume changed while the request was handled | Read the volume and send the request again |
serverless.volume_backup_no_volume | 409 | no | The container has no volume to back up | Add a volume, or restore a backup into the container |
serverless.volume_backup_volume_not_ready | 409 | yes | The volume is being created, grown or restored | Wait until it is active. A volume that is still pending after 10 minutes, with failure.operation provision, will not come up by waiting: delete it and add it again |
serverless.volume_backup_in_progress | 409 | yes | A backup of this container is being made (one at a time) | Wait until it is terminal |
serverless.volume_backup_limit_reached | 409 | no | The container keeps as many backups as it may (10) | Delete a backup |
serverless.volume_backup_deletion_pending | 409 | yes | A backup is being removed from the backup store, which takes about 5 minutes plus half a minute for every GB the backup held (an hour at most). No backup can be taken or restored meanwhile | Wait. When you are at the limit of 10 backups, delete one ahead of time |
serverless.volume_backup_too_soon | 409 | yes | The last backup was made less than 10 minutes ago | Wait |
serverless.volume_backup_daily_limit_reached | 409 | yes | 24 backups were requested in the last 24 hours, failed and deleted ones included | Try again when the oldest of them is more than 24 hours old |
serverless.volume_backup_being_restored | 409 | yes | A backup of this volume is being restored, and a deletion would pull the data from under it | Wait for the restore to finish |
serverless.volume_restore_backup_not_ready | 409 | no | The backup is still being made, failed, or is being deleted | Wait for the backup to be ready (terminal) before restoring it |
serverless.volume_restore_in_progress | 409 | no | A restore of this container is under way. Not retryable: asked again once it has finished, it would restore a second time over what was written since. meta.restore_id and meta.poll_url name it. Adding a volume to a container that has none answers this too while a restore, or the clean-up after a failed one, holds the container's place | Follow that restore |
serverless.volume_restore_cleaning_up | 409 | yes | The last restore failed and the platform is still cleaning up after it. Nothing was restored. meta.restore_id and meta.poll_url name it. Only a restore request answers this | Try again in a few minutes |
serverless.volume_restore_backup_too_large | 409 | no | The backup is of a larger volume than the container has now | Make the volume at least that large, then restore |
serverless.volume_restore_daily_limit_reached | 409 | yes | 12 restores were requested in the last 24 hours, failed ones included | Try again when the oldest of them is more than 24 hours old |
The platform cannot do it now
| Code | HTTP | Retryable | Meaning | What to do |
|---|---|---|---|---|
serverless.volume_capacity_unavailable | 503 | yes | There is no room to provision the volume, or the larger size | Try a smaller size, or later |
serverless.volume_placement_unknown | 503 | yes | We could not work out where the volume would be placed just now | Try again in a minute |
When a container with a volume does not start
These two arrive in status_details.error.code (and in findings[].code), like the other runtime failures, when the platform cannot start an instance of a container that has a volume.
| Code | Retryable | Meaning | What to do |
|---|---|---|---|
serverless.volume_node_full | yes | All instances of a container with a volume run on the server that holds the volume, and it cannot take this one right now | Lower Maximum replicas or the resource profile, or try again later |
serverless.volume_not_ready | no | The volume's claim is missing, not bound yet, or being removed | Look at the volume on the Storage tab. Delete it and add it again only if it shows Not provisioned yet: it never held your files. If the volume was working before, or you did not delete it yourself, delete nothing and contact support |
Uptime checks
An uptime failure describes your target, not the monitor. A check reporting down is working correctly.
| Code | Severity | Retryable | Meaning | What to do |
|---|---|---|---|---|
uptime.dns | fatal | yes | The hostname did not resolve | Check DNS |
uptime.timeout | fatal | yes | The target did not respond in time | Check load and timeouts |
uptime.refused | fatal | yes | The connection was refused | Check the service is listening |
uptime.http_error | fatal | yes | An unexpected HTTP status | Check the endpoint |
uptime.tls_expiring | action_required | no | The certificate expires soon | Renew it |
uptime.tls_expired | fatal | no | The certificate has expired | Renew it now |
uptime.flapping | action_required | no | The target is alternating up and down | Investigate intermittent failures |
uptime.not_running | informational | no | The check is paused | Resume it |
retryable: true here means the target may recover, not that re-reading the check will change anything.
VPS block volumes
| Code | Severity | Retryable | Meaning | What to do |
|---|---|---|---|---|
vps-volume.instance_unavailable | fatal | no | The parent VPS is stopped or gone, so the volume cannot be attached | Start the instance |
vps-volume.awaiting_instance_boot | informational | yes | The volume is attached and waiting for the guest to see it | Keep waiting |
A volume carries its own status_details. Do not poll the parent VM — it stays running throughout an attach and would look like instant success.
Cache, databases, queues and VPS
These products report failures from two sources:
- Recorded failures — written by the reconciler that observed them, coded
{product}.{category}. The category depends on what the platform observed; treat it as opaque and branch on the prefix plusretryable. - Event-derived findings — the
{product}.event_*codes above.
If you receive a code not listed on this page, apply the general rules: honour retryable, read message for the human explanation, and treat an unrecognised code as a genuine failure rather than ignoring it.
Envelope errors
These describe the request, not the resource, and appear in error.code alongside an HTTP status.
| Code | HTTP | Retryable | Meaning |
|---|---|---|---|
{product}.logs_not_supported | 400 | no | This product has no log surface — check capabilities.logs first |
{product}.logs_unavailable | 503 | yes | The log backend did not respond |
{product}.events_not_supported | 400 | no | This resource has no cluster objects to read events from |
{product}.events_unavailable | 503 | yes | Cluster events could not be read |
{product}.events_not_permitted | 503 | no | The platform is not permitted to read events for this resource |
operation.not_found | 404 | no | No operation with that id |
registry.no_project | 400 | no | No project resolved; send X-Team-Id |
approval.already_pending | 409 | no | An approval for this action is already waiting for the project owner; approval_id names it |
account_limits.request_pending | 409 | no | A limit increase request is already open for the project |
approval.not_found | 404 | no | No approval with that id in this project |
events_not_permitted is separated from events_unavailable on purpose: one is a transient upstream problem worth retrying, the other never resolves by retrying and needs an operator.
Registry preflight
GET /registry/preflight returns findings rather than failing, so these arrive with HTTP 200 in findings[].code:
| Code | Retryable | Meaning |
|---|---|---|
registry.namespace_mismatch | no | The push target is outside your project's namespace |
registry.quota_exceeded | yes | The push would exceed your storage or repository limit |
registry.manifest_not_found | no | No such tag in the repository — push it first |
registry.architecture_mismatch | no | The image was built for an architecture the platform does not run |
registry.no_credential | no | The image is private and no credential is configured |
registry.external_image | — | Informational: the image is outside your registry, so we cannot verify it |
A 503 with retryable: true means an upstream we depend on is unreachable — back off and retry. A 403 will never succeed with the same token; mint one with the right ability instead.