{"slug":"tutorial-cr-rapids-troubleshooting","title":"Troubleshooting CR + Rapids Deploys","description":"When a push or deploy goes sideways, the failure is almost always at one of three points: the","section":"Features","url":"https://docs.danubedata.ro/tutorial-cr-rapids-troubleshooting","markdown_url":"https://docs.danubedata.ro/tutorial-cr-rapids-troubleshooting.md","breadcrumbs":[{"title":"Features","slug":null},{"title":"Rapids","slug":"serverless-overview"},{"title":"Deploy with CR: Troubleshooting","slug":"tutorial-cr-rapids-troubleshooting"}],"headings":[{"level":1,"title":"Troubleshooting CR + Rapids Deploys","id":"troubleshooting-cr-rapids-deploys"},{"level":2,"title":"Push failures (docker push)","id":"push-failures-docker-push"},{"level":3,"title":"denied: requested access to the resource is denied","id":"denied-requested-access-to-the-resource-is-denied"},{"level":3,"title":"unauthorized: authentication required","id":"unauthorized-authentication-required"},{"level":3,"title":"denied: Storage quota exceeded / Repository limit reached","id":"denied-storage-quota-exceeded-repository-limit-reached"},{"level":2,"title":"Pull / deploy failures (ImagePullBackOff)","id":"pull-deploy-failures-imagepullbackoff"},{"level":2,"title":"The container starts but never becomes Ready","id":"the-container-starts-but-never-becomes-ready"},{"level":3,"title":"exec format error in the logs","id":"exec-format-error-in-the-logs"},{"level":3,"title":"The app isn't listening on $PORT","id":"the-app-isnt-listening-on-port"},{"level":3,"title":"permission denied binding the port","id":"permission-denied-binding-the-port"},{"level":3,"title":"The process crashes on startup","id":"the-process-crashes-on-startup"},{"level":2,"title":"\"My deploy didn't change anything\"","id":"my-deploy-didnt-change-anything"},{"level":2,"title":"Cold starts feel slow","id":"cold-starts-feel-slow"},{"level":2,"title":"Rolling back a bad deploy","id":"rolling-back-a-bad-deploy"},{"level":2,"title":"Still stuck?","id":"still-stuck"}],"format":"markdown","word_count":891,"content":"# Troubleshooting CR + Rapids Deploys\n\nWhen a push or deploy goes sideways, the failure is almost always at one of three points: the\n**push** to the registry, the **pull** by Rapids, or the **container starting up**. This page\nwalks the failures you're most likely to hit, in the order they happen, with the fix for each.\n\n> **First stop:** the Rapid's detail page surfaces the underlying Kubernetes event verbatim,\n> and `danube rapids show hello` plus `danube rapids deployments hello` show status, the\n> current image:tag, and per-revision traffic. Most diagnoses start there.\n\n> **You'll hear about failures automatically.** If a deploy fails — even after the first\n> rollout — Rapids flags the container **Degraded**, shows a plain-language reason (architecture\n> mismatch, image or tag not found, out of memory, startup timed out) with a fix hint, and\n> emails your team with a link to investigate. Your last healthy revision keeps serving the\n> whole time, and the container clears back to healthy on its own once a deploy succeeds.\n\n## Push failures (`docker push`)\n\n### `denied: requested access to the resource is denied`\n\nThe first path segment of your image isn't your team slug. Images **must** be tagged\n`cr.danubedata.ro/{your-team-slug}/...`.\n\n```bash\n# wrong — pushes to someone else's namespace\ndocker build -t cr.danubedata.ro/hello:1.0.0 .\n# right — acme is your team slug\ndocker build -t cr.danubedata.ro/acme/hello:1.0.0 .\n```\n\n### `unauthorized: authentication required`\n\nYour token is wrong, revoked, or expired. Re-issue a key (Container Registry → Access Keys)\nand log in again. You can verify a token directly:\n\n```bash\ncurl -i -u \"you@example.com:$DD_REGISTRY_TOKEN\" \\\n  \"https://danubedata.ro/oci/token?service=cr.danubedata.ro&scope=repository:acme/hello:pull\"\n```\n\n`200 OK` with a JSON `{\"token\":\"...\"}` body means the credentials are good; `401` means re-issue.\n\n### `denied: Storage quota exceeded` / `Repository limit reached`\n\nYou've hit your registry plan's storage cap or repository count. Delete unused tags or\nrepositories (Container Registry → Repositories), or upgrade under **Plan & Usage**. Quota\nchecks use your live `bytes_used`, so a push succeeds immediately after you free space.\n\n## Pull / deploy failures (`ImagePullBackOff`)\n\nThe image pushed fine but Rapids can't pull it into a pod. In order of likelihood:\n\n1. **The tag doesn't exist** — a typo, or the push hadn't finished. Confirm the exact tag\n   under Container Registry → Repositories → `hello` (expand the row).\n2. **Team mismatch** — first-party pulls only work when the team that owns the Rapid matches\n   the team slug in the image path. A Rapid in team `acme` cannot pull `cr.danubedata.ro/other/...`.\n3. **Stale internal credential** — if you recently renamed the team, the auto-seeded pull\n   secret can lag. Trigger a redeploy to re-materialise it: `danube rapids redeploy hello`.\n\nThe Rapid's status surface names which of these is in play.\n\n## The container starts but never becomes Ready\n\nThe image pulled, but the revision won't take traffic. Rapids only routes to a revision once\nit's listening, so a revision stuck \"not ready\" usually means a startup problem.\n\n### `exec format error` in the logs\n\nArchitecture mismatch — an `arm64` image (common when building on Apple Silicon) on the\n`linux/amd64` cluster. Rebuild for the right platform:\n\n```bash\ndocker build --platform linux/amd64 -t cr.danubedata.ro/acme/hello:1.0.1 .\ndocker push cr.danubedata.ro/acme/hello:1.0.1\ndanube rapids update hello --image cr.danubedata.ro/acme/hello --tag 1.0.1\n```\n\n### The app isn't listening on `$PORT`\n\nRapids tells the container which port to use via the `PORT` env var and routes to that port.\nIf your app hardcodes a different port, or binds `127.0.0.1` instead of `0.0.0.0`, the\nreadiness check never passes. Read `PORT` (default `8080`) and bind all interfaces.\n\n### `permission denied` binding the port\n\nContainers run as a non-root user with Linux capabilities dropped, so binding a port below\n`1024` fails even as `root`. Listen on `8080` (or any port ≥1024).\n\n### The process crashes on startup\n\nCheck the revision's logs for the stack trace (Rapid → Logs, or `danube rapids show hello`).\nA missing required env var is the usual culprit — set it with `danube rapids update hello --env KEY=VALUE`\nand a new revision rolls automatically.\n\n## \"My deploy didn't change anything\"\n\nYou pushed a new image but the running container is unchanged. Almost always this is the\n`:latest` trap — Kubernetes won't re-pull a tag it has already cached. Either deploy by\nimmutable SHA tag (recommended), or force a re-roll of the current tag:\n\n```bash\ndanube rapids redeploy hello\n```\n\nSee [CI/CD Build → Push → Deploy](https://docs.danubedata.ro/tutorial-cr-rapids-cicd) for the\nimmutable-tag workflow that avoids this.\n\n## Cold starts feel slow\n\nWith `--min-scale 0`, the first request after an idle period starts a fresh instance. To\nreduce or remove that:\n\n- Keep one instance warm: `danube rapids update hello --min-scale 1`.\n- Shrink the image so it pulls and boots faster — a distroless or alpine base (see the Go\n  example in the [first tutorial](https://docs.danubedata.ro/tutorial-cr-rapids-first-deploy))\n  starts far quicker than a full OS image.\n\n## Rolling back a bad deploy\n\n```bash\ndanube rapids deployments hello     # find the last good <sha>\ndanube rapids update hello --image cr.danubedata.ro/acme/hello --tag <good-sha>\n```\n\nTraffic shifts to the older revision once it's ready — zero downtime.\n\n## Still stuck?\n\n- [Container Registry reference](https://docs.danubedata.ro/container-registry) — full\n  registry error list and tag rules.\n- [Serverless overview](https://docs.danubedata.ro/serverless-overview) — scaling, networking,\n  and platform limits.\n- Email **support@danubedata.ro** with your container name and the event/log text — that's\n  enough for us to pinpoint it fast.\n\n---\n\n**Questions?** Contact support at support@danubedata.ro\n","prev":{"title":"Deploy with CR: Production","slug":"tutorial-cr-rapids-production","url":"https://docs.danubedata.ro/tutorial-cr-rapids-production","markdown_url":"https://docs.danubedata.ro/tutorial-cr-rapids-production.md","json_url":"https://docs.danubedata.ro/tutorial-cr-rapids-production.json"},"next":{"title":"Invoking Containers","slug":"serverless-invoking","url":"https://docs.danubedata.ro/serverless-invoking","markdown_url":"https://docs.danubedata.ro/serverless-invoking.md","json_url":"https://docs.danubedata.ro/serverless-invoking.json"},"index_url":"https://docs.danubedata.ro/index.json"}