You SSH in — or, fine, you open Cloud Run in the console — and you patch the thing. The dev worker was missing CLOUD_TASKS_LOCATION, so you set it, the worker comes back to life, the tasks flow, you close the laptop a hero.

Then someone deploys. And the env var is gone again. And the worker is broken again. And you fix it again. And someone deploys again.

Welcome to the boulder.

The setup

Our dev worker needed CLOUD_TASKS_LOCATION to know where to enqueue Cloud Tasks. It wasn’t set. So we set it — at runtime, directly on the running service. Instant fix. Worker happy. Problem “solved.”

Except “solved at runtime” and “solved” are different verbs separated by exactly one deployment.

The crime: imperative fix vs. declarative truth

deploy-dev.yml is the source of truth for what the dev service looks like. When it runs, it doesn’t merge your hand-tweaks into the service — it asserts the world it describes. It says “here is the complete set of env vars this service has,” and Cloud Run obediently makes reality match. Anything you set by hand that isn’t in the workflow gets steamrolled, because as far as the deploy is concerned, it was never supposed to exist.

So every runtime fix has a half-life of one deploy. You’re not fixing the system; you’re temporarily disagreeing with it, and the system always gets the last word.

This is the whole tragedy of patching infrastructure imperatively when something declarative owns it. Your fix is real, but it’s written in disappearing ink. The pipeline re-renders the world from its config, and your config was never in the config.

The fix: put it where the truth lives

Stop fixing the instance. Fix the definition:

# deploy-dev.yml — in the Cloud Run deploy step
--set-env-vars=CLOUD_TASKS_LOCATION=asia-south1,OTHER_VAR=...

Now the env var is part of the world the deploy asserts. Every deployment re-applies it instead of erasing it. The fix survives because it’s no longer a fix — it’s the spec.

The tell that you’re in this trap: the same fix works every time and stops working every deploy. A fix with a perfect success rate and a perfect expiration rate isn’t a fix. It’s a manual override of an automated system, and the automation has more stamina than you do.

The moral

  • If something declarative owns the resource, fix it in the declaration. Runtime patches to a Cloud Run service / Kubernetes deployment / Terraform-managed anything are temporary by design — the next reconcile wipes them.
  • A fix that needs re-applying after every deploy isn’t done. It’s a recurring chore wearing a fix costume. Find the config that the pipeline renders from and put it there.
  • Imperative and declarative management of the same resource will fight, and declarative always wins on a schedule. Pick one owner per resource.

The boulder doesn’t have to roll back down. But only if you stop pushing it by hand and write it into the hill.

Amit Jethva is the CTO and co-founder of Nuvika Technologies Pvt Ltd, makers of Fintropy, a multi-cloud FinOps platform. Learn more at nuvikatech.com.