Using just the kubernetes agent

The Prefect Kubernetes agent can start basic kubernetes jobs on your behalf.

Start a GKE cluster.

gcloud container clusters create cloud-demo-cluster --region us-east1

Create a Prefect Cloud account and a project in that account, and authenticate to it from the CLI with prefect auth login. You will need a user token.

See the Prefect docs on it here⁠

Annotate your flow to use Docker storage, and now register it to your Cloud project. You will be able to see your flow image in your Docker registry (my example is in GCP’s product for this, GCR) and the flow itself registered in Prefect Cloud.

⁠

Loading…⁠

⁠

Start a flow now, but you’ll see it won’t leave the Scheduled state until we have an agent running.

Create a runner token in the Prefect Cloud UI, and then create the Kubernetes Agent using the manifest built with prefect agent install kubernetes -t $RUNNER_TOKEN ーrbac

Save it somewhere, or pipe the above through to your Kubernetes API vi kubectl with kubectl apply -f {your file} / kubectl apply -f -

Now we can watch the agent make jobs!

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.