First off: you need to be in a Python environment that has dask.distributed installed. If you’re already in one with prefect in it, then you already have that.
Secondly, start up your scheduler from the CLI. Take note of the hostname of the scheduler so you can pass it to the workers.
dask-scheduler
and
dask-worker {your scheduler host:port}
Then annotate your flow’s configuration to use this hostname as its scheduler with a DaskExecutor object.
Loading…
If you run this, you will see your dask workers taking up the work!
Want to print your doc? This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
