Schedule Flows on Argo Workflows
Question
How can I schedule flows to run at a specific time on Argo?
Solution
There is a Metaflow decorator for that!
![](/icons/common/ob-logo-dark.png)
Outerbounds user note
This guide assumes you have a functional Metaflow deployment with Argo enabled. Outerbounds users already have a managed Argo service running and can follow this guide right away. Self-hosted Metaflow users need their Metaflow admin to first deploy Argo in the Kubernetes cluster created when they deployed Metaflow.
1Scheduling Flows
You can use Metaflow's @schedule
flow-level decorator to run on Argo Workflows, where Metaflow automatically maps a FlowSpec
to an Argo workflow template, and a run of the flow to an Argo Workflow.
After deploying the script containing your flow to Argo workflows the execution of a FlowSpec
can happen on any event-based trigger you setup or a time-based trigger defined with Metaflow's @schedule
decorator.
2Run Flow
This flow is scheduled to run daily. Notice Metaflow's @schedule
decorator has arguments that determine when the flow is run. Time based triggers you can use include:
@schedule(weekly=True)
runs the workflow on Sundays at midnight.@schedule(daily=True)
runs the workflow every day at midnight.@schedule(hourly=True)
runs the workflow every hour.@schedule(cron='0 10 * * ? *')
runs the workflow at the given Cron schedule, in this case at 10am UTC every day. You can use the rules defined here to define the schedule for the cron option.
from metaflow import FlowSpec, schedule, step
from datetime import datetime
@schedule(daily=True)
class DailyFlowArgo(FlowSpec):
@step
def start(self):
now = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print('time is %s' % now)
self.next(self.end)
@step
def end(self):
pass
if __name__ == '__main__':
DailyFlowArgo()
python schedule_flow_argo.py --with retry argo-workflows create
After running the above command your flow will be triggered daily!
3Manually Trigger Flow
You can manually trigger the flow at any time:
python schedule_flow_argo.py argo-workflows trigger
![](/icons/common/ob-logo-dark.png)
Outerbounds user note
On Outerbounds, you can view your Argo Workflows by clicking the "Deployments" tab in the navigation. Self-hosted Metaflow users can visualize and monitor the flow through the standard Argo UI.