Skip to main content

Event Triggering

Metaflow's event triggering makes it possible to start flows deployed on Argo Workflows when an event is published to an event backend.

On the backend side, the feature is implemented using Argo Events. Our default Terraform template installs a webhook event source and Jetstream as the event bus.

Publishing events​

To trigger workflows based on external events (in contrast to using internal events to connect workflows via @trigger_on_finish), you need to call ArgoEvent.publish in systems outside Metaflow.

To make publishing work, two preconditions must be met:

  1. Metaflow configuration needs to be available in the environment that uses ArgoEvent.

  2. ArgoEvent tries to call the webhook deployed in the Kubernetes cluster based on the address set in a config variable ARGO_EVENTS_WEBHOOK_URL (you can override it with the url argument in ArgoEvent). Hence this URL must be accessible from the calling environment.

    One approach is to use our port forwarding script which piggybacks on the kubectl authentication to create a secure connection to the cluster. If you don't want to use port forwarding, you must ensure that the URL is accessible to the caller otherwise.