Skip to main content

Flow Parameters

1Why Parameters?

Sometimes you need to pass a value to a flow at runtime. For example, in a machine learning system with automated aspects, you may not know how to select an input to the model training flow, such as a hyperparameter search space, until it is time to run the flow.

To address these cases you can pass values to a metaflow.Parameter in your flow. When you write a flow you can define which parameters the flow will take. Then you can pass corresponding values to the command that runs your flow:


2Write a Flow with Parameters

Using parameters is a convenient way to quickly iterate in prototyping. For example, you might want to change a hyperparameter like a model's learning rate. This flow adds a parameter called learning_rate to the MinimumFlow example from episode 1.
from metaflow import FlowSpec, step, Parameter

class ParameterizedFlow(FlowSpec):

learning_rate = Parameter('lr', default=.01)

def start(self):

def end(self):
print("Learning rate value is {}".format(self.learning_rate))

if __name__ == "__main__":

3Run the Flow

You can run the flow using the generic command and the default value will be used:

python run
     Workflow starting (run-id 1666720680013279):
[1666720680013279/start/1 (pid 52610)] Task is starting.
[1666720680013279/start/1 (pid 52610)] Task finished successfully.
[1666720680013279/end/2 (pid 52613)] Task is starting.
[1666720680013279/end/2 (pid 52613)] Learning rate value is 0.01
[1666720680013279/end/2 (pid 52613)] Task finished successfully.

Or you can pass in the parameter's value at run time:

python run --lr .001
     Workflow starting (run-id 1666720681347916):
[1666720681347916/start/1 (pid 52631)] Task is starting.
[1666720681347916/start/1 (pid 52631)] Task finished successfully.
[1666720681347916/end/2 (pid 52634)] Task is starting.
[1666720681347916/end/2 (pid 52634)] Learning rate value is 0.001
[1666720681347916/end/2 (pid 52634)] Task finished successfully.

Congratulations on finishing the first season of the tutorial!

In this season you have seen:

  • In Metaflow, a flow defines the structure of your code.
  • You can create flows by wrapping a FlowSpec object in a python script.
  • Flows can store data using the self keyword. These objects are referred to as artifacts.
  • Flow data can be accessed from previous steps, or after the flow using the client API.
  • You can pass values to your flows at run time using parameters.

In the next season, you will see how to apply many of these concepts in machine learning workflows.