Interactive Visualizations with Cards

In this episode you will use Metaflow's @card decorator to visualize the learning curves of the neural network you created in the previous episode. Specifically, you will create an in-browser visualization made from a matplotlib figure. This pattern shows how one line of code (e.g., a decorator) can afford advanced functionality in Metaflow. Note that cards are intended for quickly visualizing flow results in iterative workflows. If you are looking for a more complete dashboard, you may want to consider deploying the Metaflow UI.

1Add a Card to Your Flow

The flow is structured as follows:

• The start step loads image data from Keras.
• The build_model step builds and compiles a Keras model.
• The train step fits the neural net.
  • This is the step that contains the Metaflow card. In this instance, we append metaflow.cards.Image objects to current.card. You can use cards to create data visualizations containing plots, images, markdown, HTML, and more.

neural_net_card_flow.py

from metaflow import FlowSpec, step, card, current, Parameter
from metaflow.cards import Image

def plot_learning_curves(history):
    import matplotlib.pyplot as plt
    fig1, ax = plt.subplots(1,1)
    ax.plot(history.history['accuracy'])
    ax.plot(history.history['val_accuracy'])
    ax.set_title('model accuracy')
    ax.set_ylabel('accuracy')
    ax.set_xlabel('epoch')
    fig1.legend(['train', 'test'], loc='upper left')
    fig2, ax = plt.subplots(1,1)
    ax.plot(history.history['loss'])
    ax.plot(history.history['val_loss'])
    ax.set_title('model loss')
    ax.set_ylabel('loss')
    ax.set_xlabel('epoch')
    fig2.legend(['train', 'test'], loc='upper left')
    return fig1, fig2

class NeuralNetCardFlow(FlowSpec):
    
    epochs = Parameter('e', default=10)

    @step
    def start(self):
        import numpy as np
        from tensorflow import keras
        self.num_classes = 10
        ((x_train, y_train), 
         (x_test, y_test)) = keras.datasets.mnist.load_data()
        x_train = x_train.astype("float32") / 255
        x_test = x_test.astype("float32") / 255
        self.x_train = np.expand_dims(x_train, -1)
        self.x_test = np.expand_dims(x_test, -1)
        self.y_train = keras.utils.to_categorical(
            y_train, self.num_classes)
        self.y_test = keras.utils.to_categorical(
            y_test, self.num_classes)
        self.next(self.build_model)

    @step
    def build_model(self):
        import tempfile
        import numpy as np
        import tensorflow as tf
        from tensorflow import keras
        from tensorflow.keras import layers # pylint: disable=import-error
        self.model = keras.Sequential([
            keras.Input(shape=(28,28,1)),
            layers.Conv2D(32, kernel_size=(3, 3), 
                          activation="relu"),
            layers.MaxPooling2D(pool_size=(2, 2)),
            layers.Conv2D(64, kernel_size=(3, 3), 
                          activation="relu"),
            layers.MaxPooling2D(pool_size=(2, 2)),
            layers.Flatten(),
            layers.Dropout(0.5),
            layers.Dense(self.num_classes, activation="softmax"),
        ])
        self.model.compile(loss="categorical_crossentropy",
                           optimizer="adam", metrics=["accuracy"])
        self.next(self.train)

    @card
    @step
    def train(self):
        import tempfile
        import tensorflow as tf
        self.batch_size = 128
        history = self.model.fit(
            self.x_train, self.y_train,
            batch_size=self.batch_size, 
            epochs=self.epochs, validation_split=0.1
        )
        fig_acc, fig_loss = plot_learning_curves(history)
        current.card.append(Image.from_matplotlib(fig_acc))
        current.card.append(Image.from_matplotlib(fig_loss))
        self.next(self.end)

    @step
    def end(self):
        print("NeuralNetFlow is all done.")

if __name__ == "__main__":
    NeuralNetCardFlow()

2Run the Flow

python neural_net_card_flow.py run

     Workflow starting (run-id 1666720921633922):
     [1666720921633922/start/1 (pid 52856)] Task is starting.
     [1666720921633922/start/1 (pid 52856)] Task finished successfully.
     [1666720921633922/build_model/2 (pid 52870)] Task is starting.
     [1666720921633922/build_model/2 (pid 52870)] WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 2 of 2). These functions will not be directly callable after loading.
     [1666720921633922/build_model/2 (pid 52870)] Task finished successfully.
     [1666720921633922/train/3 (pid 52876)] Task is starting.
    422/422 [==============================] - 8s 19ms/step - loss: 0.3856 - accuracy: 0.8811 - val_loss: 0.0863 - val_accuracy: 0.97624 - loss: 2.3331 - accuracy: 0.09
    422/422 [==============================] - 7s 18ms/step - loss: 0.1164 - accuracy: 0.9650 - val_loss: 0.0579 - val_accuracy: 0.9847- loss: 0.2044 - accuracy: 0.96
    422/422 [==============================] - 7s 18ms/step - loss: 0.0874 - accuracy: 0.9732 - val_loss: 0.0475 - val_accuracy: 0.9862- loss: 0.2218 - accuracy: 0.95
    422/422 [==============================] - 7s 18ms/step - loss: 0.0709 - accuracy: 0.9778 - val_loss: 0.0418 - val_accuracy: 0.9883- loss: 0.0672 - accuracy: 0.97
    422/422 [==============================] - 8s 18ms/step - loss: 0.0658 - accuracy: 0.9794 - val_loss: 0.0398 - val_accuracy: 0.9887- loss: 0.0981 - accuracy: 0.96
    422/422 [==============================] - 8s 18ms/step - loss: 0.0595 - accuracy: 0.9816 - val_loss: 0.0366 - val_accuracy: 0.9895- loss: 0.0591 - accuracy: 0.9
    422/422 [==============================] - 8s 18ms/step - loss: 0.0542 - accuracy: 0.9829 - val_loss: 0.0346 - val_accuracy: 0.9897- loss: 0.0287 - accuracy: 0.98
    422/422 [==============================] - 8s 19ms/step - loss: 0.0503 - accuracy: 0.9836 - val_loss: 0.0333 - val_accuracy: 0.9912- loss: 0.0148 - accuracy: 1.00
    422/422 [==============================] - 9s 21ms/step - loss: 0.0480 - accuracy: 0.9848 - val_loss: 0.0325 - val_accuracy: 0.9913- loss: 0.0515 - accuracy: 0.97
    422/422 [==============================] - 8s 19ms/step - loss: 0.0436 - accuracy: 0.9866 - val_loss: 0.0323 - val_accuracy: 0.9915- loss: 0.0092 - accuracy: 1.00
     [1666720921633922/train/3 (pid 52876)] WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 2 of 2). These functions will not be directly callable after loading.
     [1666720921633922/train/3 (pid 52876)] Task finished successfully.
     [1666720921633922/end/4 (pid 52939)] Task is starting.
     [1666720921633922/end/4 (pid 52939)] NeuralNetFlow is all done.
     [1666720921633922/end/4 (pid 52939)] Task finished successfully.
     Done!

3Visualize the Card

Now you can view your cards with one command. Moreover, they are versioned like all Metaflow data. Take a look at the card containing the matplotlib figures produced in the train step with this command:

python neural_net_card_flow.py card view train

    Metaflow 2.7.12 executing NeuralNetCardFlow for user:eddie
    Resolving card: NeuralNetCardFlow/1666720921633922/train/3

You can also render cards like the one produced in this example in a Jupyter Notebook.

You can open the notebook in the command line from the directory where you ran the flow like:

jupyter lab S3E2-analysis.ipynb

In this episode you saw how using @card can help you quickly visualize images that are produced in Metaflow flows. This can help you log results and iterate quickly. In the next episode, you will see another Metaflow pattern designed to help you iterate faster. Specifically, what do you do when a task in your flow fails? See you in the next episode to find out!