Overview

Metaflow 2.9, released on March 15, 2023, introduces the Cards system for visualization and monitoring of data pipelines.

Main Features

Cards system

Cards are visual reports automatically generated for each step of a Metaflow flow, making it easier to track and debug pipelines.

python 
from metaflow import FlowSpec, step, card

class MyFlow(FlowSpec):
    @card
    @step
    def start(self):
        self.results = [1, 2, 3, 4, 5]
        self.next(self.process)

    @card
    @step
    def process(self):
        self.total = sum(self.results)
        self.next(self.end)

    @step
    def end(self):
        print(f'Total: {self.total}')

Built-in monitoring

Built-in monitoring lets you track flow execution in real time, with performance metrics and configurable alerts.

python 
from metaflow import FlowSpec, step, retry

class RobustFlow(FlowSpec):
    @retry(times=3)
    @step
    def start(self):
        import time
        self.start_time = time.time()
        self.next(self.end)

    @step
    def end(self):
        import time
        duration = time.time() - self.start_time
        print(f'Duration: {duration:.1f}s')

Sources