The Edge Executor in Airflow 3.0

Airflow 3.0 introduces the Edge Executor, a new execution mode for running tasks on edge devices. Tasks run outside the main cluster, ideal for IoT environments, remote branch offices, or constrained networks.

Architecture

A lightweight edge worker is installed on the remote device. It connects to the central scheduler via HTTPS, fetches assigned tasks, executes them locally, and sends back results. No direct database access is required.

python 
from airflow.sdk import DAG, task
from datetime import datetime


with DAG(
    dag_id='edge_iot_pipeline',
    schedule='*/15 * * * *',  # every 15 minutes
    start_date=datetime(2025, 5, 1),
) as dag:

    @task(queue='edge-factory-lyon')  # target an edge worker
    def collect_sensors() -> dict:
        """Runs on the edge device."""
        import serial
        port = serial.Serial('/dev/ttyUSB0', 9600)
        data = port.readline().decode()
        temperature, humidity = data.strip().split(',')
        return {
            'temperature': float(temperature),
            'humidity': float(humidity),
        }

    @task  # runs on the main cluster
    def analyze(measurements: dict):
        if measurements['temperature'] > 80:
            print(f"ALERT: {measurements['temperature']}C")
        print(f"Temperature: {measurements['temperature']}C")

    analyze(collect_sensors())

Use Cases

The Edge Executor is designed for scenarios where data must be collected or processed locally: industrial sensors, on-site image processing, data synchronization in disconnected or bandwidth-limited environments.

bash 
# Install the edge worker on the remote device
pip install apache-airflow-edge-worker

# Configuration and startup
export AIRFLOW_EDGE__API_URL=https://airflow.example.com/api
export AIRFLOW_EDGE__API_TOKEN=secret-token
export AIRFLOW_EDGE__QUEUE=edge-factory-lyon

# Start the edge worker
airflow edge worker
# The worker polls the scheduler and executes assigned tasks

Sources