Multi-Cluster Workflow Execution with Karmada and Argo Workflows

In our previous DaFab post, we introduced the overall multi-site orchestration vision. This entry focuses on a specific architectural building block: integrating Karmada with Argo Workflows to enable multi-cluster and multi-site workflow execution driven by rule-based placement. The key outcome is that workflow steps can be dispatched to different Kubernetes clusters (sites) based on explicit rules that reflect data locality and computing resource availability goals, without changing how users author workflows.

Karmada’s Role

Karmada provides a control plane layer for managing multiple Kubernetes clusters as one logical fleet. Instead of treating each site as an independent island, Karmada allows us to define resources once and propagate them to the appropriate member clusters based on rules. The core concept we rely on is Karmada’s propagation policies. These are rule-based decisions that determine where a given resource should be placed, meaning which sites receive which workflow objects. For DaFab, these rules are the mechanism we use to implement “optimize per step” decisions at the platform level, aligning execution with data locality and resource availability.

Critically, we configured Argo Workflows to work with Karmada’s control plane instead of directly with a single Kubernetes control plane. This architectural change is the foundation of achieving multi-cluster workflows. Argo Workflows now sees Karmada as its API surface, and Karmada handles the distribution of workflow definitions to the appropriate sites.

Multi-Site Workflow Execution Flow and Placement Rules

The end-to-end flow works as follows:

1. A user or platform component submits an Argo Workflow definition once. The workflow specification remains the same as usual, using the familiar Argo Workflows YAML format.
2. Karmada evaluates propagation rules (the policy) to choose the target site or sites for that workflow. These rules can take into account factors such as data locality, compute capacity, and site-specific capabilities.
3. For some workflow steps, the best placement is known in advance and can be encoded directly in propagation policies. For other steps, the optimal site depends on runtime conditions that are not available at submission time. To handle these cases, we use webhooks that inject placement information dynamically during workflow execution. These webhooks evaluate runtime state, such as current data locations or resource availability, and provide placement hints that guide where subsequent steps should run.
4. Karmada propagates the relevant Argo Workflows resources to the selected member clusters. Each site receives the workflow objects it is responsible for executing.
5. Execution happens on those Kubernetes clusters. The overall system behavior aligns with optimization goals, such as preferring sites that already host the needed data or sites with the right compute profile and capacity.

This approach shifts the burden of deciding where each step runs from the user or application developer to the platform itself. Rules encode locality and resource considerations for known placement decisions, while webhooks provide runtime intelligence for dynamic placement. The system transparently applies both mechanisms during workflow submission and execution.

An Illustrative Scenario

Consider a three-step Earth Observation workflow: ingest, preprocess, and model inference. Each step has different requirements and can benefit from running at different sites.

• Step 1 (ingest) runs where the input dataset already exists, minimizing data transfer overhead. This placement is known in advance, so Karmada’s propagation policy routes it directly to Site A.
• Step 2 (preprocess) depends on the output location of Step 1 and current resource availability. A webhook evaluates these runtime conditions and injects placement information, guiding the system to run Step 2 at Site B, where general-purpose compute resources are currently available and data movement is minimized.
• Step 3 (model inference) runs where accelerators or HPC-capable resources are available, ensuring the compute-intensive inference task has the right hardware. This is another policy-driven decision, routing the step to Site C.

Karmada’s rule-based propagation and runtime webhooks work together to determine these placements automatically. The user submits the workflow once, and the system routes each step to the appropriate site based on configured policies and runtime intelligence.

Figure 1 illustrates how a single workflow submission is dispatched across multiple sites with different capabilities, using both static rules and dynamic placement hints: Step 1 runs at Site A (policy-driven, data-local); Step 2 placement is determined by a webhook at runtime (routing to Site B  general compute); Step 3 runs at Site C (policy-driven, HPC-capable).

Final Thoughts

Integrating Karmada with Argo Workflows enables DaFab to orchestrate workflows across multiple Kubernetes clusters in a rule-driven, transparent way. By configuring Argo Workflows to interact with Karmada’s control plane, we shift the complexity of multi-site execution from users to the platform. This architectural approach allows the system to optimize workflow execution based on data locality and computing resource availability, as promised in our earlier vision. The result is a more efficient, scalable, and user-friendly workflow orchestration system for Earth Observation data processing across diverse computing environments.

• Tech
• Kubernetes