In our previous DaFab post, we introduced the overall multi-site orchestration vision. This entry focuses on a specific architectural building block: integrating Karmada with Argo Workflows to enable multi-cluster and multi-site workflow execution driven by rule-based placement. The key outcome is that workflow steps can be dispatched to different Kubernetes clusters (sites) based on explicit rules that reflect data locality and computing resource availability goals, without changing how users author workflows.

Navigating Strategic Pivots and Technical Milestones

DaFab has now reached half of its length, an ideal time to look back in the rear-view mirror and to reassess our roadmap. After 18 month of work we can claim substantial advancements across DaFab work packages, demonstrating agility and a solid technical foundation, despite facing initial challenges such as delayed deliverables and staffing issues. The project has successfully established a robust management structure, implemented a significant technological pivot, and achieved initial technical milestones, laying the groundwork for its full deployment.

Usability, Impact, and a New Horizon for DaFab and the Global Rucio Community

Over the past year, the DaFab project has become a catalyst for the evolution of the Rucio data management system. While initially designed to support the ATLAS experiment at Cern, today Rucio serves a far wider community of scientific collaborations with complex data needs. The DaFab initiative, centered on extracting value from massive Copernicus Earth Observation archives, has pushed Rucio into new territory, beyond file cataloguing and distributed data placement, and into the realm of rich semantic metadata and powerful filtering.