PanDA workload management system
Common software: yes
PanDA in NPPS
Participants
Experiments and projects
AID2E - AI-assisted Detector Design
ATLAS at the LHC (CERN)
Google-ATLAS HL-LHC R&D Project
Rubin Observatory (LSST)
REDWOOD
sPHENIX at RHIC (BNL)
Technical teams
AI/ML
Databases
High performance computing
Workflow and workload management
Related materials
Integrating the PanDA Workload Management System with the Vera C. Rubin Observatory,
CHEP 2023,
May 2023
Central Container Registry for ATLAS,
NPPS meeting,
Nov 2019
PanDA/Pegasus integration - a case study,
ATLAS WFMS meeting,
Nov 2019
Deployment of containers on the diverse ATLAS infrastructure,
CHEP 2019,
Nov 2019
Harnessing the power of supercomputers using the PanDA Pilot 2 in the ATLAS Experiment,
CHEP 2019,
Nov 2019
Using Kubernetes as an ATLAS computing site,
CHEP 2019,
Nov 2019
Managing the ATLAS Grid through Harvester,
CHEP 2019,
Nov 2019
Large scale fine grain simulation workflows ("Jumbo Jobs") on HPC's by the ATLAS experiment,
CHEP 2019,
Nov 2019
Enhancements in Functionality of the Interactive Visual Explorer for ATLAS Computing Metadata,
CHEP 2019,
Nov 2019
ATLAS BigPanDA monitoring and Belle II DDM,
NPPS talk,
Oct 2019
PanDA in brief,
DUNE computing model workshop,
Sep 2019
The PanDA workload management system,
NPPS talk,
Jun 2019
PanDA pilot overview,
NPPS talk,
May 2019
The Production and Distributed Analysis (PanDA) system is a data-driven workload management system engineered to operate at LHC data processing scales, including HL-LHC. The PanDA system provides a solution for scientific experiments to fully leverage their distributed heterogeneous resources, showcasing scalability, usability, flexibility, and robustness. The system has successfully proven itself through nearly two decades of steady operation in the ATLAS experiment, addressing the intricate requirements such as diverse resources distributed worldwide at about 200 sites, thousands of scientists analyzing the data remotely, the volume of processed data beyond the exabyte scale, dozens of scientific applications to support, and data processing over several billion hours of computing usage per year.
PanDA was developed primarily by NPPS and the University of Texas at Arlington. The system is open-source and has been adopted by other experiments and projects including the Rubin Observatory.
References
- PanDA: Production and Distributed Analysis System, Comput Softw Big Sci 8, 4 (2024)
- PanDA documentation
- pandawms.org
- PanDA GitHub repository
- PanDA community forum meetings
- PanDA community forum mailing list: pandawms-forum@googlegroups.com
- A click tour of the PanDA monitor (ATLAS authentication needed)
