Workflow and workload management

Participants
    Tasnuva Chowdhury
    Johannes Elmsheuser     Leadership team member     WLCG DOMA co-coordinator, USATLAS L2 Manager WBS 2.4 Computing R&D
    Wen Guan     iDDS Lead Developer
    Eddie Karavakis
    Alexei Klimentov     Former member
    Meifeng Lin (CSI)     Collaborator
    Tadashi Maeno     PanDA Project Leader, Harvester Project Leader, iDDS Project Leader
    Paul Nilsson     PanDA Pilot Project Leader
    Torre Wenaus     ePIC Deputy Software and Computing Coordinator
    Zhaoyu Yang
    Xin Zhao

Experiments and projects
    AID2E - AI-assisted Detector Design
    ATLAS at the LHC (CERN)
    Dark Energy Science Collaboration (DESC)
    DUNE Long Baseline Neutrino Facility
    ESI - EIC Simulation Infrastructure
    Google-ATLAS HL-LHC R&D Project
    HEP Software Foundation (HSF)
    Rubin Observatory (LSST)
    REDWOOD
    sPHENIX at RHIC (BNL)

Software  
    Data Carousel    
    DIRAC    
    PanDA event service    
    funcX (Globus Compute)    
    Grid and workflow/dataflow simulator    
    Harvester    
    Intelligent data delivery system (iDDS)    
    PanDA workload management system    
    PanDA monitoring system    
    PanDA Pilot    
    ATLAS production system    
    ePIC streaming workflow orchestration testbed    

Related materials
    Integrating the PanDA Workload Management System with the Vera C. Rubin Observatory,   CHEP 2023,   May 2023
    Seamless integration of commercial Clouds with ATLAS Distributed Computing,   vCHEP 2021,   May 2021
    Evolution of the ATLAS analysis model for Run-3 and prospects for HL-LHC,   CHEP 2019,   Feb 2020
    Central Container Registry for ATLAS,   NPPS meeting,   Nov 2019
    PanDA/Pegasus integration - a case study,   ATLAS WFMS meeting,   Nov 2019
    iDDS status report,   ATLAS WFMS meeting,   Nov 2019
    The ATLAS Data Carousel Project,   CHEP 2019,   Nov 2019
    Evolution of the Data Quality Monitoring and Prompt Processing System in the protoDUNE-SP experiment,   CHEP 2019,   Nov 2019
    Deployment of containers on the diverse ATLAS infrastructure,   CHEP 2019,   Nov 2019
    Harnessing the power of supercomputers using the PanDA Pilot 2 in the ATLAS Experiment,   CHEP 2019,   Nov 2019
    Using Kubernetes as an ATLAS computing site,   CHEP 2019,   Nov 2019
    Managing the ATLAS Grid through Harvester,   CHEP 2019,   Nov 2019
    Large scale fine grain simulation workflows ("Jumbo Jobs") on HPC's by the ATLAS experiment,   CHEP 2019,   Nov 2019
    Evolution of the ATLAS analysis model for Run-3 and prospects for HL-LHC,   CHEP 2019,   Nov 2019
    Event Streaming Service for ATLAS Event Processing,   CHEP 2019,   Nov 2019
    The ATLAS Data Carousel Project,   HEPiX Fall 2019,   Nov 2019
    HL-LHC Software R&D at BNL,   IRIS-HEP HL-LHC Workshop,   Oct 2019
    HL-LHC Computing R&D in US ATLAS,   IRIS-HEP HL-LHC Workshop,   Oct 2019
    PanDA in brief,   DUNE computing model workshop,   Sep 2019
    The PanDA workload management system,   NPPS talk,   Jun 2019
    Event Service - Looking forward looking back,   ATLAS Workflow Management Meeting, Lubljana,   May 2019
    PanDA pilot overview,   NPPS talk,   May 2019

Events

Google-ATLAS Technical Interchange Meeting, CERN, March 24-26 2020 24 Mar 2020

For more information see here.

Supercomputing 2019, Denver 18 Nov 2019

NPPS events at SC19 included Google - US ATLAS project planning discussions.

Conference website

HEPiX Autumn 2019 Workshop 14 Oct 2019

NPPS participated in the HEPiX Autumn 2019 workshop, with a talk on the Data Carousel R&D in ATLAS given by Alexei Klimentov.

Workshop website

NPPS authored contributions:
        The ATLAS Data Carousel Project

---