Nopayloaddb Documentation#
Nopayloaddb is a reference implementation of a Conditions Database (CDB) for High Energy Physics (HEP) experiments, developed by the HEP Software Foundation (HSF). It provides a robust, scalable RESTful API for managing time-dependent calibration and configuration data.
Warning
This documentation is under development. Some sections may be incomplete or not working as expected. Please report any issues on GitHub.
Note
Quick Start: Ready to get started? Jump to our Installation guide for Docker setup in under 5 minutes!
What is Nopayloaddb?#
Nopayloaddb serves as a conditions database service designed to handle the complex requirements of HEP experiments:
- Payload Management
Store and retrieve time-dependent calibration data, alignment parameters, and configuration settings
- Version Control
Manage different versions of conditions data with proper validity intervals (IOVs)
- Global Tags
Organize consistent sets of conditions for specific processing campaigns or data-taking periods
- RESTful API
Clean, well-documented HTTP API for easy integration with experiment frameworks
Key Features#
- Ready to Adopt
Docker containerization for easy deployment
PostgreSQL backend with connection pooling
Kubernetes and OpenShift deployment templates
- Security & Performance
JWT and token-based authentication
Bulk operations for efficient data loading
Comprehensive logging and monitoring (in development)
- Developer Friendly
Django ORM with custom database routing
RESTful API built with Django REST Framework
Documentation and examples
Core Concepts#
Understanding these key concepts will help you work effectively with Nopayloaddb:
- Payloads
The actual conditions data (e.g., calibration constants, alignment parameters) stored as references to external files or binary data.
- Payload Types
Categories that define the structure or type of payload data (e.g., ‘BeamSpot’, ‘SiPixelQuality’).
- Interval of Validity (IOV)
The time range, run number range, or other validity dimension for which a specific payload is valid.
- Global Tags
Named collections of specific payload IOVs that represent a consistent set of conditions for a particular processing campaign or data-taking period.
- Payload Lists
Intermediate entities that link Global Tags to Payload Types, serving as containers for related payload versions.
Architecture Overview#
graph TD
A["Client App<br/><small>Web/CLI Applications</small>"]
B["REST API<br/><small>Django Framework</small>"]
C["PostgreSQL Database<br/><small>Metadata Storage</small>"]
D["Payload Files<br/><small>External Storage</small>"]
A <--> B
B <--> C
B <--> D
classDef client fill:#e1f5fe,stroke:#0277bd,stroke-width:2px
classDef api fill:#1565c0,stroke:#0d47a1,stroke-width:3px,color:#fff
classDef database fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
classDef storage fill:#f1f8e9,stroke:#388e3c,stroke-width:2px
class A client
class B api
class C database
class D storage
The system follows a metadata-only approach where the database stores references to payload files rather than the files themselves, enabling efficient storage and retrieval of large datasets.
Getting Started#
Choose your preferred installation method:
Get up and running in minutes with Docker Compose. Perfect for trying out Nopayloaddb or development work.
Time: 5 minutes Requirements: Docker
Full control installation for development or custom deployments.
Time: 15-30 minutes Requirements: Python, PostgreSQL
Production-ready deployment with Kubernetes, OpenShift, or traditional servers. Includes official Helm charts for experiment-specific configurations.
Time: 1+ hours Requirements: Container orchestration Helm Charts: nopayloaddb-charts
Set up a development environment for contributing to Nopayloaddb.
Time: 30 minutes Requirements: Python, Git, PostgreSQL
Documentation Structure#
This documentation is organized into several sections:
Getting Started
User Guide
Deployment & Development
API Reference
API Quick Reference#
Here are some common API endpoints to get you started:
Global Tags
# List all global tags
curl http://localhost:8000/api/cdb_rest/globalTags
# Get specific global tag
curl http://localhost:8000/api/cdb_rest/globalTag/sPHENIX_ExampleGT_24
Payload Queries
# Query payloads by global tag and IOV
curl 'http://localhost:8000/api/cdb_rest/payloadiovs/?gtName=sPHENIX_ExampleGT_24&majorIOV=0&minorIOV=999999'
Create Resources
# Create new global tag
curl -X POST http://localhost:8000/api/cdb_rest/gt \
-H "Content-Type: application/json" \
-d '{"name": "MyNewGT", "author": "username", "description": "New global tag"}'
For complete API documentation, see API Documentation.
Example Use Cases#
- Experiment Calibration Management
Store and retrieve detector calibration constants that vary over time or run conditions.
- Configuration Management
Manage experiment configuration parameters for different data-taking periods.
- Simulation Conditions
Organize conditions data for Monte Carlo simulations with proper versioning.
- Data Processing Campaigns
Create consistent condition sets for large-scale data reprocessing efforts.
Community & Support#
Getting Help
Documentation: Check out the specific guides for detailed information
Issues: Report bugs or request features on GitHub Issues
Discussions: Join the conversation in the HEP Software Foundation activites
Contributing
Nopayloaddb is an open-source project welcoming contributions:
Code: Submit pull requests for bug fixes or new features
Documentation: Help improve these docs
Testing: Report bugs or help with testing
Ideas: Suggest new features or improvements
See our Development Guide guide for details.
License
See the LICENSE.
Acknowledgments#
Nopayloaddb is developed and maintained by the HEP Software Foundation community, with contributions from researchers and engineers across the high-energy physics community.
Special thanks to all contributors who have helped shape this project into a robust solution for conditions database management in HEP experiments.