Application Information

Architecture

A Postgres database is deployed in the rucio-db-16b namespace.

The rucio-kafka Kafka Cluster is deployed in the Rucio vClusters. The Rucio-Butler integration uses this to ingest files into Butler. Kafka MirrorMaker is deployed at the UK and France Data Facilities to synchronize Kafka messages and ingest files locally in each Data Facility into Butler. To allow this connectivity rucio-kafka is configured with external IP Addresses on the external bootstrap and brokers.

Architecture Diagram

Associated Systems

Configuration Location

Config Area

Location

Configuration

slaclab/rubin-rucio-deploy

Vault Secrets Dev

secret/rubin/usdf-rucio-dev

Vault Secrets Prod

secret/rubin/usdf-rucio

Data Flow

Dependencies - S3DF

Dependencies - External

Disaster Recovery