S3 File Notifications

Overview

The S3 File Notifications Kafka service receives S3 Notifications for new files created. This is used by Prompt Processing to identify when image files have been transferred to the USDF and for Data Transfer Monitoring to monitor file transfers.

Service Grouping	Data Curation
Operating Hours	24x7
Service Tier	Real Time
GitHub Application Code Repository	N/A
GitHub Deployment Repository	lsst-sqre/phalanx
Slack Support Channel	usdf-infra-support
Slack Alerts Channel
Prod vCluster	usdf-prompt-processing
Dev vCluster	usdf-prompt-processing-dev

• Service Information
  • Architecture
  • Architecture Diagram
  • Associated Systems
  • Configuration Location
  • Data Flow
  • Dependencies - S3DF
  • Dependencies - External
  • Disaster Recovery
• Documentation and Training
• Security
  • Requesting Access
  • Service accounts
  • Security Incident Response
  • Security Policies
• Procedures
  • Deployment
  • Maintenance
  • Backup
  • Cold Startup
  • Cold Shutdown
  • Reproduce Service
  • View Messages
  • Pulling Messages
  • Creating Ceph Bucket Notifications
  • Upgrading Kafka
  • Upgrading Strimzi Operator
• Troubleshooting
  • Known Issues
  • Monitoring
  • Identify if Kafka events are being created