Service Information¶
Architecture¶
The major components of Sasquatch are Strimzi Kafka and InfluxDB Enterprise. Kafka events from the Summit are replicated with the USDF Sasquatch MirrorMaker2. This data is accessible with InfluxDB and Chronograf. The Kafka Schema Registry is installed and stores the schemas for all topics. Sasquatch is also used to store metrics from some USDF applications. Metrics are integrated via push with Kafka events integrated into InfluxDB.
Sasquatch Kafka Brokers and InfluxDB Pods are scheduled with Kubernetes node affinities to specific nodes that have the ZFS file system for local storage. This was configured due to performance issues with Weka.
The Sasquatch URLs are here
Architecture Diagram¶
Architecture diagram is here
Associated Systems¶
Sasquatch is integrated with USDF RSP systems for metrics collection. Prompt Processing publishes metrics to Sasquatch Dev with the Sasquatch Rest Proxy. ConsDB and Obsloctap query information from Sasquatch InfluxDB.
Configuration Location¶
Config Area |
Location |
|---|---|
Configuration |
|
Vault Secrets Dev |
secret/rubin/usdf-rsp-dev/sasquatch |
Vault Secrets Int |
secret/rubin/usdf-rsp-int/sasquatch |
Vault Secrets Prod |
secret/rubin/usdf-rsp/sasquatch |
Data Flow¶
The USDF Sasquatch has Kafka installed with MirrorMaker2 installed to replicate Kafka events from the Summit using the Long Haul Network (LHN). A socat proxy is installed in the kafka-proxy namespace in the usdf-rsp vCluster to proxy connections to the Summit. Traffic is routed to socat with DNS entries in a config map in the kube-system namespace that resolve connections to the Summit Sasquatch Kafka Cluster external bootstrap and broker addresses to the socat proxy services for the bootstrap and each broker.
Dependencies - S3DF¶
Installation is on nodes with Local ZFS Storage
Dependencies - External¶
LHN Connection to the Summit
Pipeline metrics from France and UK DFs