Deep Learning-Based Anomaly Detection for Continuous Compliance Monitoring in Global Data Center Operations

Abstract

Increasingly frequent and alarming environmental events pose a challenge for many corporate organizations to demonstrate their responsible and sustainable operations. Alongside sustainability concerns, other pressing regulatory requirements—related to risk management, fraud prevention, and consumer protection—require automatic monitoring systems as the evidence for all regulatory compliance is frequently mandated. Such systems involve the trained detection of subtle and complex abnormalities within continuous multivariate time series data streams that would otherwise go unnoticed. The global scale of Data Center operations, combined with the subtleties and complexity of the compliance requirements, means that the design of such detection systems must rely on data-driven methods, underpinned with self-supervised representational learning.

Two recent case studies deployed within the operations of a large Cloud service provider demonstrate the Real-Time Inference Anomaly Detection framework applied to continuous compliance demands. The approach is inherently blind to the requirement being monitored and, when coupled with the appropriate mapping from time series patterns to specific thresholds, can continuously detect anomalies in deployed systems following a single training cycle. In these fundamentally asymmetric classification problems, training is performed using self-supervised representational learning, enabling an unlabelled dataset to be transformed into a labeled dataset with representative normal operation classes for multi-class classification. The case studies confirm that such approaches can automatically address the two transverse areas of Sustainability and Security and Access Control.