In the age of big data, it is easy to think that only machines can detect a signal amid the noise. While it’s true that big data tools can discover signals that might not be obvious, they can also create their own kind of noise in which the true signal — a true threat — can be lost.

That’s a problem anyone dealing with traditional security monitoring systems over the past few years has come to recognize. Threat detection systems have become extremely good at detecting anything that looks anomalous but, as the number of detected anomalies keeps going up, the number of actual threats is still a small fraction of those. Research indicates that less than 1% of reported anomalies represented actual threats and figuring out which detected threats constitute those dangerous few is exhausting, anxiety-inducing work.