Various aspects of working with time series data — Part 3: Anomalies, Motifs, and Signatures
This article is part of a series of articles aimed to discuss issues that every person working with time series (TS) data (especially data scientists) should know.
The topics discussed in this series are of different levels of depth and complexity. If you are just starting your journey with TS data, you might take interest in the whole series, and if you are already familiar with TS data, you might want to skip to the later parts, that include more advanced topics.
All of the code examples and discussions are in Python.
Part 1: Time formats
Part 2: Time series analysis — Seasonality, Trends, and Frequencies
Part 3: Anomalies, Motifs, and Signatures
Part 4: Time series tasks and relevant algorithms
Table of contents
· Introduction to anomalies, motifs and signatures
· Anomaly detection
· Motif Discovery
· Motif search
· Pattern Matching
· Signatures
· Summary
Introduction to anomalies, motifs and signatures
This aim of this article is to introduce some important terms in the Time Series world. TS anomalies, motifs, and signatures are involved in many TS-related data analysis tasks and getting familiar with them can be useful.
Anomalies and motifs are valuable characteristics for any sequence, and identifying them is a challenging task that can provide many insights about the data.
Anomalies are not unique to TS data, but they should be treated differently (compared to anomalies in other types of data) since their temporal location of should be accounted for. In other words, if we are looking at a sequence of TS data (for example, a week long measurements of hourly temperatures), values that can be considered as anomalies at a certain time of the day (e.g. noon) may not be anomalies during other parts of the day (e.g. night).
Motifs are patterns that are unique (not noise), but repeating throughout the sequence. For example, if we will monitor the water consumption of households, we will likely see a spike in consumption during the evening hours. This can be considered as a motif.
Motifs and anomalies can be found in both univariate and multivariate TS data, and the latter will make the task of identifying them a bit more tricky.
Motifs and anomalies are similar as they are both cases of unique events that can provide us with vital information about our data, but the main difference is that anomalies represent abnormal behaviors while motifs are patterns that represent common behaviors.
Anomaly detection
Anomaly detection in TS data is the identification of value points or patterns which are significantly different from the “normal behavior” of the data. For example:
Commonly used algorithms for anomaly detection (e.g. Isolation forest, LOF) do not consider the context of the values. Sure, some values can be considered as global anomalies (not so much in cases of a global trend), but in TS data the temporal location is very important. We would like to model the data as sequences and not as individual, independent values.
A common approach for anomaly detection in TS data is by forecasting on the data (predict the future values of the sequence). The general concept is that after forecasting we can check the error (difference between the predicted and the actual values) on the train set and use the mean error plus a certain amount of standard deviations as a threshold. This error threshold will be then used to determine when the values in our data are significantly different from what we expect, and these will be our anomalies. There are many models that can perform the task of forecasting. From the simple ARIMA model, and some ‘of-the-shelf’ models like facebook’s prophet, to deep learning models and even transformers. One example that worked well for me in the past is the LSTM AutoEncoder. It’s an AutoEncoder which has LSTM layers for its hidden layers so it can treat sequenced data.
Motif Discovery
Motifs are patterns that are unusual, but they are repeating in our sequence. When we see the same unique pattern repeating in our sequence, it usually means there is something interesting happening there. The challenge is to identify these motifs. This topic was intensively researched and discussed by the group that came up with the Matrix Profile algorithm, lead by Professor Eamonn Keogh at the University of California-Riverside and Professor Abdullah Mueen at the University of New Mexico. In one sentence, Matrix Profile is a vector that stores the (z-normalized) Euclidean distance between any subsequence within a time series sequence and its nearest neighbor. So, if there is a subsequence in our sequence which is similar to another subsequence (low euclidean distance between them), this is a potential motif. This is basically how Matrix Profile is used to identify motifs.
In the chapter above we discussed anomalies, which Matrix Profile is also able to identify. How? If the distance between a subsequence and its nearest neighbor is very big, than it is probably an anomaly because there is no other subsequence like it.
I am not going to go into the pros and cons of this approach and the developments of it but if you want to read more see the references at the end of this article. It should be noted that there are other solutions for motif discovery out there. I chose to present Matrix Profile because the concept is intuitive and easy to understand, and it works pretty well in most cases from my experience.
Motif search
Let’s say we have a motif, and now we want to search for it in another sequence in order to see when, and how many times it is repeating in this sequence. This action is also called subsequence alignment and there are a few algorithms that can do that. Matrix Profile has a solution for that as well. The basic approach here is scanning the sequence and finding the best matching subsequences.
Pattern Matching
The discussion about motifs and Matrix Profile ultimately narrows down to comparisons between subsequences (AKA patten matching). Comparing subsequences using simple euclidean distance might be a problem for subsequences representing patterns of different lengths. For example, we can compare two subsequences of the same motif, but because one of them is slightly stretched compared to the other one, the distance between them will be larger than it should. This is where Dynamic Time Warping (DTW) can help. DTW solves this issue since it allows flexible matching of data points along the two sequences. It still calculates the Euclidean distance only it can be calculated between the two best matching points.
In most cases, the best way to compare subsequences will be to compare their normalized z-scores. That way, the actual values will have less affect on the similarity score, and the pattern will have higher weight in determining that score.
Another approach for comparing subsequences is by creating signatures.
Signatures
Signatures is a very close term to motifs and they are sometimes used interchangeably, but a signature is a set of features that characterizes the motif. It could contain the subsequence itself and also other features that can be extracted from the subsequence.
Extracting features from a TS sequence is a whole subject that should be discussed separately, but here are a couple of Python libraries that can help in doing just that: TSFEL, and Catch22.
Signatures can be expressed as representation vectors, which may be used for different Time Series tasks. These tasks will be further explored and discussed in the next article of this series.
Summary
- Anomalies and Motifs are subsequences that can provide us with useful information about our sequence.
- The approach to anomaly detection in TS data is different and requires forecasting.
- Identifying motifs essentially deals with pattern matching. DTW is an important tool for pattern matching of TS data as it allows more flexibility.
- Matrix Profile provides simple tools for anomaly detection, motif discovery, and motif search (subsequence alignment).
- Signatures are a set of characterization features that represent motifs (many times in the form of representation vectors), and are useful for many TS tasks.
Matrix Profile references
The UCR Matrix Profile page
The Matrix Profile foundation
STUMPY: A MP library (article series)
If you liked this article, please give it a clap. If you want, you can also follow me to see more of my content.
To read the next article in the series: Part 4: Time series tasks and relevant algorithms (coming soon…)
