Project DetailsThis project describes a method for unsupervised anomaly detection in autonomous systems, submitted for the IEEE Signal Processing Cup 2020. The challenge involves determining the abnormality behavior of autonomous aerial vehicles using a sequence of images (video) and Inertial Measurement Unit (IMU) data. The proposed method handles this noisy highdimensional multimodal data by embedding both appearance and motion features in a reduced dimension representation, utilizing the correlation between sensors as an indicator of abnormality.
First, appearance features are extracted from each image by a pre-trained ResNet-18 deep neural network. Then, appearance and motion features are fused into a multivariate time series. Each time window of the series is embedded in a reduced dimension manifold, using a Gaussian kernel function.
The resulting kernel matrix captures the pairwise similarity between each pair of windows. We then exploit the geometric properties of the reduced dimension manifold and use its intrinsic distance as a measure of abnormality. The proposed method is attractive as it is fully unsupervised, data agnostic, and noise-robust. On the data supplied in the IEEE Signal Processing Cup 2020, it achieves impressive results with AUC (Area Under a Curve) of 0.965.
