Efficient Kalman filter based deep learning approaches for workload prediction in cloud and edge environments

Abstract

Offering cloud resources to consumers presents several difficulties for cloud service providers. When utilizing resources efficiently in cloud and edge contexts, precisely forecasting workload is a crucial problem. Accurate workload prediction allows intelligent resource allocation, preventing needless waste of computational and storage resources while meeting user’s Quality of Service(QoS). In order to mitigate this issue, Kalman filter-based novel hybrid models, including Long Short Term Memory (LSTM), Bi-directional Long Short Term Memory (BI-LSTM), and Gated Recurrent Unit (GRU), are proposed. These models utilize CNN and attention mechanisms to predict workloads at Edge Servers accurately. The proposed models were extensively evaluated on real world traces like Alibaba_v2018, Materna, Bitbrains, Microsoft Azure_2019 and Planet lab datasets at various time intervals with and without using Kalman filter. The experimental comparison shows that 97%, 82% and 90% reduction in MSE for Alibaba, 73%, 73% and 63% reduction in MSE for Materna, 72%, 63% and 40% reduction in MSE for Planet lab, 95%, 77% and 96% reduction in MSE for Microsoft Azure and 91%, 87% and 91% reduction in MSE for Bitbrains with respect to CPU utilization %. The effectiveness of the proposed forecasting model is validated through statistical analysis using the Friedman and Nemenyi post-hoc tests. © The Author(s), under exclusive licence to Springer-Verlag GmbH Austria, part of Springer Nature 2024.