A smart manufacturing framework for tool wear analysis and RUL estimation using multimodal deep learning

Abstract

The machining sector faces ongoing challenges in enhancing cutting tool efficiency and accurately forecasting tool wear, essential for increasing productivity and lowering operational expenses. This research presents an innovative multimodal deep learning architecture designed to predict the Remaining Useful Life (RUL) of cutting tools, providing a solid solution for predictive maintenance in Industry 5.0. By incorporating advanced computer vision methods with real-time sensor data evaluations, the framework delivers a thorough system for assessing tool conditions. The Mod-R2AU-Net is utilized for precise segmentation of tool wear, achieving a validation accuracy of 97.65% alongside a loss of 0.1711. At the same time, CNN models evaluate the tool wear area, yielding vital insights into degradation. These features extracted from images are merged with experimental sensor data (including speed, feed, depth of cut, force, temperature, and vibration) to form a multimodal dataset. XGBoost is applied for the classification of tool wear, attaining an accuracy of 98%, whereas Multi-Layer Perceptron (MLP) models forecast wear with an R2 of 0.9790 and an RMSE of 0.0063. Furthermore, the Differential Evolution-optimized BiLSTM (DE-BiLSTM) model provides the most precise RUL predictions, achieving an R2 of 0.9992 and surpassing conventional LSTM and BiLSTM models. This pioneering multimodal technique not only enhances predictive maintenance capabilities but also facilitates the optimization of tool usage, reduces unplanned downtimes, and produces notable economic benefits, positioning the machining industry at the leading edge of Industry 5.0. © The Author(s) 2025