A Study on the Role of Hyperparameter Optimization (HPO) iN Image Classification Problems Using Automl Models

Abstract

Hyperparameter optimization (HPO) has a profound impact on the performance of machine learning models. This work investigates various HPO techniques and offers valuable insights into their effectiveness, particularly in the context of image classification with the aid of AutoML models. The study places a special focus on Bayesian optimization and introduces the innovative application of genetic algorithms, differential evolution, and covariance matrix adaptation - evolutionary strategy for acquisition function optimization. Comparative analysis reveals that these evolutionary variants significantly enhance the performance of standard Bayesian optimization, with genetic algorithms emerging as less effective for acquisition function optimization. In real-time scenarios, where deep learning models often involve a multitude of hyperparameters, finding the optimal configuration becomes even more challenging. To illustrate the practical implications of HPO, the thesis presents two real-time case studies. The first case study centers on the analysis of land use and land cover changes in the Ernakulam district of Kerala using sentinel-2 images from 2019 and 2023. Employing machine learning models aided by evolutionary-based hyperparameter optimization, the study successfully tracks spatial and temporal changes in land use patterns. Furthermore, a post-classification change detection was carried out to elucidate the scale and pace of urban growth following the COVID-19 pandemic. The second study addresses flood vulnerability prediction in Kerala, India, leveraging machine learning models and AutoML systems. A three-dimensional convolutional neural network (CNN), coupled with Bayesian optimization and evolutionary algorithms like differential evolution and covariance matrix adaptation, leads to enhanced accuracy, precision, recall, AUC, and kappa scores compared to AutoML models. This thesis also ventures into the realm of meta-learning, a potential approach for tackling the data scarcity inherent in real-time image classification applications. Focusing on metric-based meta-learning models, it explores the integration of hyperparameter optimization techniques to enhance the performance of these models. The innovative bilevel optimization framework, combining meta-learning and hyperparameter optimization, is introduced, with Bayesian optimization and its recent variants employed i to fine-tune hyperparameters. Computational experiments conducted on Omniglot and ImageNet datasets provide insights into the impact of different hyperparameter optimization algorithms, with meta-testing accuracy serving as the basis for conclusions. In summary, the thesis focuses on the critical role of hyperparameter optimization in improving the machine learning model performance, emphasizing its practical applications through real-time case studies. Additionally, it explores the synergy of metalearning and hyperparameter optimization, offering a comprehensive view of cuttingedge techniques in the field.