Energy-Entropy-Based Intelligence Factor for Artificial Neural Networks

Hyper-parameter optimization in artificial neural networks (ANNs) is a computationally costly process that requires multiple trials to determine a feasible configuration. In this study, we propose a novel approach in which an independent external neural network is used to predict the performance and evaluate the feasibility of the first neural network's hyper-parameter settings. Our method aims to reduce the overall computational cost by utilizing only a small part of the dataset while providing a promising and effective hyperparameter tuning process. In this study, the proposed method consists of two neural networks. The first or main network was trained with various hyperparameter configurations, and the supervisory second network used the internal structural information of the main network to predict its performance. This approach provides efficient prediction of hyperparameter suitability and minimizes the number of trial-and-error cycles. The experimental results demonstrate that the proposed method achieves reliable performance predictions. The second network, trained on only 10% of the dataset, could predict the main network's error with a prediction error below 6%. Furthermore, the method achieved generalization for different neural architectures without requiring dataset-dependent tuning.