A NOVEL REINFORCEMENT LEARNING BASED ROUTING ALGORITHM FOR ENERGY MANAGEMENT IN NETWORKS

Underwater wireless sensor networks (UWSNs) have the potential to provide environmental data for various applications, including studies related to environmental changes, early warning systems, and monitoring in industry. Continuous delivery of information in these contexts is paramount. UWSNs comprise the fundamental assets in these applications. However, the peculiar characteristics of underwater require sensor nodes to rely on their limited battery reserves. Consequently, energy management in these networks becomes a critical resource allocation problem within underwater sensor networks. To address this decision making problem, cluster-based network routing protocols have been extensively explored as a technology to minimize network energy consumption. Cluster heads (CHs) are employed to aggregate data and reduce overall energy usage, thus prolonging the network’s lifespan. On the other hand, the focus on harvesting energy from ambient resources underwater has gained attention as a means to extend the operational life of sensor nodes in the distributed communication network system. This paper considers the stochastic energy harvesting process at each sensor node, specifically addressing the energy-aware routing problem in underwater acoustic sensor networks (UASNs). The contribution of this work lies in proposing a novel reinforcement learning-based algorithm for determining cluster heads (CHs), which involves not only considering the nodes’ positions and residual energy but also accounting for the expected harvested energy. Numerical results validate that our introduced approach significantly decreases energy consumption and substantially extends the network’s operational lifetime considerably.