Multi-objective Resource Allocation for 5G Using Hierarchical Reinforcement Learning

5G and beyond networks are expected to satisfy the challenging requirements of a variety of vertical services and domains. In this paper, we propose a radio access network (RAN) resource allocation method that dynamically allocates resource blocks using reinforcement learning (RL). We address Ultra-Reliable Low-Latency Communications (URLLC) and enhanced Mobile Broad-Band (eMBB) resource allocation problem aiming to maximize the throughput of eMBB service while satisfying the latency requirement of URLLC traffic. Resource allocation is optimized using a hierarchical multi-RL-Agents, where the main-Agent resides on upper layer and performs inter-slice resource allocation between URLLC and eMBB services. URLLC and eMBB sub-Agents are responsible for intra-slice resource allocation by distributing the resources among their users. The state space of the RL is reduced using the quantization of queue occupancies and grouping the users according to their channel qualities. The action space of the sub-Agents is also reduced by assigning the percentage of the resources within the URLLC and eMBB user groups. The proof of concept demonstration with three user groups shows that the proposed RL based resource allocation approach provides close approximation to the optimal solution calculated by the exhaustive search (ES).