Curriculum Learning with Heat Map-Based State Representations in Autonomous Drone Cargo Search

We address the multi-drone cargo search problem, focusing on planning and target assignment in environments with sparse rewards, limited onboard sensing, and high-dimensional state spaces. Our approach leverages reinforcement learning (RL) to coordinate drones efficiently under such constraints. We introduce a state space adaptive curriculum learning framework for RL, which employs heat map representations to overcome the fixed uniform state requirement of traditional curricula. By defining heat maps over selected subsets of the full state variables, our method enables early training stages to operate on low dimensional projections, and then progressively incorporate richer spatial information. We validate this approach on a multi drone cargo search task, where agents must locate targets under sparse rewards and limited sensing. Our experiments demonstrate that a heat map curriculum yields substantially faster convergence and higher success rates compared to both standard RL and a naive curriculum approach.