Learning to Walk on a Human Musculoskeletal Model Wearing a Knee Orthosis via Deep Reinforcement Learning

Knee orthoses aim to treat problems in the knees by customizing them in order to support the joint externally, protect the joint, provide bio-mechanical balance, eliminate dysfunctions, reduce pain, and strengthen weakened muscles. Since each case is different from each other, individual treatment is required. For this reason, measuring the performance of orthoses in a simulated environment before they are applied to the patients increases efficiency during the treatment. Musculoskeletal model simulations allow estimating how the orthosis will affect the patient's motions. In this paper, the deep reinforcement learning (DRL) method, which imitates the reference walking motion, is used in simulations for the model to learn to walk. The walking performance and muscle activation of four different musculoskeletal models that are healthy, injured in the knee but not wearing an orthosis, wearing passive orthosis, and wearing active orthosis are compared.