Adaptive Optics control with Multi-Agent Model-Free Reinforcement Learning

Posted on 14.01.2022 - 15:19
We present a novel formulation of closed-loop Adaptive Optics (AO) control as a Multi-Agent Reinforcement Learning (MARL) problem in which the controller is able to learn a non-linear policy and does not need a priori information on the dynamics of the atmosphere. We identify the different challenges of applying a Reinforcement Learning (RL) method to AO and, to solve them, propose the combination of model-free MARL for control with an autoencoder neural network to mitigate the effect of noise. Moreover, we extend current existing methods of error budget analysis to include a RL controller. The experimental results for an 8m telescope equipped with a 40x40 Shack-Hartmann system show a significant increase in performance over the integrator baseline and comparable performance to a model-based predictive approach, a linear quadratic Gaussian controller with perfect knowledge of atmospheric conditions. Finally, the error budget analysis provides evidence that the RL controller is partially compensating bandwidth error and is helping to mitigate the propagation of aliasing.


Mulet, Bartomeu; Ferreira, Florian; Quinones, Eduardo; Gratadour, Damien; Martin, Mario (2022): Adaptive Optics control with Multi-Agent Model-Free Reinforcement Learning. Optica Publishing Group. Collection.
Select your citation style and then place your mouse over the citation text to select it.



Usage metrics

Optics Express


Bartomeu Mulet
Florian Ferreira
Eduardo Quinones
Damien Gratadour
Mario Martin


need help?