Strategy Selection in Dynamic Constrained Multi-Objective Optimization via State-Augmented Deep Reinforcement Learning

What the paper says

Dynamic constrained multi-objective optimization problems are characterized by time-varying objectives, decision variables, and constraints, presenting significant challenges for evolutionary algorithms. Current dynamic constrained multi-objective evolutionary algorithms (DCMOEAs) primarily adopt multiple predetermined dynamic response strategies to address various dynamic scenarios, including changes in objectives, constraints, and environmental conditions. However, these DCMOEAs fail to refine the strategies based on how environmental changes specifically affect population characteristics such as reducing solution feasibility, diminishing population diversity, and hindering convergence toward the Pareto front, resulting in inefficient or even counterproductive responses. The response strategies should be adaptively selected based on the deterioration of the population characteristics caused by environmental changes. This selection enhances the adaptability of DCMOEAs to evolving environments, improving their performance in seeking Pareto optimal solutions. We propose a deep reinforcement learning approach that selects dynamic response strategies by analyzing deterioration patterns of the population characteristics . To enable strategy selection for different environmental scenarios, a state-augmented mechanism is designed to capture the interconnected deterioration effects among the different population characteristics, facilitating precise environmental change perception and targeted strategic responses. Experimental results demonstrate that the proposed algorithm significantly outperforms competing DCMOEAs in both convergence speed and solution quality. The results indicate that adaptively selecting dynamic response strategies based on deterioration patterns of the population characteristics leads to efficient optimization processes by avoiding unnecessary strategy executions while maintaining solution quality through targeted responses. This study provides new perspectives on developing adaptive dynamic response strategies for dynamic constrained multi-objective optimization.