ABSTRACT This study is concerned with the security control problem of multi‐agent systems (MASs). First, the multi‐agent model with unknown actuator faults and disturbances is considered, and the adverse effects of these are estimated by fuzzy logic systems (FLSs). Then, the Hamiltonian–Jacobi–Bellman equation is derived under a performance cost function with a discount term. Then, two FLSs are adopted to build an actor–critic structure to approximate the cost function and the optimal controller, respectively. Furthermore, to reduce the energy consumption, a new RL and improved artificial potential field (APF) based optimal fault‐tolerant formation controller with collision avoidance is obtained. Finally, the effectiveness of the research method is verified by the comparison of simulation results.