ABSTRACT This paper proposes an adaptive dynamic programming (ADP) method based on nonzero‐sum (NZS) game theory for the optimal cooperative control of a multi‐modular robot manipulator (MMRM). First, the system dynamics are established via the Newton‐Euler iterative algorithm, and load distribution is adopted to allocate driving forces to each module while maintaining force balance. The optimal cooperative control problem is then formulated as an NZS game, with each joint of the modular manipulators treated as a player. A radial basis function neural network (RBFNN)‐based state observer is constructed to estimate model unknowns. Moreover, a novel critic neural network weight‐adjustment rule that incorporates experience replay is proposed to relax the persistent excitation (PE) condition. Finally, Lyapunov theory proves the ultimate uniform boundedness (UUB) of the closed‐loop system errors, and experiments verify the method's effectiveness.