Existence, uniqueness and stability analysis of a nonlinear coupled system involving mixed ϕ-Riemann-Liouville and ψ-Caputo fractional derivatives

Abstract This study delves into the existence, uniqueness, and stability of solutions for a nonlinear coupled system incorporating mixed generalized fractional derivatives. The system is characterized by ψ-Caputo and ϕ-Riemann-Liouville fractional derivatives with mixed boundary conditions. We provide essential preliminaries and definitions, followed by a detailed analysis using fixed point theory to establish the main results. Furthermore, we discuss the Hyers-Ulam stability of the proposed system and illustrate the theoretical findings with several examples. This study extends and generalizes various results in the literature and provides new insights into the qualitative behavior of fractional differential systems.