• A novel coupling approach of ES-FEM-SPH is developed for solving thermal-fluid-structure interaction (TFSI) problems. • The updated Lagrangian ES-FEM is developed for solving the coupled thermal elastic problems. • The SPH method, after integrating with particle shifting technique and kernel gradient correction, is robust and effective in modeling thermal fluid flows. • The ghost particle coupling algorithm is developed for treating fluid structure conjugate heat transfer. Thermal-fluid-structure interaction (TFSI) problems are significant in science and engineering, and usually pose great challenges for numerical simulations due to the coupled effects of thermal convection, fluid flow and structure deformation. In this paper, a novel coupling approach of smoothed finite element method (ES-FEM) with an improved smoothed particle hydrodynamic (SPH) method is developed for TFSI problems. In the coupling approach, the edge based ES-FEM is used to model solid domain and the Lagrangian SPH is used to model fluid flow. In ES-FEM, the temperature and velocity gradient smoothing technique are applied over the edge-based smoothing domain for thermal structure coupling problems. In SPH, some state-of-art algorithms including kernel gradient correction (KGC) and particle shift technique (PST) are integrated to ensure computational accuracy for simulating thermal fluid flows. A ghost particle coupling algorithm is developed to handle fluid-structure interaction and fluid-structure conjugate heat transfer, and the kinematic condition, dynamics conditions and conservation of energy are satisfied. Four numerical examples are tested to demonstrate the effectiveness of the present coupling approach of ES-FEM-SPH for TFSI problems. Image, graphical abstract [ABSTRACT FROM AUTHOR]