The Legendre-based ultraspherical spectral method for ordinary differential equations (Olver, S. & Townsend, A. (2013) A fast and well-conditioned spectral method. SIAM Rev. 55 , 462–489.) is combined with a formula for the convolution of two Legendre series (Hale, N. & Townsend, A. (2014a) An algorithm for the convolution of Legendre series. SIAM J. Sci. Comput. , 36 , A1207–A1220.) to produce a new technique for solving linear Fredholm and Volterra integro-differential equations with convolution-type kernels. When the kernel and coefficient functions are sufficiently smooth, then the method is spectrally accurate and the resulting almost-banded linear systems can be solved with linear complexity. [ABSTRACT FROM AUTHOR]