In this paper, we discuss the conditions under which the coupled KdV and coupled Harry Dym hierarchies possess inverse (negative) parts. We further investigate the structure of nonlocal parts of tensor invariants of these hierarchies, in particular, the nonlocal terms of vector fields, conserved one-forms, recursion operators, Poisson and symplectic operators. We show that the invertible coupled KdV hierarchies possess Poisson structures that are at most weakly nonlocal while coupled Harry Dym hierarchies have Poisson structures with nonlocalities of the third order., Funding Agencies|Swedish Research Council|2011-52