In this paper, we introduce a new notion in a semigroup $S$ as an extension of Mary's inverse. Let $a,d\in S$. An element $a$ is called left (resp. right) invertible along $d$ if there exists $b\in S$ such that $bad=d$ (resp. $dab=b$) and $b\leq_\mathcal{L}d$ (resp. $b\leq_\mathcal{R}d$). An existence criterion of this type inverse is derived. Moreover, several characterizations of left (right) regularity, left (right) $\pi$-regularity and left (right) $*$-regularity are given in a semigroup. Further, another existence criterion of this type inverse is given by means of a left (right) invertibility of certain elements in a ring. Finally we study the (left, right) inverse along a product in a ring, and, as an application, Mary's inverse along a matrix is expressed., The authors are highly grateful to the referee for valuable comments which led to improvements of this paper. In particular, Corollaries 2.5, 2.6 and 3.6, Remarks 2.13 and 3.10 and the final remark (ii) were suggested to the authors by the referee. The first author is grateful to China Scholarship Council for giving him a purse for his further study in University of Minho, Portugal. Jianlong Chen and Huihui Zhu are financed by the National Natural Science Foundation of China (No. 11201063 and No. 11371089), the Specialized Research Fund for the Doctoral Program of Higher Education (No. 20120092110020), the Natural Science Foundation of Jiangsu Province (No. BK20141327), the Foundation of Graduate Innovation Program of Jiangsu Province(No. CXLX13-072), the Scientific Research Foundation of Graduate School of Southeast University and the Fundamental Research Funds for the Central Universities (No. 22420135011). Pedro Patr´ıcio is financed by the Research Centre of Mathematics of the University of Minho with the Portuguese Funds from the “Funda¸c˜ao para a Ciˆencia e a Tecnologia”, through the Project PEst-OE/MAT/UI0013/2014.