In 1939, L. Rédei introduced a certain triple symbol in order to generalize the Legendre symbol and Gauss' genus theory. Rédei's triple symbol $[a_1,a_2, p]$ describes the decomposition law of a prime number $p$ in a certain dihedral extension over $\mathbb{Q}$ of degree 8 determined by $a_1$ and $a_2$. In this paper, we show that the triple symbol $[-p_1,p_2, p_3]$ for certain prime numbers $p_1, p_2$ and $p_3$ can be expressed as a Fourier coefficient of a modular form of weight one. For this, we employ Hecke's theory on theta series associated to binary quadratic forms and realize an explicit version of the theorem by Weil-Langlands and Deligne-Serre for Rédei's dihedral extensions. A reciprocity law for the Rédei triple symbols yields certain reciprocal relations among Fourier coefficients.