Measurement of cos 2β in B⁰ → D(*)h⁰ with D→K⁰[subscript S]π⁺π⁻ decays by a combined time-dependent Dalitz plot analysis of BaBar and Belle data

We report measurements of sin2β and cos2β using a time-dependent Dalitz plot analysis of B⁰ → D(*)h⁰ with D→K⁰[subscript S]π⁺π⁻ decays, where the light unflavored and neutral hadron h⁰ is a π⁰, η, or ω meson. The analysis uses a combination of the final data sets of the BaBar and Belle experiments containing 471×10⁶ and 772×10⁶ BB[over ¯] pairs collected at the ϒ(4S) resonance at the asymmetric-energy B factories PEP-II at SLAC and KEKB at KEK, respectively. We measure sin2β = 0.80 ± 0.14(stat) ± 0.06(syst) ± 0.03(model) and cos2β = 0.91 ± 0.22(stat) ± 0.09(syst) ± 0.07(model). The result for the direct measurement of the angle is β=(22.5±4.4(stat) ± 1.2(syst) ± 0.6(model))°. The last quoted uncertainties are due to the composition of the D→K⁰[subscript S]π⁺π⁻ decay amplitude model, which is newly established by a Dalitz plot amplitude analysis of a high-statistics e⁺e⁻→cc[over ¯] data sample as part of this analysis. We find the first evidence for cos2β>0 at the level of 3.7 standard deviations. The measurement excludes the trigonometric multifold solution π/2-β=(68.1±0.7)° at the level of 7.3 standard deviations and therefore resolves an ambiguity in the determination of the apex of the CKM Unitarity Triangle. The hypothesis of β=0° is ruled out at the level of 5.1 standard deviations, and thus CP violation is observed in B⁰ → D(*)h⁰ decays. The measurement assumes no direct CP violation in B⁰ → D(*)h⁰ decays.