Objective To compare ordinary least squares (OLS) and instrumental variable (IV) estimates of the effects of COX-2 selective inhibitor non-steroidal anti-inflammatory drugs (COX-2s) relative to non-selective NSAIDs (non-selectives) on incidence of gastrointestinal (GI) bleeding, and acute myocardial infarction (MI). We test the validity of the IV estimates, and compare to estimates from randomised controlled trials (RCTs). Design Cohort study using administration data. Setting Primary care. Participants 93 562 patients aged over 65 years attending one of 104 general practices (GP) in the UK. Main outcomes GI bleeding and acute MI. Methods We use data from the UK general practice research database. Our outcomes are incident gastrointestinal bleeding and myocardial infarctions within 60, 120, 180 and 360 days of the first prescription. We estimate risk differences in outcomes using OLS and IVs, using physician prior prescribing as an IV, and compare them to RCT estimates. Results The OLS estimates suggested COX-2s were positively associated with GI bleeds, and were negatively associated with MI. Adjusted OLS estimates were attenuated. In contrast our IV estimates imply –0.43(–1.54,0.68) fewer GI bleeds per 100 people treated with COX-2s after 180 days. The IV estimates provide little evidence, −0.55(−1.2, 1.5), of a difference in MI between treatments. These are comparable to previously published IV estimates and RCT evidence. Conclusions Deriving causal treatment effects using observational health data has proven to be problematic. We demonstrate that in contrast to IV based estimates, conventional OLS methods fail to estimate casual effects as indicated by RCTs. [ABSTRACT FROM AUTHOR]