Background Vaccination of infants with pneumococcal conjugate vaccines is recommended by the World Health Organization. Evidence is mixed regarding the differences in immunogenicity and efficacy of the different pneumococcal vaccines. Objectives The primary objective was to compare the immunogenicity of pneumococcal conjugate vaccine-10 versus pneumococcal conjugate vaccine-13. The main secondary objective was to compare the seroefficacy of pneumococcal conjugate vaccine-10 versus pneumococcal conjugate vaccine-13. Methods We searched the Cochrane Library, EMBASE, Global Health, MEDLINE, ClinicalTrials.gov and trialsearch.who.int up to July 2022. Studies were eligible if they directly compared either pneumococcal conjugate vaccine-7, pneumococcal conjugate vaccine-10 or pneumococcal conjugate vaccine-13 in randomised trials of children under 2 years of age, and provided immunogenicity data for at least one time point. Individual participant data were requested and aggregate data used otherwise. Outcomes included the geometric mean ratio of serotype-specific immunoglobulin G and the relative risk of seroinfection. Seroinfection was defined for each individual as a rise in antibody between the post-primary vaccination series time point and the booster dose, evidence of presumed subclinical infection. Each trial was analysed to obtain the log of the ratio of geometric means and its standard error. The relative risk of seroinfection (‘seroefficacy’) was estimated by comparing the proportion of participants with seroinfection between vaccine groups. The log-geometric mean ratios, log-relative risks and their standard errors constituted the input data for evidence synthesis. For serotypes contained in all three vaccines, evidence could be synthesised using a network meta-analysis. For other serotypes, meta-analysis was used. Results from seroefficacy analyses were incorporated into a mathematical model of pneumococcal transmission dynamics to compare the differential impact of pneumococcal conjugate vaccine-10 and pneumococcal conjugate vaccine-13 introduction on invasive pneumococcal disease cases. The model estimated the impact of vaccine introduction over a 25-year time period and an economic evaluation was conducted. Results In total, 47 studies were eligible from 38 countries. Twenty-eight and 12 studies with data available were included in immunogenicity and seroefficacy analyses, respectively. Geometric mean ratios comparing pneumococcal conjugate vaccine-13 versus pneumococcal conjugate vaccine-10 favoured pneumococcal conjugate vaccine-13 for serotypes 4, 9V and 23F at 1 month after primary vaccination series, with 1.14- to 1.54-fold significantly higher immunoglobulin G responses with pneumococcal conjugate vaccine-13. Risk of seroinfection prior to the time of booster dose was lower for pneumococcal conjugate vaccine-13 for serotype 4, 6B, 9V, 18C and 23F than for pneumococcal conjugate vaccine-10. Significant heterogeneity and inconsistency were present for most serotypes and for both outcomes. Twofold higher antibody after primary vaccination was associated with a 54% decrease in risk of seroinfection (relative risk 0.46, 95% confidence interval 0.23 to 0.96). In modelled scenarios, pneumococcal conjugate vaccine-13 or pneumococcal conjugate vaccine-10 introduction in 2006 resulted in a reduction in cases that was less rapid for pneumococcal conjugate vaccine-10 than for pneumococcal conjugate vaccine-13. The pneumococcal conjugate vaccine-13 programme was predicted to avoid an additional 2808 (95% confidence interval 2690 to 2925) cases of invasive pneumococcal disease compared with pneumococcal conjugate vaccine-10 introduction between 2006 and 2030. Limitations Analyses used data from infant vaccine studies with blood samples taken prior to a booster dose. The impact of extrapolating pre-booster efficacy to post-booster time points is unknown. Network meta-analysis models contained significant heterogeneity which may lead to bias. Conclusions Serotype-specific differences were found in immunogenicity and seroefficacy between pneumococcal conjugate vaccine-13 and pneumococcal conjugate vaccine-10. Higher antibody response after vaccination was associated with a lower risk of subsequent infection. These methods can be used to compare the pneumococcal conjugate vaccines and optimise vaccination strategies. For future work, seroefficacy estimates can be determined for other pneumococcal vaccines, which could contribute to licensing or policy decisions for new pneumococcal vaccines. Study registration This study is registered as PROSPERO CRD42019124580. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 17/148/03) and is published in full in Health Technology Assessment; Vol. 28, No. 34. See the NIHR Funding and Awards website for further award information. Plain language summary Pneumococcal disease is a serious illness caused by a bacterial infection that can result in death. Children in the United Kingdom receive a vaccine to prevent this disease that protects against 13 different types of pneumococcal diseases. It is very effective, but other vaccines are also available, such as one that contains 10 types of pneumococcal diseases. Vaccines in the United Kingdom are bought by the government and the choice of which vaccine to provide is based on the cost of the vaccine as well as the benefits to our health. However, there is very little information comparing different vaccines and it is often assumed they are the same. We did a large analysis combining all studies of the two main licensed pneumococcal vaccines to determine which vaccine provides better protection against infection and how this affects costs. We used information from studies published in medical journals, and also data from studies done by the companies that own the vaccines. Our results showed that pneumococcal conjugate vaccine-13 vaccine provided better protection than pneumococcal conjugate vaccine-10 for 5 of the 10 serotypes that are contained in both vaccines. When we used these results to model what might have happened had either of these vaccines been introduced into the United Kingdom vaccination programme in 2006, we found that both vaccines caused a rapid decrease in the amount of disease, but that the decrease in disease was faster with pneumococcal conjugate vaccine-13 than pneumococcal conjugate vaccine-10. This resulted in 2808 cases of diseases prevented over a 25-year time frame with pneumococcal conjugate vaccine-13 compared with pneumococcal conjugate vaccine-10. Our methods can be used to compare other vaccines and we recommend this type of study be done in future when making decisions on vaccine product choice. Scientific summary Streptococcus pneumoniae (pneumococcus) causes severe diseases, including bacterial pneumonia, meningitis and sepsis, leading to substantial morbidity and mortality worldwide, with the highest disease burden being in young children and older adults. Three pneumococcal conjugate vaccines (PCVs) have been widely deployed worldwide in the past two decades: PCV7 (Prevnar; Pfizer, headquartered in New York City, New York, USA), PCV10 (Synflorix; GlaxoSmithKline, headquartered in Brentford, London, UK) and PCV13 (Prevenar 13; Pfizer, headquartered in New York City, New York, USA), resulting in substantial reduction in disease. Between 2009 and 2011, PCV7 was gradually replaced by PCV13 and PCV10 and is no longer available. The World Health Organization (WHO) does not preferentially endorse one PCV over another. Both PCV13 and PCV10 have been shown to provide both direct and indirect protection against pneumococcal pneumonia, invasive pneumococcal disease and nasopharyngeal carriage. Although there are 10 common serotypes in these 2 vaccines, the components of the vaccines differ, with different carrier proteins used in the conjugation process, as well as different amounts of polysaccharide, and these differences may contribute to differences in protection. Large randomised controlled trials directly comparing different PCVs with invasive pneumococcal disease as the primary outcome are not feasible. We previously used ‘seroinfection’ as an outcome for analysis of PCVs, where seroinfection is defined as an increase in antibody levels between the primary vaccination series (typically complete at 5–7 months of age) and the booster dose (typically administered at 9–18 months of age). Seroinfection can be regarded as evidence of exposure to the pathogen and a resultant subclinical infection, given antibody responses wane rapidly during this period otherwise. Seroinfection rates for different vaccines can be compared by calculating the relative risk (RR) of seroinfection, referred to herein as ‘seroefficacy’. We meta-analysed data from studies of PCVs to compare the immunogenicity and seroefficacy of PCV10 with PCV13 for each serotype. We aimed to determine if serotype-specific immune responses were higher for either vaccine and whether this resulted in greater protection again seroinfection. In addition, we explored the overall relationship between the higher immune response and protection against seroinfection in infants. Following this, we show how serotype-specific estimates of seroefficacy can be incorporated in vaccine cost-effectiveness models. Objectives The primary objective of the systematic review was to compare the immunogenicity of PCV10 versus PCV13 for each serotype contained in the vaccines. The secondary objectives were: to compare the seroefficacy of PCV10 versus PCV13 for each serotype contained in the vaccines for PCV10 and PCV13 separately, to estimate immunogenicity and seroefficacy in comparison with the older PCV7 vaccine to determine how the comparisons of immunogenicity and efficacy of PCV10 to PCV13 are affected by the co-administration of different routine vaccines. Methods Systematic review We conducted a systematic review identifying studies that compared the immunogenicity of licensed PCVs in trials which randomised children to one of two different PCVs. The PCVs included in the review were PCV7 (Prevnar; Pfizer), PCV10 (Synflorix; GlaxoSmithKline) and PCV13 (Prevenar 13; Pfizer); PCV7 was included even though no longer available, so that we could compare PCV13 and PCV10 indirectly through them each being compared with PCV7 for the same serotypes. Data sources The databases searched were Cochrane Database of Systematic Reviews and Cochrane Central Register of Controlled Trials, EMBASE, Global Health and MEDLINE. The trial registers searched were ClinicalTrials.gov (https://clinicaltrials.gov/) and WHO International Clinical Trials Registry Platform (https://trialsearch.who.int/). The search comprised title/abstract keywords and subject headings for pneumococcal vaccines and children. A methodological search filter for randomised controlled trials taken from the Cochrane Handbook was used to limit to randomised controlled trials. Pharmaceutical company websites (GlaxoSmithKline and Pfizer) were also hand-searched for relevant studies. No date or language limits were applied. Study selection Randomised controlled trials were included if they provided direct comparisons of either PCV7, PCV10 or PCV13 among infants and children ˂ 2 years of age, and if they provided estimates of antibody responses [serotype-specific anti-pneumococcal immunoglobulin G (IgG) to PCVs for at least one time point of 1] between 4 and 6 weeks after the primary vaccination series and/or 1 month after a booster vaccination. Individual participant-level data were retrieved if available. Aggregate data from publications were extracted if individual participant data were not available. Risk of bias in results of the included studies was assessed independently by two reviewers using the Cochrane Risk of Bias Tool. Data synthesis Each trial with individual participant-level data available was analysed to obtain the log of the ratio of geometric means (log-GMR) and its standard error (SE) for each serotype and time point of interest. The RR of seroinfection was estimated by comparing the proportion of participants with seroinfection between vaccine groups. When no seroinfection occurred in any group (numerator of absolute risk was 0), a small non-zero value (0.5) was added to both sero-infected and sero-non-infected groups to allow estimation of the RR. The log-GMRs, log-RRs and their SEs constituted the input data for evidence synthesis. Only trials supplying individual participant data were included in seroefficacy analyses. For serotypes contained in all three vaccines, evidence could be synthesised using a network meta-analysis (NMA) of all comparisons. For other serotypes, meta-analysis was used for evidence synthesis. To estimate the overall association between antibody geometric mean ratio (GMR) and RR across all serotypes, we fitted a mixed-effect model regressing study-level RRs of seroinfection on GMRs across serotypes, weighted by the sample size of each study. Fixed effects included GMR, serotype and interactions between GMR and serotype (allowing serotype-specific association), while study was included as a random effect. Mathematical modelling and retrospective economic evaluation To illustrate the use of serotype-specific estimates of seroefficacy in modelling vaccine impact and cost-effectiveness, we developed a serotype-specific mathematical model of pneumococcal transmission dynamics to compare the differential impact of PCV10 and PCV13 introduction on invasive pneumococcal disease cases with vaccine serotypes in England and Wales. The model estimated the impact over a 25-year time period from 2006 to 2030. We subsequently assessed the cost-effectiveness of introducing infant vaccination with PCV13 compared with introducing PCV10 from a healthcare payer perspective in England and Wales. More specifically, we retrospectively estimated the additional threshold price per dose below which PCV13 would be more cost-effective than PCV10 had they both been available at the time of introduction of the PCV vaccine programme in England and Wales in 2006. Results Database registry and hand searches identified 4699 publication records of which 47 studies (78 publication reports) satisfied our eligibility criteria. Nineteen studies (24 publication reports) were excluded from the analysis: 6 studies did not provide individual patient or aggregate data and 13 studies (18 publication reports) were studies with the vaccines of interest, but it was not possible to form a loop within the NMA to provide indirect evidence. The remaining 28 studies (54 publication records) from 2009 to 2023 were included in the NMAs. Twenty-two studies provided individual participant data with a further five studies reporting aggregate data. Immunogenicity Geometric mean ratios for comparisons between PCV13 versus PCV10 for any primary series schedule were higher for PCV13 for serotypes 4, 7F, 9V and 23F at 1 month after primary vaccination series, with 1.14- to 1.54-fold higher IgG responses with PCV13. Additional serotypes contained only in the PCV13 vaccine (3, 6A and 19A) also favoured PCV13 as expected. GMRs were similar for the remaining serotypes (1, 5, 6B, 14, 18C and 19F). GMRs favoured PCV7 over either PCV13 or PCV10 for serotypes 4, 6B, 9V, 14 and 23F. There was no difference in GMRs for serotypes 18C and 19F across three vaccines. At the pre-booster time point, data were available from 18 cohorts. IgG responses were lower with PCV13 compared with PCV10 for all PCV7 serotypes except for serotype 14, with the point estimates of GMRs comparing PCV13 versus PCV10 ranging from 0.44 to 0.78. IgG responses were higher for PCV13 for serotypes 1, 5 and 7F. GMRs comparing PCV13 versus PCV7 showed higher IgG with PCV7 for serotypes 4, 6B, 9V, 14 and 23F and higher IgG with PCV13 for serotype 19F. At 28 days post booster, data were available from 26 cohorts. GMRs favoured PCV13 over PCV10 for serotype 6B, 9V, 14 and 23F and favoured PCV10 over PCV13 for serotype 18C. For serotype 1, 5 and 7F, antibody responses were higher in PCV13 compared with PCV10. PCV7 recipients had higher geometric mean concentrations (GMCs) compared with PCV13 for all PCV7 serotypes except 6B for which there was no difference, and 19F, which favoured PCV13. For PCV13-only serotypes (3, 6A and 19A), GMRs favour PCV13 at all three time points. Substantial heterogeneity and network inconsistency were present for most serotypes at all three time points. To explore potential reasons for the observed heterogeneity, we summarised cohort-level GMRs and RRs for each vaccine comparison. These descriptive analyses revealed a lack of consistency in the direction of study-level estimates within each vaccine comparison, resulting in the significant heterogenicity. There was also no observable pattern in any trial-level variable (region, co-administered vaccines, vaccine schedule), from which one might propose a mechanism that would adequately explain this variation in GMRs. Seroefficacy There were 12 studies (15 cohorts) with available individual participant antibody data at both post-primary and prior to the booster dose, allowing serotype-specific estimation of seroefficacy from a total of 5152 participants. Of these 15 cohorts, 6 compared PCV10 versus PCV7, 3 compared PCV13 versus PCV7 and 6 compared PCV13 versus PCV10. Among PCV7 serotypes, the risk of seroinfection was lower with PCV13 than PCV10 for serotypes 4, 6B, 9V, 18C and 23F, while no difference was seen for serotype 14 and 19F. The RRs of seroinfection (PCV13 vs. PCV10) for PCV7 serotypes ranged from 0.32 (95% CI 0.19 to 0.52) for serotype 4 to 1.28 (95% CI 0.95 to 1.74) for serotype 14. For serotypes 1, 5 and 7F, evidence was summarised from six studies directly comparing PCV13 with PCV10. Comparisons between PCV13 and PCV7 favoured neither vaccine over the other, whereas comparisons between PCV7 and PCV10 favoured PCV7 for serotypes 5, 6B, 9V, 18C and 23F. The I2 and p-values indicated some heterogeneity for all PCV7 serotypes except for serotype 4 and 19F. In the mixed-effects model of all serotypes combined, vaccines that produced the same amount of antibody (GMR = 1) had very similar protection (adjusted RR 0.80, 95% CI 0.41 to 1.58). The model estimate indicates that for each twofold increase in antibody response, the risk of seroinfection was halved (GMR of 2.0; RR 0.46, 95% CI 0.23 to 0.96). Mathematical model and economic evaluation Mathematical model results showed that in the absence of any vaccine programme, an increase in invasive pneumococcal disease cases caused by all five serotypes would be seen over the 25-year time frame. With the introduction of either PCV13 or PCV10 vaccine programmes in 2006, case counts would have decreased, achieving near eradication of all serotypes within the time frame modelled. The decrease in cases was most rapid for serotype 6B and least rapid for serotype 4. The decrease in cases was less rapid for PCV10 than for PCV13 due to the lower seroefficacy. The introduction of an infant PCV13 programme was predicted to avoid an additional 2808 (95% CI 2690 to 2925) cases of invasive pneumococcal disease compared with PCV10 introduction between 2006 and 2030. This includes an estimated 326 cases of meningitis, 578 cases of sepsis, 1770 cases of invasive pneumonia and 30,680 cases of non-invasive pneumonia. Under base-case assumptions, this resulted in discounted healthcare savings of £13 million (95% CI £12 to £14 million). Including non-invasive pneumonia increased the savings to £27 million (95% CI £25 to £29 million). Conclusions In our study, we used a novel methodology to define seroinfection from immunogenicity data to compare the relative efficacy of PCVs in preventing infection. Our results using individual-level data from a global meta-analysis provide the first estimates of the comparative protection afforded by different pneumococcal vaccines and show that for many serotypes, carriage events are less common after PCV13 than PCV10, likely due to a higher antibody response. In addition, we quantify the relationship between the immune response to vaccination and protection against infection, measured serologically, and show that higher antibody responses in infants are associated with greater protection from infection. Licensure of new vaccines is based on non-inferiority comparisons with current vaccines and the proportion of antibody responses above the agreed threshold as a minimum requirement. Once a vaccine meets this ‘at-least-as-good-as’ immunogenicity criteria, it has previously not been clear whether exceeding it is of benefit, and the WHO position paper on pneumococcal vaccines states ‘It is unknown whether a lower serotype-specific GMC of antibody indicates less efficacy’. Our results show that lower protection against subclinical infection does indeed follow from lower antibody production and that two vaccines that produce a similar level of antibody will provide similar levels of protection. The implications of these findings are of greatest importance when a new vaccine roll-out is being considered. Lower antibody production or lower seroefficacy for one vaccine product does not necessarily imply limited effectiveness against invasive pneumococcal diseases when considering vaccines such as PCV10 and PCV13 which are highly effective vaccines in many settings. Instead, lower antibody responses lead to less rapidly observed indirect protection after implementation into a national programme as a smaller proportion of transmission events are blocked by the vaccine. This is evident in the mathematical modelling which showed less rapid decreases in the number of cases of invasive disease when introducing PCV10 compared with PCV13. Implications for practice This evidence of differences in serotype-specific protection can be incorporated into cost-effectiveness models used to compare vaccine products. Cost-effectiveness studies have highlighted the lack of evidence of comparative efficacy for different PCVs, resulting in previous cost-effectiveness models that ignore serotype-specific differences and assume equivalent efficacy for all serotypes covered by different PCVs. Our study fills this evidence gap and allows researchers and policy-makers to use more accurate vaccine-specific models in decision-making. Our cost-effectiveness analysis of a hypothetical scenario showed that introducing infant PCV13 was predicted to avert a higher burden of pneumococcal disease compared with PCV10. This would have realised a small saving of £13 million discounted over 24 years. When considering the introduction of new pneumococcal vaccines into the routine immunisation schedule, we recommend that differences in antibody responses for different vaccines be considered in modelling scenarios as higher antibody responses result in reduced transmission and greater impact on invasive diseases. Vaccine-specific threshold prices can then be determined for cost-effective vaccines. Our analysis showed that due to its higher efficacy against some serotypes, a higher threshold price per dose could be paid for PCV13 while remaining cost-effective. Study registration This study is registered as PROSPERO CRD42019124580. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 17/148/03) and is published in full in Health Technology Assessment; Vol. 28, No. 34. See the NIHR Funding and Awards website for further award information.