Search

Background: Advances in biotechnology have raised expectations that biomarkers, including genetic profiles, will yield information to accurately predict outcomes for individuals. However, results to date have been disappointing. In addition, statistical methods to quantify the predictive information in markers have not been standardized.Methods: We discuss statistical techniques to summarize predictive information, including risk distribution curves and measures derived from them, that relate to decision making. Attributes of these measures are contrasted with alternatives such as receiver operating characteristic curves, R2, percent reclassification, and net reclassification index. Data are generated from simple models of risk conferred by genetic profiles for individuals in a population. Statistical techniques are illustrated, and the risk prediction capacities of different risk models are quantified.Results: Risk distribution curves are most informative and relevant to clinical practice. They show proportions of subjects classified into clinically relevant risk categories. In a population in which 10% have the outcome event and subjects are categorized as high risk if their risk exceeds 20%, we identified some settings where more than half of those destined to have an event were classified as high risk by the risk model. Either 150 genes each with odds ratio of 1.5 or 250 genes each with odds ratio of 1.25 were required when the minor allele frequencies are 10%. We show that conclusions based on receiver operating characteristic curves may not be the same as conclusions based on risk distribution curves.Conclusions: Many highly predictive genes will be required to identify substantial numbers of subjects at high risk. Cancer Epidemiol Biomarkers Prev; 19(3); 655–65

Supplementary Appendix from The Potential of Genes and Other Markers to Inform about Risk

The HIV Vaccine Trials Network (HVTN) 087 vaccine trial assessed the effect of increasing doses of pIL-12 (interleukin-12 delivered as plasmid DNA) adjuvant on the immunogenicity of an HIV-1 multiantigen (MAG) DNA vaccine delivered by electroporation and boosted with a vaccine comprising an attenuated vesicular stomatitis virus expressing HIV-1 Gag (VSV-Gag). We randomized 100 healthy adults to receive placebo or 3 mg HIV-MAG DNA vaccine (ProfectusVax HIV-1 gag / pol or ProfectusVax nef / tat / vif , env ) coadministered with pIL-12 at 0, 250, 1,000, or 1,500 μg intramuscularly by electroporation at 0, 1, and 3 months followed by intramuscular inoculation with 3.4 × 10 7 PFU VSV-Gag vaccine at 6 months. Immune responses were assessed after the prime and boost and 6 months after the last vaccination. High-dose pIL-12 increased the magnitude of CD8 + T-cell responses postboost compared to no pIL-12 ( P = 0.02), while CD4 + T-cell responses after the prime were higher in the absence of pIL-12 than with low- and medium-dose pIL-12 ( P ≤ 0.05). The VSV boost increased Gag-specific CD4 + and CD8 + T-cell responses in all groups ( P < 0.001 for CD4 + T cells), inducing a median of four Gag epitopes in responders. Six to 9 months after the boost, responses decreased in magnitude, but CD8 + T-cell response rates were maintained. The addition of a DNA prime dramatically improved responses to the VSV vaccine tested previously in the HVTN 090 trial, leading to broad epitope targeting and maintained CD8 + T-cell response rates at early memory. The addition of high-dose pIL-12 given with a DNA prime by electroporation and boosted with VSV-Gag increased the CD8 + T-cell responses but decreased the CD4 + responses. This approach may be advantageous in reshaping the T-cell responses to a variety of chronic infections or tumors. (This study has been registered at ClinicalTrials.gov under registration no. NCT01578889.)

Comprehensive assessment of cellular responses to the RTS,S/AS01E vaccine is needed to understand potential correlates and ultimately mechanisms of protection against malaria disease. Cellular responses recognizing the RTS,S/AS01E-containing circumsporozoite protein (CSP) and Hepatitis B surface antigen (HBsAg) were assessed before and 1 month after primary vaccination by intracellular cytokine staining and 16-color flow cytometry in 105 RTS,S/AS01-vaccinated and 74 rabies-vaccinated participants (controls) in a pediatric phase III trial in Africa. RTS,S/AS01E-vaccinated children had significantly higher frequencies of CSP- and HBsAg-specific CD4+ T cells producing IL-2, TNF-alpha, and CD40L and HBsAg-specific CD4+ T producing IFN-gamma and IL-17 than baseline and the control group. Vaccine-induced responses were identified in both central and effector memory (EM) compartments. EM CD4+ T cells expressing IL-4 and IL-21 were detected recognizing both vaccine antigens. Consistently higher response rates to both antigens in RTS,S/AS01E-vaccinated than comparator-vaccinated children were observed. RTS,S/AS01E induced polyfunctional CSP- and HBsAg-specific CD4+ T cells, with a greater degree of polyfunctionality in HBsAg responses. In conclusion, RTS,S/AS01E vaccine induces T cells of higher functional heterogeneity and polyfunctionality than previously characterized. Responses detected in memory CD4+ T cell compartments may provide correlates of RTS,S/AS01-induced immunity and duration of protection in future correlates of immunity studies.

Despite substantial clinical benefits, complete eradication of HIV has not been possible using antiretroviral therapy (ART) alone. Strategies that can either eliminate persistent viral reservoirs or boost host immunity to prevent rebound of virus from these reservoirs after discontinuation of ART are needed; one possibility is therapeutic vaccination. We report the results of a randomized, placebo-controlled trial of a therapeutic vaccine regimen in patients in whom ART was initiated during the early stage of HIV infection and whose immune system was anticipated to be relatively intact. The objectives of our study were to determine whether the vaccine was safe and could induce an immune response that would maintain suppression of plasma viremia after discontinuation of ART. Vaccinations were well tolerated with no serious adverse events but produced only modest augmentation of existing HIV-specific CD4+ T cell responses, with little augmentation of CD8+ T cell responses. Compared with placebo, the vaccination regimen had no significant effect on the kinetics or magnitude of viral rebound after interruption of ART and no impact on the size of the HIV reservoir in the CD4+ T cell compartment. Notably, 26% of subjects in the placebo arm exhibited sustained suppression of viremia (

Background: Advances in biotechnology have raised expectations that biomarkers, including genetic profiles, will yield information to accurately predict outcomes for individuals. However, results to date have been disappointing. In addition, statistical methods to quantify the predictive information in markers have not been standardized. Methods: We discuss statistical techniques to summarize predictive information, including risk distribution curves and measures derived from them, that relate to decision making. Attributes of these measures are contrasted with alternatives such as receiver operating characteristic curves, R2, percent reclassification, and net reclassification index. Data are generated from simple models of risk conferred by genetic profiles for individuals in a population. Statistical techniques are illustrated, and the risk prediction capacities of different risk models are quantified. Results: Risk distribution curves are most informative and relevant to clinical practice. They show proportions of subjects classified into clinically relevant risk categories. In a population in which 10% have the outcome event and subjects are categorized as high risk if their risk exceeds 20%, we identified some settings where more than half of those destined to have an event were classified as high risk by the risk model. Either 150 genes each with odds ratio of 1.5 or 250 genes each with odds ratio of 1.25 were required when the minor allele frequencies are 10%. We show that conclusions based on receiver operating characteristic curves may not be the same as conclusions based on risk distribution curves. Conclusions: Many highly predictive genes will be required to identify substantial numbers of subjects at high risk. Cancer Epidemiol Biomarkers Prev; 19(3); 655–65

Background. Statistical evaluation of medical imaging tests used for diagnostic and prognostic purposes often employs receiver operating characteristic (ROC) curves. Two methods for ROC analysis are popular. The ordinal regression method is the standard approach used when evaluating tests with ordinal values. The direct ROC modeling method is a more recently developed approach, motivated by applications to tests with continuous values. Objective. The authors compare the methods in terms of model formulations, interpretations of estimated parameters, the ranges of scientific questions that can be addressed with them, their computational algorithms, and the efficiencies with which they use data. Results. The authors show that a strong relationship exists between the methods by demonstrating that they fit the same models when only a single test is evaluated. The ordinal regression models are typically alternative parameterizations of the direct ROC models and vice versa. The direct method has two major advantages over the ordinal regression method: 1) estimated parameters relate directly to ROC curves, facilitating interpretations of covariate effects on ROC performance, and 2) comparisons between tests can be done directly in this framework. Comparisons can be made while accommodating covariate effects and even between tests that have values on different scales, such as between a continuous biomarker test and an ordinal valued imaging test. The ordinal regression method provides slightly more precise parameter estimates from data in our simulated data models. Conclusion. Although the ordinal regression method is slightly more efficient, the direct ROC modeling method has important advantages in regard to interpretation, and it offers a framework to address a broader range of scientific questions, including the facility to compare tests.

The RV144 HIV-1 vaccine trial demonstrated partial efficacy of 31% against HIV-1 infection. Studies into possible correlates of protection found that antibodies specific to the V1 and V2 (V1/V2) region of envelope correlated inversely with infection risk and that viruses isolated from trial participants contained genetic signatures of vaccine-induced pressure in the V1/V2 region. We explored the hypothesis that the genetic signatures in V1 and V2 could be partly attributed to selection by vaccine-primed T cells. We performed a T-cell-based sieve analysis of breakthrough viruses in the RV144 trial and found evidence of predicted HLA binding escape that was greater in vaccine versus placebo recipients. The predicted escape depended on class I HLA A*02- and A*11-restricted epitopes in the MN strain rgp120 vaccine immunogen. Though we hypothesized that this was indicative of postacquisition selection pressure, we also found that vaccine efficacy (VE) was greater in A*02-positive (A*02 + ) participants than in A*02 − participants (VE = 54% versus 3%, P = 0.05). Vaccine efficacy against viruses with a lysine residue at site 169, important to antibody binding and implicated in vaccine-induced immune pressure, was also greater in A*02 + participants (VE = 74% versus 15%, P = 0.02). Additionally, a reanalysis of vaccine-induced immune responses that focused on those that were shown to correlate with infection risk suggested that the humoral responses may have differed in A*02 + participants. These exploratory and hypothesis-generating analyses indicate there may be an association between a class I HLA allele and vaccine efficacy, highlighting the importance of considering HLA alleles and host immune genetics in HIV vaccine trials. IMPORTANCE The RV144 trial was the first to show efficacy against HIV-1 infection. Subsequently, much effort has been directed toward understanding the mechanisms of protection. Here, we conducted a T-cell-based sieve analysis, which compared the genetic sequences of viruses isolated from infected vaccine and placebo recipients. Though we hypothesized that the observed sieve effect indicated postacquisition T-cell selection, we also found that vaccine efficacy was greater for participants who expressed HLA A*02, an allele implicated in the sieve analysis. Though HLA alleles have been associated with disease progression and viral load in HIV-1 infection, these data are the first to suggest the association of a class I HLA allele and vaccine efficacy. While these statistical analyses do not provide mechanistic evidence of protection in RV144, they generate testable hypotheses for the HIV vaccine community and they highlight the importance of assessing the impact of host immune genetics in vaccine-induced immunity and protection. (This study has been registered at ClinicalTrials.gov under registration no. NCT00223080.)

The phase III RV144 HIV-1 vaccine trial estimated vaccine efficacy (VE) to be 31.2%. This trial demonstrated that the presence of HIV-1-specific IgG-binding Abs to envelope (Env) V1V2 inversely correlated with infection risk, while the presence of Env-specific plasma IgA Abs directly correlated with risk of HIV-1 infection. Moreover, Ab-dependent cellular cytotoxicity responses inversely correlated with risk of infection in vaccine recipients with low IgA; therefore, we hypothesized that vaccine-induced Fc receptor-mediated (FcR-mediated) Ab function is indicative of vaccine protection. We sequenced exons and surrounding areas of FcR-encoding genes and found one FCGR2C tag SNP (rs114945036) that associated with VE against HIV-1 subtype CRF01_AE, with lysine at position 169 (169K) in the V2 loop (CRF01_AE 169K). Individuals carrying CC in this SNP had an estimated VE of 15%, while individuals carrying CT or TT exhibited a VE of 91%. Furthermore, the rs114945036 SNP was highly associated with 3 other FCGR2C SNPs (rs138747765, rs78603008, and rs373013207). Env-specific IgG and IgG3 Abs, IgG avidity, and neutralizing Abs inversely correlated with CRF01_AE 169K HIV-1 infection risk in the CT- or TT- carrying vaccine recipients only. These data suggest a potent role of Fc-γ receptors and Fc-mediated Ab function in conferring protection from transmission risk in the RV144 VE trial.

The RV144 clinical trial of a prime/boost immunizing regimen using recombinant canary pox (ALVAC-HIV) and two gp120 proteins (AIDSVAX B and E) was previously shown to have a 31.2% efficacy rate. Plasma specimens from vaccine and placebo recipients were used in an extensive set of assays to identify correlates of HIV-1 infection risk. Of six primary variables that were studied, only one displayed a significant inverse correlation with risk of infection: the antibody (Ab) response to a fusion protein containing the V1 and V2 regions of gp120 (gp70-V1V2). This finding prompted a thorough examination of the results generated with the complete panel of 13 assays measuring various V2 Abs in the stored plasma used in the initial pilot studies and those used in the subsequent case-control study. The studies revealed that the ALVAC-HIV/AIDSVAX vaccine induced V2-specific Abs that cross-react with multiple HIV-1 subgroups and recognize both conformational and linear epitopes. The conformational epitope was present on gp70-V1V2, while the predominant linear V2 epitope mapped to residues 165–178, immediately N-terminal to the putative α4β7 binding motif in the mid-loop region of V2. Odds ratios (ORs) were calculated to compare the risk of infection with data from 12 V2 assays, and in 11 of these, the ORs were ≤1, reaching statistical significance for two of the variables: Ab responses to gp70-V1V2 and to overlapping V2 linear peptides. It remains to be determined whether anti-V2 Ab responses were directly responsible for the reduced infection rate in RV144 and whether anti-V2 Abs will prove to be important with other candidate HIV vaccines that show efficacy, however, the results support continued dissection of Ab responses to the V2 region which may illuminate mechanisms of protection from HIV-1 infection and may facilitate the development of an effective HIV-1 vaccine.

T-cell based sieve analysis ties HLA A*02 to vaccine efficacy and IgA-C1 immune correlate in RV144 Thai trial T Hertz, A Gartland, H Janes, S Li, Y Fong, GD Tomaras, D Morris, D Geraghty, GH Kijak, PT Edlefsen, M Rolland, BB Larsen, S Tovanabutra, E Sanders-Buell, AC DeCamp, CA Magaret, H Ahmed, S Nariya, K Wong, H Zhao, W Deng, BS Maust, M Bose, S Howell, M Lazzaro, A Bates, E Lei, A Bradfield, G Ibitamuno, V Assawadarachai, RJ O’Connel, MS deSouza, S Nitayaphan, S Rerks-Ngarm, ML Robb, MJ McElrath, BF Haynes, NL Michael, PB Gilbert, JI Mullins, JH Kim

Background There are currently many different methods for processing and summarizing probe-level data from Affymetrix oligonucleotide arrays. It is of great interest to validate these methods and identify those that are most effective. There is no single best way to do this validation, and a variety of approaches is needed. Moreover, gene expression data are collected to answer a variety of scientific questions, and the same method may not be best for all questions. Only a handful of validation studies have been done so far, most of which rely on spike-in datasets and focus on the question of detecting differential expression. Here we seek methods that excel at estimating relative expression. We evaluate methods by identifying those that give the strongest linear association between expression measurements by array and the "gold-standard" assay. Quantitative reverse-transcription polymerase chain reaction (qRT-PCR) is generally considered the "gold-standard" assay for measuring gene expression by biologists and is often used to confirm findings from microarray data. Here we use qRT-PCR measurements to validate methods for the components of processing oligo array data: background adjustment, normalization, mismatch adjustment, and probeset summary. An advantage of our approach over spike-in studies is that methods are validated on a real dataset that was collected to address a scientific question. Results We initially identify three of six popular methods that consistently produced the best agreement between oligo array and RT-PCR data for medium- and high-intensity genes. The three methods are generally known as MAS5, gcRMA, and the dChip mismatch mode. For medium- and high-intensity genes, we identified use of data from mismatch probes (as in MAS5 and dChip mismatch) and a sequence-based method of background adjustment (as in gcRMA) as the most important factors in methods' performances. However, we found poor reliability for methods using mismatch probes for low-intensity genes, which is in agreement with previous studies. Conclusion We advocate use of sequence-based background adjustment in lieu of mismatch adjustment to achieve the best results across the intensity spectrum. No method of normalization or probeset summary showed any consistent advantages.

UnlabelledIn the RV144 HIV-1 vaccine efficacy trial, IgG antibody (Ab) binding levels to variable regions 1 and 2 (V1V2) of the HIV-1 envelope glycoprotein gp120 were an inverse correlate of risk of HIV-1 infection. To determine if V1V2-specific Abs cross-react with V1V2 from different HIV-1 subtypes, if the nature of the V1V2 antigen used to asses cross-reactivity influenced infection risk, and to identify immune assays for upcoming HIV-1 vaccine efficacy trials, new V1V2-scaffold antigens were designed and tested. Protein scaffold antigens carrying the V1V2 regions from HIV-1 subtypes A, B, C, D or CRF01_AE were assayed in pilot studies, and six were selected to assess cross-reactive Abs in the plasma from the original RV144 case-control cohort (41 infected vaccinees, 205 frequency-matched uninfected vaccinees, and 40 placebo recipients) using ELISA and a binding Ab multiplex assay. IgG levels to these antigens were assessed as correlates of risk in vaccine recipients using weighted logistic regression models. Levels of Abs reactive with subtype A, B, C and CRF01_AE V1V2-scaffold antigens were all significant inverse correlates of risk (p-values of 0.0008-0.05; estimated odds ratios of 0.53-0.68 per 1 standard deviation increase). Thus, levels of vaccine-induced IgG Abs recognizing V1V2 regions from multiple HIV-1 subtypes, and presented on different scaffolds, constitute inverse correlates of risk for HIV-1 infection in the RV144 vaccine trial. The V1V2 antigens provide a link between RV144 and upcoming HIV-1 vaccine trials, and identify reagents and methods for evaluating V1V2 Abs as possible correlates of protection against HIV-1 infection.Trial registrationClinicalTrials.gov NCT00223080.

ObjectiveDetermine the role of V2 antibodies (Abs) in protecting vaccinees from HIV infection after immunization with ALVAC/HIV and clade E and B recombinant gp120 (RV144 vaccine regimen). ELISA reactivity to a V1V2-gp70 fusion protein and a cyclic V2 peptide was assessed using coded plasma specimen