Search

Background We demonstrate the feasibility of creating a pair of reference samples to be used as surrogates for clinical samples measured in either a research or clinical laboratory setting. The reference sample paradigm presented and evaluated here is designed to assess the capability of a measurement process to detect true differences between two biological samples. Cell-based reference samples can be created with a biomarker signature pattern designed in silico. Clinical laboratories working in regulated applications are required to participate in proficiency testing programs; research laboratories doing discovery typically do not. These reference samples can be used in proficiency tests or as process controls that allow a laboratory to evaluate and optimize its measurement systems, monitor performance over time (process drift), assess changes in protocols, reagents, and/or personnel, maintain standard operating procedures, and most importantly, provide evidence for quality results. Results The biomarkers of interest in this study are microRNAs (miRNAs), small non-coding RNAs involved in the regulation of gene expression. Multiple lung cancer associated cell lines were determined by reverse transcription (RT)-PCR to have sufficiently different miRNA profiles to serve as components in mixture designs as reference samples. In silico models based on the component profiles were used to predict miRNA abundance ratios between two different cell line mixtures, providing target values for profiles obtained from in vitro mixtures. Two reference sample types were tested: total RNA mixed after extraction from cell lines, and intact cells mixed prior to RNA extraction. MicroRNA profiling of a pair of samples composed of extracted RNA derived from these cell lines successfully replicated the target values. Mixtures of intact cells from these lines also approximated the target values, demonstrating potential utility as mimics for clinical specimens. Both designs demonstrated their utility as reference samples for inter- or intra-laboratory testing. Conclusions Cell-based reference samples can be created for performance assessment of a measurement process from biomolecule extraction through quantitation. Although this study focused on miRNA profiling with RT-PCR using cell lines associated with lung cancer, the paradigm demonstrated here should be extendable to genome-scale platforms and other biomolecular endpoints. Electronic supplementary material The online version of this article (10.1186/s12896-018-0423-4) contains supplementary material, which is available to authorized users.

Background Highly multiplexed assays for quantitation of RNA transcripts are being used in many areas of biology and medicine. Using data generated by these transcriptomic assays requires measurement assurance with appropriate controls. Methods to prototype and evaluate multiple RNA controls were developed as part of the External RNA Controls Consortium (ERCC) assessment process. These approaches included a modified Latin square design to provide a broad dynamic range of relative abundance with known differences between four complex pools of ERCC RNA transcripts spiked into a human liver total RNA background. Results ERCC pools were analyzed on four different microarray platforms: Agilent 1- and 2-color, Illumina bead, and NIAID lab-made spotted microarrays; and two different second-generation sequencing platforms: the Life Technologies 5500xl and the Illumina HiSeq 2500. Individual ERCC controls were assessed for reproducible performance in signal response to concentration among the platforms. Most demonstrated linear behavior if they were not located near one of the extremes of the dynamic range. Performance issues with any individual ERCC transcript could be attributed to detection limitations, platform-specific target probe issues, or potential mixing errors. Collectively, these pools of spike-in RNA controls were evaluated for suitability as surrogates for endogenous transcripts to interrogate the performance of the RNA measurement process of each platform. The controls were useful for establishing the dynamic range of the assay, as well as delineating the useable region of that range where differential expression measurements, expressed as ratios, would be expected to be accurate. Conclusions The modified Latin square design presented here uses a composite testing scheme for the evaluation of multiple performance characteristics: linear performance of individual controls, signal response within dynamic range pools of controls, and ratio detection between pairs of dynamic range pools. This compact design provides an economical sample format for the evaluation of multiple external RNA controls within a single experiment per platform. These results indicate that well-designed pools of RNA controls, spiked into samples, provide measurement assurance for endogenous gene expression studies. Electronic supplementary material The online version of this article (doi:10.1186/s12896-016-0281-x) contains supplementary material, which is available to authorized users.

Universal approaches for assessing the diagnostic performance of microarray assays are essential for the application of microarray technology to clinical and regulatory settings. Reference systems for diagnostic assays in laboratory medicine typically involve the utilization of reference samples, metrics, and reference datasets to ensure that measurements are comparable and true. For microarray performance evaluation and process improvement, reference samples can be composed of mixes of different tissue or cell line RNAs that contain tissue-selective analytes at defined target ratios. The diagnostic accuracy of detected changes in expression, measured as the area under the curve from receiver-operating characteristic plots, can provide a single commutable value for comparing assay specificity and sensitivity. Examples of applying this method for assessing overall performance are provided using public datasets generated on five commercial human whole genome microarray platforms for the MicroArray Quality Control project, a community-wide effort to address issues surrounding microarray data reliability.

We have assessed the utility of RNA titration samples for evaluating microarray platform performance and the impact of different normalization methods on the results obtained. As part of the MicroArray Quality Control project, we investigated the performance of five commercial microarray platforms using two independent RNA samples and two titration mixtures of these samples. Focusing on 12,091 genes common across all platforms, we determined the ability of each platform to detect the correct titration response across the samples. Global deviations from the response predicted by the titration ratios were observed. These differences could be explained by variations in relative amounts of messenger RNA as a fraction of total RNA between the two independent samples. Overall, both the qualitative and quantitative correspondence across platforms was high. In summary, titration samples may be regarded as a valuable tool, not only for assessing microarray platform performance and different analysis methods, but also for determining some underlying biological features of the samples.

Over the last decade, the introduction of microarray technology has had a profound impact on gene expression research. The publication of studies with dissimilar or altogether contradictory results, obtained using different microarray platforms to analyze identical RNA samples, has raised concerns about the reliability of this technology. The MicroArray Quality Control (MAQC) project was initiated to address these concerns, as well as other performance and data analysis issues. Expression data on four titration pools from two distinct reference RNA samples were generated at multiple test sites using a variety of microarray-based and alternative technology platforms. Here we describe the experimental design and probe mapping efforts behind the MAQC project. We show intraplatform consistency across test sites as well as a high level of interplatform concordance in terms of genes identified as differentially expressed. This study provides a resource that represents an important first step toward establishing a framework for the use of microarrays in clinical and regulatory settings.

A significant limitation to the analytical accuracy and precision of dual-labeled spotted cDNA microarrays is the signal error due to dye bias. Transcript-dependent dye bias may be due to gene-specific differences of incorporation of two distinctly different chemical dyes and the resultant differential hybridization efficiencies of these two chemically different targets for the same probe. Several approaches were used to assess and minimize the effects of dye bias on fluorescent hybridization signals and maximize the experimental design efficiency of a cell culture experiment. Dye bias was measured at the individual transcript level within each batch of simultaneously processed arrays by replicate dual-labeled split-control sample hybridizations and accounted for a significant component of fluorescent signal differences. This transcript-dependent dye bias alone could introduce unacceptably high numbers of both false-positive and false-negative signals. We found that within a given set of concurrently processed hybridizations, the bias is remarkably consistent and therefore measurable and correctable. The additional microarrays and reagents required for paired technical replicate dye-swap corrections commonly performed to control for dye bias could be costly to end users. Incorporating split-control microarrays within a set of concurrently processed hybridizations to specifically measure dye bias can eliminate the need for technical dye swap replicates and reduce microarray and reagent costs while maintaining experimental accuracy and technical precision. These data support a practical and more efficient experimental design to measure and mathematically correct for dye bias.

There is a critical need for standard approaches to assess, report, and compare the technical performance of genome-scale differential gene expression experiments. We assess technical performance with a proposed "standard" dashboard of metrics derived from analysis of external spike-in RNA control ratio mixtures. These control ratio mixtures with defined abundance ratios enable assessment of diagnostic performance of differentially expressed transcript lists, limit of detection of ratio (LODR) estimates, and expression ratio variability and measurement bias. The performance metrics suite is applicable to analysis of a typical experiment, and here we also apply these metrics to evaluate technical performance among laboratories. An interlaboratory study using identical samples shared amongst 12 laboratories with three different measurement processes demonstrated generally consistent diagnostic power across 11 laboratories. Ratio measurement variability and bias were also comparable amongst laboratories for the same measurement process. Different biases were observed for measurement processes using different mRNA enrichment protocols., Comment: 65 pages, 6 Main Figures, 33 Supplementary Figures

Background DNA microarrays, which have been increasingly used to monitor mRNA transcripts at a global level, can provide detailed insight into cellular processes involved in response to drugs and toxins. This is leading to new understandings of signaling networks that operate in the cell, and the molecular basis of diseases. Custom printed oligonucleotide arrays have proven to be an effective way to facilitate the applications of DNA microarray technology. A successful microarray experiment, however, involves many steps: well-designed oligonucleotide probes, printing, RNA extraction and labeling, hybridization, and imaging. Optimization is essential to generate reliable microarray data. Results Hybridization and washing steps are crucial for a successful microarray experiment. By following the hybridization and washing conditions recommended by an oligonucleotide provider, it was found that the expression ratios were compressed greater than expected and data analysis revealed a high degree of non-specific binding. A series of experiments was conducted using rat mixed tissue RNA reference material (MTRRM) and other RNA samples to optimize the hybridization and washing conditions. The optimized hybridization and washing conditions greatly reduced the non-specific binding and improved the accuracy of spot intensity measurements. Conclusion The results from the optimized hybridization and washing conditions greatly improved the reproducibility and accuracy of expression ratios. These experiments also suggested the importance of probe designs using better bioinformatics approaches and the need for common reference RNA samples for platform performance evaluation in order to fulfill the potential of DNA microarray technology.

Within the International Life Sciences Institute Committee on Genomics, a working group was formed to focus on the application of microarray technology to preclinical assessments of drug-induced nephrotoxicity. As part of this effort, Sprague-Dawley rats were treated with the nephrotoxicant cisplatin at doses of 0.3-5 mg/kg over a 4- to 144-hr time course. RNA prepared from these animals was run on a variety of microarray formats at multiple sites. A set of 93 differentially expressed genes associated with cisplatin-induced renal injury was identified on the National Institute of Environmental Health Sciences (NIEHS) custom cDNA microarray platform using quadruplicate measurements of pooled animal RNA. The reproducibility of this profile of statistically significant gene changes on other platforms, in pooled and individual animal replicate samples, and in an independent study was investigated. A good correlation in response between platforms was found among the 48 genes in the NIEHS data set that could be matched to probes on the Affymetrix RGU34A array by UniGene identifier or sequence alignment. Similar results were obtained with genes that could be linked between the NIEHS and Incyte or PHASE-1 arrays. The degree of renal damage induced by cisplatin in individual animals was commensurate with the number of differentially expressed genes in this data set. These results suggest that gene profiles linked to specific types of tissue injury or mechanisms of toxicity and identified in well-performed replicated microarray experiments may be extrapolatable across platform technologies, laboratories, and in-life studies.

Background The interpretability of microarray data can be affected by sample quality. To systematically explore how RNA quality affects microarray assay performance, a set of rat liver RNA samples with a progressive change in RNA integrity was generated by thawing frozen tissue or by ex vivo incubation of fresh tissue over a time course. Results Incubation of tissue at 37°C for several hours had little effect on RNA integrity, but did induce changes in the transcript levels of stress response genes and immune cell markers. In contrast, thawing of tissue led to a rapid loss of RNA integrity. Probe sets identified as most sensitive to RNA degradation tended to be located more than 1000 nucleotides upstream of their transcription termini, similar to the positioning of control probe sets used to assess sample quality on Affymetrix GeneChip® arrays. Samples with RNA integrity numbers less than or equal to 7 showed a significant increase in false positives relative to undegraded liver RNA and a reduction in the detection of true positives among probe sets most sensitive to sample integrity for in silico modeled changes of 1.5-, 2-, and 4-fold. Conclusion Although moderate levels of RNA degradation are tolerated by microarrays with 3'-biased probe selection designs, in this study we identify a threshold beyond which decreased specificity and sensitivity can be observed that closely correlates with average target length. These results highlight the value of annotating microarray data with metrics that capture important aspects of sample quality.

The comparability and reliability of data generated using microarray technology would be enhanced by use of a common set of standards that allow accuracy, reproducibility and dynamic range assessments on multiple formats. We designed and tested a complex biological reagent for performance measurements on three commercial oligonucleotide array formats that differ in probe design and signal measurement methodology. The reagent is a set of two mixtures with different proportions of RNA for each of four rat tissues (brain, liver, kidney and testes). The design provides four known ratio measurements of >200 reference probes, which were chosen for their tissue-selectivity, dynamic range coverage and alignment to the same exemplar transcript sequence across all three platforms. The data generated from testing three biological replicates of the reagent at eight laboratories on three array formats provides a benchmark set for both laboratory and data processing performance assessments. Close agreement with target ratios adjusted for sample complexity was achieved on all platforms and low variance was observed among platforms, replicates and sites. The mixed tissue design produces a reagent with known gene expression changes within a complex sample and can serve as a paradigm for performance standards for microarrays that target other species.