Search

BACKGROUND: With the use of artificial intelligence and machine learning techniques for biomedical informatics, security and privacy concerns over the data and subject identities have also become an important issue and essential research topic. Without intentional safeguards, machine learning models may find patterns and features to improve task performance that are associated with private personal information. OBJECTIVE: The privacy vulnerability of deep learning models for information extraction from medical textural contents needs to be quantified since the models are exposed to private health information and personally identifiable information. The objective of the study is to quantify the privacy vulnerability of the deep learning models for natural language processing and explore a proper way of securing patients’ information to mitigate confidentiality breaches. METHODS: The target model is the multitask convolutional neural network for information extraction from cancer pathology reports, where the data for training the model are from multiple state population-based cancer registries. This study proposes the following schemes to collect vocabularies from the cancer pathology reports; (a) words appearing in multiple registries, and (b) words that have higher mutual information. We performed membership inference attacks on the models in high-performance computing environments. RESULTS: The comparison outcomes suggest that the proposed vocabulary selection methods resulted in lower privacy vulnerability while maintaining the same level of clinical task performance.

Bidirectional Encoder Representations from Transformers (BERT) and BERT-based approaches are the current state-of-the-art in many natural language processing (NLP) tasks; however, their application to document classification on long clinical texts is limited. In this work, we introduce four methods to scale BERT, which by default can only handle input sequences up to approximately 400 words long, to perform document classification on clinical texts several thousand words long. We compare these methods against two much simpler architectures - a word-level convolutional neural network and a hierarchical self-attention network - and show that BERT often cannot beat these simpler baselines when classifying MIMIC-III discharge summaries and SEER cancer pathology reports. In our analysis, we show that two key components of BERT - pretraining and WordPiece tokenization - may actually be inhibiting BERT's performance on clinical text classification tasks where the input document is several thousand words long and where correctly identifying labels may depend more on identifying a few key words or phrases rather than understanding the contextual meaning of sequences of text.

Population cancer registries can benefit from Deep Learning (DL) to automatically extract cancer characteristics from the high volume of unstructured pathology text reports they process annually. The success of DL to tackle this and other real-world problems is proportional to the availability of large labeled datasets for model training. Although collaboration among cancer registries is essential to fully exploit the promise of DL, privacy and confidentiality concerns are main obstacles for data sharing across cancer registries. Moreover, DL for natural language processing (NLP) requires sharing a vocabulary dictionary for the embedding layer which may contain patient identifiers. Thus, even distributing the trained models across cancer registries causes a privacy violation issue. In this article, we propose DL NLP model distribution via privacy-preserving transfer learning approaches without sharing sensitive data. These approaches are used to distribute a multitask convolutional neural network (MT-CNN) NLP model among cancer registries. The model is trained to extract six key cancer characteristics – tumor site, subsite, laterality, behavior, histology, and grade – from cancer pathology reports. Using 410,064 pathology documents from two cancer registries, we compare our proposed approach to conventional transfer learning without privacy-preserving, single-registry models, and a model trained on centrally hosted data. The results show that transfer learning approaches including data sharing and model distribution outperform significantly the single-registry model. In addition, the best performing privacy-preserving model distribution approach achieves statistically indistinguishable average micro- and macro-F1 scores across all extraction tasks (0.823,0.580) as compared to the centralized model (0.827,0.585).

BackgroundAutomated text classification has many important applications in the clinical setting; however, obtaining labelled data for training machine learning and deep learning models is often difficult and expensive. Active learning techniques may mitigate this challenge by reducing the amount of labelled data required to effectively train a model. In this study, we analyze the effectiveness of 11 active learning algorithms on classifying subsite and histology from cancer pathology reports using a Convolutional Neural Network as the text classification model.ResultsWe compare the performance of each active learning strategy using two differently sized datasets and two different classification tasks. Our results show that on all tasks and dataset sizes, all active learning strategies except diversity-sampling strategies outperformed random sampling, i.e., no active learning. On our large dataset (15K initial labelled samples, adding 15K additional labelled samples each iteration of active learning), there was no clear winner between the different active learning strategies. On our small dataset (1K initial labelled samples, adding 1K additional labelled samples each iteration of active learning), marginal and ratio uncertainty sampling performed better than all other active learning techniques. We found that compared to random sampling, active learning strongly helps performance on rare classes by focusing on underrepresented classes.ConclusionsActive learning can save annotation cost by helping human annotators efficiently and intelligently select which samples to label. Our results show that a dataset constructed using effective active learning techniques requires less than half the amount of labelled data to achieve the same performance as a dataset constructed using random sampling.

We aim to reduce overfitting and model overconfidence by distilling the knowledge of an ensemble of deep learning models into a single model for the classification of cancer pathology reports.We consider the text classification problem that involves 5 individual tasks. The baseline model consists of a multitask convolutional neural network (MtCNN), and the implemented ensemble (teacher) consists of 1000 MtCNNs. We performed knowledge transfer by training a single model (student) with soft labels derived through the aggregation of ensemble predictions. We evaluate performance based on accuracy and abstention rates by using softmax thresholding.The student model outperforms the baseline MtCNN in terms of abstention rates and accuracy, thereby allowing the model to be used with a larger volume of documents when deployed. The highest boost was observed for subsite and histology, for which the student model classified an additional 1.81% reports for subsite and 3.33% reports for histology.Ensemble predictions provide a useful strategy for quantifying the uncertainty inherent in labeled data and thereby enable the construction of soft labels with estimated probabilities for multiple classes for a given document. Training models with the derived soft labels reduce model confidence in difficult-to-classify documents, thereby leading to a reduction in the number of highly confident wrong predictions.Ensemble model distillation is a simple tool to reduce model overconfidence in problems with extreme class imbalance and noisy datasets. These methods can facilitate the deployment of deep learning models in high-risk domains with low computational resources where minimizing inference time is required.

Objectives The International Classification of Childhood Cancer (ICCC) facilitates the effective classification of a heterogeneous group of cancers in the important pediatric population. However, there has been no development of machine learning models for the ICCC classification. We developed deep learning-based information extraction models from cancer pathology reports based on the ICD-O-3 coding standard. In this article, we describe extending the models to perform ICCC classification. Materials and Methods We developed 2 models, ICD-O-3 classification and ICCC recoding (Model 1) and direct ICCC classification (Model 2), and 4 scenarios subject to the training sample size. We evaluated these models with a corpus consisting of 29 206 reports with age at diagnosis between 0 and 19 from 6 state cancer registries. Results Our findings suggest that the direct ICCC classification (Model 2) is substantially better than reusing the ICD-O-3 classification model (Model 1). Applying the uncertainty quantification mechanism to assess the confidence of the algorithm in assigning a code demonstrated that the model achieved a micro-F1 score of 0.987 while abstaining (not sufficiently confident to assign a code) on only 14.8% of ambiguous pathology reports. Conclusions Our experimental results suggest that the machine learning-based automatic information extraction from childhood cancer pathology reports in the ICCC is a reliable means of supplementing human annotators at state cancer registries by reading and abstracting the majority of the childhood cancer pathology reports accurately and reliably.

The Social Security Administration Service to Epidemiological Researchers (SSA-SER) can help central cancer registries meet the contractual follow-up requirements of the Surveillance, Epidemiology, and End Results (SEER) Program and improve survival estimate accuracy. We evaluated the impact of first-time SSA-SER linkage on follow-up rates and survival estimates for 2 SEER registries. Methods: In May 2019, cancer registries in Idaho (Cancer Data Registry of Idaho [CDRI]) and New York (New York State Cancer Registry [NYSCR]) used results from an SSA-SER linkage to update date of last contact and vital status for patients with a SEER-reportable tumor diagnosed during 2000-2016. We compared follow-up completeness through 2017 between pre-SSA-SER linkage and post-SSA-SER linkage data. Among individuals with a first primary tumor diagnosed during 2009-2015, we calculated 60-month age-standardized all sites and site-specific relative survival ratio (RSR) estimates via the presumed alive method using pre-SSA linkage data, and survival time calculated from last known date of contact using post-SSA linkage data. Results: SSA-SER linkage improved overall followup completeness from 79.0% to 97.4% and 55.7% to 92.6% for CDRI and NYSCR, respectively. Follow-up completeness improved most for laboratory-only reported tumors, in situ tumors, melanomas of the skin, prostate cancers, and benign and borderline brain and other central nervous system tumors. Post-SSA linkage RSRs were lower than pre-SSA presumed alive RSRs by an average -0.47% and -2.16% for Idaho and New York, respectively. Conclusions: SSA-SER linkage greatly and efficiently improved follow-up completeness for the 2 participating registries and revealed small difference in survival estimates by method. Use of the SSA-SER by all US registries would standardize and improve US survival estimates.

Background Cancer recurrence is an important measure of the impact of cancer treatment. However, no population-based data on recurrence are available. Pathology reports could potentially identify cancer recurrences. Their utility to capture recurrences is unknown. Objective This analysis assesses the sensitivity of pathology reports to identify patients with cancer recurrence and the stage at recurrence. Subjects The study includes patients with recurrent breast (n=214) or colorectal (n=203) cancers. Research design This retrospective analysis included patients from a population-based cancer registry who were part of the Patient-Centered Outcomes Research (PCOR) Study, a project that followed cancer patients in-depth for 5 years after diagnosis to identify recurrences. Measures Information abstracted from pathology reports for patients with recurrence was compared with their PCOR data (gold standard) to determine what percent had a pathology report at the time of recurrence, the sensitivity of text in the report to identify recurrence, and if the stage at recurrence could be determined from the pathology report. Results One half of cancer patients had a pathology report near the time of recurrence. For patients with a pathology report, the report's sensitivity to identify recurrence was 98.1% for breast cancer cases and 95.7% for colorectal cancer cases. The specific stage at recurrence from the pathology report had a moderate agreement with gold-standard data. Conclusions Pathology reports alone cannot measure population-based recurrence of solid cancers but can identify specific cohorts of recurrent cancer patients. As electronic submission of pathology reports increases, these reports may identify specific recurrent patients in near real-time.

In the last decade, the widespread adoption of electronic health record documentation has created huge opportunities for information mining. Natural language processing (NLP) techniques using machine and deep learning are becoming increasingly widespread for information extraction tasks from unstructured clinical notes. Disparities in performance when deploying machine learning models in the real world have recently received considerable attention. In the clinical NLP domain, the robustness of convolutional neural networks (CNNs) for classifying cancer pathology reports under natural distribution shifts remains understudied. In this research, we aim to quantify and improve the performance of the CNN for text classification on out-of-distribution (OOD) datasets resulting from the natural evolution of clinical text in pathology reports. We identified class imbalance due to different prevalence of cancer types as one of the sources of performance drop and analyzed the impact of previous methods for addressing class imbalance when deploying models in real-world domains. Our results show that our novel class-specialized ensemble technique outperforms other methods for the classification of rare cancer types in terms of macro F1 scores. We also found that traditional ensemble methods perform better in top classes, leading to higher micro F1 scores. Based on our findings, we formulate a series of recommendations for other ML practitioners on how to build robust models with extremely imbalanced datasets in biomedical NLP applications.

The coronavirus disease 2019 (COVID-19) pandemic led to delayed medical care in the United States. We examined changes in patterns of cancer diagnosis and surgical treatment between January 1 and December 31 in 2020 and 2019 with real-time electronic pathology report data from population-based Surveillance, Epidemiology, and End Results cancer registries from Georgia and Louisiana. During 2020, there were 29 905 fewer pathology reports than in 2019, representing a 10.2% decline. Declines were observed in all age groups, including children and adolescents younger than 18 years. The nadir was early April 2020, with 42.8% fewer reports than in April 2019. Numbers of reports through December 2020 never consistently exceeded those in 2019 after first declines. Patterns were similar by age group and cancer site. Findings suggest substantial delays in diagnosis and treatment services for cancers during the pandemic. Ongoing evaluation can inform public health efforts to minimize any lasting adverse effects of the pandemic on cancer diagnosis, stage, treatment, and survival.

BACKGROUND: Chemotherapy information in the population-based cancer registries is underascertained and lacks detail. We conducted a pilot study in the Georgia SEER Cancer Registry (GCR) to investigate the feasibility of supplementing chemotherapy information using billing claims from six private oncology practices (OP). METHODS: To assess cancer patients’ representativeness from OP, we compared individuals with invasive first primary cancers diagnosed during 2013–2015 in the GCR (cohort 1) with those who had at least one OP claim in the 12 months after diagnosis (cohort 2). To assess completeness of OP claims to capture chemotherapy (yes or no), we further restricted cohort 2 to patients ages 65 years and older enrolled in fee-for-service Medicare Part A and B from the diagnosis date through 12 months follow-up or to the date of death. With Medicare data serving as the gold standard, sensitivity, specificity, and kappa statistics for the receipt of chemotherapy per OP claims were calculated by demographic and clinical characteristics. RESULTS: Cancer patients seeking care in the OP included in our analysis were not representative of the underlying patient population in the GCR. The practices underrepresented minorities and uninsured while overrepresenting females, persons with high socioeconomic status, patients residing outside the metropolitan Atlanta area, and persons with advance staged disease. The ability of practice claims to identify chemotherapy receipt was moderate (76.1% sensitivity) but varied by demographic and clinical characteristics (76.1–83.0%). CONCLUSIONS: Given the limited ability of OP claims to identify chemotherapy receipt, we suggest analyzing these data for hypothesis generation, but inference should be limited to this patient cohort.

BACKGROUND Chemotherapy information in the population-based cancer registries is underascertained and lacks detail. We conducted a pilot study in the Georgia SEER Cancer Registry (GCR) to investigate the feasibility of supplementing chemotherapy information using billing claims from six private oncology practices (OP). METHODS To assess cancer patients' representativeness from OP, we compared individuals with invasive first primary cancers diagnosed during 2013-2015 in the GCR (cohort 1) with those who had at least one OP claim in the 12 months after diagnosis (cohort 2). To assess completeness of OP claims to capture chemotherapy (yes or no), we further restricted cohort 2 to patients ages 65 years and older enrolled in fee-for-service Medicare Part A and B from the diagnosis date through 12 months follow-up or to the date of death. With Medicare data serving as the gold standard, sensitivity, specificity, and kappa statistics for the receipt of chemotherapy per OP claims were calculated by demographic and clinical characteristics. RESULTS Cancer patients seeking care in the OP included in our analysis were not representative of the underlying patient population in the GCR. The practices underrepresented minorities and uninsured while overrepresenting females, persons with high socioeconomic status, patients residing outside the metropolitan Atlanta area, and persons with advance staged disease. The ability of practice claims to identify chemotherapy receipt was moderate (76.1% sensitivity) but varied by demographic and clinical characteristics (76.1-83.0%). CONCLUSIONS Given the limited ability of OP claims to identify chemotherapy receipt, we suggest analyzing these data for hypothesis generation, but inference should be limited to this patient cohort.

Collaboration among cancer registries is essential to develop accurate, robust, and generalizable deep learning models for automated information extraction from cancer pathology reports. Sharing data presents a serious privacy issue, especially in biomedical research and healthcare delivery domains. Distributing pretrained deep learning (DL) models has been proposed to avoid critical data sharing. However, there is growing recognition that collaboration among clinical institutes through DL model distribution exposes new security and privacy vulnerabilities. These vulnerabilities increase in natural language processing (NLP) applications, in which the dataset vocabulary with word vector representations needs to be associated with the other model parameters. In this paper, we propose a novel privacy-preserving DL model distribution across cancer registries for information extraction from cancer pathology reports with privacy and confidentiality considerations. The proposed approach exploits the adversarial training framework to distinguish private features from shared features among different datasets. It only shares registry-invariant model parameters, without sharing raw data nor registry-specific model parameters among cancer registries. Thus, it protects both the data and the trained model simultaneously. We compare our proposed approach to single-registry models, and a model trained on centrally hosted data from different cancer registries. The results show that the proposed approach significantly outperforms the single-registry models and achieves statistically indistinguishable micro and macro F1-score as compared to the centralized model.

We introduce a deep learning architecture, hierarchical self-attention networks (HiSANs), designed for classifying pathology reports and show how its unique architecture leads to a new state-of-the-art in accuracy, faster training, and clear interpretability. We evaluate performance on a corpus of 374,899 pathology reports obtained from the National Cancer Institute's (NCI) Surveillance, Epidemiology, and End Results (SEER) program. Each pathology report is associated with five clinical classification tasks – site, laterality, behavior, histology, and grade. We compare the performance of the HiSAN against other machine learning and deep learning approaches commonly used on medical text data – Naive Bayes, logistic regression, convolutional neural networks, and hierarchical attention networks (the previous state-of-the-art). We show that HiSANs are superior to other machine learning and deep learning text classifiers in both accuracy and macro F-score across all five classification tasks. Compared to the previous state-of-the-art, hierarchical attention networks, HiSANs not only are an order of magnitude faster to train, but also achieve about 1% better relative accuracy and 5% better relative macro F-score.

1514 Background: The COVID-19 pandemic led to delays in medical care in the United States. We examined changes in patterns of cancer diagnosis and surgical treatment in 2020 using real-time electronic pathology report data from population-based SEER cancer registries in Georgia and Louisiana. Methods: Bi-weekly numbers, distributions, and patterns of pathology reports were compared between January 1st and December 31st in 2020 and the same period in 2019 by age group and cancer site. Results: During 2020, there were 29,905 fewer pathology reports than in 2019, representing a 10.2% decline. Absolute declines were greatest among adults aged ≥50 years (N=23,065); percentage declines were greatest among children and young adults ≤18 years (38.3%). By cancer site, percentage declines were greatest for lung cancer (17.4%), followed by colorectal (12.0%), breast (9.0%) and prostate (5.8%) cancers. Biweekly reports were statistically significantly lower in 2020 than in 2019 from late March through the end of December in most biweekly periods. The nadir was the month of April 2020 – the number of reports was at least 40% lower than in April 2019. The number of reports in 2020 compared with 2019 also declined sharply in early November (26.8%) and late December (32.0%). Numbers of reports in 2020 never consistently exceeded those in 2019 after the first decline. Patterns were similar by cancer site, with variation in magnitude and duration of declines. Conclusions: Significant declines in cancer pathology reports from population-based registries during 2020 suggest substantial delays in screening, evaluation of signs and symptoms, diagnosis, and treatment services for cancers with effective screening tests as well as in cancer sites and age groups without effective screening tests as an indirect result of the COVID-19 pandemic. Ongoing evaluation will be critical for informing public health efforts to minimize any lasting adverse effects of the pandemic on cancer screening, diagnosis, treatment, and survival.[Table: see text]

The COVID-19 pandemic has interrupted oncology services including screening, diagnosis, and treatment. In this study pathology services were investigated as a proxy for cancer screening, diagnosis, and treatment delays to assess the effect of the pandemic on oncology services. In order to quantify the impact of COVID-19 we reviewed all pathology reports from January 1 through November 30 in 2018, 2019, and 2020 from 5 central registries in the National Cancer Institute Surveillance, Epidemiology, and End Results (SEER) Program. Using the SEER Data Management System and SAS, pathology reports were counted by cancer site (breast, lung, colorectal, prostate, other), patient age category (75 years), bi-weekly collection interval, and registry. Only pathology laboratories with monthly electronic reporting to their central registry throughout the study period were included. The reporting counts from 2020 were compared to those of the prior two years. In comparison to the 2019 baseline volume, March through November 2020 had an 11.2% decrease in pathology volume (48,779 fewer reports) in the 5 central registries. Over 75% of this decrease in volume occurred over a 10-week period, March through May 2020, with 30.2% fewer (37,127 reports) than expected pathology reports during this initial peak of the COVID-19 pandemic. The maximum bi-weekly decrease observed was 41.7% (>10,000 reports) from the 2019 baseline, which occurred in April 2020. During this 10-week period the decrease in volume by age category ranged 29.6-32.2%, cancer site ranged 27.5-39.1% and central registry ranged 26.1-36.8%. Two additional periods of volume decrease also occurred, both of shorter duration and magnitude, mid-July through August and mid-October through mid-November. Since the onset of the pandemic there remains a deficit (>11%) in the expected pathology volume compared to 2019, reflecting the continued impact of COVID-19 across oncology services. The majority of the pathology volume decrease in 2020 occurred during the initial peak of the COVID-19 pandemic. This period of decreased reporting volume along with two smaller decreases align with the 3 peaks of COVID-19 new cases reported by the CDC COVID Data Tracker. Prior to the onset of the pandemic the 2020 volume started at a bi-weekly rate greater than 2019, and the 2019 volume was consistently greater than 2018. Therefore, the 2020 expected values would be greater than those of the 2019 comparison year, i.e., the volume decreases reported would likely represent conservative estimates of the actual volume decrease. Continued longitudinal monitoring and the addition of more registries to this analysis are planned as well as pathology report type categorization (screening, surveillance, or treatment). This study also demonstrates the ability of the NCI SEER program to assist with near real time reporting of cancer data in quantifying the effect on the healthcare system of the COVID-19 pandemic. Citation Format: Todd Golden, Linda Coyle, Jennifer Stevens, Serban Negoita. Quantifying the impact of COVID-19 on cancer care: Pathology volume change in SEER cancer registries [abstract]. In: Proceedings of the AACR Virtual Meeting: COVID-19 and Cancer; 2021 Feb 3-5. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(6_Suppl):Abstract nr S12-01.

BACKGROUND Researchers have used prostate‐specific antigen (PSA) values collected by central cancer registries to evaluate tumors for potential aggressive clinical disease. An independent study collecting PSA values suggested a high error rate (18%) related to implied decimal points. To evaluate the error rate in the Surveillance, Epidemiology, and End Results (SEER) program, a comprehensive review of PSA values recorded across all SEER registries was performed. METHODS Consolidated PSA values for eligible prostate cancer cases in SEER registries were reviewed and compared with text documentation from abstracted records. Four types of classification errors were identified: implied decimal point errors, abstraction or coding implementation errors, nonsignificant errors, and changes related to “unknown” values. RESULTS A total of 50,277 prostate cancer cases diagnosed in 2012 were reviewed. Approximately 94.15% of cases did not have meaningful changes (85.85% correct, 5.58% with a nonsignificant change of, Researchers have used prostate‐specific antigen (PSA) values collected by central cancer registries to evaluate tumors for potential aggressive clinical disease. The current study describes the comprehensive review of PSA laboratory values recorded across all Surveillance, Epidemiology, and End Results registries for 2012 prostate cancer cases and the proposed mechanism for reviewing and correcting PSA laboratory values.

While machine learning (ML) has shown some promise in medical research, its actual impact has been limited relative to other application domains. One reason for this disparity is the lack of high-quality, patient-level data available to the broader ML research community. Such datasets are often not made available due to protections around patient privacy. To overcome these obstacles, high-quality, synthetic datasets could be leveraged to accelerate methodological developments in the application of ML to biomedical research. Clinical data in the form of electronic health records present a rich data source to be used for synthetic data generation. Such data can be high dimensional and predominantly categorical, which poses multiple challenges from a modeling perspective. In this paper, we evaluate four classes of synthetic data generation techniques, as well as several metrics for evaluating the quality of the synthetic data. While the results and discussions are broadly applicable to medical data, for demonstration purposes we generate synthetic datasets from the publicly available Surveillance Epidemiology and End Results (SEER) program. Specifically, our cohort consists of breast cancer cases diagnosed in the year of 2010, which includes over 26000 individual cases. Finally, we discuss the trade-offs of the different methods and metrics, providing guidance on considerations for the generation and usage of synthetic medical data. Citation Format: Andre R. Goncalves, Priyadip Ray, Braden Soper, Madhumita Myneni, Jennifer L. Stevens, Linda M. Coyle, Ana Paula Sales. Generation and evaluation of medical synthetic data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 1641.

e18317 Background: The National Cancer Institute (NCI) Surveillance, Epidemiology, and End Results (SEER) Program curates population-based cancer data representing 34% of the US population. CancerLinQ is an ASCO initiative that collects and analyzes electronic health record (EHR) data to give oncologists opportunities to improve the quality of patient care. With the shared goal of understanding care delivery, NCI SEER and CancerLinQ launched a pilot linkage. Purpose: Establish data exchange between registries and oncology practices to a) provide clinicians with SEER data to more effectively evaluate care within their practices, and b) enhance ability of SEER registries to capture cancer-related data and facilitate compliance of legally mandated public health reporting requirements while supporting metrics for quality reporting to providers. Methods: The SEER Iowa Cancer Registry is developing bidirectional linkages with CancerLinQ practices. The initial pilot in Iowa establishes connectivity and a data pipeline to capture discrete data elements in EHRs. The linkage methods are securely conducted by IMS, an honest broker for the Registry and ASCO. Patterns of care will be evaluated in the matched patient population. Analysis of shared data elements will provide comparative validation of data captured electronically (EHR) and manually (abstraction). Enhancing the patient care quality through efficient utilization of shared data was paramount when selecting treatment-related Quality Oncology Practice Initiative (QOPI) measures for calculation focusing on breast (QPP 449, QPP 450) and prostate cancer (QPP 102, QPP 104). Results: Publicly available SEER data for cohort evaluation is available to providers via SEERLinQ. The two-way exchange data pipeline complies with reporting requirements. Validation of shared data elements, statistics for matched patients, improved data completeness measures, and automated calculation of QOPI measures will be demonstrated. Conclusions: This collaboration builds an initial foundation of curated Registry-EHR linked data to automate cancer reporting to lower the physician burden, improve SEER evaluation of clinical care patterns, and enhance patient care quality.

STEM Pathways offer students additional opportunities to explore and prepare for successful careers in new and emerging STEM fields. To breakdown existing silos between K12 and higher education, community college-centered programs function as conduits between high schools, universities and jobs. An online social collaboration system called the Arizona STEM Network enables rich collaboration and shared ownership of the students' education with results that attract local industry partners. The STEM Pathways Guide, hosted on the Arizona STEM Network, provides a comprehensive tool to design, implement, and validate STEM Pathway Programs, measure program effectiveness and share best practices. Learning's from four other STEM Guides developed by SFAz informed the structuring and content of the STEM Pathways Guide. The Pathways Guide is based on a model validated by research. The model has seven primary components. College owned components include Education Outreach and Career Exploration, Foundational Knowledge and Skills, and Transferable Certifications and Degrees. Interdependency components include Student Support Strategies, Industry Engagement, Technology Integration, and Curricular Alignment. Currently, eight rural community colleges in Arizona and eleven Hispanic Serving Community Colleges (HSCCs) in California, Texas, Florida, Arizona, and New Mexico are participating in the online community and using the STEM Pathways Guide.

In 1995-1996, the authors mailed a food frequency questionnaire to 3.5 million American Association of Retired Persons members who were aged 50-69 years and who resided in one of six states or two metropolitan areas with high-quality cancer registries. In establishing a cohort of 567,169 persons (340,148 men and 227,021 women), the authors were fortunate in that a less-than-anticipated baseline response rate (threatening inadequate numbers of respondents in the intake extremes) was offset by both a shifting and a widening of the intake distributions among those who provided satisfactory data. Reported median intakes for the first and fifth intake quintiles, respectively, were 20.4 and 40.1 (men) and 20.1 and 40.0 (women) percent calories from fat, 10.3 and 32.0 (men) and 8.7 and 28.7 (women) g per day of dietary fiber, 3.1 and 11.6 (men) and 2.8 and 11.3 (women) servings per day of fruits and vegetables, and 20.7 and 156.8 (men) and 10.5 and 97.0 (women) g per day of red meat. After 5 years of follow-up, the cohort is expected to yield nearly 4,000 breast cancers, more than 10,000 prostate cancers, more than 4,000 colorectal cancers, and more than 900 pancreatic cancers. The large size and wide intake range of the cohort will provide ample power for examining a number of important diet and cancer hypotheses.

The software for analysis of the Health Habits and History Questionnaire (HHHQ) has been revised and is available to researchers. As in earlier versions of the software, questionnaires other than the standard National Cancer Institute versions can be analyzed. Foods can be added or dropped, nutrients can be added or changed, and many other revisions and options are facilitated. Estimates of 33 nutrients and up to 20 user-defined food groups are produced. The validity is unchanged from the previous software. Other features include a data entry key-and-verify system, standardized editing, a computer-assisted interview, and the calculation of health indices including pack-years of smoking and social network index.

The purpose of this study was to compare the level of HIV-related knowledge, attitudes, and risk behaviors among teens in a southern state. A 40-item instrument, developed by the Division of Adolescent and School Health, Centers for Disease Control, that assessed HIV-related knowledge, attitudes and risk behaviors was administered to rural and urban teens (N = 180)in a southern state, Spring, 1990. Data were analyzed using the t-test for independent means. No significant differences existed when comparing rural and urban teens' mean knowledge and behavior scores. Additionally, no significant differences were found when controlling for gender. Data analysis did reveal a significant difference in the attitude scores of rural and urban teens. Urban teens had a lower mean score, indicating more positive HIV-related attitudes, beliefs, perceptions, and communication experiences. Overall, the study population exhibited a high degree of HIV-related knowledge, yet engaged in sexual behaviors that put the...

The characteristics of "effective" schools can now be summarized

Educators are confronted with more mandated testing now than at any other time in the history of public education in the United States. An integral part of our educational system, testing has become our most accepted as well as most expected method of evaluation. With the increase in testing, teachers often must spend instructional time preparing students to take standardized tests (knowing, of course, that the test results will be used for an assessment of their own effectiveness). Many teachers now construct their classroom tests in a format that allows students to become comfortable with the standardized testing format. Well-constructed teachermade tests can serve as measures of knowledge as well as practice for standardized testing. According to Carey (1988), the most popular format for standardized tests is the multiple-choice format. Teachers who wish to use that format should be aware of the strengths, limitations, and factors to consider in the construction of multiple-choice items.

The authors report the adaptation of a dietary and risk factor questionnaire and analysis system for use with IBM (or compatible) personal computers. This system includes a flexible computer-assisted interview program which may be modified to suit investigator needs while preserving a standard output format. It also includes a nutrient analysis program for calculation of usual dietary intake. Each of these two major components has its own set of utilities and options, so that investigators may tailor the system to particular research areas. The nature and capabilities of the two main programs are described, as well as the development of the underlying system and the general flow of data in the system.