Search

Official price statistics in the Philippines are mainly sourced from the conduct of regular surveys and censuses which entail high costs. As businesses move into digital platforms, alternatives to these traditional data sources have become more available; one of which is web scraping, a process of collecting information from the web. As digital and online platforms become increasingly utilized for commerce, web scraping offers a way to increase the frequency of data collection while reducing its cost compared to price surveys. This paper provides a survey of experiences of various government statistical agencies in their conduct of web scraping for the Consumer Price Index (CPI). Moreover, it details the Philippines' experience using web scraped data to estimate the food and alcoholic beverages CPI of the National Capital Region in the Philippines, and that is compared to the official CPI estimate of the Philippine Statistics Authority. Finally, this paper discusses the challenges encountered and the recommendations for enhancing the approach. [ABSTRACT FROM AUTHOR]

Collecting data from businesses faces ever-larger challenges, some of them calling for an overhaul of underlying methodology, e.g. motivation for participating is low; technology is shaping data collection processes; response processes within businesses are imperfectly understood while alternative data sources originating from digitalization processes push the response process (thus also response quality) further out of our sight. The paper reviews these challenges, discusses them in light of new developments in the field, and proposes directions for future research. This review may help those that collect data from businesses (e.g. national statistical institutes, academia, and private statistical agencies) to reconsider their current approaches in light of what promises to work (or not) in today's environment and to build their toolkit of business data collection methods. [ABSTRACT FROM AUTHOR]

As mobile and wireless technologies continually improve and become more affordable, reliable, powerful and user-friendly, Computer Assisted Personal Interviewing (CAPI) is expected to become the one of the effective approaches in field-based census data collection even in those countries where access to infrastructure for information and communication technologies is limited. African countries are at such a case. The introduction of electronic data capture into the business cycle of the census provides cost and time savings, and also allows users to take advantage of added features that can be programmed into mobile devices or linked to the data collection process. These features include, among others, integrated maps and Global Positioning System (GPS) and real time monitoring of fieldwork. This paper shows how the Italian National Institute of Statistics (Istat) has supported the Ethiopia Central Statistics Agency (CSA) in designing and implementing such a monitoring system, which has been fully integrated with the census data collection process managed by CSPro, the public domain software package developed by the U.S. Census Bureau (USCB). The proposed architecture is generalised and provides a simple solution for monitoring electronic data collection operations, particularly in cases where the technical and financial resources to implement such a system from the ground up are lacking. [ABSTRACT FROM AUTHOR]

The Dutch 2011 Census tables were produced by combining existing register and sample survey data. Since the last census based on a complete enumeration was held in 1971, the willingness of the population to participate has fallen sharply. Statistics Netherlands no longer uses census questionnaires and has found an alternative in the register-based census, using only existing data. The register-based census is cheaper and more socially acceptable. The table results of the Netherlands are not only comparable with earlier Dutch censuses, but also with those of the other countries in the 2011 European Census Round, although most other countries do not yet have register-based censuses. The Dutch Census 2011 cannot be considered in isolation from its predecessors. This most recent census builds on a rich tradition of traditional population and housing censuses carried out in the period 1829–1971. For the European Union censuses of 1981 and 1991, Statistics Netherlands provided alternative census data consisting of a combination of register and survey data, but without complete numerical consistency between census tables and without detailed information about regions and specific categories. This combined data provision was continued for the censuses of 2001 and 2011, by which time it was possible to compile completely consistent data at a very detailed level. The 2011 Census data are more detailed than those of 2001, and include the hypercubes (high dimensional tables) that are mandatory for all EU countries under the 2008 European Census Act. The challenge for the future is to keep comparability, but to improve the timeliness of the European census results. [ABSTRACT FROM AUTHOR]

Monitoring and addressing racial and ethnic disparities in health and health care require the collection and analysis of reliable demographic information. With the intention of improving the measurement and monitoring of disparities, certain provisions of the Patient Protection and Affordable Care Act (ACA) of 2010 require states to collect, report, and analyze data on demographic characteristics of applicants and participants in Medicaid and other federally supported programs. In this large-scale study we link Medicaid records to the Census 2000, the 2010 Census, and the American Community Survey for years 2001–2009 to explore the extent to which pre-ACA Medicaid administrative records matched self-reported race and Hispanic origin in Census Bureau data. Record linkage allows comparison of demographic, socioeconomic and neighborhood characteristics between Medicaid participants with matching and non-matching race and Hispanic origin in census. Identification of the groups most likely to have non-matching and missing race and Hispanic origin data in Medicaid relative to census can inform strategies to improve the quality of demographic data collected from the Medicaid population. [ABSTRACT FROM AUTHOR]

Previous studies have shown that modeling based on administrative records can be predictive of Nonresponse Followup (NRFU) enumeration outcomes in U.S. Census Bureau Decennial data collection operations. We compare model predictive power when varying training data sources and evaluate the extent to which survey data can be used to reduce enumerator workload when combined with available administrative data. We perform the evaluation using the 2010 Census and the 2014 American Community Survey. Our main finding is that a large survey-based training dataset, such as the American Community Survey, can provide results comparable to Census data. Robustness checks then illustrate that even small sample survey-based training datasets can also yield comparable predictions. We also discuss a broader role for use of existing survey data in NRFU operations of statistical agencies outside of the United States when national Census or administrative data sources have only incomplete coverage of the population. [ABSTRACT FROM AUTHOR]

The aim of this paper is to show a Palestinian Central Bureau of Statistics (PCBS) [1] experiment in administrative data records linkage. We focused in this paper on PCBS experiment in matching different data sources from different ministries, municipalities and other partners with PCBS Establishments Census 2012. Different matching algorithms and tools were used in the experiment. We started our experiment by using the Fuzzy Lookup [2]. It is an add-in for Excel developed by Microsoft Research. It performs fuzzy matching of textual data in Microsoft Excel. The tool uses the Jaccard Index of Similarity and Levenshtein distance; a statistical way to measure similarities between sample sets. In order to compare data and try to find out matching data, we also used Duke, see Lars [3] which is an existing and flexible deduplication (or entity resolution, or record linkage) engine written in Java. By using Duke Engine, we wrote our matching algorithm and comparators to increase the matching results and matching accuracy. We also wrote some data-cleaning functions for matching variables (Commercial Name, Owner Name and Telephone) in order to standardize each matching variable to get improved results. Different matching algorithms were used in the experiment such as Hamming Distance, e.g. Mohammad [4], Levenshtein distance, Mark [5], Jaccard Similarity, e.g. Suphakit et al. [6], exact match and multiple match. The results showed that after cleaning the identification variables, the number of matches rises significantly. We also noted that there is an improvement in matching rates when going from the matching based only on phone numbers to the matching based on Telephone, Commercial Name and Owner Name. [ABSTRACT FROM AUTHOR]

Previous studies have shown that modeling based on administrative records can be predictive of Nonresponse Followup (NRFU) enumeration outcomes in U.S. Census Bureau Decennial data collection operations. We compare model predictive power when varying training data sources and evaluate the extent to which survey data can be used to reduce enumerator workload when combined with available administrative data. We perform the evaluation using the 2010 Census and the 2014 American Community Survey. Our main finding is that a large survey-based training dataset, such as the American Community Survey, can provide results comparable to Census data. Robustness checks then illustrate that even small sample survey-based training datasets can also yield comparable predictions. We also discuss a broader role for use of existing survey data in NRFU operations of statistical agencies outside of the United States when national Census or administrative data sources have only incomplete coverage of the population. [ABSTRACT FROM AUTHOR]

Under the Micro-Economic Platform Programme, we are transforming how micro-economic surveys are processed at Statistics New Zealand. This is part of our organisation-wide business transformation programme, Statistics 2020 Te Kāpehu Whetū (Stats 2020), which aims to increase our relevance to government and public data users, and reduce the cost of producing statistical outputs. I report on insights into this programme and outline what the platform team has learned during the projects included in the Micro-Economic Platform. In particular, I report on the challenges involved in using standard methods and tools, increasing the level of automation in the system, introducing survey teams to the platform, and collaborating to achieve the migration of each survey to the platform. I also discuss what we learned from these challenges as part of achieving Statistics New Zealand's statistical system of the future. [ABSTRACT FROM AUTHOR]

National Statistical Offices (NSO's) need data for the production of statistics. Apart from data obtained through surveys, NSO's are increasingly using other data sources (for example, data collected by other organizations). Disadvantages here come from the fact that very often an NSO uses an administrative data source for a purpose different than the one for which the data was originally collected. As a result of this difference, the 'statistical' usability of a data source needs to be thoroughly studied by an NSO before its use. Some more possible problems of statistics are the following: data sources, recent data, regularity, timeliness of data, updating data, time series, data frequency (annual, quarterly, monthly or daily data required), data comparability (is it possible to compare one country's performance against another or group of countries etc.), data costs. Sometimes, the required data do not exist; data from different sources are not always comparable. Also, the quality of some information needs an improvement. It is therefore of vital importance that a systematic approach is available to determine the quality of statistical data. For this purpose a quality framework have been developed by authors to determine whether statistical data is of quality required for their intended use. [ABSTRACT FROM AUTHOR]