Search

Abstract Background Measuring palliative care quality requires the application of evaluation methods to compare clinically meaningful groups of patients across different settings. Such protocols are currently lacking in Poland. The Australian Palliative Care Outcome Collaboration (PCOC) concept of Palliative phases precisely defines patients, enables episodes of care extraction for benchmarking and further assessment of service delivery. The present study is aimed at developing cross-cultural adaptation and psychometric validation for a Polish translation of Palliative phases. Methods Forward and backward translation was performed to obtain a Polish draft version regarding definitions of the PCOC phases. The draft was then subjected to linguistic and graphical transformations in the process of cognitive interviewing. The acceptability of the Polish version was assessed based on staff perceptions of fit, ease of assignment and familiarity with the patient’s and family’s situation. Finally, cross-sectional analysis was conducted among 313 hospice and home-care palliative patients. The attending doctor and nurse independently evaluated the same patients using the Polish version of the PCOC phases to establish inter-rater reliability values. Then, to determine its construct validity, the PCOC indicators were referred to patients’ prognosis, functioning level and PALCOM scale scores. Results A Polish draft version of the PCOC phases was prepared. Seven of the 13 interviewees reported problems with comprehending this proposal. This prompted changes being made to linguistic and graphical aspects of the tool. The majority of respondents preferred the final graphical scheme of phases, prepared after round-two of interviews. Scheme application acceptability was confirmed in a practical trial. The respondents’ overall conviction regarding degree of fit and assignment ease was high. Seventy percent of PCOC phase assignments was in agreement, and a moderate level or inter-rater reliability was obtained (kappa 0.573). The highest proportion of totally bed-bound patients with the shortest survival prognosis was observed for the terminal phase, while the highest complexity of palliative care needs was noted for the unstable one. Conclusions The PCOC phases tool appears to be a valuable resource for specialists in palliative care settings to support audit measures. Practical training is recommended prior to its implementation in routine practice.

Abstract Background Sexual and reproductive empowerment (SRE) is an important determinant of women’s and girls’ health yet measuring it is complex due to cultural and domain-specific variations. This study describes the process of adapting an SRE scale consisting of four domains (self-efficacy; future orientation; social support; and safety) and testing its psychometric properties among Arabic speaking adolescent girls in Lebanon. Methods An SRE scale developed in a Western context was adapted in four steps: (1) reviewing the scale and selecting culturally appropriate domains for translation to standard Arabic; (2) conducting cognitive interviews with 30 11-17-year-old adolescent girls in Lebanon; (3) administering the scale to 339 refugee adolescent girls who participated in an early marriage intervention; and (4) conducting confirmatory factor analysis (CFA) on the data to assess the scale’s psychometric properties. Results The original model for the 13-item, four-domain adapted scale demonstrated poor fit in CFA. After iteratively removing two items, scale properties were improved, albeit were not optimal. The validity and reliability results for the self-efficacy domain were acceptable. Cognitive interview data revealed that Arab adolescent girls understood self-efficacy in relational terms, recognizing that autonomous decision-making is not necessarily favored but is influenced by parents and family. Conclusions This study presents an effort to customize an SRE scale for use in studies on the health of adolescent girls in an Arab cultural context. Findings from cognitive interviews highlight the importance of taking into consideration relationality in adolescent sexual and reproductive decision-making. The self-efficacy domain in the adapted scale demonstrates acceptable psychometric properties and is recommended for use in health studies to capture SRE.

Background: The neurogenic bladder symptom score (NBSS) has been widely used to specifically measure symptoms and consequences of neurogenic bladder (NB). The cognitive interviewing (CI) is effective in assessing item clarity and identifying key issues related to the comprehension of the instrument. We aim to translate the NBSS into Chinese and use the CI approach to explore the thought processes of patients with NB in responding the Chinese Version of the NBSS, identify and modify the factors hinder the thought processes to enhance the face validity of the NBSS. Methods: The translation of the NBSS into Chinese was conducted with the guidance of the recommended frameworks. Patients with NB were recruited by purpose sampling. CI with the combination of thinking aloud and verbal probing techniques were used to explore thought processes. The interviews were transcribed and analyzed based on Tourangeau four-stage response model. Results: Two rounds of CI were carried out. The problems of comprehension, judgement and response mapping were identified in 8 items. Four items were revised based on the results of the interview. The revised items were verified and eventually integrated into the final version. Conclusion: The Chinese Version of the NBSS was easy to comprehend and use. The use of CI methodologies can increase the comprehensibility and cultural applicability of the NBSS, providing the evidence for the development of a clearer and more appropriate questionnaire.

Abstract Aim To translate the Empowerment Scale for Pregnant Women (ESPW) into Chinese and to assess its linguistic validity. Methods The integrative method of the translation process, the Delphi technique, and cognitive interviews were used to implement cross‐cultural adaptation and enhance comprehensibility and linguistic validation. This study recruited 14 experts in the expert review and cognitively reviewed 15 pregnant women. Results The two‐round Delphi method created agreement on cultural applicability. The results of content validity achieved good levels: The item‐level content validity index (CVI) ranged from 0.78 to 1.00, and the scale‐level content validity index, calculated using two different formulas, were 0.97 and 0.81, respectively. Kappa values ranged from 0.74 to 1.00. Pregnant women could understand most of the items and response options in the cognitive interview. The revisions to the wording were made based on suggestions from experts and pregnant women. Conclusion The prefinal simplified Chinese ESPW was semantically and conceptually equivalent to the English version, which was well prepared for further psychometric tests in the next stage of cross‐cultural adaptation. Patient or Public Contribution This comprehensive method successfully developed a Chinese tool to measure the empowerment of pregnant women, indicating the international applicability of this tool and the methodological scientific nature. The simplified Chinese ESPW has the potential to support the identification of empowerment levels of pregnant women and the evaluation of the effectiveness of health education and promotion programmes.

Abstract: The Icelandic versions of the HEXACO-100 and HEXACO-60 were evaluated using confirmatory factor analysis. The sample for HEXACO-100 consisted of N = 716, and the sample for HEXACO-60 of N = 319. Six respondents were recruited for cognitive interviews. Modification indices, factor loadings, and cognitive interviews were used to identify items in need of revision, identifying a total of 56 items. Examples of the types of information gained using a holistic approach during item identification are presented. While widely used in psychometric research, the combination of confirmatory factor analysis with cognitive interviews and probing is underutilized in research on personality assessment. A holistic approach toward psychometric evaluation provides valuable information and is recommended as an integral part of the validation process of personality instruments.

Abstract Objective/Background The Consensus Sleep Diary (CSD) is widely used to assess subjective sleep. Psychometric evaluations and focus-groups support its validity and clinical usefulness, but further research into its validity is needed. The aim of the study was to evaluate a Swedish translation of the CSD regarding test content and response processes in patients with insomnia. Patients/Methods In connection with translating the CSD into Swedish, we used cognitive interviewing to evaluate test content and the response process, that is, how people make decisions when responding to survey items. Cognitive interviews were conducted with 13 primary health care patients with insomnia disorder (mean age, 49 years; SD 15.5). Iterative, reparative analysis was used to investigate test content. Descriptive deductive analysis was used to investigate interview transcripts for the themes of the cognitive model: comprehension, retrieval, decision process, and judgement. Together, the themes explain the response process when responding to a patient-reported outcome measure. Results The overall comprehension of the CSD could be affected by poor adherence to the instructions (comprehension). Patients had difficulty with recall if they did not complete the diary immediately in the morning and just before bedtime (retrieval). They could have problems deciding how to respond to certain items because they imbued sleep-related concepts with extra meaning (decision process), and had trouble finding response alternatives nuanced enough to describe their experience of sleep and tiredness (judgement). Conclusions This study contributes knowledge on how the instrument is perceived and used by care-seeking patients with insomnia. In this context, the CSD exhibits known flaws such as memory lapses if the diary is not filled in directly in the morning. To increase the accuracy of patients’ responses, therapists should support patients in reading the instructions.

Paranormal health beliefs denote the inclination to endorse illusory supernatural notions about well-being and treatment. These ideations are important since they potentially influence perceptions of health and allied behaviors. Noting this, researchers in Italy developed and verified the Paranormal Health Beliefs Scale (PHBS). Despite initial promising outcomes, the construct and measurement properties of the PHBS have remained under investigated. This is likely due to the fact that the instrument draws heavily on traditional Italian social, political, and religious influences and is overly culturally specific. Hence, items do not generalize well across populations and nationalities. Acknowledging these factors, this study used cognitive interviewing (think aloud protocol and concurrent probing) to assess the suitability of the PHBS for general use. Concurrently, the intention was to identify necessary modifications that would enhance scale performance. Fourteen interviewees (eight males and six females), evenly distributed across two rounds, participated. Round 1 focused on comprehension and perception of the PHBS. Cognitive interviews identified issues with culturally particular content/points of reference, phraseology, and wording. To address these a modified version of the PHBS was produced. Round 2 then examined the effectiveness of changes. Analysis revealed fewer concerns, although difficulties with ambiguity, complex terminology, and response scale appropriateness persisted. Overall, interviews indicated that a focus on illusory (rather than paranormal) health beliefs would improve scale utility. Methodologically, cognitive interviewing provided invaluable insights into the effectiveness of the PHBS and identified ways in which researchers could adapt the instrument for use with other cultures.

Abstract Background International attention is being paid to the issue of making evidence sustainable after implementation. Developing an identification model is essential to promote and monitor the sustainability of evidence implementation. However, this model is not available in Chinese. This study aims to translate the National Health Service Sustainability Model into Chinese and to verify whether the model is adapted to the Chinese healthcare environment. Methods This study follows the translation and validation guidelines developed by Sousa and Rojjanasrirat. The translations include forward and backward translations and their comparison. Expert reviews were used to validate the content validity of the Chinese version of the National Health Service sustainability model. Cognitive interviews were used to assess the validity of the language in the Chinese setting. Results The translation was conducted by a bilingual research team and took 12 months. Expert reviews were undertaken with eight experts, and cognitive interviews with six participants. The content validity of the model is excellent, but at least 20% of the experts still felt that items one, three, five and nine needed refinements. In the cognitive interviews, most items, instructions and response options were well understood by the participants responsible for the evidence-based practice project. However, some language issues were still identified in items one, three, four, five, seven, nine, and ten. Participants reported that the sustainability results of the model assessment were consistent with their previous judgments of the items. Based on the expert review and interview results, items one, three, four, five, seven, nine and ten require further refinement. In summary, seven of the ten items have been amended. Conclusions This study provides insight into how the National Health Service sustainability model can be used in the Chinese healthcare setting and paves the way for future large-scale psychometric testing.

Abstract Background Organizational readiness is a key factor for successful implementation of evidence-based interventions (EBIs), but a valid and reliable measure to assess readiness across contexts and settings is needed. The R = MC2 heuristic posits that organizational readiness stems from an organization’s motivation, capacity to implement a specific innovation, and its general capacity. This paper describes a process used to examine the face and content validity of items in a readiness survey developed to assess organizational readiness (based on R = MC2) among federally qualified health centers (FQHC) implementing colorectal cancer screening (CRCS) EBIs. Methods We conducted 20 cognitive interviews with FQHC staff (clinical and non-clinical) in South Carolina and Texas. Participants were provided a subset of items from the readiness survey to review. A semi-structured interview guide was developed to elicit feedback from participants using “think aloud” and probing techniques. Participants were recruited using a purposive sampling approach and interviews were conducted virtually using Zoom and WebEx. Participants were asked 1) about the relevancy of items, 2) how they interpreted the meaning of items or specific terms, 3) to identify items that were difficult to understand, and 4) how items could be improved. Interviews were transcribed verbatim and coded in ATLAS.ti. Findings were used to revise the readiness survey. Results Key recommendations included reducing the survey length and removing redundant or difficult to understand items. Additionally, participants recommended using consistent terms throughout (e.g., other units/teams vs. departments) the survey and changing pronouns (e.g., people, we) to be more specific (e.g., leadership, staff). Moreover, participants recommended specifying ambiguous terms (e.g., define what “better” means). Conclusion Use of cognitive interviews allowed for an engaged process to refine an existing measure of readiness. The improved and finalized readiness survey can be used to support and improve implementation of CRCS EBIs in the clinic setting and thus reduce the cancer burden and cancer-related health disparities.

Abstract Background Use of patient-reported outcome measures in clinical settings facilitate the delivery of better health care to improve patient health outcomes. Previously collected qualitative data indicated themes that could inform items for a health-related quality of life measure. This study investigated the content validity of items for inclusion in a new health-related quality of life measure suitable for patients with ovarian cancer. Methods Cognitive interviewing techniques were used with fourteen women diagnosed with ovarian cancer and at different times since diagnosis, to evaluate items derived from the previously collected qualitative dataset. A set of draft items was administered via telephone, Zoom and WhatsApp app together with questions on item meaning and wording. Interviews were transcribed and thematically analysed. Results Four broad themes emerged in relation to the questionnaire construction and comprehension of items: intent and clarity, wording, relevance and context, and overall questionnaire construct. All draft items were adjusted based on the interview findings. A final set of 38 health-related quality of life items comprised 7 items describing physical health and functioning, 21 describing emotional wellbeing and 10 items describing social wellbeing; each rated on a five-point frequency response scale. Conclusion The items reflected a range of personal experiences associated with the patient clinical journey, creating a health-related quality of life tool specific to women diagnosed with ovarian cancer. The cognitive interviewing process established content validity for the tool, thereby, preparing it for field testing and evaluation of its psychometric properties. This study highlighted the fundamental role of cognitive interviewing during health-related quality of life questionnaire development to ensure that item content is grounded in patient feelings, functioning and meaning.

Background: The diversity in characteristics of cannabis products and behavior patterns make evaluation of cannabis exposure in population-based, self-report surveys challenging. Accurate identification of cannabis exposure and related outcomes necessitates a thorough understanding of participants’ interpretations of survey questions assessing cannabis consumption behaviors. Objectives: The current study utilized cognitive interviewing to gain insight on participants’ interpretation of items in a self-reported survey instrument used to estimate the quantity of THC consumed in population samples. Methods: Cognitive interviewing was used to evaluate survey items assessing cannabis use frequency, routes of administration, quantity, potency, and perceived “typical patterns” of use. Ten participants ≥18 years (n = 4 cisgender-men; n = 3 cisgender-women; n = 3 non-binary/transgender) who had used cannabis plant material or concentrates in the past week were recruited to take a self-administered questionnaire and subsequently answer a series of scripted probes regarding survey items. Results: While most items presented no issues with comprehension, participants identified several areas of ambiguity in question or response item wording or in visual cues included in the survey. Generally, participants with irregular use patterns (i.e., non-daily use) reported more difficulty recalling the time or quantity of cannabis use. Findings resulted in several changes to the updated survey, including updated reference images and new quantity/frequency of use items specific to the route of administration. Conclusion: Incorporating cognitive interviewing into cannabis measurement development among a sample of knowledgeable cannabis consumers led to improvements in assessing cannabis exposure in population surveys, which may otherwise have been missed.

Introduction. The issue of the readiness of adolescents with chronic kidney disease (CKD) for the transition to the adult service in the Russian healthcare system remains unexplored. One of the reasons of this problem is the lack of Russian tools for evaluating patient transition readiness.The goal of this study is to develop Russian-language questionnaires to evaluate the readiness of adolescents for the transition to an adult health service based on the English-language originals.Materials and methods. A literature search was carried out to identify English questionnaires, as well as recommendations for their translation and adaptation. Development of the Russian versions of the questionnaires included the following major stages: preparation, two forward translations, their reconciliation, adaptation and creation of an intermediate version, back-translation into English, cognitive interviewing of the adolescents, proofreading and approval of the final version by the authors of the original English questionnaires. Cognitive interviewing was performed with 10 adolescents with CKD aged 15–17 years.Results. Four English questionnaires were identified. They were studied and comparatively analysed. Translated versions of these questionnaires into foreign languages were revealed. Two most promising questionnaires for translation into Russian were identified: UNC TRxANSITION Index and STARx. Russian versions of these questionnaires were developed based on the international recommendations. Cognitive interviewing provided additional adaptation and linguistic validation of the translated versions based on the adolescents’ opinions. UNC TRxANSITION Index is a provider-administered questionnaire and includes 32 questions. STARx is a self-report questionnaire and includes 18 questions.Conclusions. Russian versions of questionnaires for evaluating adolescents’ transition readiness are available for use by health professionals and social workers. A further study of their psychometric properties and confirmation of the validity of their results in the context of the Russian healthcare system is required.

Abstract Aim The aim of this study is to explore the use of the Questionnaire Appraisal System with a combination of qualitative and quantitative methods in cognitive interviewing for cross‐cultural adaptation. Design This is a descriptive methodological study. Methods Using the Mandarin version of the Post‐Stroke Checklist as an example, cognitive interviews were conducted with 27 stroke survivors in Guangzhou between November 2020 and February 2021. The Questionnaire Appraisal System was applied as a codebook in focus group discussions to perform quantitative data collections and quantitative content analysis. Results Thirty‐eight problems were proposed in focus group discussions and identified all but four of the 30 questions that emerged in the cognitive interviews. A new item was added to the Questionnaire Appraisal System for better categorization. Four categories and six subcategories of problems in the checklist were revealed.

Plain language summary Sexual and reproductive health receives a lot of attention, globally. Much of this attention focuses on research and clinical services to address health outcomes. Behavioural issues known to be closely linked to health outcomes—that is people’s sexual activity and sexuality—tend to be overlooked. In many countries, there is very little data about people’s sexual practices and behaviours. If this information were available, then sexual and reproductive health programmes could be planned to better meet the needs of populations. Therefore, the World Health Organization developed a survey instrument which can be used to ask people around the world about their sexual practice and behaviours. The protocol described here will test the draft survey instrument in different countries to make sure that people from diverse backgrounds understand each survey question in a comparable way. The study uses ‘cognitive interviewing’, a qualitative method where researchers ask participants about their thought processes in reading a question and providing an answer while they complete a survey. Up to 20 countries will participate in this study, with five or more completing the research at a time in each study wave. Country teams in each wave will translate the survey instrument (if necessary), conduct the cognitive interviews, revise the instrument based on the findings, and if needed, conduct more interviews in their setting with the revised survey instrument. The result of this research will be a survey instrument that has been tested in a variety of languages and settings and is ready to be used in different contexts, globally.

The need for high-quality, real-time data has never presented itself as clearly as it did during the COVID-19 pandemic. Responding to the COVID-19 pandemic, from both a policy and a public health perspective, required timely, accurate data about the public's attitudes and behaviors from health surveillance, monitoring, and public opinion surveys. The uniqueness of the COVID-19 pandemic also created particular challenges for survey data collection, specifically, how to develop high quality survey questions on topics that had never been previously fielded. To account for this challenge, the National Center for Health Statistics adopted an iterative, two-component, mixed-method approach to question design and evaluation. The first, a cognitive interviewing study using virtual, online interviews was used to produce interpretative schemata of the response processes underlying the survey questions. The second, a two-round, mixed method survey using a statistically-sampled panel, was designed to further develop the interpretive schemata and to allow for detailed subgroup analyses. To increase the usefulness of the survey's second round, cognitive interview findings and results from the survey's first round were used to develop both open- and close-ended embedded probes. Taken together, the studies reveal the specific problems for question-design during such a novel, quickly-evolving event: 1) a lack of shared understanding of novel concepts and vocabulary, 2) the shifting reference period respondents use to think about attitudes and behaviors during a multi-year event, 3) the pervasive nature of the event that therefore frames how respondents conceptualize and process questions about unrelated topics. This iterative approach to understanding question-design problems not only allowed for the continuing improvement of COVID-19 survey items, going forward, it also provided a methodological foundation for question development for high quality, real-time data collection.

Abstract Background Palliative care phases (stable, unstable, deteriorating, terminal and bereavement) are routinely used in Australia and the UK to describe the clinical situation of patients and their families and to evaluate the associated care plan. In addition, it serves as a benchmark developed by the Australian Palliative Care Outcome Collaboration (PCOC) and is used nationwide for comparisons between services. In Germany, the concept is not used consistently due to various translations. Furthermore, there is no nationwide systematic approach to routinely assess clinical outcomes in palliative care. The study aims to develop a German version of the palliative care phase definitions by adapting them culturally, and to examine the inter-rater reliability of the adjusted definitions with healthcare professionals. Methods Mixed-methods approach: Cognitive interview study using ‘think aloud’ and verbal probing techniques and a consecutive multi-center cross-sectional study with two clinicians independently assigning the phase definitions. Interviewees/participants were selected through convenience and purposive sampling in specialist palliative care inpatient units, advisory and community services and in three specialist palliative care units with doctors, nursing staff and allied health professionals. Results Fifteen interviews were conducted. Identified difficulties were: Some translated terms were 1) not self-explanatory (e.g. ‘family/carer’ or ‘care plan’) and (2) too limited to the medical dimension neglecting the holistic approach of palliative care. (3) Problems of comprehension regarding the concept in general occurred, e.g. in differentiating between the ‘unstable’ and ‘deteriorating’ phase. Inter-rater reliability was moderate (kappa = 0.44; 95% CI = 0.39–0.52). The assignment of the phase ‘deteriorating’ has caused the most difficulties. Conclusion Overall, the adapted palliative care phases are suitable to use in the German specialist palliative care setting. However, the concept of the phases is not self-explanatory. To implement it nationwide for outcome measurement/benchmarking, it requires further education, on-the-job training and experience as well as the involvement of healthcare professionals in implementation process. For the use of international concepts in different healthcare systems, a deeper discussion and cultural adaptation is necessary besides the formal translation.

Abstract Background Chronic Hypersensitivity Pneumonitis (CHP) is caused by an immune mediated response in the lung tissue after exposure to an inhaled environmental antigenic stimulant. We previously documented the ways in which CHP impacts patients’ lives and have now developed a disease-specific instrument, the CHP-HRQOL instrument, to measure health-related quality of life (HRQOL). The objective of this study was to assess content validity for the CHP-HRQOL. Methods Cognitive interviews were conducted among adults with CHP. The instrument was revised and refined between each round of interviews. Feedback was obtained on the instructions, items, response options, and recall period. Items where participants had difficulty with comprehension, wording, or misinterpretation were marked by the interviewer and participant feedback was reviewed to make revisions, add or delete items when appropriate. Readability statistics were calculated using Flesch-Kincaid grade level and reading ease scores. Results Ten participants were interviewed over three rounds, with revisions made to the questionnaire in an iterative process. In the initial 39 item instrument, we identified 7 items where two or more participants reported difficulty. Participants preferred a four-week recall period (compared to a two-week recall period) and response options with a 5-point response scale. The final version of the CHP-HRQOL includes 40 items with a median reading level between 6th and 7th grade. Conclusion The CHP-HRQOL instrument demonstrated high content validity and is ready for psychometric testing in further validation studies.

Abstract Background Accurate diagnosis of carpal tunnel syndrome (CTS) is essential for directing appropriate treatment; and for making decisions about work injury claims. The Kamath and Stothard Questionnaire (KSQ) is a self-reported tool used for the diagnosis of CTS. Comprehensibility and comprehensiveness of this questionnaire are critical to diagnostic performance and need to be established. The purpose of the study was to describe how potential respondents, clinicians, and measurement researchers interpret KSQ questions in order to identify and resolve potential sources of misclassification. Methods Hand therapists, measurement researchers, participants with CTS, and a control group were interviewed using cognitive interviewing techniques (talk aloud, semi-structured interview probes) in Hamilton, Canada. All interviews were recorded and transcribed verbatim. A directed content analysis was done to analyze the interviews using a previously established framework. Findings Eighteen participants were interviewed. Areas, where questions were unclear to some participants, were recorded and categorized into five themes: Clarity and Comprehension (52%), Relativeness (38%), Inadequate Response Definition (4%), Perspective Modifiers (4%), and Reference Point (2%). Respondents also identified several symptoms of CTS that are not covered by the KSQ that might be of diagnostic value, e.g., weakness and dropping items. Conclusion The content validity of the current iteration of the KSQ was not established. The problematic questions identified in the study have been reported to have low specificity and negative predictive values in a previous quantitative study. The content validity issues identified may explain the poor performance. Recommendations were made to modify the wording of the KSQ and the potential addition of three new questions. Future studies should determine whether the modified questionnaire can provide better diagnostic accuracy and psychometric properties. The results of this study may assist in ruling in or out CTS diagnosis to a wide variety of target audience, such as hand specialists, physical and occupational therapists, as well as family doctors.

Content validity is a fundamental requirement of outcome measures. After reviewing operational needs and existing definitions, content validity we as defined as: the extent to which a measure provides a comprehensive and true assessment of the key relevant elements of a specified construct or attribute across a defined range, clearly and equitably for a stated target audience and context. ICF linkage rules from 2002, 2005, and 2019 have provide increasingly clear processes for describing and evaluating content of outcome measures. ICF Core Sets provide international reference standards of the core constructs of importance for different health conditions. Both are important as reference standards during content validation. To summarize their use as reference standards, the following summary indicators were proposed: (1) Measure to ICF linkage, (2) Measure to (Brief or Comprehensive) Core Set Absolute Linkage, (3) Measure to (Brief or Comprehensive) Core Set Unique Linkage, (4) Core Set Representation, and (5) Core Set Unique Disability Representation. Methods to assess how respondents engage with content are needed to complement ICF-linking. Cognitive interviewing is an ideal method since it used to explore how respondents interpret and calibrate response to individual items on an outcome measure. We proposed a framework for classifying these responses: Clarity/Comprehension, Relevance, Inadequate response definition, Reference Point, Perspective modification, and Calibration Across Items. Our analysis of 24 manuscripts that used ICF linking for content validation since updated linking rules were published found that authors typically used linking to validate existing measures, involved multiple raters, used 2005 linking rules, summarized content at a concept level (e.g., impairment, activity, participation) and/or use core sets as a reference standard. Infrequently, ICF linking was used to create item pools/conceptual frameworks for new measures, applied the full scope of the 2019 linking rules, used summary indicators, or integrated ICF-linking with qualitative methods like cognitive interviews. We conclude that ICF linkage is a powerful tool for content validity during development or validation of PROM. Best practices include use of updated ICF linking rules, triangulation of ICF linking with participant assessments of clarity and relevance preferably obtained using cognitive interview methods, and application of defined summary indicators.

Abstract Background Health literacy (HL) is defined as the knowledge and competences of people to meet the complex demands of health in modern society. It is an important factor in ensuring positive health outcomes, yet Iceland is one of many countries with limited knowledge of HL and no valid HL measurement. The aim of this study was to translate the European Health Literacy Survey Questionnaire- short version (HLS-EU-Q16) into Icelandic, adapt the version, explore its psychometric properties and establish preliminary norms. Methods The HLS-EU-Q16 translation model included three steps: 1) translation-back-translation of HLS-EU-Q16 including specialists’ review (n = 6); 2) cognitive interviewing of lay people (n = 17); and 3) psychometric analysis with survey participants. The HLS-EU-Q16 includes 16 items, with scores ranges from zero (low/no HL) to 16 (high HL). Statistics included were descriptive, internal consistency measured by Cronbach’s α, exploratory factor analysis, and multivariate linear regression. Results After the translation and cognitive interviewing, 11 of the HLS-EU-Q16 items were reworded to adapt the instrument to Icelandic culture while maintaining their conceptual objectives. Survey participants were 251. Internal consistency of the translated and adapted instrument was α = .88. Four factors with eigenvalues > 1.0 explained 62.6% of variance. Principal component analysis with Oblimin rotation presented four latent constructs, “Processing and Using Information from the Doctor” (4 items, α = .77), “Processing and Using Information from the Family and Media” (4 items, α = .85), “Processing Information in Connection to Healthy Lifestyle” (5 items, α = .76), and “Finding Information about Health Problems/Illnesses” (3 items, α = .73). Lower self-rated health was an independent predictor of lower HL (β = −.484, p = .008). Preliminary norms for HL ranged from five to 16 (M 13.7, SD ± 2.6) with 72.5% with sufficient HL (score 13–16), 22% with problematic HL (score 9–12) and 5.5% with inadequate HL (score 0–8). Conclusions The Icelandic version of HLS-EU-Q16 is psychometrically sound, with reasonably clear factor structure, and comparable to the original model. This opens possibilities to study HL in Iceland and compare the results internationally. The translation model introduced might be helpful for other countries where information on HL is missing based on lack of validated tools.

Abstract Background Identification and prevention of mobility limitations in older adults is important to reduce adverse health outcomes. The Life-Space Assessment (LSA) provides a single measure of mobility including environmental and social resources of the older adult. Availability of the LSA for non-English speaking countries is still sparse. Therefore, we translated the LSA into Danish and performed a content validity analysis of the translation in older adults with mobility limitations. Methods After translation into Danish, the Danish version (LSA-DK) was content validated using cognitive interviewing in older mobility limited adults (+ 65) from an outpatient rehabilitation center (n = 12), medical wards at a university hospital (n = 11), and an assisted living facility (n = 7). The interviews were transcribed and analyzed according to the four stages of the Information Processing Model. Based on the analyses, recommendations for changes to the LSA-DK and to the manual were made and presented to the developers of the LSA. Results Consensus was reached on the LSA-DK. Thirty cognitive interviews were carried out. A wide range of sources of error primarily related to the comprehension, memory and decision process were identified. The frequency and type of error sources were most prevalent among assisted living facility informants and included difficulties in defining the geographical extension of neighborhood, town and outside town. The results led to adaptations to the questionnaire and manual to support implementation of the LSA-DK in clinical practice. Conclusions The Life-Space Assessment was translated into Danish and content validated based on cognitive interviews. Adaptations were made to support that the Danish version can be implemented in clinical practice and used in the assessment of mobility in older Danish adults.

Validated self-report measures of post-stroke fatigue are lacking. The Dutch Multifactor Fatigue Scale (DMFS) was translated into Danish, and response process evidence of validity was evaluated. DMFS consists of 38 Likert-rated items distributed on five subscales: Impact of fatigue (11 items), Signs and direct consequences of fatigue (9), Mental fatigue (7), Physical fatigue (6), and Coping with fatigue (5). Response processes to DMFS were investigated using a Three-Step Test-Interview (TSTI) protocol, and data were analyzed using Framework Analysis. Response processes were indexed on the following categories: (i) “congruent,” response processes were related to the subscale construct; (ii) “incongruent,” response processes were not related to the subscale construct; (iii) “ambiguous,” response processes were both congruent and incongruent or insufficient to evaluate congruency; and (iv) “confused,” participants did not understand the item. Nine adults were recruited consecutively 10–34 months post-stroke (median = 26.5) at an outpatient brain injury rehabilitation center in 2019 [five females, mean age = 55 years (SD = 6.3)]. Problematic items were defined as

Objective: The objective of this paper is to provide a practical illustration of methods useful for translating and testing questionnaire instruments for nursing and healthcare to ensure reliability, validity, and appropriateness for the target culture. Methods: We present the process used to create the Japanese version of a well-established quality of life (QOL) instrument, originally developed in American English. The Ferrans and Powers Quality of Life Index (QLI)-Cancer Version III was translated into Japanese by a team of bilingual translators and tested using an iterative process involving cognitive interviewing with monolingual Japanese cancer patients. Results: Discussions among the translation team made it possible to find and resolve linguistic, cultural, and practical issues regarding the translation. Problems stemming from question interpretation and information retrieval were resolved through the cognitive interviewing process. One problem related to response editing could not be remedied with altered phrasing, namely a question referring to the respondents' sex lives. This item was retained in the Japanese version of the QLI as an indispensable component of QOL, particularly in a healthcare context. Conclusions: The final Japanese version captured the intended meaning of the original, and also was culturally appropriate and clearly understood by Japanese cancer patients.

Abstract Background Theories, models, and frameworks (TMF) are foundational for generalizing implementation efforts and research findings. However, TMF and the criteria used to select them are not often described in published articles, perhaps due in part to the challenge of selecting from among the many TMF that exist in the field. The objective of this international study was to develop a user-friendly tool to help scientists and practitioners select appropriate TMF to guide their implementation projects. Methods Implementation scientists across the USA, the UK, and Canada identified and rated conceptually distinct categories of criteria in a concept mapping exercise. We then used the concept mapping results to develop a tool to help users select appropriate TMF for their projects. We assessed the tool’s usefulness through expert consensus and cognitive and semi-structured interviews with implementation scientists. Results Thirty-seven implementation scientists (19 researchers and 18 practitioners) identified four criteria domains: usability, testability, applicability, and familiarity. We then developed a prototype of the tool that included a list of 25 criteria organized by domain, definitions of the criteria, and a case example illustrating an application of the tool. Results of cognitive and semi-structured interviews highlighted the need for the tool to (1) be as succinct as possible; (2) have separate versions to meet the unique needs of researchers versus practitioners; (3) include easily understood terms; (4) include an introduction that clearly describes the tool’s purpose and benefits; (5) provide space for noting project information, comparing and scoring TMF, and accommodating contributions from multiple team members; and (6) include more case examples illustrating its application. Interview participants agreed that the tool (1) offered them a way to select from among candidate TMF, (2) helped them be explicit about the criteria that they used to select a TMF, and (3) enabled them to compare, select from among, and/or consider the usefulness of combining multiple TMF. These revisions resulted in the Theory Comparison and Selection Tool (T-CaST), a paper and web-enabled tool that includes 16 specific criteria that can be used to consider and justify the selection of TMF for a given project. Criteria are organized within four categories: applicability, usability, testability, and acceptability. Conclusions T-CaST is a user-friendly tool to help scientists and practitioners select appropriate TMF to guide implementation projects. Additionally, T-CaST has the potential to promote transparent reporting of criteria used to select TMF within and beyond the field of implementation science.

Abstract Background Noncommunicable diseases represents long term medical conditions, which often puts the patients under enormous demands when following treatment, exposing them to experiencing treatment burden. The Patient Experience with Treatment and Self-Management (PETS) questionnaire was developed as a patient-reported measure to identify treatment burden of chronic illness, using modern measurement theory and tested in a variety of settings. Developed in English, this set of measures had not been previously translated into Norwegian. The objective of this study was to develop a Norwegian version of the PETS and to pretest the translated measures through a cognitive debriefing methodology. Methods A rigorous translation approach was applied, guided by Functional Assessment of Chronic Illness Therapy methodology. Bilingual teams from Norway and the United States reviewed the translation to develop a provisional version, which was evaluated for test content validity with cognitive interviews by probing 12 native Norwegian patients with noncommunicable diseases. The interviews applied both concurrent and retrospective verbal probing techniques, guided by a question route. Audio-recorded interviews were transcribed verbatim and analysed using systematic text condensation. Results Assessment of translatability identified the need for cultural adaptation on several core words, balanced with the need to keep close to the original literal meaning. Seven patients with colorectal cancer and five patients with heart failure participated in cognitive testing of the Norwegian version of the PETS. The analytical process of the cognitive interviews identified two emergent main themes, ‘comprehension and readability’ and ‘relevance of the PETS’, with seven corresponding subthemes. Most items, response options and instructions were well understood by the patients. Revisions were made concerning cultural relevance. Conclusions PETS items were semantically equivalent to the original. The patients with colorectal cancer and heart failure were able to comprehend the PETS and found it to express their experience with treatment burden in chronic illness. Future work will focus on psychometric construct validation and reliability testing of the PETS.

Abstract Background In Germany, there are no measurement tools to assess the general health literacy of adolescents. The aim of the study “Measurement of Health Literacy Among Adolescents” (MOHLAA) is to develop such a tool for use among adolescents aged 14–17. The German version of the European Health Literacy Survey Questionnaire (HLS-EU-Q47-GER) served as a blueprint for the development of the tool. The present study examined the extent to which the HLS-EU-Q47-GER can be applied to the measurement of general health literacy in adolescents. Methods The applicability of the HLS-EU-Q47-GER for adolescents was tested qualitatively using cognitive interviewing (CI). Purposive sampling was used to achieve an equal distribution of participants regarding age groups, educational backgrounds and gender. CI was standardized on the basis of an interview guide. Verbal probing and the retrospective think-aloud technique were applied. The interviews were audio-recorded, transcribed and analyzed using the criteria of theory-based analysis, which were derived from the model of cognitive processes. The analysis focused on identifying terms and questions that were difficult to understand and on scrutinizing the extent to which the content of the items is appropriate for assessing adolescents’ health literacy. Results Adolescent respondents were unfamiliar with some terms of the HLS-EU-Q47-GER or provided heterogeneous interpretations of the terms. They had limited or no experience regarding some health-related tasks in health care and disease prevention that are addressed by HLS-EU-Q-items. A few items seemed to be too “difficult” to answer due to a high abstraction level or because they lacked any reference to the everyday lives of youth. Despite comprehension problems with some of the HLS-EU items, the respondents assessed the covered health-related tasks as “very easy” or “fairly easy”. CI stressed the importance of interpersonal agents, especially parents, in helping adolescents understand and judge the reliability of health information. Conclusions The results of CI indicated that the applicability of the HLS-EU-Q47-GER to the measurement of general health literacy among adolescents aged 14–17 is limited. In order to prevent biased data, some items of the questionnaire should be adjusted to adolescents’ state of development and experiences with health care and disease prevention.

One of the core components of the TRAPD (Translation, Review, Adjudication, Pretesting and Documentation) team approach to translation in survey research is pretesting. Cognitive interviewing is increasingly being used for pretesting survey questionnaires adapted to different populations. Exploring the issue of question sensitivity is particularly relevant when adapting a questionnaire to a population different than the one for which it was designed. However, little guidance exists on the use of cognitive interviewing, and specifically, the types of verbal probes, to elicit respondent feedback on question sensitivity. In preparation for the Saudi National Mental Health Survey, cognitive interviewing was carried out to pretest the Arabic version of the World Mental Health survey instrument (CIDI 3.0). Different types of cognitive probes: proactive direct, proactive indirect and general probes were randomly assigned to survey questions to investigate differences in the feedback elicited by each type of probe. Findings suggest that different types of cognitive probes that are designed to explore perceived sensitivity of the survey questions elicit different amounts and types of feedback. An indirect cognitive probe identified a topic to be sensitive in more instances than a direct probe or a general probe. A general probe, on the other hand, elicited more non-codable feedback especially when paired with a survey question that asks about a more abstract concept such as the respondent’s feelings.

Abstract Background Self-rated health (SRH) is widely used to measure subjective health. Yet it is unclear what underlies health ratings, with implications for understanding the validity of SRH overall and across sociodemographic characteristics. We analyze participants’ explanations of how they formulated their SRH answer in addition to which health factors they considered and examine group differences in these processes. Methods Cognitive interviews were conducted with 64 participants in a convenience quota sample crossing dimensions of race/ethnicity (white, Latino, black, American Indian), gender, age, and education. Participants rated their health then described their thoughts when answering SRH. We coded participants’ answers in an inductive, iterative, and systematic process from interview transcripts, developing analytic categories (i.e., themes) and subdimensions within. We examined whether the presence of each dimension of an analytic category varied across sociodemographic groups. Results Our qualitative analysis led to the identification and classification of various subdimensions of the following analytic categories: types of health factors mentioned, valence of health factors, temporality of health factors, conditional health statements, and descriptions and definitions of health. We found differences across groups in some types of health factors mentioned—corresponding, conflicting, or novel with respect to prior research. Furthermore, we also documented various processes through which respondents integrate seemingly disparate health factors to formulate an answer through valence and conditional health statements. Finally, we found some evidence of sociodemographic group differences with respect to types of health factors mentioned, valence of health factors, and conditional health statements, highlighting avenues for future research. Conclusion This study provides a description of how participants rate their general health status and highlights potential differences in these processes across sociodemographic groups, helping to provide a more comprehensive understanding of how SRH functions as a measure of health.

Abstract Background To expand our clinical and scientific knowledge about holistic outcomes within palliative care, there is a need for agreed-upon patient-reported outcome measures. These patient-reported outcome measures then require translation and cultural adaptation, either from country-specific languages to English, or the other way around. The aim of this study was to translate and cross-culturally adapt the Integrated Palliative care Outcome Scale (IPOS) to the Swedish care context. Methods Swedish versions of IPOS Patient and IPOS Staff were developed and culturally adapted using recommended guidelines including cognitive interviews with patients (n = 13) and staff (n = 15) from different care contexts including general and specialised palliative care. Results The comprehension and judgement difficulties identified in the pre-final patient and staff versions were successfully solved during the cognitive interviewing process. IPOS was well accepted by both patients and staff, none of the questions were experienced as inappropriate, and all questions were judged important. Conclusions In this study, we translated and culturally adapted the patient and staff versions of IPOS, and demonstrated face and content validity and acceptability of the scale through cognitive interviewing with patients and staff within residential care facility, surgical and specialised palliative home care units. Cognitive interviewing in parallel with patients and staff in rounds, with tentative analysis in between, was a suitable method for identifying and solving challenges with comprehension and evaluation in the pre-final version of IPOS. The Swedish IPOS is now available for use in a variety of clinical care settings.

Background: A questionnaire specific to community pharmacy characteristics, such as staffing models for clinical activities and business operations, does not exist. As community pharmacy practice expands, it is important to characterize how pharmacies are changing for outcomes research. The aim of this study was to conduct cognitive interviewing with community pharmacists to gain feedback on the formatting, readability, and content of items measuring community pharmacy characteristics to develop such a questionnaire. Methods: National surveys and previously developed survey work were reviewed to identify the following question categories: business operations, human resource management, division of clinical responsibilities, technology, and enhanced services. Questions for each domain were drafted and assessed for applicability across different states and level of importance by researchers in 3 different states. Using the “think aloud” method of cognitive interviewing to evaluate clarity in instructions, question items and response entry, an iterative process was established that included 3 rounds of interviews with discussion and modifications made by the research team between each round. Results: A total of thirteen cognitive interviews across 3 rounds were conducted via telephone and lasted between 30 and 60 minutes. Time for participant pharmacists to complete the questionnaire ranged from 12 minutes to 30 minutes. The interviews revealed areas of ambiguity, and missing response options for the variety of business structures. The question categories with the most problematic items were business operations, human resource management, and division of clinical responsibilities. Conclusion: Using cognitive interviewing, a community pharmacy questionnaire focusing on operational characteristics was developed. Future research is warranted to test the organizational characteristics defined in this paper with a larger sample size representing multiple states. Article Type: Original Research

While racism is widely recognised as a complex social phenomenon, the basis for defining and identifying everyday racism from a lay perspective is not well understood. This exploration of factors used to frame everyday racism draws on seven cognitive interviews and four focus groups conducted in November 2010 and January 2011 with Australian adults predominantly from Anglo ethnic and cultural backgrounds. The study reveals lay theorising centring on tropes of intentionality, effect of speech, relationality and acceptability. Participants were more likely to think of racism as having negative, overtly offensive and emotional connotations. Racialised speech that was not considered to be blatantly racist was more contested, with participants engaging in complex theorising to determine whether or not such speech constituted racism. The study also highlights the potential of qualitative research to inform survey development as an unobtrusive method for in-depth participant reflection. The ambiguous nature of everyday racism demonstrated in this paper indicates a need to foster more nuanced lay understandings of racism that encompass the subtle, rational and complementary expressions that can be situated within institutions and society.

Participatory approaches to environmental research and decision-making require that all social stakeholders are involved from the onset of the debate. In such a setting, communication among different expertise is crucial, but language and technicalities may represent a barrier. In the clinical setting, decisions regarding treatment preferences may be influenced by the summary statistics used, but, according to the literature, no study has compared different statistical indicators for risk communication in environmental epidemiology. In this paper, we report on the qualitative results of the cognitive interviews conducted for assessing two questionnaires devoted to investigating risk perception when selected epidemiological results are communicated, by using different statistical indicators of health impact and uncertainty. The initial questionnaires were tested on 15 people residing in the high environmental risk area of Livorno (Italy). Cognitive interviewing led to substantial revision of the initial drafts. Moreover, it highlighted the difficulty of communicating statistical uncertainty and the need to account for the complex interaction between mathematical skills, affective factors and individual a priori knowledge on environmental risk perception.

Abstract Background Validated instruments collecting data on health-related resource use are lacking, but required, for example, to investigate predictors of healthcare use or for health economic evaluation. The objective of the study was to develop, test and refine a questionnaire collecting data on health-related resource use and expenditure in patients with diabetes. Methods The questionnaire was tested in 43 patients with diabetes mellitus types 1 and 2 in Germany. Response behaviour suggestive of problems with questions (item non-response, request for clarification, comments, inadequate answer, “don’t know”) was systematically registered. Cognitive interviews focusing on information retrieval and comprehension problems were carried out. Results Many participants had difficulties answering questions pertaining to frequency of visits to the general practitioner (26%), time spent receiving healthcare services (39%), regular medication currently taken (35%) and out of pocket expenditure on medication (42%). These difficulties seem to result mainly from poor memory. A number of comprehension problems were established and relevant questions were revised accordingly. Conclusion The questionnaire on health-related resource use and expenditure for use in diabetes research in Germany was developed and refined after careful testing. Ideally, the questionnaire should be externally validated for different modes of administration and recall periods within a variety of populations.

Abstract Background Available measures of patient-reported outcomes for complementary and alternative medicine (CAM) inadequately capture the range of patient-reported treatment effects. The Self-Assessment of Change questionnaire was developed to measure multi-dimensional shifts in well-being for CAM users. With content derived from patient narratives, items were subsequently focused through interviews on a new cohort of participants. Here we present the development of the final version in which the content and format is refined through cognitive interviews. Methods We conducted cognitive interviews across five iterations of questionnaire refinement with a culturally diverse sample of 28 CAM users. In each iteration, participant critiques were used to revise the questionnaire, which was then re-tested in subsequent rounds of cognitive interviews. Following all five iterations, transcripts of cognitive interviews were systematically coded and analyzed to examine participants' understanding of the format and content of the final questionnaire. Based on this data, we established summary descriptions and selected exemplar quotations for each word pair on the final questionnaire. Results The final version of the Self-Assessment of Change questionnaire (SAC) includes 16 word pairs, nine of which remained unchanged from the original draft. Participants consistently said that these stable word pairs represented opposite ends of the same domain of experience and the meanings of these terms were stable across the participant pool. Five pairs underwent revision and two word pairs were added. Four word pairs were eliminated for redundancy or because participants did not agree on the meaning of the terms. Cognitive interviews indicate that participants understood the format of the questionnaire and considered each word pair to represent opposite poles of a shared domain of experience. Conclusions We have placed lay language and direct experience at the center of questionnaire revision and refinement. In so doing, we provide an innovative model for the development of truly patient-centered outcome measures. Although this instrument was designed and tested in a CAM-specific population, it may be useful in assessing multi-dimensional shifts in well-being across a broader patient population.