Search

Ensuring teacher effectiveness within higher education contexts continues to shape institutional policy discourse and practice. However, there is limited research exploring how assessment practices correspond with teachers' and students' perceptions of teacher effectiveness, particularly in vocational higher education settings. Acknowledging the complexities of assessing vocational competence and the dearth of literature exploring teacher and student views on vocational assessment practices, this study interrogates the perceptions of 13 teachers and 15 students from five Indonesian vocational higher education institutions to better understand what constitutes fair assessment as an aspect of effective teaching. This study examines how teachers and students are positioned within the assessment process and the existing opportunities to make decisions regarding assessment design and processes. It shows (i) discrepancies in teachers' and students' perceptions of student positionality within assessment processes; (ii) blurry lines between ideas of 'assessment contract' and 'assessment practice'; and (iii) contested views on student learning processes and attitudes as attributes of vocational competence. The study illuminates the tensions between students' and teachers' positionality within assessment practices, highlighting the need for increased student participation while maintaining teachers' desire to make assessment decisions. [ABSTRACT FROM AUTHOR]

Abstract Background The Findable, Accessible, Interoperable and Reusable(FAIR) Principles explicitly require the use of FAIR vocabularies, but what precisely constitutes a FAIR vocabulary remains unclear. Being able to define FAIR vocabularies, identify features of FAIR vocabularies, and provide assessment approaches against the features can guide the development of vocabularies. Results We differentiate data, data resources and vocabularies used for FAIR, examine the application of the FAIR Principles to vocabularies, align their requirements with the Open Biomedical Ontologies principles, and propose FAIR Vocabulary Features. We also design assessment approaches for FAIR vocabularies by mapping the FVFs with existing FAIR assessment indicators. Finally, we demonstrate how they can be used for evaluating and improving vocabularies using exemplary biomedical vocabularies. Conclusions Our work proposes features of FAIR vocabularies and corresponding indicators for assessing the FAIR levels of different types of vocabularies, identifies use cases for vocabulary engineers, and guides the evolution of vocabularies.

Background: The Findable, Accessible, Interoperable and Reusable(FAIR) Principles explicitly require the use of FAIR vocabularies, but what precisely constitutes a FAIR vocabulary remains unclear. Being able to define FAIR vocabularies, identify features of FAIR vocabularies, and provide assessment approaches against the features can guide the development of vocabularies. Results: We differentiate data, data resources and vocabularies used for FAIR, examine the application of the FAIR Principles to vocabularies, align their requirements with the Open Biomedical Ontologies principles, and propose FAIR Vocabulary Features. We also design assessment approaches for FAIR vocabularies by mapping the FVFs with existing FAIR assessment indicators. Finally, we demonstrate how they can be used for evaluating and improving vocabularies using exemplary biomedical vocabularies. Conclusions: Our work proposes features of FAIR vocabularies and corresponding indicators for assessing the FAIR levels of different types of vocabularies, identifies use cases for vocabulary engineers, and guides the evolution of vocabularies. [ABSTRACT FROM AUTHOR]

The FAIR principles provide guidance for improving the findability, accessibility, interoperability and reuse of research data and other digital objects. The Agricultural Research Federation (AgReFed) has developed a FAIR implementation consisting of policies, resources and tools to enable FAIR agricultural data in Australia. It prescribes minimum acceptable data standards and an associated set of FAIR metrics for agricultural research data. Existing FAIR assessment tools were examined and found to have limitations in serving the purposes of AgReFed. The AgReFed FAIR assessment tool addresses these needs with novel features to help improve FAIRness of datasets and other digital resources. By providing the ability to assess and subsequently re-assess datasets while also enabling users to add comments, it facilitates building work lists that support and document the improvement in FAIRness. Other innovative features include customisable FAIR metrics, storage of assessment results, a versioning system and inbuilt help resources. In addition to user-friendly reporting of AgReFed standards compliance, the integrated automated F-UJI FAIR assessment API provides a supplementary set of machine-readable FAIR scores. The AgReFed FAIR Assessment Tool has been deployed for public use and released as open source to help data custodians enact FAIR principles in domains and data communities within and beyond Australian agriculture.

A Digital Object (DO) "is a sequence of bits, incorporating a work or portion of a work or other information in which a party has rights or interests, or in which there is value". DOs should have persistent identifiers, meta-data and be readable by both humans and machines. A FAIR Digital Object is a DO able to interact with automated data processing systems (De Smedt et al. 2020) while following the FAIR (Findable, Accessible, Interoperable and Reusable principles) principles (Wilkinson et al. 2016).Although FAIR was originally targeted towards data artifacts, new initiatives have emerged to adapt other research digital resources such as software (Katz et al. 2021) (Lamprecht et al. 2020), ontologies (Poveda-Villalón et al. 2020), virtual research environments and even DOs (Collins et al. 2018). In this paper, we describe the challenges when assessing the level of compliance of a DO with the FAIR principles (i.e., its FAIRness), assuming that a DO contains multiple resources and captures their relationships. We explore different methods to calculate an evaluation score, and we discuss the challeneges and importance of providing explanations and guidelines for authors.FAIR assessment tools There are a growing number of tools used to assess the FAIRness of DOs. Community groups like FAIRassist.org have compiled lists of guidelines and tools for assessing the FAIRness of digital resources. These range from self assessment tools like questionnaires and checklists to semi-automated validators (Devaraju et al. 2021). Examples of automated validation tools include the F-UJI Automated FAIR Data Assessment Tool (Devaraju and Huber 2020), FAIR Evaluator and FAIR Checker for datasets or individual DOs; HowFairIs (Spaaks et al. 2021) for code repositories; and and FOOPS (Garijo et al. 2021) to assess ontologies.When it comes to assessing FDOs, we find two main challenges:Resource score discrepancy: Different FAIR assessment tools for the same type of resource produce different scores. For example, a recent study over datasets showcases differences in scores for the same resource due to how the FAIR principles are interpreted by different authors (Dumontier 2022).Heterogeneous FDO metadata: Validators include tests that explore metadata of the digital object. However, there is no agreed metadata schema to represent FDO metadata, which complicates this operation. In addition, metadata may be specific to a certain domain (De Smedt et al. 2020). To address this challenge, we need i) to agree on minimum common set of metadata to measure the FAIRness of DOs and ii) propose a framework to describe extensions for specialized digital objects (datasets, software, ontologies, VRE, etc.).In (Wilkinson et al. 2019), the authors propose a community-driven framework to assess the FAIRness of individual digital objects. This framework is based on:a collection of maturity indicators,principle compliance tests, anda module to apply those tests to digital resources.The proposed indicators may be a starting point to define which tests are needed for each type of resource (de Miranda Azevedo and Dumontier 2020).Aggregation of FAIR metrics Another challenge is the best way to produce an assessment score for a FDO, independently of the tests that are run to assess it. For example, each of the four dimensions of FAIR (Findable, Accessible, Interoperable and Reusable) usually have a different number of associated assessment tests. If the final score is presented based on the number of tests, then by default some dimensions may have more importance than others. Similarly, not all tests may have the same importance for some specific resources (e.g., in some cases having a license in a resource may be considered more important than having its full documentation).In our work we consider a FDO as an aggregation of resources, and therefore we face the additional challenge of creating an aggregated FAIRness score for the whole FDO. We consider the following aggregation scores:Global score: calculated by formula (see Fig. 1-1). It represents the percentage of total passed tests. It doesn’t take into account the principle to which a test belongs.FAIR average score: calculated by formula (see Fig. 1- 2). It represents the average of the passed tests ratios for each principle plus the ratio of passed tests used to evaluate the Research Object itself.Both metrics are agnostic to the kind of resource analyzed. The score they produce ranges from [0 - 100].Discussion A FDO has metadata records that describe it. Some records are common for all DOs, and others are specific to a DO. This makes it difficult to assess some FAIR principles like "F2: "data are described with rich metadata". Therefore, we believe a discussion of a minimal set of FAIR metadata should be addressed by the community.In addition, a FAIR assessment score can change significantly depending on the formula used for aggregating all metrics. Therefore, it is key to explain to users the method and provenance used to produce such score. Different communities should agree on the best scoring mechanism for their FDOs, e.g., by adding a weight to each principle and figuring out the right number of tests for each principle, which may give more importance to the principles with tests.We believe that the objective of a FAIR scoring system should not be to produce a ranking, but become a mechanism to improve the FAIRness of a FDO.

The scientific community's efforts have increased regarding the application and assessment of the FAIR principles on Digital Objects (DO) such as publications, datasets, or research software. Consequently, openly available automated FAIR assessment services have been working on standardization, such as FAIR enough, the FAIR evaluator or FAIRsFAIR's F-UJI. Digital Competence Centers such as University Libraries have been paramount in this process by facilitating a range of activities, such as awareness campaigns, trainings, or systematic support. However, in practice, using the FAIR assessment tools is still an intricate process for the average researcher. It requires a steep learning curve since it involves performing a series of manual processes requiring specific knowledge when learning the frameworks, disengaging some some researchers in the process.We aim to use technology to close this gap and make this process more accessible by bringing the FAIR assessment to the researcher's profiles. We will develop "The FAIR extension", an open-source, user-friendly web browser extension that allows researchers to make FAIR assessment directly at the web source. Web browser extensions have been an accessible digital tool for libraries supporting scholarship (De Sarkar 2015). A remarkable example is the lightweight version of reference managers deployed as a browser service (Ferguson 2019). Moreover, it has been demonstrated that they can be a vehicle for open access, such as Lean Library Browser Extension.The FAIR extension is a service that builds on top of the community-accepted FAIR evaluator APIs, i.e. it does not intend to create yet another FAIR assessment framework from scratch. The objective of the FAIR Digital Objects Framework (FDOF) is for objects published in a digital environment to comply with a set of requirements, such as identifiability, and the use of a rich metadata record (Santos 2021, Schultes and Wittenburg 2019). The FAIR extension will connect via REST-like operations to individual FAIR metrics test endpoints, according to Wilkinson et al. (2018), Wilkinson et al. (2019) and ultimately display the FAIR metrics on the client side (Fig. 1). Ultimately, the user will get FAIR scores of articles, datasets and other DOs in real-time on a web source, such as a scholarly platform or DO repository. With the possibility of creating simple reports of the assessment.It is acknowledged that the development of web-based tools carries some constraints regarding platform versions releases, e.g. Chromium Development Calendar. Nevertheless, we are optimistic about the potential use cases. For example,A student wanting to make use of a DO (e.g. software package), but doesn't know which to choose. The FAIR extension will indicate which one is more FAIR and aid the decision making processA Data steward recommending sourcesA researcher who wants to display all FAIR metrics of her DOs on a research profileA PI that wants to evaluate an aggregated metric for a project.These use cases can be the means to bringing the open source community and FAIR DO interest groups to work together.

A Digital Object (DO) "is a sequence of bits, incorporating a work or portion of a work or other information in which a party has rights or interests, or in which there is value". DOs should have persistent identifiers, meta-data and be readable by both humans and machines. A FAIR Digital Object is a DO able to interact with automated data processing systems (De Smedt et al. 2020) while following the FAIR (Findable, Accessible, Interoperable and Reusable principles) principles (Wilkinson et al. 2016). Although FAIR was originally targeted towards data artifacts, new initiatives have emerged to adapt other research digital resources such as software (Katz et al. 2021) (Lamprecht et al. 2020), ontologies (Poveda-Villalón et al. 2020), virtual research environments and even DOs (Collins et al. 2018). In this paper, we describe the challenges when assessing the level of compliance of a DO with the FAIR principles (i.e., its FAIRness), assuming that a DO contains multiple resources and captures their relationships. We explore different methods to calculate an evaluation score, and we discuss the challeneges and importance of providing explanations and guidelines for authors. FAIR assessment tools: There are a growing number of tools used to assess the FAIRness of DOs. Community groups like FAIRassist.org have compiled lists of guidelines and tools for assessing the FAIRness of digital resources. These range from self assessment tools like questionnaires and checklists to semi-automated validators (Devaraju et al. 2021). Examples of automated validation tools include the F-UJI Automated FAIR Data Assessment Tool (Devaraju and Huber 2020), FAIR Evaluator and FAIR Checker for datasets or individual DOs; HowFairIs (Spaaks et al. 2021) for code repositories; and and FOOPS (Garijo et al. 2021) to assess ontologies. When it comes to assessing FDOs, we find two main challenges: • Resource score discrepancy: Different FAIR assessment tools for the same type of resource produce different scores. For example, a recent study over datasets showcases differences in scores for the same resource due to how the FAIR principles are interpreted by different authors (Dumontier 2022). • Heterogeneous FDO metadata: Validators include tests that explore metadata of the digital object. However, there is no agreed metadata schema to represent FDO metadata, which complicates this operation. In addition, metadata may be specific to a certain domain (De Smedt et al. 2020). To address this challenge, we need i) to agree on minimum common set of metadata to measure the FAIRness of DOs and ii) propose a framework to describe extensions for specialized digital objects (datasets, software, ontologies, VRE, etc.). In (Wilkinson et al. 2019), the authors propose a community-driven framework to assess the FAIRness of individual digital objects. This framework is based on: 1. a collection of maturity indicators, 2. principle compliance tests, and 3. a module to apply those tests to digital resources. The proposed indicators may be a starting point to define which tests are needed for each type of resource (de Miranda Azevedo and Dumontier 2020). Aggregation of FAIR metrics: Another challenge is the best way to produce an assessment score for a FDO, independently of the tests that are run to assess it. For example, each of the four dimensions of FAIR (Findable, Accessible, Interoperable and Reusable) usually have a different number of associated assessment tests. If the final score is presented based on the number of tests, then by default some dimensions may have more importance than others. Similarly, not all tests may have the same importance for some specific resources (e.g., in some cases having a license in a resource may be considered more important than having its full documentation). In our work we consider a FDO as an aggregation of resources, and therefore we face the additional challenge of creating an aggregated FAIRness score for the whole FDO. We consider the following aggregation scores: • Global score: calculated by formula (see Fig. 1-1). It represents the percentage of total passed tests. It doesn't take into account the principle to which a test belongs. • FAIR average score: calculated by formula (see Fig. 1-2). It represents the average of the passed tests ratios for each principle plus the ratio of passed tests used to evaluate the Research Object itself. Both metrics are agnostic to the kind of resource analyzed. The score they produce ranges from [0 - 100]. Discussion: A FDO has metadata records that describe it. Some records are common for all DOs, and others are specific to a DO. This makes it difficult to assess some FAIR principles like "F2: "data are described with rich metadata". Therefore, we believe a discussion of a minimal set of FAIR metadata should be addressed by the community. In addition, a FAIR assessment score can change significantly depending on the formula used for aggregating all metrics. Therefore, it is key to explain to users the method and provenance used to produce such score. Different communities should agree on the best scoring mechanism for their FDOs, e.g., by adding a weight to each principle and figuring out the right number of tests for each principle, which may give more importance to the principles with tests. We believe that the objective of a FAIR scoring system should not be to produce a ranking, but become a mechanism to improve the FAIRness of a FDO. [ABSTRACT FROM AUTHOR]

Fairness in assessment has become increasingly topical and controversial in recent years. Assessment theoreticians are writing more about fairness and assessment practitioners have developed processes and good practice to minimise unfairness. There is also increased scrutiny by students, parents and the wider public – not only of the fairness of assessments themselves and their outcomes, but of their use, notably for selection for college or university. This is in a context of continued awareness of inequalities in society and their impact on education and assessment. And on top of all these questions has been the impact – and the continuing long shadow – of Covid. Can there be fair assessment in such an unfair world? We consider three types of challenge to fair assessment: •Theoretical challenges •Challenges from thinking about social justice •Challenges from the way that statistics were used to award assessment outcomes in 2020 (particularly in England) [ABSTRACT FROM AUTHOR]

The scientific community's efforts have increased regarding the application and assessment of the FAIR principles on Digital Objects (DO) such as publications, datasets, or research software. Consequently, openly available automated FAIR assessment services have been working on standardization, such as FAIR enough, the FAIR evaluator or FAIRsFAIR's F-UJI. Digital Competence Centers such as University Libraries have been paramount in this process by facilitating a range of activities, such as awareness campaigns, trainings, or systematic support. However, in practice, using the FAIR assessment tools is still an intricate process for the average researcher. It requires a steep learning curve since it involves performing a series of manual processes requiring specific knowledge when learning the frameworks, disengaging some some researchers in the process. We aim to use technology to close this gap and make this process more accessible by bringing the FAIR assessment to the researcher's profiles. We will develop "The FAIR extension", an open-source, user-friendly web browser extension that allows researchers to make FAIR assessment directly at the web source. Web browser extensions have been an accessible digital tool for libraries supporting scholarship (De Sarkar 2015). A remarkable example is the lightweight version of reference managers deployed as a browser service (Ferguson 2019). Moreover, it has been demonstrated that they can be a vehicle for open access, such as Lean Library Browser Extension. The FAIR extension is a service that builds on top of the community-accepted FAIR evaluator APIs, i.e. it does not intend to create yet another FAIR assessment framework from scratch. The objective of the FAIR Digital Objects Framework (FDOF) is for objects published in a digital environment to comply with a set of requirements, such as identifiability, and the use of a rich metadata record (Santos 2021, Schultes and Wittenburg 2019). The FAIR extension will connect via REST-like operations to individual FAIR metrics test endpoints, according to Wilkinson et al. (2018), Wilkinson et al. (2019) and ultimately display the FAIR metrics on the client side (Fig. 1). Ultimately, the user will get FAIR scores of articles, datasets and other DOs in real-time on a web source, such as a scholarly platform or DO repository. With the possibility of creating simple reports of the assessment. It is acknowledged that the development of web-based tools carries some constraints regarding platform versions releases, e.g. Chromium Development Calendar. Nevertheless, we are optimistic about the potential use cases. For example, 1. A student wanting to make use of a DO (e.g. software package), but doesn't know which to choose. The FAIR extension will indicate which one is more FAIR and aid the decision making process 2. A Data steward recommending sources 3. A researcher who wants to display all FAIR metrics of her DOs on a research profile 4. A PI that wants to evaluate an aggregated metric for a project. These use cases can be the means to bringing the open source community and FAIR DO interest groups to work together. [ABSTRACT FROM AUTHOR]

Empirical and theoretical studies have highlighted the need to investigate the implications of the introduction of the issue of justice in education. However, little is known about the specific field of learning assessment and about the possible enhancement for assessment processes when inspired by discourses about justice, equity and fairness. What does it mean to rethink assessment through the lens of justice? The present paper aims to uncover key information related to this issue with the aim to provide greater understanding about how to build more equitable assessment practices. The PRISMA guidelines were adopted. Internet-based bibliographic searches were conducted via 2 major electronic databases (ERIC and Education Source) to access studies examining the association between the issues of justice, fairness and equity in assessment. A total of 26 empirical studies meeting the inclusion criteria were identified. The studies reported the attention to both cultures and practices related to fairness in assessment, as well as a focus on addressing diversity in the classroom moving away from a model of adjustments and reasonable accommodations towards an equitable and universal assessment. Additional research is important to clarify these issues and an important effort should be made to construct better assessment practices based on students and teachers' perceptions of justice. [ABSTRACT FROM AUTHOR]

Multiple-choice tests do not assess examinees’ knowledge in accord with reality. In fact, the partial knowledge of examinees is not assessed. Providing a new approach to the assessment of reading comprehension in the framework of fuzzy logic, this study aims to measure this partial knowledge. In this approach, participants have to choose as many correct options as there are considering the stem. Therefore, the correct answer to each question can range from one option to all options. For the first session, an expository and an argumentative genre, and for the second session, the reading section of a TOEFL test was used. The results showed that the approach is fairer as it considers the partial knowledge of the examinees while in other common multiple-choice tests this is ignored. Also, the use of idea units as the units comprising a text gives us clues regarding the degree of difficulty of different parts of a text as well as clues about why misunderstanding may occur of the same text among different people.

The article describes the principles of development of psychological tests and achievement tests that are referred to as «universal design evaluation». It reveals the main elements of this approach. The authors introduce the concept of accommodation as a shift in testing procedure without changing the measurable construct. They illustrate how to create a test in the paradigm of the universal evaluation design through the development of accommodation of testing procedures for children with reading difficulties.

Copyright of GIST: Education & Learning Research Journal is the property of Institucion Universitaria Colombo Americana (UNICA) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Classrooms are the most diverse and complex social structure for teaching-learning process where teachers practice differentiated instructions, assess students with fairness and interact with just. Sans doubt, potency in teaching always lies with teachers' equity based interaction, assessment and pedagogy. Quality of student-teacher interaction is vital and fundamental to understand students' engagement in classroom, fulfillment of students differentiated learning needs and their assessment that does justice with them. Teachers are considered as a catalytic agent who can play a pivotal role in students' classroom to narrow the achievement gaps among students by bringing equity in classrooms. The purpose of the study undertaken was to examine teachers' differentiated instructions, student-teacher fair interaction and fair assessment during teaching-learning process in classrooms. The study was quantitative and descriptive in nature and data were collected through self-developed questionnaire administered on primary school teachers. By using convenience sampling, 300 teachers were selected as sample from four tehsils of district Sargodha. The study found that albeit, teachers in primary schools are using differentiated instructions but not to the degree of satisfaction. They need more knowledge and orientation to understand the need and importance for using differentiated instructions in their classrooms. Also the study found that teachers are less fair in their assessment of students as well as do less justice with students while interacting with them in classrooms and schools. [ABSTRACT FROM AUTHOR]