Search

Key Points This article argues that while there is great promise in the everyday automation of higher education to create benefits for students, efficiencies for instructors, and cost savings for institutions, it is important to critically consider how AI-based assessment will transform the role of teachers and the relationship between teachers and students. The focus of the work is on exploring whether and to what extent the requirements set forth in Article 22 of the General Data Protection Regulation (GDPR) apply within the context of AI-based automatic assessment systems, in particular the legal obligation to ensure that a teacher remains in the loop, for example being capable of overseeing and overriding decisions when necessary. Educational judgments involving automatic assessments frequently occur in a complicated decision-making environment that is framed by institutional processes which are multi-step, hierarchical, and bureaucratic. This complexity makes it challenging to determine whether the output of an AI-based automatic assessment system represents an ‘individual decision’ about a data subject within the meaning of Article 22. It is also unclear whether AI-based assessments involve decisions based ‘solely’ on automatic processing or whether teachers provide decisional support, excluding the application of Article 22. According to recent enforcement decisions, human oversight is entangled with institutional procedures and safeguards as well as system design.

To contribute to the emerging discipline of Responsible Artificial Intelligence (AI), this paper seeks to determine in more detail what responsibility means within the context of the deployment of AI in the Higher Education (HE) context. More, specifically, it seeks to disentangle the boundaries of legal responsibilities within a complex system of humans and technology to understand more clearly who is responsible and for what under the law when it comes to use of facial recognition technology (FRT) in this context. The focus of the paper is on examining the critical role and distinct nature of Ed Tech in providing FRT to the HE. Apply relational ethics theory, it asks what the legal obligations of Ed Tech product and service developers (private organizations) are in relation to the universities (public and private authorities involved in teaching and research), teachers, students and other stakeholders who utilize these AI-driven tools.

This article explores the specific roles and responsibilities of technology providers when it comes to implementing Data Protection by Design (“DPbD”) and Data Protection by Default (“DPbDf”). As an example, it looks at the Education Technology (“Ed Tech”) sector and the complexities of the supply chains that exist therein to highlight that, in addition to the Higher Education (“HE”) institutions that procure products and services for advancing teaching and learning, Ed Tech vendors may also have responsibility and liability for the processing of student’s personal data. Ultimately, this paper asks whether there are any legal gaps, ambiguities, or normative conflicts to the extent that technology providers can have responsibility in contemporary data processing activities yet escape potential liability where it concerns issues of General Data Protection Regulation (“GDPR”) compliance. This paper argues that there is befuddlement concerning the determination of which parties are responsible for meeting DPbD and DPbDf obligations, as well as with regards to the extent of this responsibility. In some cases, an Ed Tech provider is a controller or processor in practice together with a HE institution, yet, in others it, may not have any legal responsibility to support the development of privacy and data-protection preserving systems, notwithstanding the fact it might be much more knowledgeable than a HE institution that has procured the Ed Tech product or service about the state-of-the art of the technology. Even in cases where it is clear that an Ed Tech provider does have responsibility as a controller or processor, it is unclear how it should share DPbD obligations and coordinate actions with HE institutions, especially when the Ed Tech supplier may only be involved in a limited way or at a minor phase in the processing of student data. There is an urgent need to recognize the complex, interdependent, and nonlinear context of contemporary data processing where there ex

In this position paper, we have used Alan Cooper's persona technique to illustrate the utility of audio- and video-based AAL technologies. Therefore, two primary examples of potential audio- and video-based AAL users, Anna and Irakli, serve as reference points for describing salient ethical, legal and social challenges related to use of AAL. These challenges are presented on three levels: individual, societal, and regulatory. For each challenge, a set of policy recommendations is suggested.

Main Findings: AI systems are increasingly being used to shift decisions made by humans over to automated systems, potentially limiting the space for democratic participation. The risk that AI erodes democracy is exacerbated where most people are excluded from the ownership and production of AI technologies that will impact them. AI learns through datasets but, very often, that data excludes key parts of the population. Where marginalized groups are considered, datasets often contain derogatory terms, or exclude explanatory contextual information, that is hard to accurately categorise in a format that AI can process. Resulting biases within AI design raise concerns as to the quality and representativeness of AI-based decisions and their impact on society. There is very little two-way communication between the developers and users of AI-technologies such that the latter function only as personal data providers. Being largely excluded from the development of AI’s role in human decision-making, everyday individuals may feel more marginalized and disinterested in building a healthy and sustainable society. Yet, AI’s capacity for seeing patterns in big data provides new ways to reach parts of the population excluded from traditional policymaking. It can serve to identify structural discrimination and include information from those otherwise ignored in important decisions. AI could enhance public participation by both providing decision-makers with better data and helping to communicate complex decisions – and their consequences – to wider parts of the population., The section titled "The Participation Paradox in the Politics of AI" (pages 7-8) details the main findings of a workshop organised by the authors with a range of experts. Roundtable experts included (please note that the text does not necessarily reflect the views of everyone listed):Malvika Sharan, The Turing Institute, UK; Pedro Sanches, Umeå University, Sweden; ​Sunny Dosanjh, Deloitte MCS Limited​, UK; Aleks Berditchevskaia, NESTA, UK;​Henrik Björklund, Umeå University, Sweden; ​Birgit Schippers, University of Strathclyde Glasgow, UK​; Rachel Foley, DeepMind, UK/USA; Ratidzo Njagu, Kunashe Foundation, Zimbabwe.

This paper explores the extent to which technology providers are responsible to end users for embedding data protection rules in the AI systems they design and develop, so as to safeguard the fundamental rights to privacy and data protection. The main argument set forth is that a relational rationale, requiring a broader range of actors in the supply chain to share legal responsibility for Data Protection by Design (DPbD) is better suited to address infringements to these fundamental rights than the current model that assigns responsibility mainly to the data controller or data processor. Reconceptualizing the law in a more future-oriented, relational, and distributed way would make it possible to adapt legal rules – including those within the GDPR and the continuously evolving EU acquis – to the complex reality of technology development, at least partly addressing the responsibility gap in DPbD. A future-oriented conception of responsibility would require technology providers to adopt more proactive approaches to DPbD, even where they are unlikely to qualify as a controller. A relational approach to DPbD would require technology providers to bear greater responsibilities to those individuals or groups that are affected by their design choices. A distributed approach to DPbD would allow for downstream actors in the supply chain to bear part of the legal responsibility for DPbD by relying on legal requirements that are applicable to various actors in the supply chain supporting DPbD such as those found in contract law, liability law, and the emerging EU acquis governing AI, data, and information security.

Active and Assisted Living (AAL) technologies and services are a possible solution to address the crucial challenges regarding health and social care resulting from demographic changes and current economic conditions. AAL systems aim to improve quality of life and support independent and healthy living of older and frail people. AAL monitoring systems are composed of networks of sensors (worn by the users or embedded in their environment) processing elements and actuators that analyse the environment and its occupants to extract knowledge and to detect events, such as anomalous behaviours, launch alarms to tele-care centres, or support activities of daily living, among others. Therefore, innovation in AAL can address healthcare and social demands while generating economic opportunities. Recently, there has been far-reaching advancements in the development of video-based devices with improved processing capabilities, heightened quality, wireless data transfer, and increased interoperability with Internet of Things (IoT) devices. Computer vision gives the possibility to monitor an environment and report on visual information, which is commonly the most straightforward and human-like way of describing an event, a person, an object, interactions and actions. Therefore, cameras can offer more intelligent solutions for AAL but they may be considered intrusive by some end users. The General Data Protection Regulation (GDPR) establishes the obligation for technologies to meet the principles of data protection by design and by default. More specifically, Article 25 of the GDPR requires that organizations must "implement appropriate technical and organizational measures [...] which are designed to implement data protection principles [...] , in an effective manner and to integrate the necessary safeguards into [data] processing.” Thus, AAL solutions must consider privacy-by-design methodologies in order to protect the fundamental rights of those being monitored. Different met

Main Findings: AI systems are increasingly being used to shift decisions made by humans over to automated systems, potentially limiting the space for democratic participation. The risk that AI erodes democracy is exacerbated where most people are excluded from the ownership and production of AI technologies that will impact them. AI learns through datasets but, very often, that data excludes key parts of the population. Where marginalized groups are considered, datasets often contain derogatory terms, or exclude explanatory contextual information, that is hard to accurately categorise in a format that AI can process. Resulting biases within AI design raise concerns as to the quality and representativeness of AI-based decisions and their impact on society. There is very little two-way communication between the developers and users of AI-technologies such that the latter function only as personal data providers. Being largely excluded from the development of AI’s role in human decision-making, everyday individuals may feel more marginalized and disinterested in building a healthy and sustainable society. Yet, AI’s capacity for seeing patterns in big data provides new ways to reach parts of the population excluded from traditional policymaking. It can serve to identify structural discrimination and include information from those otherwise ignored in important decisions. AI could enhance public participation by both providing decision-makers with better data and helping to communicate complex decisions – and their consequences – to wider parts of the population., The section titled "The Participation Paradox in the Politics of AI" (pages 7-8) details the main findings of a workshop organised by the authors with a range of experts. Roundtable experts included (please note that the text does not necessarily reflect the views of everyone listed):Malvika Sharan, The Turing Institute, UK; Pedro Sanches, Umeå University, Sweden; ​Sunny Dosanjh, Deloitte MCS Limited​, UK; Aleks Berditchevskaia, NESTA, UK;​Henrik Björklund, Umeå University, Sweden; ​Birgit Schippers, University of Strathclyde Glasgow, UK​; Rachel Foley, DeepMind, UK/USA; Ratidzo Njagu, Kunashe Foundation, Zimbabwe.

Population aging resulting from demographic changes requires some challenging decisions and necessary steps to be taken by different stakeholders to manage current and future demand for assistance and support. The consequences of population aging can be mitigated to some extent by assisting technologies that can support the autonomous living of older individuals and persons in need of care in their private environments as long as possible. A variety of technical solutions are already available on the market, but privacy protection is a serious, often neglected, issue when using such (assisting) technology. Thus, privacy needs to be thoroughly taken under consideration in this context. In a three-year project PAAL (‘Privacy-Aware and Acceptable Lifelogging Services for Older and Frail People’), researchers from different disciplines, such as law, rehabilitation, human-computer interaction, and computer science, investigated the phenomenon of privacy when using assistive lifelogging technologies. In concrete terms, the concept of Privacy by Design was realized using two exemplary lifelogging applications in private and professional environments. A user-centered empirical approach was applied to the lifelogging technologies, investigating the perceptions and attitudes of (older) users with different health-related and biographical profiles. The knowledge gained through the interdisciplinary collaboration can improve the implementation and optimization of assistive applications. In this paper, partners of the PAAL project present insights gained from their cross-national, interdisciplinary work regarding privacy-aware and acceptable lifelogging technologies.

The objective of Working Group (WG) 4 of the COST Action NET4Age-Friendly is to examine existing policies, advocacy, and funding opportunities and to build up relations with policy makers and funding organisations. Also, to synthesize and improve existing knowledge and models to develop from effective business and evaluation models, as well as to guarantee quality and education, proper dissemination and ensure the future of the Action. The Working Group further aims to enable capacity building to improve interdisciplinary participation, to promote knowledge exchange and to foster a cross-European interdisciplinary research capacity, to improve cooperation and co-creation with cross-sectors stakeholders and to introduce and educate students SHAFE implementation and sustainability (CB01, CB03, CB04, CB05). To enable the achievement of the objectives of Working Group 4, the Leader of the Working Group, the Chair and Vice-Chair, in close cooperation with the Science Communication Coordinator, developed a template (see annex 1) to map the current state of SHAFE policies, funding opportunities and networking in the COST member countries of the Action. On invitation, the Working Group lead received contributions from 37 countries, in a total of 85 Action members. The contributions provide an overview of the diversity of SHAFE policies and opportunities in Europe and beyond. These were not edited or revised and are a result of the main areas of expertise and knowledge of the contributors; thus, gaps in areas or content are possible and these shall be further explored in the following works and reports of this WG. But this preliminary mapping is of huge importance to proceed with the WG activities. In the following chapters, an introduction on the need of SHAFE policies is presented, followed by a summary of the main approaches to be pursued for the next period of work. The deliverable finishes with the opportunities of capacity building, networking and funding that will be

Ambient assisted living (AAL) technologies are increasingly presented and sold as essential smart additions to daily life and home environments that will radically transform the healthcare and wellness markets of the future. An ethical approach and a thorough understanding of all ethics in surveillance/monitoring architectures are therefore pressing. AAL poses many ethical challenges raising questions that will affect immediate acceptance and long-term usage. Furthermore, ethical issues emerge from social inequalities and their potential exacerbation by AAL, accentuating the existing access gap between high-income countries (HIC) and low and middle-income countries (LMIC). Legal aspects mainly refer to the adherence to existing legal frameworks and cover issues related to product safety, data protection, cybersecurity, intellectual property, and access to data by public, private, and government bodies. Successful privacy-friendly AAL applications are needed, as the pressure to bring Internet of Things (IoT) devices and ones equipped with artificial intelligence (AI) quickly to market cannot overlook the fact that the environments in which AAL will operate are mostly private (e.g., the home). The social issues focus on the impact of AAL technologies before and after their adoption. Future AAL technologies need to consider all aspects of equality such as gender, race, age and social disadvantages and avoid increasing loneliness and isolation among, e.g. older and frail people. Finally, the current power asymmetries between the target and general populations should not be underestimated nor should the discrepant needs and motivations of the target group and those developing and deploying AAL systems. Whilst AAL technologies provide promising solutions for the health and social care challenges, they are not exempt from ethical, legal and social issues (ELSI). A set of ELSI guidelines is needed to integrate these factors at the research and development stage.

This paper examines the field of Artificial Intelligence (AI) and Law and offers some broad reflections on its current state. First, the paper introduces the concept of AI, paying particular attention to the distinction between hard and soft AI. Next, it considers how AI can be used to support (or replace!) legal work and legal reasoning. The paper goes on to explore applications of AI in the legal domain and concludes with some critical reflections on the use of AI in the legal context.

Universities around the globe have been profoundly affected by stay-at-home orders, which have required them to close their doors and shift to online teaching and learning. In an effort to avoid delaying or postponing examinations amid the Covid-19 outbreak, many higher-education institutions have turned to online proctoring tools, raising complex questions about how they can ensure the integrity of online assessments while at the same time respect ethical and legal constraints, especially regarding students’ fundamental rights to privacy, data protection and non-discrimination. In particular, universities are increasingly relying on AI-based facial recognition technologies (FRT) that can be used to authenticate remote users that connect from offsite the campus as well as to identify cheating and other dubious behavior throughout the online exam process., Publicerad: 2021-05-05

Lifelogging technologies promise to manage many of the concerns raised by population aging. The technology can be used to predict and prevent disease, provide personalized healthcare, and to give support to formal and informal caregivers. Although lifelogging technologies offer major opportunities to improve efficiency and care in the healthcare setting, there are many aspects of these devices that raise serious privacy concerns that can undercut their use and further development. One way to manage privacy concerns raised by lifelogging technologies is through the application of Privacy by Design, an approach that involves embedding legal rules into information systems at the outset of their development. Many current approaches to Privacy by Design, however, lack methodological rigor, leaving stakeholders perplexed about how to achieve the objectives underlying the concept in practice. This paper will explore ways to develop a Privacy by Design methodology within the context of Ambient Assistive Living (AAL) technologies like lifelogging. It will set forth a concrete, methodological approach towards incorporating privacy into all stages of a lifelogging system's development. The methodology begins with a contextual understanding of privacy, relying on theoretical and empirical studies conducted by experts in humancomputer relations. It then involves an analysis of the relevant black-letter law. A systematic approach as to how to incorporate the requisite legal rules into lifelogging devices is then presented, taking into the account the specific design elements of these kinds of systems.

This paper explores a specific risk-mitigation strategy to reduce privacy concerns in the Internet of Health Things (IoHT): data anonymization. It contributes to the current academic debate surrounding the role of anonymization in the IoHT by evaluating how data controllers can balance privacy risks against the quality of output data and select the appropriate privacy model that achieves the aims underlying the concept of Privacy by Design. It sets forth several approaches for identifying the risk of re-identification in the IoHT as well as explores the potential for synthetic data generation to be used as an alternative method to anonymization for data sharing.

Lifelogging technologies promise to deliver an array of applications that can revolutionise the way that healthcare is currently administered. While lifelogging technologies provide great opportunity to improve efficiency and care in the healthcare settings, the ubiquity and invisibility of lifelogging devices raise critical privacy concerns. One way to cope with the privacy challenges that arise from the use of lifelogging devices is to find technological means to guarantee the protection of personal data. However, because of the nascent nature of the industry, there is a lack of guidance as to how to build lifelogging devices in a way that anticipates data protection concerns. This chapter seeks to explore the concept of Privacy by Design within the context of lifelogging. It uses the core data protection principles set forth in the GDPR to evaluate which aspects of lifelogging have potential for privacy breaches. It then makes suggestions about how the concept of Privacy by Design can be implemented in a way to alleviate those concerns, pointing to various transparency and embedded legal measures available in order to strengthen the data protection rights of individuals within the context of lifelogging.