Search

This article discusses how an analogy between algorithms and bureaucratic decision-making could help conceptualize error management in algorithmic systems. It argues that a view of algorithms as irreflexive bureaucratic processes is insufficient as an account of errors in complex public sector contexts, where algorithms operate jointly with other organizational work practices. To conceptualize such contexts, the article proposes that algorithms could be viewed as analogous to work routines in bureaucratic organizations. Doing so helps clarify that algorithmic irreflexivity becomes problematic when the coordination of routine work around automation fails. Thus, also the challenges of error management come to concern the wider context of organized work. This argument is illustrated using known examples from the critical literature on algorithms. Finally, drawing on recent studies in routine dynamics, the article formulates empirical research directions on error management in algorithmic systems. This article discusses how an analogy between algorithms and bureaucratic decision-making could help conceptualize error management in algorithmic systems. It argues that a view of algorithms as irreflexive bureaucratic processes is insufficient as an account of errors in complex public sector contexts, where algorithms operate jointly with other organizational work practices. To conceptualize such contexts, the article proposes that algorithms could be viewed as analogous to more traditional work routines in bureaucratic organizations. Doing so helps clarify that algorithmic irreflexivity becomes problematic when the coordination of routine work around automation fails. Thus, also the challenges of error management come to concern the wider context of organized work. This argument is illustrated using known examples from the critical literature on algorithms. Finally, drawing on recent studies in routine dynamics, the article formulates empirical research directions on error management in algorithmic systems.

The future is likely to see an increase in the public-sector use of automated decision-making systems which employ machine learning techniques. However, there is no clear understanding of how English administrative law will apply to this kind of decision-making. This paper seeks to address the problem by bringing together administrative law, data protection law, and a technical understanding of automated decision-making systems in order to identify some of the questions to ask and factors to consider when reviewing the use of these systems. Due to the relative novelty of automated decision-making in the public sector, this kind of study has not yet been undertaken elsewhere. As a result, this paper provides a starting point for judges, lawyers, and legal academics who wish to understand how to legally assess or review automated decision-making systems and identifies areas where further research is required.

This paper raises the question of whether recent dynamics in the governance of artificial intelligence (AI) and automated decision making (ADM) in the United Kingdom (UK) are paternalistic in nature. An initial introduction sets out relevant background (Section 1), followed by details of the narrative review methods used (Section 2). Our analysis is presented in two parts. The first sets out general observations on UK AI/ADM governance (Section 3), including: an apparent dislocation between relevant measures and harms; and the existence of a trade-off between participation and confidentiality/ secrecy. The second analytical part (Section 4) then suggests three distinct ‘agendas’ in UK AI/ADM governance, demonstrating that both the Industrial Strategy’s approach to AI and recent efforts to develop standards for AI in the UK public sector are paternalistic. This part also shows the dominance of Industrial Strategy and highlighting the significance of the role of the Centre for Data Ethics and Innovation (CDEI). A conclusion (Section 5) then summarises the discussion and proposes specific ways for researchers, civil society actors and policymakers to address the problematic paternalism of current AI/ADM governance in the UK.This paper raises the question of whether recent dynamics in the governance of artificial intelligence (AI) and automated decision making (ADM) in the United Kingdom (UK) are paternalistic in nature. An initial introduction sets out relevant background (Section 1), followed by details of the narrative review methods used (Section 2). Our analysis is presented in two parts. The first sets out general observations on UK AI/ADM governance (Section 3), including: an apparent dislocation between relevant measures and harms; and the existence of a trade-off between participation and confidentiality/ secrecy. The second analytical part (Section 4) then suggests three distinct ‘agendas’ in UK AI/ADM governance, demonstrating that both the Industrial Strategy’s approach to AI and recent efforts to develop standards for AI in the UK public sector are paternalistic. This part also shows the dominance of Industrial Strategy and highlighting the significance of the role of the Centre for Data Ethics and Innovation (CDEI). A conclusion (Section 5) then summarises the discussion and proposes specific ways for researchers, civil society actors and policymakers to address the problematic paternalism of current AI/ADM governance in the UK.

In the EU, different measures have been adopted with regard to the storage and exchange of personal data of third-country nationals for external border controls. Large-scale databases and risk assessment are used to facilitate the entry of those considered as ‘bona fide travelers’ and to identify those considered as a risk of irregular migration or security threat. The purposes of existing databases have been gradually extended, blurring the line between the objectives of immigration control and security and law enforcement. Emphasizing the non-discriminatory approach of data protection and applying criteria from the case-law of the Court of Justice of the European Union (CJEU), this contribution questions the legitimacy of these measures from the perspective of the principles of necessity and proportionality, purpose limitation, and the prohibition of automated-decision making. large-scale databases, interoperability, biometrics, third-country nationals, non-discrimination, data protection, necessity and proportionality, purpose limitation, automated decision-making

The normative attractivity of transparency is beyond compare. No wonder it is one of the main principles in the EU’s General Data Protection Regulation. It also features in a majority of AI ethics codes. Transparency is called for because it is assumed that it will solve the so-called ‘black box problem’ (uncertainty about how inputs translate into outputs in algorithmic systems) and by so doing legitimize automated decision-making (computer-based decisionmaking without human influence; ADM). In this paper, the legitimizing effect of transparency in ADM is discussed. I argue that transparency cannot deliver in its quest to resolve the black box problem. The main claim is that transparency is inherently performative in nature and cannot but be so. This performativity goes against the promise of unmediated visibility, vested in transparency. As demonstrated, when transparency is brought into the context of ADM, its hidden functioning logic becomes visible in a new way.

In this article, we develop the concept of Transparency by Design that serves as practical guidance in helping promote the beneficial functions of transparency while mitigating its challenges in automated-decision making (ADM) environments. With the rise of artificial intelligence (AI) and the ability of AI systems to make automated and self-learned decisions, a call for transparency of how such systems reach decisions has echoed within academic and policy circles. The term transparency, however, relates to multiple concepts, fulfills many functions, and holds different promises that struggle to be realized in concrete applications. Indeed, the complexity of transparency for ADM shows tension between transparency as a normative ideal and its translation to practical application. To address this tension, we first conduct a review of transparency, analyzing its challenges and limitations concerning automated decision-making practices. We then look at the lessons learned from the development of Privacy by Design, as a basis for developing the Transparency by Design principles. Finally, we propose a set of nine principles to cover relevant contextual, technical, informational, and stakeholder-sensitive considerations. Transparency by Design is a model that helps organizations design transparent AI systems, by integrating these principles in a step-by-step manner and as an ex-ante value, not as an afterthought.

This paper aims to present a new application of TOPSIS with an automated decision-making process for the analysis of drinking water. For this purpose, the algorithm was modified with a fuzzy disjunction, and the maximal output values were set to one. The properties of drinking water, such as total dissolved solids, hardness, electrical conductivity, and cost, were the criteria analyzed in this study. These criteria were analyzed with unmodified and modified algorithms. Therefore, the modified TOPSIS was also used to optimize the parameters of the candidates. The appearance of the value of 1.0 in the algorithm’s output was due to the confusion of an individual’s categories of drinking water and undrinkable water. The advantage of this investigation was that, for the first time, it allowed automated decision-making to detect the drinking water in different samples and analyze them according to their characteristics. This would be important in developing new technologies for detecting and analyzing drinking water in the environment. The results of this paper can be applied in materials sciences and engineering.

As the use of digitalization and automated decision-making becomes more common in the public sector, civil servants and clients find themselves in an environment where automation and robot technology can be expected to make dramatic changes. Social service delivery in Trelleborg, Sweden, is the setting for a case study of the goals, policies, procedures, and responses to a change in how social assistance is delivered using automated decision-making. Interviews with politicians and professionals complemented with government documents and reports provide the empirical data for the analysis. Four value positions: Professionalism, Efficiency, Service, and Engagement, are used as the analytical framework. The findings reveal that the new technology in some respects has increased accountability, decreased costs, and enhanced efficiency, in association with a focus on citizen centricity. While the findings establish a congruence among instances of some value positions, a divergence is observed among others. Examples of divergence are professional knowledge vs. automated treatment, a decrease in costs vs. the need to share costs, and citizen trust vs. the lack of transparency. The study confirms the power of applying the value positions lens in e-Government research, As the use of digitalization and automated decision-making becomes more common in the public sector, civil servants and clients find themselves in an environment where automation and robot technology can be expected to make dramatic changes. Social service delivery in Trelleborg, Sweden, is the setting for a case study of the goals, policies, procedures, and responses to a change in how social assistance is delivered using automated decision-making. Interviews with politicians and professionals complemented with government documents and reports provide the empirical data for the analysis. Four value positions: Professionalism, Efficiency, Service, and Engagement, are used as the analytical framework. The findings reveal that the new technology in some respects has increased accountability, decreased costs, and enhanced efficiency, in association with a focus on citizen centricity. While the findings establish a congruence among instances of some value positions, a divergence is observed among others. Examples of divergence are professional knowledge vs. automated treatment, a decrease in costs vs. the need to share costs, and citizen trust vs. the lack of transparency. The study confirms the power of applying the value positions lens in e-Government research.

El objetivo de este trabajo es analizar si el artículo 22 del Reglamento General de Protección de Datos proporciona las herramientas apropiadas para proteger a los individuos ante posibles injerencias en los derechos humanos producidas por decisiones basadas en el tratamiento automatizado, incluida la elaboración de perfiles. The purpose of this dissertation is to analyse whether article 22 of the General Protection Regulation provides appropriate safeguards to protect individuals from possible human rights infringements caused by decisions based solely on automated processing, including profiling. L'objectiu d'aquest treball és analitzar si l'article 22 del Reglament General de Protecció de Dades proporciona les eines apropiades per a protegir als individus davant possibles ingerències en els drets humans produïdes per decisions basades en el tractament automatitzat, inclosa l'elaboració de perfils.

Big data technologies and predictive analysis have changed the policing methods, as well as police management in the United States, since the early 2000s. The fact that predictive policing activities have been primarily deployed in the US is no coincidence. Even if crime in France and Europe is significant, especially in the presence of a high terrorist risk for several years, the United States is seeing the spread of large-scale crime and, above all, very significant armed urban violence.Today, however, predictive policing tools are no longer limited to the United States and are also being introduced in several European Union Member States. Nevertheless, the methods used are not identical. Generally speaking, "predictive policing" consists for the police (in France, police or gendarmerie) in using data and other indicators to calculate the probability of the risk of a crime occurring in the future. It may also be a probability calculation that a person, previously identified due to suspected illegal activities, may take action. Such information is useful, on the one hand, to prevent the commission of offences and, on the other hand, to better deploy police forces. Predictive policing tools can target places or individuals.This study, funded by CHEMI (French Ministère de l'intérieur), aims to provide a technical analysis of predictive policing tools used in the United States and Europe, as well as to measure their impact on the population, taking into account very different legislative frameworks. Recommendations were made to the French Ministère de l'intérieur to ensure the effectiveness of algorithmic tools while limiting the risks of violating the fundamental rights of individuals.; Les technologies des données massives et les analyses prédictives ont fait évoluer les méthodes de maintien de la paix et sécurité publique, ainsi que du management des forces de police aux États-Unis, dès le début des années 2000. Le fait que les activités de police prédictive se soient développées en premier lieu sur le territoire des États-Unis n'est pas l'effet du hasard. Même si la criminalité en France et en Europe n'est pas négligeable, en particulier en présence d'un risque terroriste élevé depuis plusieurs années, les États-Unis voient s'étendre une criminalité de grande ampleur et surtout une violence urbaine armée très significative.Aujourd'hui, les outils de police prédictive ne sont cependant plus l'apanage des États-Unis et sont aussi introduits dans plusieurs États membres de l'Union européenne. Toutefois, les méthodes utilisées ne sont pas identiques. De façon générale, la « police prédictive » consiste pour la police (en France, police ou gendarmerie) à utiliser des données et autres indicateurs pour calculer la probabilité du risque de survenance d'un crime dans le futur. Il peut également s'agir d'un calcul de probabilité qu'une personne, précédemment repérée en raison d'activités suspectées d'illégalité, puisse passer à l'action. De telles informations sont utiles, d'une part, pour prévenir la commission d'infractions et, d'autre part, pour mieux déployer les forces de police. Les outils de police prédictive peuvent cibler des lieux ou individus.Cette étude, financée par le CHEMI (Ministère de l'intérieur français), vise à faire une analyse technique des outils de police prédictive utilisé aux États-Unis et en Europe, ainsi que d'en mesurer l'impact sur la population, en tenant compte de cadres légaux très différents. Des recommandations ont été faites au Ministère de l'intérieur, afin de garantir l'efficacité des outils algorithmiques tout en limitant les risques d'atteinte aux droits fondamentaux des individus.

Big data technologies and predictive analysis have changed the policing methods, as well as police management in the United States, since the early 2000s. The fact that predictive policing activities have been primarily deployed in the US is no coincidence. Even if crime in France and Europe is significant, especially in the presence of a high terrorist risk for several years, the United States is seeing the spread of large-scale crime and, above all, very significant armed urban violence.Today, however, predictive policing tools are no longer limited to the United States and are also being introduced in several European Union Member States. Nevertheless, the methods used are not identical. Generally speaking, "predictive policing" consists for the police (in France, police or gendarmerie) in using data and other indicators to calculate the probability of the risk of a crime occurring in the future. It may also be a probability calculation that a person, previously identified due to suspected illegal activities, may take action. Such information is useful, on the one hand, to prevent the commission of offences and, on the other hand, to better deploy police forces. Predictive policing tools can target places or individuals.This study, funded by CHEMI (French Ministère de l'intérieur), aims to provide a technical analysis of predictive policing tools used in the United States and Europe, as well as to measure their impact on the population, taking into account very different legislative frameworks. Recommendations were made to the French Ministère de l'intérieur to ensure the effectiveness of algorithmic tools while limiting the risks of violating the fundamental rights of individuals.; Les technologies des données massives et les analyses prédictives ont fait évoluer les méthodes de maintien de la paix et sécurité publique, ainsi que du management des forces de police aux États-Unis, dès le début des années 2000. Le fait que les activités de police prédictive se soient développées en premier lieu sur le territoire des États-Unis n'est pas l'effet du hasard. Même si la criminalité en France et en Europe n'est pas négligeable, en particulier en présence d'un risque terroriste élevé depuis plusieurs années, les États-Unis voient s'étendre une criminalité de grande ampleur et surtout une violence urbaine armée très significative.Aujourd'hui, les outils de police prédictive ne sont cependant plus l'apanage des États-Unis et sont aussi introduits dans plusieurs États membres de l'Union européenne. Toutefois, les méthodes utilisées ne sont pas identiques. De façon générale, la « police prédictive » consiste pour la police (en France, police ou gendarmerie) à utiliser des données et autres indicateurs pour calculer la probabilité du risque de survenance d'un crime dans le futur. Il peut également s'agir d'un calcul de probabilité qu'une personne, précédemment repérée en raison d'activités suspectées d'illégalité, puisse passer à l'action. De telles informations sont utiles, d'une part, pour prévenir la commission d'infractions et, d'autre part, pour mieux déployer les forces de police. Les outils de police prédictive peuvent cibler des lieux ou individus.Cette étude, financée par le CHEMI (Ministère de l'intérieur français), vise à faire une analyse technique des outils de police prédictive utilisé aux États-Unis et en Europe, ainsi que d'en mesurer l'impact sur la population, en tenant compte de cadres légaux très différents. Des recommandations ont été faites au Ministère de l'intérieur, afin de garantir l'efficacité des outils algorithmiques tout en limitant les risques d'atteinte aux droits fondamentaux des individus.

New technologies allow tax authorities to carry out faster and automated analysis of large amounts of data, minimising errors and saving time. Some of these technologies enable tax administrations to identify and cluster taxpayers based on the risk of noncompliance. Consequently, “high risk” taxpayers will be audited. The European Union General Data Protection Regulation (GDPR) has introduced new provisions on automated decision making and how individuals can be profiled - technology, such as the one implemented by tax administrations, could present difficulties in this area. Even if profiling and automated decision making in tax matters are included in the broader public interest exception, safeguards to taxpayers’ privacy rights need to be in place.

The aim of this thesis is to gain knowledge of how automated decision-making in Swedish public authorities affects the ability to comply with the Swedish principle of public access to official records. This is done through a Posthumanist analysis of seven qualitative interviews. With the help of the actor-network theory concept of the nonhuman actor the underlying computer programs of automated decision-making is understood as an actor who, in interaction with human actors, affects the public sector´s ability to provide the public with valuable information about how they run their organizations. The interviews were conducted with employees at two Swedish public organizations: The Swedish Board of Student Finance and Swedish Association of Local Authorities and Regions. The respondents were asked what kind of insight in the underlying computer programs for automated decision-making that they themselves and the public thought was needed. The interviews were recorded and transcribed, and provided the material for the analysis. The study shows that the employees share the opinion that insight in the automated decision-making systems are not necessary for the public. As a consequence of this, the system developers are left with exclusive knowledge of how the public sector is making decisions and the decision-making becomes invisible for a majority of the public. This is not in line with the aim of the principle of public access to official records. This is a two years master’s thesis in Archival Science.

This paper analyzes the importance of Indonesia's comprehensive legal framework on automated decision-making empowered by Artificial Intelligence, comparing it to the European Union, the United States, and China. Specifically, this paper inquires about the status quo of the legal protection of automated decision-making In Indonesia. The analysis highlights profiling in an automated decision-making system with the following discussion about personal data protection. In this context, the European Union's member states set out the General Data Protection Regulation (GDPR) that prohibits automated decision-making to a certain extent. In the United States, the practice of automated decision-making is rather usual. Simultaneously, China takes an exceptional measure instead and develops this automation through a social credit system. The analysis concludes that Indonesia has weak legal protection towards personal data and profiling, which later becomes the basis in facilitating automated decision-making. The provision of automated decision-making and profiling is the absolute bare minimum to Indonesia's Personal Data Protection Bill due to insufficient legal certainty. In the end, it is paramount for lawmakers to consider a comprehensive regulation on automated decision-making by adopting the European Union's GDPR framework. KEYWORDS: Artificial Intelligence, Automated Decision-Making, Personal Data Protection.

Digitalna tehnologija je postala nepogrešljiv del vseh področij in vidikov tako zasebnega kot javnega življenja in posega tako v prosti čas kot v sfero podjetništva. Takšne avtomatizirane sisteme uporabljajo banke, delodajalci, zdravstvene ustanove, podjetja idr. Posamezniki so vse pogosteje podvrženi avtomatizaciji, vprašanje pa je, v kolikšni meri se tega zavedajo. Z vnosom zahtevanih podatkov v posamezne obrazce npr. na banki, računalnik sam na podlagi algoritmov odloči o tem, ali je nekdo upravičen da pridobi kredit ali ne. Pri tem se zastavlja vprašanje, v kolikšni meri lahko pride pri takšnem odločanju do diskriminacije na podlagi posameznih vnešenih podatkov, ki jih algoritem ovrednoti različno. Razvoj avtomatizacije in umetne inteligence pa gre celo tako daleč, da lahko pričakujemo avtomatizacijo tudi na sodiščih, vprašanje pa je, v kolikšni meri lahko do tega pride. Mnogi so pri tem zaskrbljeni, kaj avtomatizacija odločanja pomeni za pravnike in ali lahko ogrozi poklic pravnika, saj npr. v nekaterih odvetniških pisarnah v Združenih državah Amerike že uporabljajo sistem algoritmov, ki stranki preko izpolnjenega spletnega obrazca sporoči, ali je v določeni zadevi primeren posvet pri odvetniku in pravno postopanje ali ne. Splošna uredba o varstvu podatkov (GDPR - General Data Protection Regulation), ki je bila sprejeta 27. aprila 2016 in se je pričela uporabljati 25. maja 2018, vsebuje pomembne določbe, ki urejajo področje avtomatiziranega odločanja. Oddelek 4 drugega poglavja ima naslov Pravica do ugovora in avtomatizirano sprejemanje posameznih odločitev, ki v členih 21 in 22 določa, kdaj lahko posameznik ugovarja obdelavi osebnih podatkov in kdaj je mogoče avtomatizirano sprejemanje posameznih odločitev, vključno z oblikovanjem profilov. V skladu z 22. členom Splošne uredbe o varstvu podatkov ima posameznik, na katerega se nanašajo osebni podatki, pravico, da zanj ne velja odločitev, ki temelji zgolj na avtomatizirani obdelavi, vključno z oblikovanjem profilov, ki ima pravne učinke v zvezi z njim ali na podoben način nanj znatno vpliva. Nadalje pa je določeno, da se omejitev iz prvega odstavka ne uporablja, če veljajo določene izjeme. Moja magistrska naloga bo napisana na podlagi predhodno opravljene raziskave o tem, kako se avtomatizacija in avtomatizirano odločanje uporablja v praksi, kje se že uporablja in v katerih postopkih se že uporablja, kakšne so njene prednosti in slabosti, kakšne spremembe na tem področju lahko pričakujemo po pričetku uporabe Splošne uredbe o varstvu podatkov in ali je možna avtomatizacija odločanja na sodiščih ter v kolikšni meri. Digital technology has become significant part in all aspects of both private and public life, not just in leisure but also in business. Automated-decision making systems are used in banking system, healthcare system, in business etc. Individuals are often subject to automated decision-making and the question is to what extent they are aware of it. By entering the required data into individual forms, for example in the bank, the computer itself decides on the basis of algorithm whether someone is entitled to get a loan or not. This raises the question of the extent in which such a decision can lead to discrimination based on individual’s data entered that algorithm evaluates differently. The development of automation and artificial intelligence is even so far that automation can be expected on the courts, but the question is to what extent is this possible. Some people are worried about the impact automated decision-making will have for lawyers and whether it can jeopardise the profession of a lawyer. In some law firms in the United States they are already using algorithmic systems to inform a client, through a completed online form, whether or not a consultation with a lawyer and legal action is appropriate in particular case. The General Data Protection Regulation (GDPR) was adopted on 27 April 2016 and applied on 25 May 2018. GDPR contains important provisions governing the field of automated decision-making. Section 4 of the second chapter has a title “Right to object and automated decision-making” which in Articles 21 and 22 determines when an individual have the right object to the processing of personal data concerning him/her and when the data subject can be subject to a decision reached in automated decision-making. In accordance with Article 22 of the GDPR the data subject has the right not to be subject to the decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her. Furthermore, it is provided that the restriction referred to in paragraph 1 shall not apply in certain exceptions. The master’s thesis focus on the previously conducted research on how automation and automated decision-making are used in practice, where this kind of system is already used and in which procedures. Furthermore, it focus on advantages and disadvantages of such automated decision-making systems and what changes can be expected in this field after the application of the GDPR and whether automated decision-making is possible on the courts and to what extent.

The present article argues that the fact that personal data holds great value, in combination with a lack of transparency in its commercial use, leads to a need for consumer policy that strengthens consumer protection. The widespread practice of user agreements and consent-based regulation of personal data collection is not satisfactory for balancing these information-asymmetric markets. The lack of transparency deriving from the complex and massive datafication of consumers – where consumers are profiled, data is brokered and the algorithmically automated decision-making is opaque – speaks to the need for improved supervision at a more structural level above and beyond the individual consumer's choices, preferably by more active consumer protection authorities.

A survey of the German population addressed attitudes towards scenarios of big data practices, i.e. price discrimination in retail, credit scoring, differentiations in health insurance and in employment, with features of using internet data, automated decision-making, and selling of data. The study analysed behavioural adaptations, protection measures, relations to demographics, personal value orientations, knowledge about computers, and attitudes about privacy and data protection.

The UK government announced in March 2020 that it would create an NHS Covid-19 ‘Data Store’ from information routinely collected as part of the health service. This ‘Store’ would use a number of sources of population data to provide a ‘single source of truth’ about the spread of the coronavirus in England. The initiative illustrates the difficulty of relying on automated processing when making healthcare decisions under the General Data Protection Regulation (GDPR). The end-product of the store, a number of ‘dashboards’ for decision-makers, was intended to include models and simulations developed through artificial intelligence. Decisions made on the basis of these dashboards would be significant, even (it was suggested) to the point of diverting patients and critical resources between hospitals based on their predictions. How these models will be developed, and externally validated, remains unclear. This is an issue if they are intended to be used for decisions which will affect patients so directly and acutely. We have (by default) a right under the GDPR not to be subject to significant decisions based solely on automated decision-making. It is not obvious, at present, whether resource allocation within the NHS could take place in reliance on this automated modelling. The recent A Level debacle illustrates, in the context of education, the risks of basing life-changing decisions on the national application of a single equation. It is worth considering the potential consequences for the health service if the NHS Data Store is used for resource planning as part of the Covid-19 response.