Inevitable challenges of autonomy: ethical concerns in personalized algorithmic decision-making

Abstract Breakthroughs in Artificial Intelligence (AI) have transformed human decision-making processes. Individuals increasingly rely on algorithms to assist with various tasks such as searching and shopping. However, the widespread use of personalized algorithmic decision-making has raised numerous ethical concerns, specifically its impact on user autonomy. This article examines these concerns and argues that algorithmic decision-making presents several challenges to user autonomy that are difficult to eliminate. These challenges include the fact that algorithms deviate from a user’s authentic self, create self-reinforcing loops that narrow the user’s self, and lead to a decline in the user’s capacities. This article attributes these challenges to the ontological differences between users and AI, as well as their unawareness regarding the ethical risks associated with algorithms, asserting that the sense of autonomy users experience in algorithmic decision-making is merely an illusion. From the perspective of human-AI interaction, it is proposed a strategy that achieves a balance between human insight and AI through a human-in-the-loop approach to mitigate these challenges. This approach maximizes the strengths of AI in processing and analyzing data while ensuring that users retain agency.