Generating Terror: The Risks of Generative AI Exploitation

With the arrival and rapid adoption of sophisticated deep-learning models such as ChatGPT, there is growing concern that terrorists and violent extremists could use these tools to enhance their operations online and in the real world. Large language models have the potential to enable terrorists to learn, plan, and propagate their activities with greater efficiency, accuracy, and impact than ever before. As such, there is a significant need to research the security implications of these deep-learning models. Findings from this research will prove integral to the development of effective countermeasures to prevent and detect the misuse and abuse of these platforms by terrorists and violent extremists. In this paper, the authors offer an early exploration of how these large language models could be exploited by terrorists or other violent extremists. Specifically, the authors investigated the potential implications of commands that can be input into these systems that eff23ectively 'jailbreak' the model, allowing it to remove many of its standards and policies that prevent the base model23 from providing extremist, illegal, or unethical content. Using multiple accounts, the authors explored the different ways that extremists could potentially utilize five different large language models to support their efforts in training, conducting operational planning, and developing propaganda. The article discusses the potential implications and suggests recommendations for policymakers to address these issues.
'Artificial intelligence poses threats to humanity's survival on par with nuclear warfare and global pandemics... My worst fear is that we, the industry, cause significant harm to the world. I [...]