Skip to search
Skip to main content
Back to Search
Start Over
The Llama 3 Herd of Models
Authors :
Dubey, Abhimanyu Jauhri, Abhinav Pandey, Abhinav Kadian, Abhishek Al-Dahle, Ahmad Letman, Aiesha Mathur, Akhil Schelten, Alan Yang, Amy Fan, Angela Goyal, Anirudh Hartshorn, Anthony Yang, Aobo Mitra, Archi Sravankumar, Archie Korenev, Artem Hinsvark, Arthur Rao, Arun Zhang, Aston Rodriguez, Aurelien Gregerson, Austen Spataru, Ava Roziere, Baptiste Biron, Bethany Tang, Binh Chern, Bobbie Caucheteux, Charlotte Nayak, Chaya Bi, Chloe Marra, Chris McConnell, Chris Keller, Christian Touret, Christophe Wu, Chunyang Wong, Corinne Ferrer, Cristian Canton Nikolaidis, Cyrus Allonsius, Damien Song, Daniel Pintz, Danielle Livshits, Danny Esiobu, David Choudhary, Dhruv Mahajan, Dhruv Garcia-Olano, Diego Perino, Diego Hupkes, Dieuwke Lakomkin, Egor AlBadawy, Ehab Lobanova, Elina Dinan, Emily Smith, Eric Michael Radenovic, Filip Zhang, Frank Synnaeve, Gabriel Lee, Gabrielle Anderson, Georgia Lewis Nail, Graeme Mialon, Gregoire Pang, Guan Cucurell, Guillem Nguyen, Hailey Korevaar, Hannah Xu, Hu Touvron, Hugo Zarov, Iliyan Ibarra, Imanol Arrieta Kloumann, Isabel Misra, Ishan Evtimov, Ivan Copet, Jade Lee, Jaewon Geffert, Jan Vranes, Jana Park, Jason Mahadeokar, Jay Shah, Jeet van der Linde, Jelmer Billock, Jennifer Hong, Jenny Lee, Jenya Fu, Jeremy Chi, Jianfeng Huang, Jianyu Liu, Jiawen Wang, Jie Yu, Jiecao Bitton, Joanna Spisak, Joe Park, Jongsoo Rocca, Joseph Johnstun, Joshua Saxe, Joshua Jia, Junteng Alwala, Kalyan Vasuden Upasani, Kartikeya Plawiak, Kate Li, Ke Heafield, Kenneth Stone, Kevin El-Arini, Khalid Iyer, Krithika Malik, Kshitiz Chiu, Kuenley Bhalla, Kunal Rantala-Yeary, Lauren van der Maaten, Laurens Chen, Lawrence Tan, Liang Jenkins, Liz Martin, Louis Madaan, Lovish Malo, Lubo Blecher, Lukas Landzaat, Lukas de Oliveira, Luke Muzzi, Madeline Pasupuleti, Mahesh Singh, Mannat Paluri, Manohar Kardas, Marcin Oldham, Mathew Rita, Mathieu Pavlova, Maya Kambadur, Melanie Lewis, Mike Si, Min Singh, Mitesh Kumar Hassan, Mona Goyal, Naman Torabi, Narjes Bashlykov, Nikolay Bogoychev, Nikolay Chatterji, Niladri Duchenne, Olivier Çelebi, Onur Alrassy, Patrick Zhang, Pengchuan Li, Pengwei Vasic, Petar Weng, Peter Bhargava, Prajjwal Dubal, Pratik Krishnan, Praveen Koura, Punit Singh Xu, Puxin He, Qing Dong, Qingxiao Srinivasan, Ragavan Ganapathy, Raj Calderer, Ramon Cabral, Ricardo Silveira Stojnic, Robert Raileanu, Roberta Girdhar, Rohit Patel, Rohit Sauvestre, Romain Polidoro, Ronnie Sumbaly, Roshan Taylor, Ross Silva, Ruan Hou, Rui Wang, Rui Hosseini, Saghar Chennabasappa, Sahana Singh, Sanjay Bell, Sean Kim, Seohyun Sonia Edunov, Sergey Nie, Shaoliang Narang, Sharan Raparthy, Sharath Shen, Sheng Wan, Shengye Bhosale, Shruti Zhang, Shun Vandenhende, Simon Batra, Soumya Whitman, Spencer Sootla, Sten Collot, Stephane Gururangan, Suchin Borodinsky, Sydney Herman, Tamar Fowler, Tara Sheasha, Tarek Georgiou, Thomas Scialom, Thomas Speckbacher, Tobias Mihaylov, Todor Xiao, Tong Karn, Ujjwal Goswami, Vedanuj Gupta, Vibhor Ramanathan, Vignesh Kerkez, Viktor Gonguet, Vincent Do, Virginie Vogeti, Vish Petrovic, Vladan Chu, Weiwei Xiong, Wenhan Fu, Wenyin Meers, Whitney Martinet, Xavier Wang, Xiaodong Tan, Xiaoqing Ellen Xie, Xinfeng Jia, Xuchao Wang, Xuewei Goldschlag, Yaelle Gaur, Yashesh Babaei, Yasmine Wen, Yi Song, Yiwen Zhang, Yuchen Li, Yue Mao, Yuning Coudert, Zacharie Delpierre Yan, Zheng Chen, Zhengxing Papakipos, Zoe Singh, Aaditya Grattafiori, Aaron Jain, Abha Kelsey, Adam Shajnfeld, Adam Gangidi, Adithya Victoria, Adolfo Goldstand, Ahuva Menon, Ajay Sharma, Ajay Boesenberg, Alex Vaughan, Alex Baevski, Alexei Feinstein, Allie Kallet, Amanda Sangani, Amit Yunus, Anam Lupu, Andrei Alvarado, Andres Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Andrew Ramchandani, Ankit Franco, Annie Saraf, Aparajita Chowdhury, Arkabandhu Gabriel, Ashley Bharambe, Ashwin Eisenman, Assaf Yazdan, Azadeh James, Beau Maurer, Ben Leonhardi, Benjamin Huang, Bernie Loyd, Beth De Paola, Beto Paranjape, Bhargavi Liu, Bing Wu, Bo Ni, Boyu Hancock, Braden Wasti, Bram Spence, Brandon Stojkovic, Brani Gamido, Brian Montalvo, Britt Parker, Carl Burton, Carly Mejia, Catalina Wang, Changhan Kim, Changkyu Zhou, Chao Hu, Chester Chu, Ching-Hsiang Cai, Chris Tindal, Chris Feichtenhofer, Christoph Civin, Damon Beaty, Dana Kreymer, Daniel Li, Daniel Wyatt, Danny Adkins, David Xu, David Testuggine, Davide David, Delia Parikh, Devi Liskovich, Diana Foss, Didem Wang, Dingkang Le, Duc Holland, Dustin Dowling, Edward Jamil, Eissa Montgomery, Elaine Presani, Eleonora Hahn, Emily Wood, Emily Brinkman, Erik Arcaute, Esteban Dunbar, Evan Smothers, Evan Sun, Fei Kreuk, Felix Tian, Feng Ozgenel, Firat Caggioni, Francesco Guzmán, Francisco Kanayet, Frank Seide, Frank Florez, Gabriela Medina Schwarz, Gabriella Badeer, Gada Swee, Georgia Halpern, Gil Thattai, Govind Herman, Grant Sizov, Grigory Guangyi Zhang Lakshminarayanan, Guna Shojanazeri, Hamid Zou, Han Wang, Hannah Zha, Hanwen Habeeb, Haroun Rudolph, Harrison Suk, Helen Aspegren, Henry Goldman, Hunter Damlaj, Ibrahim Molybog, Igor Tufanov, Igor Veliche, Irina-Elena Gat, Itai Weissman, Jake Geboski, James Kohli, James Asher, Japhet Gaya, Jean-Baptiste Marcus, Jeff Tang, Jeff Chan, Jennifer Zhen, Jenny Reizenstein, Jeremy Teboul, Jeremy Zhong, Jessica Jin, Jian Yang, Jingyi Cummings, Joe Carvill, Jon Shepard, Jon McPhie, Jonathan Torres, Jonathan Ginsburg, Josh Wang, Junjie Wu, Kai U, Kam Hou Saxena, Karan Prasad, Karthik Khandelwal, Kartikay Zand, Katayoun Matosich, Kathy Veeraraghavan, Kaushik Michelena, Kelly Li, Keqian Huang, Kun Chawla, Kunal Lakhotia, Kushal Huang, Kyle Chen, Lailin Garg, Lakshya A, Lavender Silva, Leandro Bell, Lee Zhang, Lei Guo, Liangpeng Yu, Licheng Moshkovich, Liron Wehrstedt, Luca Khabsa, Madian Avalani, Manav Bhatt, Manish Tsimpoukelli, Maria Mankus, Martynas Hasson, Matan Lennie, Matthew Reso, Matthias Groshev, Maxim Naumov, Maxim Lathi, Maya Keneally, Meghan Seltzer, Michael L. Valko, Michal Restrepo, Michelle Patel, Mihir Vyatskov, Mik Samvelyan, Mikayel Clark, Mike Macey, Mike Wang, Mike Hermoso, Miquel Jubert Metanat, Mo Rastegari, Mohammad Bansal, Munish Santhanam, Nandhini Parks, Natascha White, Natasha Bawa, Navyata Singhal, Nayan Egebo, Nick Usunier, Nicolas Laptev, Nikolay Pavlovich Dong, Ning Zhang, Ning Cheng, Norman Chernoguz, Oleg Hart, Olivia Salpekar, Omkar Kalinli, Ozlem Kent, Parkin Parekh, Parth Saab, Paul Balaji, Pavan Rittner, Pedro Bontrager, Philip Roux, Pierre Dollar, Piotr Zvyagina, Polina Ratanchandani, Prashant Yuvraj, Pritish Liang, Qian Alao, Rachad Rodriguez, Rachel Ayub, Rafi Murthy, Raghotham Nayani, Raghu Mitra, Rahul Li, Raymond Hogan, Rebekkah Battey, Robin Wang, Rocky Maheswari, Rohan Howes, Russ Rinott, Ruty Bondu, Sai Jayesh Datta, Samyak Chugh, Sara Hunt, Sara Dhillon, Sargun Sidorov, Sasha Pan, Satadru Verma, Saurabh Yamamoto, Seiji Ramaswamy, Sharadh Lindsay, Shaun Feng, Sheng Lin, Shenghao Zha, Shengxin Cindy Shankar, Shiva Zhang, Shuqiang Wang, Sinong Agarwal, Sneha Sajuyigbe, Soji Chintala, Soumith Max, Stephanie Chen, Stephen Kehoe, Steve Satterfield, Steve Govindaprasad, Sudarshan Gupta, Sumit Cho, Sungmin Virk, Sunny Subramanian, Suraj Choudhury, Sy Goldman, Sydney Remez, Tal Glaser, Tamar Best, Tamara Kohler, Thilo Robinson, Thomas Li, Tianhe Zhang, Tianjun Matthews, Tim Chou, Timothy Shaked, Tzook Vontimitta, Varun Ajayi, Victoria Montanez, Victoria Mohan, Vijai Kumar, Vinay Satish Mangla, Vishal Albiero, Vítor Ionescu, Vlad Poenaru, Vlad Mihailescu, Vlad Tiberiu Ivanov, Vladimir Li, Wei Wang, Wenchen Jiang, Wenwen Bouaziz, Wes Constable, Will Tang, Xiaocheng Wang, Xiaofang Wu, Xiaojian Wang, Xiaolan Xia, Xide Wu, Xilun Gao, Xinbo Chen, Yanjun Hu, Ye Jia, Ye Qi, Ye Li, Yenda Zhang, Yilin Zhang, Ying Adi, Yossi Nam, Youngjin Yu Wang Hao, Yuchen Qian, Yundi He, Yuzi Rait, Zach DeVito, Zachary Rosnbrick, Zef Wen, Zhaoduo Yang, Zhenyu Zhao, Zhiwei
Publication Year :
2024
Abstract
Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.
Details
Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2407.21783
Document Type :
Working Paper