Back to Search Start Over

Why is the winner the best?

Authors :
Eisenmann, Matthias
Reinke, Annika
Weru, Vivienn
Tizabi, Minu Dietlinde
Isensee, Fabian
Adler, Tim J.
Ali, Sharib
Andrearczyk, Vincent
Aubreville, Marc
Baid, Ujjwal
Bakas, Spyridon
Balu, Niranjan
Bano, Sophia
Bernal, Jorge
Bodenstedt, Sebastian
Casella, Alessandro
Cheplygina, Veronika
Daum, Marie
Bruijne, Marleen de
Depeursinge, Adrien
Dorent, Reuben
Egger, Jan
Ellis, David G.
Engelhardt, Sandy
Ganz, Melanie
Ghatwary, Noha
Girard, Gabriel
Godau, Patrick
Gupta, Anubha
Hansen, Lasse
Harada, Kanako
Heinrich, Mattias
Heller, Nicholas
Hering, Alessa
Huaulmé, Arnaud
Jannin, Pierre
Kavur, Ali Emre
Kodym, Oldřich
Kozubek, Michal
Li, Jianning
Li, Hongwei
Ma, Jun
Martín-Isla, Carlos
Menze, Bjoern
Noble, Alison
Oreiller, Valentin
Padoy, Nicolas
Pati, Sarthak
Payette, Kelly
Rädsch, Tim
Rafael-Patiño, Jonathan
Bawa, Vivek Singh
Speidel, Stefanie
Sudre, Carole H.
Wijnen, Kimberlin van
Wagner, Martin
Wei, Donglai
Yamlahi, Amine
Yap, Moi Hoon
Yuan, Chun
Zenk, Maximilian
Zia, Aneeq
Zimmerer, David
Aydogan, Dogu Baran
Bhattarai, Binod
Bloch, Louise
Brüngel, Raphael
Cho, Jihoon
Choi, Chanyeol
Dou, Qi
Ezhov, Ivan
Friedrich, Christoph M.
Fuller, Clifton
Gaire, Rebati Raman
Galdran, Adrian
Faura, Álvaro García
Grammatikopoulou, Maria
Hong, SeulGi
Jahanifar, Mostafa
Jang, Ikbeom
Kadkhodamohammadi, Abdolrahim
Kang, Inha
Kofler, Florian
Kondo, Satoshi
Kuijf, Hugo
Li, Mingxing
Luu, Minh Huan
Martinčič, Tomaž
Morais, Pedro
Naser, Mohamed A.
Oliveira, Bruno
Owen, David
Pang, Subeen
Park, Jinah
Park, Sung-Hong
Płotka, Szymon
Puybareau, Elodie
Rajpoot, Nasir
Ryu, Kanghyun
Saeed, Numan
Shephard, Adam
Shi, Pengcheng
Štepec, Dejan
Subedi, Ronast
Tochon, Guillaume
Torres, Helena R.
Urien, Helene
Vilaça, João L.
Wahid, Kareem Abdul
Wang, Haojie
Wang, Jiacheng
Wang, Liansheng
Wang, Xiyue
Wiestler, Benedikt
Wodzinski, Marek
Xia, Fangfang
Xie, Juanying
Xiong, Zhiwei
Yang, Sen
Yang, Yanwu
Zhao, Zixuan
Maier-Hein, Klaus
Jäger, Paul F.
Kopp-Schneider, Annette
Maier-Hein, Lena
Eisenmann, Matthias
Reinke, Annika
Weru, Vivienn
Tizabi, Minu Dietlinde
Isensee, Fabian
Adler, Tim J.
Ali, Sharib
Andrearczyk, Vincent
Aubreville, Marc
Baid, Ujjwal
Bakas, Spyridon
Balu, Niranjan
Bano, Sophia
Bernal, Jorge
Bodenstedt, Sebastian
Casella, Alessandro
Cheplygina, Veronika
Daum, Marie
Bruijne, Marleen de
Depeursinge, Adrien
Dorent, Reuben
Egger, Jan
Ellis, David G.
Engelhardt, Sandy
Ganz, Melanie
Ghatwary, Noha
Girard, Gabriel
Godau, Patrick
Gupta, Anubha
Hansen, Lasse
Harada, Kanako
Heinrich, Mattias
Heller, Nicholas
Hering, Alessa
Huaulmé, Arnaud
Jannin, Pierre
Kavur, Ali Emre
Kodym, Oldřich
Kozubek, Michal
Li, Jianning
Li, Hongwei
Ma, Jun
Martín-Isla, Carlos
Menze, Bjoern
Noble, Alison
Oreiller, Valentin
Padoy, Nicolas
Pati, Sarthak
Payette, Kelly
Rädsch, Tim
Rafael-Patiño, Jonathan
Bawa, Vivek Singh
Speidel, Stefanie
Sudre, Carole H.
Wijnen, Kimberlin van
Wagner, Martin
Wei, Donglai
Yamlahi, Amine
Yap, Moi Hoon
Yuan, Chun
Zenk, Maximilian
Zia, Aneeq
Zimmerer, David
Aydogan, Dogu Baran
Bhattarai, Binod
Bloch, Louise
Brüngel, Raphael
Cho, Jihoon
Choi, Chanyeol
Dou, Qi
Ezhov, Ivan
Friedrich, Christoph M.
Fuller, Clifton
Gaire, Rebati Raman
Galdran, Adrian
Faura, Álvaro García
Grammatikopoulou, Maria
Hong, SeulGi
Jahanifar, Mostafa
Jang, Ikbeom
Kadkhodamohammadi, Abdolrahim
Kang, Inha
Kofler, Florian
Kondo, Satoshi
Kuijf, Hugo
Li, Mingxing
Luu, Minh Huan
Martinčič, Tomaž
Morais, Pedro
Naser, Mohamed A.
Oliveira, Bruno
Owen, David
Pang, Subeen
Park, Jinah
Park, Sung-Hong
Płotka, Szymon
Puybareau, Elodie
Rajpoot, Nasir
Ryu, Kanghyun
Saeed, Numan
Shephard, Adam
Shi, Pengcheng
Štepec, Dejan
Subedi, Ronast
Tochon, Guillaume
Torres, Helena R.
Urien, Helene
Vilaça, João L.
Wahid, Kareem Abdul
Wang, Haojie
Wang, Jiacheng
Wang, Liansheng
Wang, Xiyue
Wiestler, Benedikt
Wodzinski, Marek
Xia, Fangfang
Xie, Juanying
Xiong, Zhiwei
Yang, Sen
Yang, Yanwu
Zhao, Zixuan
Maier-Hein, Klaus
Jäger, Paul F.
Kopp-Schneider, Annette
Maier-Hein, Lena
Source :
Eisenmann , M , Reinke , A , Weru , V , Tizabi , M D , Isensee , F , Adler , T J , Ali , S , Andrearczyk , V , Aubreville , M , Baid , U , Bakas , S , Balu , N , Bano , S , Bernal , J , Bodenstedt , S , Casella , A , Cheplygina , V , Daum , M , Bruijne , M D , Depeursinge , A , Dorent , R , Egger , J , Ellis , D G , Engelhardt , S , Ganz , M , Ghatwary , N , Girard , G , Godau , P , Gupta , A , Hansen , L , Harada , K , Heinrich , M , Heller , N , Hering , A , Huaulmé , A , Jannin , P , Kavur , A E , Kodym , O , Kozubek , M , Li , J , Li , H , Ma , J , Martín-Isla , C , Menze , B , Noble , A , Oreiller , V , Padoy , N , Pati , S , Payette , K , Rädsch , T , Rafael-Patiño , J , Bawa , V S , Speidel , S , Sudre , C H , Wijnen , K V , Wagner , M , Wei , D , Yamlahi , A , Yap , M H , Yuan , C , Zenk , M , Zia , A , Zimmerer , D , Aydogan , D B , Bhattarai , B , Bloch , L , Brüngel , R , Cho , J , Choi , C , Dou , Q , Ezhov , I , Friedrich , C M , Fuller , C , Gaire , R R , Galdran , A , Faura , Á G , Grammatikopoulou , M , Hong , S , Jahanifar , M , Jang , I , Kadkhodamohammadi , A , Kang , I , Kofler , F , Kondo , S , Kuijf , H , Li , M , Luu , M H , Martinčič , T , Morais , P , Naser , M A , Oliveira , B , Owen , D , Pang , S , Park , J , Park , S-H , Płotka , S , Puybareau , E , Rajpoot , N , Ryu , K , Saeed , N , Shephard , A , Shi , P , Štepec , D , Subedi , R , Tochon , G , Torres , H R , Urien , H , Vilaça , J L , Wahid , K A , Wang , H , Wang , J , Wang , L , Wang , X , Wiestler , B , Wodzinski , M , Xia , F , Xie , J , Xiong , Z , Yang , S , Yang , Y , Zhao , Z , Maier-Hein , K , Jäger , P F , Kopp-Schneider , A & Maier-Hein , L 2023 , Why is the winner the best? in Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 . IEEE Computer Society Press , pp. 19955-19966 , 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , Vancouve , Canada , 18/06/2023 .
Publication Year :
2023

Abstract

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multi-center study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and postprocessing (66%). The "typical" lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work.

Details

Database :
OAIster
Journal :
Eisenmann , M , Reinke , A , Weru , V , Tizabi , M D , Isensee , F , Adler , T J , Ali , S , Andrearczyk , V , Aubreville , M , Baid , U , Bakas , S , Balu , N , Bano , S , Bernal , J , Bodenstedt , S , Casella , A , Cheplygina , V , Daum , M , Bruijne , M D , Depeursinge , A , Dorent , R , Egger , J , Ellis , D G , Engelhardt , S , Ganz , M , Ghatwary , N , Girard , G , Godau , P , Gupta , A , Hansen , L , Harada , K , Heinrich , M , Heller , N , Hering , A , Huaulmé , A , Jannin , P , Kavur , A E , Kodym , O , Kozubek , M , Li , J , Li , H , Ma , J , Martín-Isla , C , Menze , B , Noble , A , Oreiller , V , Padoy , N , Pati , S , Payette , K , Rädsch , T , Rafael-Patiño , J , Bawa , V S , Speidel , S , Sudre , C H , Wijnen , K V , Wagner , M , Wei , D , Yamlahi , A , Yap , M H , Yuan , C , Zenk , M , Zia , A , Zimmerer , D , Aydogan , D B , Bhattarai , B , Bloch , L , Brüngel , R , Cho , J , Choi , C , Dou , Q , Ezhov , I , Friedrich , C M , Fuller , C , Gaire , R R , Galdran , A , Faura , Á G , Grammatikopoulou , M , Hong , S , Jahanifar , M , Jang , I , Kadkhodamohammadi , A , Kang , I , Kofler , F , Kondo , S , Kuijf , H , Li , M , Luu , M H , Martinčič , T , Morais , P , Naser , M A , Oliveira , B , Owen , D , Pang , S , Park , J , Park , S-H , Płotka , S , Puybareau , E , Rajpoot , N , Ryu , K , Saeed , N , Shephard , A , Shi , P , Štepec , D , Subedi , R , Tochon , G , Torres , H R , Urien , H , Vilaça , J L , Wahid , K A , Wang , H , Wang , J , Wang , L , Wang , X , Wiestler , B , Wodzinski , M , Xia , F , Xie , J , Xiong , Z , Yang , S , Yang , Y , Zhao , Z , Maier-Hein , K , Jäger , P F , Kopp-Schneider , A & Maier-Hein , L 2023 , Why is the winner the best? in Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 . IEEE Computer Society Press , pp. 19955-19966 , 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , Vancouve , Canada , 18/06/2023 .
Notes :
application/pdf, English
Publication Type :
Electronic Resource
Accession number :
edsoai.on1382500378
Document Type :
Electronic Resource