Search

Your search keyword '"Varoquaux, G"' showing total 169 results
Start Over Author "Varoquaux, G"
169 results on '"Varoquaux, G"'

1. Confidence intervals for performance estimates in 3D medical image segmentation

Author

Jurdi, R. El, Varoquaux, G., and Colliot, O.

Subjects
Electrical Engineering and Systems Science - Image and Video Processing, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning
Abstract

Medical segmentation models are evaluated empirically. As such an evaluation is based on a limited set of example images, it is unavoidably noisy. Beyond a mean performance measure, reporting confidence intervals is thus crucial. However, this is rarely done in medical image segmentation. The width of the confidence interval depends on the test set size and on the spread of the performance measure (its standard-deviation across of the test set). For classification, many test images are needed to avoid wide confidence intervals. Segmentation, however, has not been studied, and it differs by the amount of information brought by a given test image. In this paper, we study the typical confidence intervals in medical image segmentation. We carry experiments on 3D image segmentation using the standard nnU-net framework, two datasets from the Medical Decathlon challenge and two performance measures: the Dice accuracy and the Hausdorff distance. We show that the parametric confidence intervals are reasonable approximations of the bootstrap estimates for varying test set sizes and spread of the performance metric. Importantly, we show that the test size needed to achieve a given precision is often much lower than for classification tasks. Typically, a 1% wide confidence interval requires about 100-200 test samples when the spread is low (standard-deviation around 3%). More difficult segmentation tasks may lead to higher spreads and require over 1000 samples., Comment: 10 pages

Published
2023

2. Comparing distributions: $\ell_1$ geometry improves kernel two-sample testing

Author

Scetbon, M. and Varoquaux, G.

Subjects
Statistics - Machine Learning, Computer Science - Machine Learning
Abstract

Are two sets of observations drawn from the same distribution? This problem is a two-sample test. Kernel methods lead to many appealing properties. Indeed state-of-the-art approaches use the $L^2$ distance between kernel-based distribution representatives to derive their test statistics. Here, we show that $L^p$ distances (with $p\geq 1$) between these distribution representatives give metrics on the space of distributions that are well-behaved to detect differences between distributions as they metrize the weak convergence. Moreover, for analytic kernels, we show that the $L^1$ geometry gives improved testing power for scalable computational procedures. Specifically, we derive a finite dimensional approximation of the metric given as the $\ell_1$ norm of a vector which captures differences of expectations of analytic functions evaluated at spatial locations or frequencies (i.e, features). The features can be chosen to maximize the differences of the distributions and give interpretable indications of how they differs. Using an $\ell_1$ norm gives better detection because differences between representatives are dense as we use analytic kernels (non-zero almost everywhere). The tests are consistent, while much faster than state-of-the-art quadratic-time kernel-based tests. Experiments on artificial and real-world problems demonstrate improved power/time tradeoff than the state of the art, based on $\ell_2$ norms, and in some cases, better outright power than even the most expensive quadratic-time tests.

Published
2019

3. The Past, Present, and Future of the Brain Imaging Data Structure (BIDS)

Author

Poldrack, R, Markiewicz, C, Appelhoff, S, Ashar, Y, Auer, T, Baillet, S, Bansal, S, Beltrachini, L, Benar, C, Bertazzoli, G, Bhogawar, S, Blair, R, Bortoletto, M, Boudreau, M, Brooks, T, Calhoun, V, Castelli, F, Clement, P, Cohen, A, Cohen-Adad, J, D'Ambrosio, S, de Hollander, G, de la Iglesia-Vayá, M, de la Vega, A, Delorme, A, Devinsky, O, Draschkow, D, Duff, E, Dupre, E, Earl, E, Esteban, O, Feingold, F, Flandin, G, Galassi, A, Gallitto, G, Ganz, M, Gau, R, Gholam, J, Ghosh, S, Giacomel, A, Gillman, A, Gleeson, P, Gramfort, A, Guay, S, Guidali, G, Halchenko, Y, Handwerker, D, Hardcastle, N, Herholz, P, Hermes, D, Honey, C, Innis, R, Ioanas, H, Jahn, A, Karakuzu, A, Keator, D, Kiar, G, Kincses, B, Laird, A, Lau, J, Lazari, A, Legarreta, J, Li, A, Li, X, Love, B, Lu, H, Marcantoni, E, Maumet, C, Mazzamuto, G, Meisler, S, Mikkelsen, M, Mutsaerts, H, Nichols, T, Nikolaidis, A, Nilsonne, G, Niso, G, Norgaard, M, Okell, T, Oostenveld, R, Ort, E, Park, P, Pawlik, M, Pernet, C, Pestilli, F, Petr, J, Phillips, C, Poline, J, Pollonini, L, Raamana, P, Ritter, P, Rizzo, G, Robbins, K, Rockhill, A, Rogers, C, Rokem, A, Rorden, C, Routier, A, Saborit-Torres, J, Salo, T, Schirner, M, Smith, R, Spisak, T, Sprenger, J, Swann, N, Szinte, M, Takerkart, S, Thirion, B, Thomas, A, Torabian, S, Varoquaux, G, Voytek, B, Welzel, J, Wilson, M, Yarkoni, T, Gorgolewski, K, Poldrack, Russell A., Markiewicz, Christopher J., Appelhoff, Stefan, Ashar, Yoni K., Auer, Tibor, Baillet, Sylvain, Bansal, Shashank, Beltrachini, Leandro, Benar, Christian G., Bertazzoli, Giacomo, Bhogawar, Suyash, Blair, Ross W., Bortoletto, Marta, Boudreau, Mathieu, Brooks, Teon L., Calhoun, Vince D., Castelli, Filippo Maria, Clement, Patricia, Cohen, Alexander L., Cohen-Adad, Julien, D'Ambrosio, Sasha, de Hollander, Gilles, de la Iglesia-Vayá, María, de la Vega, Alejandro, Delorme, Arnaud, Devinsky, Orrin, Draschkow, Dejan, Duff, Eugene Paul, DuPre, Elizabeth, Earl, Eric, Esteban, Oscar, Feingold, Franklin W., Flandin, Guillaume, Galassi, Anthony, Gallitto, Giuseppe, Ganz, Melanie, Gau, Rémi, Gholam, James, Ghosh, Satrajit S., Giacomel, Alessio, Gillman, Ashley G., Gleeson, Padraig, Gramfort, Alexandre, Guay, Samuel, Guidali, Giacomo, Halchenko, Yaroslav O., Handwerker, Daniel A., Hardcastle, Nell, Herholz, Peer, Hermes, Dora, Honey, Christopher J., Innis, Robert B., Ioanas, Horea-Ioan, Jahn, Andrew, Karakuzu, Agah, Keator, David B., Kiar, Gregory, Kincses, Balint, Laird, Angela R., Lau, Jonathan C., Lazari, Alberto, Legarreta, Jon Haitz, Li, Adam, Li, Xiangrui, Love, Bradley C., Lu, Hanzhang, Marcantoni, Eleonora, Maumet, Camille, Mazzamuto, Giacomo, Meisler, Steven L., Mikkelsen, Mark, Mutsaerts, Henk, Nichols, Thomas E., Nikolaidis, Aki, Nilsonne, Gustav, Niso, Guiomar, Norgaard, Martin, Okell, Thomas W., Oostenveld, Robert, Ort, Eduard, Park, Patrick J., Pawlik, Mateusz, Pernet, Cyril R., Pestilli, Franco, Petr, Jan, Phillips, Christophe, Poline, Jean-Baptiste, Pollonini, Luca, Raamana, Pradeep Reddy, Ritter, Petra, Rizzo, Gaia, Robbins, Kay A., Rockhill, Alexander P., Rogers, Christine, Rokem, Ariel, Rorden, Chris, Routier, Alexandre, Saborit-Torres, Jose Manuel, Salo, Taylor, Schirner, Michael, Smith, Robert E., Spisak, Tamas, Sprenger, Julia, Swann, Nicole C., Szinte, Martin, Takerkart, Sylvain, Thirion, Bertrand, Thomas, Adam G., Torabian, Sajjad, Varoquaux, Gael, Voytek, Bradley, Welzel, Julius, Wilson, Martin, Yarkoni, Tal, Gorgolewski, Krzysztof J., Poldrack, R, Markiewicz, C, Appelhoff, S, Ashar, Y, Auer, T, Baillet, S, Bansal, S, Beltrachini, L, Benar, C, Bertazzoli, G, Bhogawar, S, Blair, R, Bortoletto, M, Boudreau, M, Brooks, T, Calhoun, V, Castelli, F, Clement, P, Cohen, A, Cohen-Adad, J, D'Ambrosio, S, de Hollander, G, de la Iglesia-Vayá, M, de la Vega, A, Delorme, A, Devinsky, O, Draschkow, D, Duff, E, Dupre, E, Earl, E, Esteban, O, Feingold, F, Flandin, G, Galassi, A, Gallitto, G, Ganz, M, Gau, R, Gholam, J, Ghosh, S, Giacomel, A, Gillman, A, Gleeson, P, Gramfort, A, Guay, S, Guidali, G, Halchenko, Y, Handwerker, D, Hardcastle, N, Herholz, P, Hermes, D, Honey, C, Innis, R, Ioanas, H, Jahn, A, Karakuzu, A, Keator, D, Kiar, G, Kincses, B, Laird, A, Lau, J, Lazari, A, Legarreta, J, Li, A, Li, X, Love, B, Lu, H, Marcantoni, E, Maumet, C, Mazzamuto, G, Meisler, S, Mikkelsen, M, Mutsaerts, H, Nichols, T, Nikolaidis, A, Nilsonne, G, Niso, G, Norgaard, M, Okell, T, Oostenveld, R, Ort, E, Park, P, Pawlik, M, Pernet, C, Pestilli, F, Petr, J, Phillips, C, Poline, J, Pollonini, L, Raamana, P, Ritter, P, Rizzo, G, Robbins, K, Rockhill, A, Rogers, C, Rokem, A, Rorden, C, Routier, A, Saborit-Torres, J, Salo, T, Schirner, M, Smith, R, Spisak, T, Sprenger, J, Swann, N, Szinte, M, Takerkart, S, Thirion, B, Thomas, A, Torabian, S, Varoquaux, G, Voytek, B, Welzel, J, Wilson, M, Yarkoni, T, Gorgolewski, K, Poldrack, Russell A., Markiewicz, Christopher J., Appelhoff, Stefan, Ashar, Yoni K., Auer, Tibor, Baillet, Sylvain, Bansal, Shashank, Beltrachini, Leandro, Benar, Christian G., Bertazzoli, Giacomo, Bhogawar, Suyash, Blair, Ross W., Bortoletto, Marta, Boudreau, Mathieu, Brooks, Teon L., Calhoun, Vince D., Castelli, Filippo Maria, Clement, Patricia, Cohen, Alexander L., Cohen-Adad, Julien, D'Ambrosio, Sasha, de Hollander, Gilles, de la Iglesia-Vayá, María, de la Vega, Alejandro, Delorme, Arnaud, Devinsky, Orrin, Draschkow, Dejan, Duff, Eugene Paul, DuPre, Elizabeth, Earl, Eric, Esteban, Oscar, Feingold, Franklin W., Flandin, Guillaume, Galassi, Anthony, Gallitto, Giuseppe, Ganz, Melanie, Gau, Rémi, Gholam, James, Ghosh, Satrajit S., Giacomel, Alessio, Gillman, Ashley G., Gleeson, Padraig, Gramfort, Alexandre, Guay, Samuel, Guidali, Giacomo, Halchenko, Yaroslav O., Handwerker, Daniel A., Hardcastle, Nell, Herholz, Peer, Hermes, Dora, Honey, Christopher J., Innis, Robert B., Ioanas, Horea-Ioan, Jahn, Andrew, Karakuzu, Agah, Keator, David B., Kiar, Gregory, Kincses, Balint, Laird, Angela R., Lau, Jonathan C., Lazari, Alberto, Legarreta, Jon Haitz, Li, Adam, Li, Xiangrui, Love, Bradley C., Lu, Hanzhang, Marcantoni, Eleonora, Maumet, Camille, Mazzamuto, Giacomo, Meisler, Steven L., Mikkelsen, Mark, Mutsaerts, Henk, Nichols, Thomas E., Nikolaidis, Aki, Nilsonne, Gustav, Niso, Guiomar, Norgaard, Martin, Okell, Thomas W., Oostenveld, Robert, Ort, Eduard, Park, Patrick J., Pawlik, Mateusz, Pernet, Cyril R., Pestilli, Franco, Petr, Jan, Phillips, Christophe, Poline, Jean-Baptiste, Pollonini, Luca, Raamana, Pradeep Reddy, Ritter, Petra, Rizzo, Gaia, Robbins, Kay A., Rockhill, Alexander P., Rogers, Christine, Rokem, Ariel, Rorden, Chris, Routier, Alexandre, Saborit-Torres, Jose Manuel, Salo, Taylor, Schirner, Michael, Smith, Robert E., Spisak, Tamas, Sprenger, Julia, Swann, Nicole C., Szinte, Martin, Takerkart, Sylvain, Thirion, Bertrand, Thomas, Adam G., Torabian, Sajjad, Varoquaux, Gael, Voytek, Bradley, Welzel, Julius, Wilson, Martin, Yarkoni, Tal, and Gorgolewski, Krzysztof J.

Abstract

The Brain Imaging Data Structure (BIDS) is a community-driven standard for the organization of data and metadata from a growing range of neuroscience modalities. This paper is meant as a history of how the standard has developed and grown over time. We outline the principles behind the project, the mechanisms by which it has been extended, and some of the challenges being addressed as it evolves. We also discuss the lessons learned through the project, with the aim of enabling researchers in other domains to learn from the success of BIDS.

Published
2024

4. The Past, Present, and Future of the Brain Imaging Data Structure (BIDS)

Author

Poldrack, R. A., Markiewicz, C. J., Appelhoff, S., Ashar, Y. K., Auer, T., Baillet, S., Bansal, S., Beltrachini, L., Bertazzoli, G., Bhogawar, S., Blair, R. W., Bortoletto, M., Boudreau, M., Brooks, T. L., Bénar, C. G., Calhoun, V. D., Castelli, F. M., Clement, P., Cohen, A. L., Cohen-Adad, J., Dambrosio, S., Delorme, A., Devinsky, O., Draschkow, D., Duff, E. P., Dupre, E., Earl, E., Esteban, O., Feingold, F. W., Flandin, G., Galassi, A., Gallitto, G., Ganz, M., Gholam, J., Ghosh, S. S., Giacomel, A., Gillman, A. G., Gleeson, P., Gramfort, A., Guay, S., Guidali, G., Halchenko, Y. O., Handwerker, D. A., Hardcastle, N., Herholz, P., Hermes, D., Honey, C. J., Innis, R. B., Ioanas, H.-I., Jahn, A., Karakuzu, A., Keator, D. B., Kiar, G., Kincses, B., Laird, A. R., Lau, J. C., Lazari, A., Legarreta, J. H., Li, A., Li, X., Love, B. C., Lu, H., Maumet, C., Mazzamuto, G., Meisler, S. L., Mikkelsen, M., Mutsaerts, H., Nichols, T. E., Nikolaidis, A., Nilsonne, G., Niso, G., Norgaard, M., Okell, T. W., Oostenveld, R., Ort, E., Park, P. J., Pawlik, M., Pernet, C. R., Pestilli, F., (0000-0002-3201-6002) Petr, J., Phillips, C., Poline, J.-B., Pollonini, L., Raamana, P. R., Ritter, P., Rizzo, G., Robbins, K. A., Rockhill, A. P., Rogers, C., Rokem, A., Rorden, C., Routier, A., Saborit-Torres, J. M., Salo, T., Schirner, M., Smith, R. E., Spisak, T., Sprenger, J., Swann, N. C., Szinte, M., Takerkart, S., Thirion, B., Thomas, A. G., Torabian, S., Varoquaux, G., Vaya, M. D. L. I., Voytek, B., Welzel, J., Wilson, M., Hollander, G., Vega, A., Gorgolewski, K. J., Poldrack, R. A., Markiewicz, C. J., Appelhoff, S., Ashar, Y. K., Auer, T., Baillet, S., Bansal, S., Beltrachini, L., Bertazzoli, G., Bhogawar, S., Blair, R. W., Bortoletto, M., Boudreau, M., Brooks, T. L., Bénar, C. G., Calhoun, V. D., Castelli, F. M., Clement, P., Cohen, A. L., Cohen-Adad, J., Dambrosio, S., Delorme, A., Devinsky, O., Draschkow, D., Duff, E. P., Dupre, E., Earl, E., Esteban, O., Feingold, F. W., Flandin, G., Galassi, A., Gallitto, G., Ganz, M., Gholam, J., Ghosh, S. S., Giacomel, A., Gillman, A. G., Gleeson, P., Gramfort, A., Guay, S., Guidali, G., Halchenko, Y. O., Handwerker, D. A., Hardcastle, N., Herholz, P., Hermes, D., Honey, C. J., Innis, R. B., Ioanas, H.-I., Jahn, A., Karakuzu, A., Keator, D. B., Kiar, G., Kincses, B., Laird, A. R., Lau, J. C., Lazari, A., Legarreta, J. H., Li, A., Li, X., Love, B. C., Lu, H., Maumet, C., Mazzamuto, G., Meisler, S. L., Mikkelsen, M., Mutsaerts, H., Nichols, T. E., Nikolaidis, A., Nilsonne, G., Niso, G., Norgaard, M., Okell, T. W., Oostenveld, R., Ort, E., Park, P. J., Pawlik, M., Pernet, C. R., Pestilli, F., (0000-0002-3201-6002) Petr, J., Phillips, C., Poline, J.-B., Pollonini, L., Raamana, P. R., Ritter, P., Rizzo, G., Robbins, K. A., Rockhill, A. P., Rogers, C., Rokem, A., Rorden, C., Routier, A., Saborit-Torres, J. M., Salo, T., Schirner, M., Smith, R. E., Spisak, T., Sprenger, J., Swann, N. C., Szinte, M., Takerkart, S., Thirion, B., Thomas, A. G., Torabian, S., Varoquaux, G., Vaya, M. D. L. I., Voytek, B., Welzel, J., Wilson, M., Hollander, G., Vega, A., and Gorgolewski, K. J.

Abstract

The Brain Imaging Data Structure (BIDS) is a community-driven standard for the organization of data and metadata from a growing range of neuroscience modalities. This paper is meant as a history of how the standard has developed and grown over time. We outline the principles behind the project, and the mechanisms by which it has been extended. We also discuss the lessons learned through the project, with the aim of enabling researchers in other domains to learn from the success of BIDS.

Published
2024

5. Metrics reloaded: recommendations for image analysis validation

Author

Maier-Hein, L., Reinke, A., Godau, P., Tizabi, M.D., Buettner, F., Christodoulou, E., Glocker, B., Isensee, F., Kleesiek, J., Kozubek, M., Reyes, M., Riegler, M.A., Wiesenfarth, M., Kavur, A.E., Sudre, C.H., Baumgartner, M., Eisenmann, M., Heckmann-Notzel, D., Radsch, T., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Benis, A., Blaschko, M.B., Cardoso, M.J., Cheplygina, V., Cimini, B.A., Collins, G.S., Farahani, K., Ferrer, L., Galdran, A., Ginneken, B. van, Haase, R., Hashimoto, D.A., Hoffman, M.M., Huisman, M., Jannin, P., Kahn, C.E., Kainmueller, D., Kainz, B., Karargyris, A., Karthikesalingam, A., Kofler, F., Kopp-Schneider, A., Kreshuk, A., Kurc, T., Landman, B.A., Litjens, G.J., Madani, A., Maier-Hein, K., Martel, A.L., Mattson, P., Meijering, E., Menze, B., Moons, K.G., Muller, H., Nichyporuk, B., Nickel, F., Petersen, J., Rajpoot, N., Rieke, N., Saez-Rodriguez, J., Sanchez, C.I., Shetty, S., Smeden, M. van, Summers, R.M., Taha, A.A., Tiulpin, A., Tsaftaris, S.A., Calster, B. van, Varoquaux, G., Jager, P.F., Maier-Hein, L., Reinke, A., Godau, P., Tizabi, M.D., Buettner, F., Christodoulou, E., Glocker, B., Isensee, F., Kleesiek, J., Kozubek, M., Reyes, M., Riegler, M.A., Wiesenfarth, M., Kavur, A.E., Sudre, C.H., Baumgartner, M., Eisenmann, M., Heckmann-Notzel, D., Radsch, T., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Benis, A., Blaschko, M.B., Cardoso, M.J., Cheplygina, V., Cimini, B.A., Collins, G.S., Farahani, K., Ferrer, L., Galdran, A., Ginneken, B. van, Haase, R., Hashimoto, D.A., Hoffman, M.M., Huisman, M., Jannin, P., Kahn, C.E., Kainmueller, D., Kainz, B., Karargyris, A., Karthikesalingam, A., Kofler, F., Kopp-Schneider, A., Kreshuk, A., Kurc, T., Landman, B.A., Litjens, G.J., Madani, A., Maier-Hein, K., Martel, A.L., Mattson, P., Meijering, E., Menze, B., Moons, K.G., Muller, H., Nichyporuk, B., Nickel, F., Petersen, J., Rajpoot, N., Rieke, N., Saez-Rodriguez, J., Sanchez, C.I., Shetty, S., Smeden, M. van, Summers, R.M., Taha, A.A., Tiulpin, A., Tsaftaris, S.A., Calster, B. van, Varoquaux, G., and Jager, P.F.

Abstract

Contains fulltext : 305368.pdf (Publisher’s version ) (Closed access), Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.

Published
2024

6. Understanding metric-related pitfalls in image analysis validation

Author

Reinke, A., Tizabi, M.D., Baumgartner, M., Eisenmann, M., Heckmann-Notzel, D., Kavur, A.E., Radsch, T., Sudre, C.H., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Benis, A., Buettner, F., Cardoso, M.J., Cheplygina, V., Chen, J., Christodoulou, E., Cimini, B.A., Farahani, K., Ferrer, L., Galdran, A., Ginneken, B. van, Glocker, B., Godau, P., Hashimoto, D.A., Hoffman, M.M., Huisman, M., Isensee, F., Jannin, P., Kahn, C.E., Kainmueller, D., Kainz, B., Karargyris, A., Kleesiek, J., Kofler, F., Kooi, T., Kopp-Schneider, A., Kozubek, M., Kreshuk, A., Kurc, T., Landman, B.A., Litjens, G.J., Madani, A., Maier-Hein, K., Martel, A.L., Meijering, E., Menze, B., Moons, K.G., Muller, H., Nichyporuk, B., Nickel, F., Petersen, J., Rafelski, S.M., Rajpoot, N., Reyes, M., Riegler, M.A., Rieke, N., Saez-Rodriguez, J., Sanchez, C.I., Shetty, S., Summers, R.M., Taha, A.A., Tiulpin, A., Tsaftaris, S.A., Calster, B. van, Varoquaux, G., Yaniv, Z.R., Jager, P.F., Maier-Hein, L., Reinke, A., Tizabi, M.D., Baumgartner, M., Eisenmann, M., Heckmann-Notzel, D., Kavur, A.E., Radsch, T., Sudre, C.H., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Benis, A., Buettner, F., Cardoso, M.J., Cheplygina, V., Chen, J., Christodoulou, E., Cimini, B.A., Farahani, K., Ferrer, L., Galdran, A., Ginneken, B. van, Glocker, B., Godau, P., Hashimoto, D.A., Hoffman, M.M., Huisman, M., Isensee, F., Jannin, P., Kahn, C.E., Kainmueller, D., Kainz, B., Karargyris, A., Kleesiek, J., Kofler, F., Kooi, T., Kopp-Schneider, A., Kozubek, M., Kreshuk, A., Kurc, T., Landman, B.A., Litjens, G.J., Madani, A., Maier-Hein, K., Martel, A.L., Meijering, E., Menze, B., Moons, K.G., Muller, H., Nichyporuk, B., Nickel, F., Petersen, J., Rafelski, S.M., Rajpoot, N., Reyes, M., Riegler, M.A., Rieke, N., Saez-Rodriguez, J., Sanchez, C.I., Shetty, S., Summers, R.M., Taha, A.A., Tiulpin, A., Tsaftaris, S.A., Calster, B. van, Varoquaux, G., Yaniv, Z.R., Jager, P.F., and Maier-Hein, L.

Abstract

Contains fulltext : 305370.pdf (Publisher’s version ) (Closed access), Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.

Published
2024

7. Scale-Free and Multifractal Time Dynamics of fMRI Signals during Rest and Task

Author

Ciuciu, P., Varoquaux, G., Abry, P., Sadaghiani, S., and Kleinschmidt, A.

Subjects
Mathematics - Statistics Theory, Quantitative Biology - Neurons and Cognition
Abstract

Scaling temporal dynamics in functional MRI (fMRI) signals have been evidenced for a decade as intrinsic characteristics of ongoing brain activity (Zarahn et al., 1997). Recently, scaling properties were shown to fluctuate across brain networks and to be modulated between rest and task (He, 2011): notably, Hurst exponent, quantifying long memory, decreases under task in activating and deactivating brain regions. In most cases, such results were obtained: First, from univariate (voxelwise or regionwise) analysis, hence focusing on specific cognitive systems such as Resting-State Networks (RSNs) and raising the issue of the specificity of this scale-free dynamics modulation in RSNs. Second, using analysis tools designed to measure a single scaling exponent related to the second order statistics of the data, thus relying on models that either implicitly or explicitly assume Gaussianity and (asymptotic) self-similarity, while fMRI signals may significantly depart from those either of those two assumptions (Ciuciu et al., 2008; Wink et al., 2008). To address these issues, the present contribution elaborates on the analysis of the scaling properties of fMRI temporal dynamics by proposing two significant variations. First, scaling properties are technically investigated using the recently introduced Wavelet Leader-based Multifractal formalism (WLMF; Wendt et al., 2007). This measures a collection of scaling exponents, thus enables a richer and more versatile description of scale invariance (beyond correlation and Gaussianity), referred to as multifractality. Also, it benefits from improved estimation performance compared to tools previously used in the literature. Second, scaling properties are investigated in both RSN and non-RSN structures (e.g., artifacts), at a broader spatial scale than the voxel one, using a multivariate approach, namely the Multi-Subject Dictionary Learning (MSDL) algorithm (Varoquaux et al., 2011) that produces a set of spatial components that appear more sparse than their Independent Component Analysis (ICA) counterpart. These tools are combined and applied to a fMRI dataset comprising 12 subjects with resting-state and activation runs (Sadaghiani et al., 2009). Results stemming from those analysis confirm the already reported task-related decrease of long memory in functional networks, but also show that it occurs in artifacts, thus making this feature not specific to functional networks. Further, results indicate that most fMRI signals appear multifractal at rest except in non-cortical regions. Task-related modulation of multifractality appears only significant in functional networks and thus can be considered as the key property disentangling functional networks from artifacts. These finding are discussed in the light of the recent literature reporting scaling dynamics of EEG microstate sequences at rest and addressing non-stationarity issues in temporally independent fMRI modes.

Published
2013
Full Text
View/download PDF

8. A group model for stable multi-subject ICA on fMRI datasets

Author

Varoquaux, G., Sadaghiani, S., Pinel, P., Kleinschmidt, A., Poline, J. B., and Thirion, B.

Subjects
Statistics - Applications, Statistics - Methodology
Abstract

Spatial Independent Component Analysis (ICA) is an increasingly used data-driven method to analyze functional Magnetic Resonance Imaging (fMRI) data. To date, it has been used to extract sets of mutually correlated brain regions without prior information on the time course of these regions. Some of these sets of regions, interpreted as functional networks, have recently been used to provide markers of brain diseases and open the road to paradigm-free population comparisons. Such group studies raise the question of modeling subject variability within ICA: how can the patterns representative of a group be modeled and estimated via ICA for reliable inter-group comparisons? In this paper, we propose a hierarchical model for patterns in multi-subject fMRI datasets, akin to mixed-effect group models used in linear-model-based analysis. We introduce an estimation procedure, CanICA (Canonical ICA), based on i) probabilistic dimension reduction of the individual data, ii) canonical correlation analysis to identify a data subspace common to the group iii) ICA-based pattern extraction. In addition, we introduce a procedure based on cross-validation to quantify the stability of ICA patterns at the level of the group. We compare our method with state-of-the-art multi-subject fMRI ICA methods and show that the features extracted using our procedure are more reproducible at the group level on two datasets of 12 healthy controls: a resting-state and a functional localizer study.

Published
2010
Full Text
View/download PDF

9. The Sagnac effect in superfluids

Author

Varoquaux, E. and Varoquaux, G.

Subjects
General Relativity and Quantum Cosmology, Condensed Matter - Other Condensed Matter
Abstract

We comment on the interpretation of absolute rotation measurements with superfluids, which involve the Sagnac effect. This effect, first observed in rotating optical interferometers, has now been seen in a host of different physical systems, ranging from atomic clocks to electrons, neutrons, atomic beams, ... We show that the time-honored explanation of this effect based on general relativity provides a common explanation for all these systems, including superfluids, in contradistinction with the views expressed in a recent review article., Comment: Letter to the Editors of Uspheki Fisiki Nauk - 4 pages

Published
2007

11. Body-weight variability and risk of cardiovascular outcomes in patients with type 1 diabetes: a retrospective observational analysis of data from the DCCT/EDIC population

Author

Petria, I, Albuquerque, S, Varoquaux, G, Vie, J, Venteclef, N, Mohammedi, K, Roussel, R, Camoin, M, Perseghin, G, Velho, G, Potier, L, Petria, Iulia, Albuquerque, Samuel, Varoquaux, Gaël, Vie, Jill-Jênn, Venteclef, Nicolas, Mohammedi, Kamel, Roussel, Ronan, Camoin, Marion, Perseghin, Gianluca, Velho, Gilberto, Potier, Louis, Petria, I, Albuquerque, S, Varoquaux, G, Vie, J, Venteclef, N, Mohammedi, K, Roussel, R, Camoin, M, Perseghin, G, Velho, G, Potier, L, Petria, Iulia, Albuquerque, Samuel, Varoquaux, Gaël, Vie, Jill-Jênn, Venteclef, Nicolas, Mohammedi, Kamel, Roussel, Ronan, Camoin, Marion, Perseghin, Gianluca, Velho, Gilberto, and Potier, Louis

Abstract

Background: Cardiovascular risk and body-weight management are both emerging challenges of type 1 diabetes care. We evaluated the association between intraindividual variability of body-weight and risk of cardiovascular events in people with type 1 diabetes. Methods: We analyzed 1,398 participants from the DCCT/EDIC studies. Five indices of intraindividual variability of body-weight were calculated for each participant taking into account body-weight measures obtained during the DCCT follow-up (average 6 ± 2 years). The Average Successive Variability (ASV) index, the main variable of interest, was defined as the average absolute difference between successive body-weight measures. The primary outcome was a composite of major adverse cardiovascular events (MACE: nonfatal myocardial infarction or stroke, or cardiovascular death) occurring during the subsequent EDIC follow-up (20 ± 3 years). All-cause death was a secondary outcome. Risk of outcomes were assessed by Cox proportional hazards regression analyses, adjusted for traditional cardiovascular risks factors, including BMI. Results: The cumulative incidence of MACE and all-cause death during follow-up were 5.6% (n = 79) and 6.8% (n = 95), respectively. The adjusted Hazard Ratio (HR) for MACE by every increase of 1 standard deviation (SD) of ASV was 1.34 (95% CI, 1.06–1.66), p = 0.01. For all-cause death, the adjusted HR for 1 SD increase of ASV was 1.25 (1.03–1.50), p = 0.03. Similar results were observed when considering the other indices of intraindividual variability of body-weight. Conclusions: High body-weight variability (body-weight cycling) is associated with increased risk of MACE and all-cause death in people with type 1 diabetes, independently of the BMI and traditional cardiovascular risk factors.

Published
2022

12. Centering inclusivity in the design of online conferences-An OHBM-Open Science perspective

Author

Levitis, E, van Praag, CDG, Gau, R, Heunis, S, DuPre, E, Kiar, G, Bottenhorn, KL, Glatard, T, Nikolaidis, A, Whitaker, KJ, Mancini, M, Niso, G, Afyouni, S, Alonso-Ortiz, E, Appelhoff, S, Arnatkeviciute, A, Atay, SM, Auer, T, Baracchini, G, Bayer, JMM, Beauvais, MJS, Bijsterbosch, JD, Bilgin, IP, Bollmann, S, Botvinik-Nezer, R, Bright, MG, Calhoun, VD, Chen, X, Chopra, S, Hu, C-P, Close, TG, Cookson, SL, Craddock, RC, De la Vega, A, De Leener, B, Demeter, D, Di Maio, P, Dickie, EW, Eickhoff, SB, Esteban, O, Finc, K, Frigo, M, Ganesan, S, Ganz, M, Garner, KG, Garza-Villarreal, EA, Gonzalez-Escamilla, G, Goswami, R, Griffiths, JD, Grootswagers, T, Guay, S, Guest, O, Handwerker, DA, Herholz, P, Heuer, K, Huijser, DC, Iacovella, V, Joseph, MJE, Karakuzu, A, Keator, DB, Kobeleva, X, Kumar, M, Laird, AR, Larson-Prior, LJ, Lautarescu, A, Lazari, A, Legarreta, JH, Li, X-Y, Lv, J, Mansour, SL, Meunier, D, Moraczewski, D, Nandi, T, Nastase, SA, Nau, M, Noble, S, Norgaard, M, Obungoloch, J, Oostenveld, R, Orchard, ER, Poldrack, RA, Qiu, A, Raamana, PR, Rokem, A, Rutherford, S, Sharan, M, Shaw, TB, Syeda, WT, Testerman, MM, Toro, R, Valk, SL, Van den Bossche, S, Varoquaux, G, Vasa, F, Veldsman, M, Vohryzek, J, Wagner, AS, Walsh, RJ, White, T, Wong, F-T, Xie, X, Yan, C-G, Yang, Y-F, Yee, Y, Zanitti, GE, Van Gulick, AE, Duff, E, Maumet, C, Levitis, E, van Praag, CDG, Gau, R, Heunis, S, DuPre, E, Kiar, G, Bottenhorn, KL, Glatard, T, Nikolaidis, A, Whitaker, KJ, Mancini, M, Niso, G, Afyouni, S, Alonso-Ortiz, E, Appelhoff, S, Arnatkeviciute, A, Atay, SM, Auer, T, Baracchini, G, Bayer, JMM, Beauvais, MJS, Bijsterbosch, JD, Bilgin, IP, Bollmann, S, Botvinik-Nezer, R, Bright, MG, Calhoun, VD, Chen, X, Chopra, S, Hu, C-P, Close, TG, Cookson, SL, Craddock, RC, De la Vega, A, De Leener, B, Demeter, D, Di Maio, P, Dickie, EW, Eickhoff, SB, Esteban, O, Finc, K, Frigo, M, Ganesan, S, Ganz, M, Garner, KG, Garza-Villarreal, EA, Gonzalez-Escamilla, G, Goswami, R, Griffiths, JD, Grootswagers, T, Guay, S, Guest, O, Handwerker, DA, Herholz, P, Heuer, K, Huijser, DC, Iacovella, V, Joseph, MJE, Karakuzu, A, Keator, DB, Kobeleva, X, Kumar, M, Laird, AR, Larson-Prior, LJ, Lautarescu, A, Lazari, A, Legarreta, JH, Li, X-Y, Lv, J, Mansour, SL, Meunier, D, Moraczewski, D, Nandi, T, Nastase, SA, Nau, M, Noble, S, Norgaard, M, Obungoloch, J, Oostenveld, R, Orchard, ER, Poldrack, RA, Qiu, A, Raamana, PR, Rokem, A, Rutherford, S, Sharan, M, Shaw, TB, Syeda, WT, Testerman, MM, Toro, R, Valk, SL, Van den Bossche, S, Varoquaux, G, Vasa, F, Veldsman, M, Vohryzek, J, Wagner, AS, Walsh, RJ, White, T, Wong, F-T, Xie, X, Yan, C-G, Yang, Y-F, Yee, Y, Zanitti, GE, Van Gulick, AE, Duff, E, and Maumet, C

Abstract

As the global health crisis unfolded, many academic conferences moved online in 2020. This move has been hailed as a positive step towards inclusivity in its attenuation of economic, physical, and legal barriers and effectively enabled many individuals from groups that have traditionally been underrepresented to join and participate. A number of studies have outlined how moving online made it possible to gather a more global community and has increased opportunities for individuals with various constraints, e.g., caregiving responsibilities. Yet, the mere existence of online conferences is no guarantee that everyone can attend and participate meaningfully. In fact, many elements of an online conference are still significant barriers to truly diverse participation: the tools used can be inaccessible for some individuals; the scheduling choices can favour some geographical locations; the set-up of the conference can provide more visibility to well-established researchers and reduce opportunities for early-career researchers. While acknowledging the benefits of an online setting, especially for individuals who have traditionally been underrepresented or excluded, we recognize that fostering social justice requires inclusivity to actively be centered in every aspect of online conference design. Here, we draw from the literature and from our own experiences to identify practices that purposefully encourage a diverse community to attend, participate in, and lead online conferences. Reflecting on how to design more inclusive online events is especially important as multiple scientific organizations have announced that they will continue offering an online version of their event when in-person conferences can resume.

Published
2021

15. Manifold-regression to predict from MEG/EEG brain signals without source modeling

Author

Sabbagh, D., Ablin, P., Varoquaux, G., Gramfort, A., Denis-Alexander Engemann, Modelling brain structure, function and variability based on high-field MRI data (PARIETAL), Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Service NEUROSPIN (NEUROSPIN), Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA), Service NEUROSPIN (NEUROSPIN), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Inria Saclay - Ile de France, and Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)

Subjects
Signal Processing (eess.SP), FOS: Computer and information sciences, Computer Science - Machine Learning, Quantitative Biology::Neurons and Cognition, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, [SCCO.NEUR]Cognitive science/Neuroscience, FOS: Electrical engineering, electronic engineering, information engineering, [INFO.INFO-IM]Computer Science [cs]/Medical Imaging, Machine Learning (stat.ML), Electrical Engineering and Systems Science - Signal Processing, Machine Learning (cs.LG)
Abstract

International audience; Magnetoencephalography and electroencephalography (M/EEG) can reveal neuronal dynamics non-invasively in real-time and are therefore appreciated methods in medicine and neuroscience. Recent advances in modeling brain-behavior relationships have highlighted the effectiveness of Riemannian geometry for summarizing the spatially correlated time-series from M/EEG in terms of their covariance. However , after artefact-suppression, M/EEG data is often rank deficient which limits the application of Riemannian concepts. In this article, we focus on the task of regression with rank-reduced covariance matrices. We study two Riemannian approaches that vectorize the M/EEG covariance between-sensors through projection into a tangent space. The Wasserstein distance readily applies to rank-reduced data but lacks affine-invariance. This can be overcome by finding a common sub-space in which the covariance matrices are full rank, enabling the affine-invariant geometric distance. We investigated the implications of these two approaches in synthetic generative models, which allowed us to control estimation bias of a linear model for prediction. We show that Wasserstein and geometric distances allow perfect out-of-sample prediction on the generative models. We then evaluated the methods on real data with regard to their effectiveness in predicting age from M/EEG covariance matrices. The findings suggest that the data-driven Riemannian methods outperform different sensor-space estimators and that they get close to the performance of biophysics-driven source-localization model that requires MRI acquisitions and tedious data processing. Our study suggests that the proposed Riemannian methods can serve as fundamental building-blocks for automated large-scale analysis of M/EEG.

Published
2019

17. Learning to Discover Sparse Graphical Models

Author

Belilovsky, E., Varoquaux, G., Kastner, K., Matthew Blaschko, Centre de vision numérique (CVN), Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec, Organ Modeling through Extraction, Representation and Understanding of Medical Image Content (GALEN), Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Ecole Centrale Paris, Departement Elektrotechniek - ESAT [leuven], Catholic University of Leuven - Katholieke Universiteit Leuven (KU Leuven), Université de Montréal (UdeM), Service NEUROSPIN (NEUROSPIN), Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA), Modelling brain structure, function and variability based on high-field MRI data (PARIETAL), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), Ecole Centrale Paris-Inria Saclay - Ile de France, Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Université Paris-Saclay, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Service NEUROSPIN (NEUROSPIN), Precup, D, and Teh, YW

Subjects
FOS: Computer and information sciences, PSI_MBL, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Machine Learning (stat.ML)
Abstract

International audience; We consider structure discovery of undirected graphical models from observational data. Inferring likely structures from few examples is a complex task often requiring the formulation of priors and sophisticated inference procedures. Popular methods rely on estimating a penalized maximum likelihood of the precision matrix. However, in these approaches structure recovery is an indirect consequence of the data-fit term, the penalty can be difficult to adapt for domain-specific knowledge, and the inference is computationally demanding. By contrast, it may be easier to generate training samples of data that arise from graphs with the desired structure properties. We propose here to leverage this latter source of information as training data to learn a function, parametrized by a neural network that maps empirical covariance matrices to estimated graph structures. Learning this function brings two benefits: it implicitly models the desired structure or sparsity properties to form suitable priors, and it can be tailored to the specific problem of edge structure discovery, rather than maximizing data likelihood. Applying this framework, we find our learnable graph-discovery method trained on synthetic data generalizes well: identifying relevant edges in both synthetic and real data, completely unknown at training time. We find that on genetics, brain imaging, and simulation data we obtain performance generally superior to analytical methods.

Published
2017

18. Testing for Differences in Gaussian Graphical Models: Applications to Brain Connectivity

Author

Belilovsky, E., Varoquaux, G., Matthew Blaschko, Departement Elektrotechniek - ESAT [leuven], Catholic University of Leuven - Katholieke Universiteit Leuven (KU Leuven), Organ Modeling through Extraction, Representation and Understanding of Medical Image Content (GALEN), Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Ecole Centrale Paris, Centre de vision numérique (CVN), Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec, Modelling brain structure, function and variability based on high-field MRI data (PARIETAL), Service NEUROSPIN (NEUROSPIN), Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), Lee, DD, Sugiyama, M, Luxburg, UV, Guyon, I, Garnett, R, Ecole Centrale Paris-Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Service NEUROSPIN (NEUROSPIN), and Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)

Subjects
FOS: Computer and information sciences, PSI_MBL, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, [SCCO.NEUR]Cognitive science/Neuroscience, Machine Learning (stat.ML), Statistics::Computation
Abstract

© 2016 NIPS Foundation - All Rights Reserved. Functional brain networks are well described and estimated from data with Gaussian Graphical Models (GGMs), e.g. using sparse inverse covariance estimators. Comparing functional connectivity of subjects in two populations calls for comparing these estimated GGMs. Our goal is to identify differences in GGMs known to have similar structure. We characterize the uncertainty of differences with confidence intervals obtained using a parametric distribution on parameters of a sparse estimator. Sparse penalties enable statistical guarantees and interpretable models even in high-dimensional and low-sample settings. Characterizing the distributions of sparse models is inherently challenging as the penalties produce a biased estimator. Recent work invokes the sparsity assumptions to effectively remove the bias from a sparse estimator such as the lasso. These distributions can be used to give confidence intervals on edges in GGMs, and by extension their differences. However, in the case of comparing GGMs, these estimators do not make use of any assumed joint structure among the GGMs. Inspired by priors from brain functional connectivity we derive the distribution of parameter differences under a joint penalty when parameters are known to be sparse in the difference. This leads us to introduce the debiased multi-task fused lasso, whose distribution can be characterized in an efficient manner. We then show how the debiased lasso and multi-task fused lasso can be used to obtain confidence intervals on edge differences in GGMs. We validate the techniques proposed on a set of synthetic examples as well as neuro-imaging dataset created for the study of autism. Belilovsky E., Varoquaux G., Blaschko M., ''Testing for differences in Gaussian graphical models: Applications to brain connectivity'', 30th annual conference on neural information processing systems - NIPS 2016, 13 pp., December 4-9, 2016, Barcelona, Spain. ispartof: pages:595-603 ispartof: Proceedings NIPS 2016 vol:29 pages:595-603 ispartof: Annual conference on neural information processing systems - NIPS 2016 location:Barcelona, Spain date:4 Dec - 9 Dec 2016 status: published

Published
2016

19. Big data and extreme-scale computing

Author

Asch, M, primary, Moore, T, additional, Badia, R, additional, Beck, M, additional, Beckman, P, additional, Bidot, T, additional, Bodin, F, additional, Cappello, F, additional, Choudhary, A, additional, de Supinski, B, additional, Deelman, E, additional, Dongarra, J, additional, Dubey, A, additional, Fox, G, additional, Fu, H, additional, Girona, S, additional, Gropp, W, additional, Heroux, M, additional, Ishikawa, Y, additional, Keahey, K, additional, Keyes, D, additional, Kramer, W, additional, Lavignon, J-F, additional, Lu, Y, additional, Matsuoka, S, additional, Mohr, B, additional, Reed, D, additional, Requena, S, additional, Saltz, J, additional, Schulthess, T, additional, Stevens, R, additional, Swany, M, additional, Szalay, A, additional, Tang, W, additional, Varoquaux, G, additional, Vilotte, J-P, additional, Wisniewski, R, additional, Xu, Z, additional, and Zacharov, I, additional

Published
2018
Full Text
View/download PDF

20. BIDS apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods

Author

Schneidman, D, Gorgolewski, KJ, Alfaro-Almagro, F, Auer, T, Bellec, P, Capota, M, Chakravarty, MM, Churchill, NW, Cohen, AL, Craddock, RC, Devenyi, GA, Eklund, A, Esteban, O, Flandin, G, Ghosh, SS, Guntupalli, JS, Jenkinson, M, Keshavan, A, Kiar, G, Liem, F, Raamana, PR, Raffelt, D, Steele, CJ, Quirion, P-O, Smith, RE, Strother, SC, Varoquaux, G, Wang, Y, Yarkoni, T, Poldrack, RA, Schneidman, D, Gorgolewski, KJ, Alfaro-Almagro, F, Auer, T, Bellec, P, Capota, M, Chakravarty, MM, Churchill, NW, Cohen, AL, Craddock, RC, Devenyi, GA, Eklund, A, Esteban, O, Flandin, G, Ghosh, SS, Guntupalli, JS, Jenkinson, M, Keshavan, A, Kiar, G, Liem, F, Raamana, PR, Raffelt, D, Steele, CJ, Quirion, P-O, Smith, RE, Strother, SC, Varoquaux, G, Wang, Y, Yarkoni, T, and Poldrack, RA

Abstract

The rate of progress in human neurosciences is limited by the inability to easily apply a wide range of analysis methods to the plethora of different datasets acquired in labs around the world. In this work, we introduce a framework for creating, testing, versioning and archiving portable applications for analyzing neuroimaging data organized and described in compliance with the Brain Imaging Data Structure (BIDS). The portability of these applications (BIDS Apps) is achieved by using container technologies that encapsulate all binary and other dependencies in one convenient package. BIDS Apps run on all three major operating systems with no need for complex setup and configuration and thanks to the comprehensiveness of the BIDS standard they require little manual user input. Previous containerized data processing solutions were limited to single user environments and not compatible with most multi-tenant High Performance Computing systems. BIDS Apps overcome this limitation by taking advantage of the Singularity container technology. As a proof of concept, this work is accompanied by 22 ready to use BIDS Apps, packaging a diverse set of commonly used neuroimaging algorithms.

Published
2017

21. I.C.E.: a transportable atomic inertial sensor for test in microgravity

Author

Nyman, R. A., primary, Varoquaux, G., additional, Clement, J.-F., additional, Bouyer, P., additional, Santarelli, G., additional, Pereira Dos Santos, F., additional, Clairon, A., additional, Landragin, A., additional, Chambon, D., additional, Lienhart, F., additional, Boussen, S., additional, and Bresson, A., additional

Published
2017
Full Text
View/download PDF

23. Group-PCA for very large fMRI datasets

Author

Smith, S.M., Hyvarinen, A., Varoquaux, G., Miller, K.L., Beckmann, C.F., Department of Computer Science, and Neuroinformatics research group / Aapo Hyvärinen

Subjects
PCA, Big data, Neurology, HUMAN CONNECTOME PROJECT, Cognitive Neuroscience, fMRI, Stress-related disorders Donders Center for Medical Neuroscience [Radboudumc 13], 3112 Neurosciences, 220 Statistical Imaging Neuroscience, ICA, 113 Computer and information sciences
Abstract

Contains fulltext : 135326.pdf (Publisher’s version ) (Closed access) Increasingly-large datasets (for example, the resting-state fMRI data from the Human Connectome Project) are demanding analyses that are problematic because of the sheer scale of the aggregate data. We present two approaches for applying group-level PCA; both give a close approximation to the output of PCA applied to full concatenation of all individual datasets, while having very low memory requirements regardless of the number of datasets being combined. Across a range of realistic simulations, we find that in most situations, both methods are more accurate than current popular approaches for analysis of multi-subject resting-state fMRI studies. The group-PCA output can be used to feed into a range of further analyses that are then rendered practical, such as the estimation of group-averaged voxelwise connectivity, group-level parcellation, and group-ICA. Copyright 2014 Elsevier Inc. All rights reserved.

Published
2014

24. Scikit-learn

Author

Varoquaux, G., primary, Buitinck, L., additional, Louppe, G., additional, Grisel, O., additional, Pedregosa, F., additional, and Mueller, A., additional

Published
2015
Full Text
View/download PDF

25. I.C.E.: An Ultra-Cold Atom Source for Long-Baseline Interferometric Inertial Sensors in Reduced Gravity

Author

Varoquaux, G., Zahzam, N., Chaibi, W., Clément, J-F., Carraz, O., Brantut, Jean-Philippe, Nyman, R.A., Pereira Dos Santos, F., Mondin, L., Rouzé, M., Bidel, Y., Bresson, A., Landragin, A., Bouyer, P., laboratoire Charles Fabry de l'Institut d'Optique / Optique atomique, Laboratoire Charles Fabry de l'Institut d'Optique (LCFIO), Centre National de la Recherche Scientifique (CNRS)-Institut d'Optique Graduate School (IOGS)-Université Paris-Sud - Paris 11 (UP11)-Centre National de la Recherche Scientifique (CNRS)-Institut d'Optique Graduate School (IOGS)-Université Paris-Sud - Paris 11 (UP11), ONERA - The French Aerospace Lab [Palaiseau], ONERA-Université Paris Saclay (COmUE), Systèmes de Référence Temps Espace (SYRTE), Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut national des sciences de l'Univers (INSU - CNRS)-Observatoire de Paris, Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Centre National de la Recherche Scientifique (CNRS), Centre National d'Études Spatiales [Toulouse] (CNES), ICE, and Université Paris-Sud - Paris 11 (UP11)-Institut d'Optique Graduate School (IOGS)-Centre National de la Recherche Scientifique (CNRS)-Université Paris-Sud - Paris 11 (UP11)-Institut d'Optique Graduate School (IOGS)-Centre National de la Recherche Scientifique (CNRS)

Subjects
Condensed Matter - Other Condensed Matter, [PHYS.PHYS.PHYS-OPTICS]Physics [physics]/Physics [physics]/Optics [physics.optics], [PHYS.PHYS.PHYS-ATOM-PH]Physics [physics]/Physics [physics]/Atomic Physics [physics.atom-ph], Atomic Physics (physics.atom-ph), degenerate gas, [PHYS.COND.CM-GEN]Physics [physics]/Condensed Matter [cond-mat]/Other [cond-mat.other], gravimeter, 32.80.Pj, 32.80.Qk, 39.20.+q, 91.10.Pp, 91.10.Fc, FOS: Physical sciences, Atom interferometry, microgravity, Other Condensed Matter (cond-mat.other), Physics - Atomic Physics
Abstract

International audience; The accuracy and precision of current atom-interferometric inertialsensors rival state-of-the-art conventional devices using artifact-based test masses . Atomic sensors are well suited for fundamental measurements of gravito-inertial fields. The sensitivity required to test gravitational theories can be achieved by extending the baseline of the interferometer. The I.C.E. (Interférométrie Cohérente pour l'Espace) interferometer aims to achieve long interrogation times in compact apparatus via reduced gravity. We have tested a cold-atom source during airplane parabolic flights. We show that this environment is compatible with free-fall interferometric measurements using up to 4 second interrogation time. We present the next-generation apparatus using degenerate gases for low release-velocity atomic sources in space-borne experiments.

Published
2007

26. Changing computational research. The challenges ahead

Author

Neylon, Cameron, Aerts, J., Brown, C.T., Coles, S.J., Hatton, L., Lemire, D., Millman, K.J., Murray-Rust, P., Perez, F., Saunders, N., Shah, N., Smith, A., Varoquaux, G., Willighagen, E., Neylon, Cameron, Aerts, J., Brown, C.T., Coles, S.J., Hatton, L., Lemire, D., Millman, K.J., Murray-Rust, P., Perez, F., Saunders, N., Shah, N., Smith, A., Varoquaux, G., and Willighagen, E.

Abstract

EDITORIAL The past year has been an interesting one for those interested in reproducible research. There have been great examples of replicability [1, 2] in research communication, and examples of horrifying failure of reproducibility (as described in [3]) with serious questions being raised on the ability of our current system of research communication to guarantee, or even encourage, that published research be reproducible or replicable. When we launched the call for papers for Open Research Computation in late 2010 we saw a clear need for higher standards. Computational research should stand out as an exemplar of just how reproducible research can be, yet it falls short more often than not. With modern computational tools it is entirely possible to provide packages which allow direct replication of results. It is possible to provide data and code in the form of a functional virtual machine image along with automated tests to ensure everything is working as expected. But alongside this we can support the reader’s ability to modify and re-purpose tools, to run them against new data, indeed to support efforts to deliberately break the system to identify its limitations. In short, to do what we are supposed to do as scientists – replicate, reproduce, and test the limits of our models and understanding.

Published
2012

36. Scale-free and multifractal time dynamics of fMRI signals during rest and task.

Author

Ciuciu, P., Varoquaux, G., Abry, P., Sadaghiani, S., and Kleinschmidt, A.

Subjects
MULTIFRACTALS, MAGNETIC resonance imaging, BIOLOGICAL neural networks, NEURAL circuitry, UNIVARIATE analysis, IMMUNOSPECIFICITY
Abstract

Scaling temporal dynamics in functional MRI (fMRI) signals have been evidenced for a decade as intrinsic characteristics of ongoing brain activity (Zarahn et al., 1997). Recently, scaling properties were shown to fluctuate across brain networks and to be modulated between rest and task (He, 2011): notably, Hurst exponent, quantifying long memory, decreases under task in activating and deactivating brain regions. In most cases, such results were obtained: First, from univariate (voxelwise or regionwise) analysis, hence focusing on specific cognitive systems such as Resting-State Networks (RSNs) and raising the issue of the specificity of this scale-free dynamics modulation in RSNs. Second, using analysis tools designed to measure a single scaling exponent related to the second order statistics of the data, thus relying on models that either implicitly or explicitly assume Gaussianity and (asymptotic) self-similarity, while fMRI signals may significantly depart from those either of those two assumptions (Ciuciu et al., 2008; Wink et al., 2008). To address these issues, the present contribution elaborates on the analysis of the scaling properties of fMRI temporal dynamics by proposing two significant variations. First, scaling properties are technically investigated using the recently introduced Wavelet Leader-based Multifractal formalism (WLMF; Wendt et al., 2007). This measures a collection of scaling exponents, thus enables a richer and more versatile description of scale invariance (beyond correlation and Gaussianity), referred to as multifractality. Also, it benefits from improved estimation performance compared to tools previously used in the literature. Second, scaling properties are investigated in both RSN and non-RSN structures (e.g., artifacts), at a broader spatial scale than the voxel one, using a multivariate approach, namely the Multi-Subject Dictionary Learning (MSDL) algorithm (Varoquaux et al., 2011) that produces a set of spatial components that appear more sparse than their Independent Component Analysis (ICA) counterpart. These tools are combined and applied to a fMRI dataset comprising 12 subjects with resting-state and activation runs (Sadaghiani et al., 2009). Results stemming from those analysis confirm the already reported task-related decrease of long memory in functional networks, but also show that it occurs in artifacts, thus making this feature not specific to functional networks. Further, results indicate that most fMRI signals appear multifractal at rest except in non-cortical regions. Task-related modulation of multifractality appears only significant in functional networks and thus can be considered as the key property disentangling functional networks from artifacts. These finding are discussed in the light of the recent literature reporting scaling dynamics of EEG microstate sequences at rest and addressing non-stationarity issues in temporally independent fMRI modes. [ABSTRACT FROM AUTHOR]

Published
2012
Full Text
View/download PDF

37. Scale-free and multifractal time dynamics of fMRI signals during rest and task.

Author

Ciuciua, P., Varoquaux, G., Abry, P., Sadaghiani, S., and Kleinschmidt, A.

Subjects
MAGNETIC resonance imaging, BRAIN research, MULTIFRACTALS, CENTRAL nervous system, QUANTITATIVE research
Abstract

Scale-free (or 1/f spectral) properties in functional MRI (fMRI) signals have been evidenced for a decade as intrinsic characteristics of ongoing brain activity [70]. Recently, they were shown to fluctuate across brain networks and to be modulated between rest and task [37]: Notably, Hurst exponent, quantifying long memory, decreases during task in activating and deactivating brain regions. Such results were obtained using tools assuming Gaussianity and self-similar models, while fMRI signals may significantly depart from those two assumptions [24] and were often based on voxelwise or regionwise independent analysis, hence showing large inter-subject variability. The present article aims at contributing to the analysis of temporal scale-free properties of fMRI signals in two respects. First, a recent statistical analysis tool, the Wavelet Leader-based Multifractal formalism (WLMF), is used [68, 3]. It shows improved estimation performance, and it is framed within a rich and versatile class of models, that of multifractal processes. Second, scalefree properties are not investigated at the voxel or region scale. Instead, resting-state brain networks are extracted using a Multi- Subject Dictionary Learning (MSDL) algorithm [63]: It produces both a set of spatial components, and a set of times series, for each component and each subject, that conveys ongoing dynamics in functional networks but also in artifacts. These combined tools are applied to an fMRI dataset comprising 12-subjects with resting-state and activation runs [55]. This dataset enabled us to confirm the task-related decrease of long memory in functional networks but also in artifacts making this feature not specific to functional networks. Also, most of fMRI signals appear multifractal at rest except in non-cortical regions. Task-related modulation of multifractality appears only significant in functional networks and thus can be considered the key property to disentangle functional networks from artifacts. Our results are finally discussed in the light of the recent literature dealing with i) the scale-free dynamics of EEG microstate sequences at rest and ii) non-stationarities in temporally independent fMRI modes. [ABSTRACT FROM AUTHOR]

Published
2012
Full Text
View/download PDF

38. Sagnac effect in superfluid liquids

Author

Varoquaux, E and Varoquaux, G

Abstract

The interpretation of the Sagnac effect is re-examined in the context of recent cold atomic beam and superfluid experiments. A widespread misconception concerning the understanding of this effect in a superfluid liquid is discussed.

Published
2008

39. Brain covariance selection: Better individual functional connectivity models using population prior

Author

Varoquaux, G., Alexandre Gramfort, Poline, J. -B, Thirion, B., Laboratoire de Neuroimagerie Assistée par Ordinateur (LNAO), Commissariat à l'énergie atomique et aux énergies alternatives (CEA), Modelling brain structure, function and variability based on high-field MRI data (PARIETAL), Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Service NEUROSPIN (NEUROSPIN), Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Université Paris-Saclay, John Lafferty, Richard Zemel and John Shawe-Taylor, INRIA-INSERM collaboration, ANR-08-BLAN-0250,ViMAGINE,Neuroimagerie Multimodale des Processus Cérébraux Rapides(2008), Service NEUROSPIN (NEUROSPIN), Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Inria Saclay - Ile de France, and Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)

Subjects
FOS: Computer and information sciences, ACM: I.: Computing Methodologies/I.4: IMAGE PROCESSING AND COMPUTER VISION/I.4.9: Applications, Quantitative Biology::Neurons and Cognition, [SCCO.NEUR]Cognitive science/Neuroscience, functional connectivity, ACM: G.: Mathematics of Computing/G.3: PROBABILITY AND STATISTICS/G.3.5: Multivariate statistics, Machine Learning (stat.ML), [STAT.OT]Statistics [stat]/Other Statistics [stat.ML], neuroscience, Statistics - Machine Learning, FOS: Biological sciences, Quantitative Biology - Neurons and Cognition, [INFO.INFO-IM]Computer Science [cs]/Medical Imaging, Neurons and Cognition (q-bio.NC), covariance selection, ACM: J.: Computer Applications/J.3: LIFE AND MEDICAL SCIENCES/J.3.0: Biology and genetics
Abstract

Spontaneous brain activity, as observed in functional neuroimaging, has been shown to display reproducible structure that expresses brain architecture and carries markers of brain pathologies. An important view of modern neuroscience is that such large-scale structure of coherent activity reflects modularity properties of brain connectivity graphs. However, to date, there has been no demonstration that the limited and noisy data available in spontaneous activity observations could be used to learn full-brain probabilistic models that generalize to new data. Learning such models entails two main challenges: i) modeling full brain connectivity is a difficult estimation problem that faces the curse of dimensionality and ii) variability between subjects, coupled with the variability of functional signals between experimental runs, makes the use of multiple datasets challenging. We describe subject-level brain functional connectivity structure as a multivariate Gaussian process and introduce a new strategy to estimate it from group data, by imposing a common structure on the graphical model in the population. We show that individual models learned from functional Magnetic Resonance Imaging (fMRI) data using this population prior generalize better to unseen data than models based on alternative regularization schemes. To our knowledge, this is the first report of a cross-validated model of spontaneous brain activity. Finally, we use the estimated graphical model to explore the large-scale characteristics of functional architecture and show for the first time that known cognitive networks appear as the integrated communities of functional connectivity graph., in Advances in Neural Information Processing Systems, Vancouver : Canada (2010)

40. Body-weight variability and risk of cardiovascular outcomes in patients with type 1 diabetes: a retrospective observational analysis of data from the DCCT/EDIC population

Author

Petria, Iulia, Albuquerque, Samuel, Varoquaux, Gaël, Vie, Jill-Jênn, Venteclef, Nicolas, Mohammedi, Kamel, Roussel, Ronan, Camoin, Marion, Perseghin, Gianluca, Velho, Gilberto, Potier, Louis, Università degli Studi di Milano-Bicocca = University of Milano-Bicocca (UNIMIB), Institut Necker Enfants-Malades (INEM - UM 111 (UMR 8253 / U1151)), Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-Université Paris Cité (UPCité), Méthodes computationnelles et mathématiques pour comprendre la société et la santé à partir de données (SODA), Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), Biologie des maladies cardiovasculaires = Biology of Cardiovascular Diseases, Université de Bordeaux (UB)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS), Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP), Petria, I, Albuquerque, S, Varoquaux, G, Vie, J, Venteclef, N, Mohammedi, K, Roussel, R, Camoin, M, Perseghin, G, Velho, G, and Potier, L

Subjects
Type 1 diabete, Endocrinology, Diabetes and Metabolism, Body Weight, Myocardial Infarction, Cardiovascular risk, Cardiovascular System, Article clinique, Diabetes Mellitus, Type 1, Weight variation, Risk Factors, Diabetes Mellitus, Humans, Mortality, MED/13 - ENDOCRINOLOGIA, Cardiology and Cardiovascular Medicine, [SDV.MHEP]Life Sciences [q-bio]/Human health and pathology, Type 1, Retrospective Studies
Abstract

Background Cardiovascular risk and body-weight management are both emerging challenges of type 1 diabetes care. We evaluated the association between intraindividual variability of body-weight and risk of cardiovascular events in people with type 1 diabetes. Methods We analyzed 1,398 participants from the DCCT/EDIC studies. Five indices of intraindividual variability of body-weight were calculated for each participant taking into account body-weight measures obtained during the DCCT follow-up (average 6 ± 2 years). The Average Successive Variability (ASV) index, the main variable of interest, was defined as the average absolute difference between successive body-weight measures. The primary outcome was a composite of major adverse cardiovascular events (MACE: nonfatal myocardial infarction or stroke, or cardiovascular death) occurring during the subsequent EDIC follow-up (20 ± 3 years). All-cause death was a secondary outcome. Risk of outcomes were assessed by Cox proportional hazards regression analyses, adjusted for traditional cardiovascular risks factors, including BMI. Results The cumulative incidence of MACE and all-cause death during follow-up were 5.6% (n = 79) and 6.8% (n = 95), respectively. The adjusted Hazard Ratio (HR) for MACE by every increase of 1 standard deviation (SD) of ASV was 1.34 (95% CI, 1.06–1.66), p = 0.01. For all-cause death, the adjusted HR for 1 SD increase of ASV was 1.25 (1.03–1.50), p = 0.03. Similar results were observed when considering the other indices of intraindividual variability of body-weight. Conclusions High body-weight variability (body-weight cycling) is associated with increased risk of MACE and all-cause death in people with type 1 diabetes, independently of the BMI and traditional cardiovascular risk factors.

Published
2022

41. Individual Brain Charting dataset extension, third release for movie watching and retinotopy data.

Author

Pinho AL, Richard H, Ponce AF, Eickenberg M, Amadon A, Dohmatob E, Denghien I, Torre JJ, Shankar S, Aggarwal H, Thual A, Chapalain T, Ginisty C, Becuwe-Desmidt S, Roger S, Lecomte Y, Berland V, Laurier L, Joly-Testault V, Médiouni-Cloarec G, Doublé C, Martins B, Varoquaux G, Dehaene S, Hertz-Pannier L, and Thirion B

Subjects
Humans, Motion Pictures, Visual Cortex physiology, Visual Cortex diagnostic imaging, Brain physiology, Brain diagnostic imaging, Brain Mapping, Magnetic Resonance Imaging
Abstract

The Individual Brain Charting (IBC) is a multi-task functional Magnetic Resonance Imaging dataset acquired at high spatial-resolution and dedicated to the cognitive mapping of the human brain. It consists in the deep phenotyping of twelve individuals, covering a broad range of psychological domains suitable for functional-atlasing applications. Here, we present the inclusion of task data from both naturalistic stimuli and trial-based designs, to uncover structures of brain activation. We rely on the Fast Shared Response Model (FastSRM) to provide a data-driven solution for modelling naturalistic stimuli, typically containing many features. We show that data from left-out runs can be reconstructed using FastSRM, enabling the extraction of networks from the visual, auditory and language systems. We also present the topographic organization of the visual system through retinotopy. In total, six new tasks were added to IBC, wherein four trial-based retinotopic tasks contributed with a mapping of the visual field to the cortex. IBC is open access: source plus derivatives imaging data and meta-data are available in public repositories., (© 2024. Crown.)

Published
2024
Full Text
View/download PDF

42. The past, present, and future of the brain imaging data structure (BIDS).

Author

Poldrack RA, Markiewicz CJ, Appelhoff S, Ashar YK, Auer T, Baillet S, Bansal S, Beltrachini L, Benar CG, Bertazzoli G, Bhogawar S, Blair RW, Bortoletto M, Boudreau M, Brooks TL, Calhoun VD, Castelli FM, Clement P, Cohen AL, Cohen-Adad J, D'Ambrosio S, de Hollander G, de la Iglesia-Vayá M, de la Vega A, Delorme A, Devinsky O, Draschkow D, Duff EP, DuPre E, Earl E, Esteban O, Feingold FW, Flandin G, Galassi A, Gallitto G, Ganz M, Gau R, Gholam J, Ghosh SS, Giacomel A, Gillman AG, Gleeson P, Gramfort A, Guay S, Guidali G, Halchenko YO, Handwerker DA, Hardcastle N, Herholz P, Hermes D, Honey CJ, Innis RB, Ioanas HI, Jahn A, Karakuzu A, Keator DB, Kiar G, Kincses B, Laird AR, Lau JC, Lazari A, Legarreta JH, Li A, Li X, Love BC, Lu H, Marcantoni E, Maumet C, Mazzamuto G, Meisler SL, Mikkelsen M, Mutsaerts H, Nichols TE, Nikolaidis A, Nilsonne G, Niso G, Norgaard M, Okell TW, Oostenveld R, Ort E, Park PJ, Pawlik M, Pernet CR, Pestilli F, Petr J, Phillips C, Poline JB, Pollonini L, Raamana PR, Ritter P, Rizzo G, Robbins KA, Rockhill AP, Rogers C, Rokem A, Rorden C, Routier A, Saborit-Torres JM, Salo T, Schirner M, Smith RE, Spisak T, Sprenger J, Swann NC, Szinte M, Takerkart S, Thirion B, Thomas AG, Torabian S, Varoquaux G, Voytek B, Welzel J, Wilson M, Yarkoni T, and Gorgolewski KJ

Abstract

The Brain Imaging Data Structure (BIDS) is a community-driven standard for the organization of data and metadata from a growing range of neuroscience modalities. This paper is meant as a history of how the standard has developed and grown over time. We outline the principles behind the project, the mechanisms by which it has been extended, and some of the challenges being addressed as it evolves. We also discuss the lessons learned through the project, with the aim of enabling researchers in other domains to learn from the success of BIDS., Competing Interests: Gaia Rizzo is an employee of Invicro. No other authors have competing interests to declare., (© 2024 Massachusetts Institute of Technology. Published under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.)

Published
2024
Full Text
View/download PDF

43. Understanding metric-related pitfalls in image analysis validation.

Author

Reinke A, Tizabi MD, Baumgartner M, Eisenmann M, Heckmann-Nötzel D, Kavur AE, Rädsch T, Sudre CH, Acion L, Antonelli M, Arbel T, Bakas S, Benis A, Blaschko M, Buettner F, Cardoso MJ, Cheplygina V, Chen J, Christodoulou E, Cimini BA, Collins GS, Farahani K, Ferrer L, Galdran A, van Ginneken B, Glocker B, Godau P, Haase R, Hashimoto DA, Hoffman MM, Huisman M, Isensee F, Jannin P, Kahn CE, Kainmueller D, Kainz B, Karargyris A, Karthikesalingam A, Kenngott H, Kleesiek J, Kofler F, Kooi T, Kopp-Schneider A, Kozubek M, Kreshuk A, Kurc T, Landman BA, Litjens G, Madani A, Maier-Hein K, Martel AL, Mattson P, Meijering E, Menze B, Moons KGM, Müller H, Nichyporuk B, Nickel F, Petersen J, Rafelski SM, Rajpoot N, Reyes M, Riegler MA, Rieke N, Saez-Rodriguez J, Sánchez CI, Shetty S, van Smeden M, Summers RM, Taha AA, Tiulpin A, Tsaftaris SA, Calster BV, Varoquaux G, Wiesenfarth M, Yaniv ZR, Jäger PF, and Maier-Hein L

Abstract

Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.

Published
2024

44. Metrics reloaded: recommendations for image analysis validation.

Author

Maier-Hein L, Reinke A, Godau P, Tizabi MD, Buettner F, Christodoulou E, Glocker B, Isensee F, Kleesiek J, Kozubek M, Reyes M, Riegler MA, Wiesenfarth M, Kavur AE, Sudre CH, Baumgartner M, Eisenmann M, Heckmann-Nötzel D, Rädsch T, Acion L, Antonelli M, Arbel T, Bakas S, Benis A, Blaschko MB, Cardoso MJ, Cheplygina V, Cimini BA, Collins GS, Farahani K, Ferrer L, Galdran A, van Ginneken B, Haase R, Hashimoto DA, Hoffman MM, Huisman M, Jannin P, Kahn CE, Kainmueller D, Kainz B, Karargyris A, Karthikesalingam A, Kofler F, Kopp-Schneider A, Kreshuk A, Kurc T, Landman BA, Litjens G, Madani A, Maier-Hein K, Martel AL, Mattson P, Meijering E, Menze B, Moons KGM, Müller H, Nichyporuk B, Nickel F, Petersen J, Rajpoot N, Rieke N, Saez-Rodriguez J, Sánchez CI, Shetty S, van Smeden M, Summers RM, Taha AA, Tiulpin A, Tsaftaris SA, Van Calster B, Varoquaux G, and Jäger PF

Subjects
Machine Learning, Semantics, Image Processing, Computer-Assisted, Algorithms
Abstract

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases., (© 2024. Springer Nature America, Inc.)

Published
2024
Full Text
View/download PDF

45. Predicting future cognitive decline from non-brain and multimodal brain imaging data in healthy and pathological aging.

Author

Hebling Vieira B, Liem F, Dadi K, Engemann DA, Gramfort A, Bellec P, Craddock RC, Damoiseaux JS, Steele CJ, Yarkoni T, Langer N, Margulies DS, and Varoquaux G

Subjects
Activities of Daily Living, Aged, Aged, 80 and over, Brain diagnostic imaging, Cognitive Dysfunction pathology, Humans, Magnetic Resonance Imaging methods, Middle Aged, Neuroimaging, Aging pathology, Aging physiology, Brain pathology, Cognitive Dysfunction diagnostic imaging
Abstract

Previous literature has focused on predicting a diagnostic label from structural brain imaging. Since subtle changes in the brain precede a cognitive decline in healthy and pathological aging, our study predicts future decline as a continuous trajectory instead. Here, we tested whether baseline multimodal neuroimaging data improve the prediction of future cognitive decline in healthy and pathological aging. Nonbrain data (demographics, clinical, and neuropsychological scores), structural MRI, and functional connectivity data from OASIS-3 (N = 662; age = 46-96 years) were entered into cross-validated multitarget random forest models to predict future cognitive decline (measured by CDR and MMSE), on average 5.8 years into the future. The analysis was preregistered, and all analysis code is publicly available. Combining non-brain with structural data improved the continuous prediction of future cognitive decline (best test-set performance: R2 = 0.42). Cognitive performance, daily functioning, and subcortical volume drove the performance of our model. Including functional connectivity did not improve predictive accuracy. In the future, the prognosis of age-related cognitive decline may enable earlier and more effective individualized cognitive, pharmacological, and behavioral interventions., Competing Interests: Disclosure statement The authors declare no conflict of interest., (Copyright © 2022. Published by Elsevier Inc.)

Published
2022
Full Text
View/download PDF

46. Insights from an autism imaging biomarker challenge: Promises and threats to biomarker discovery.

Author

Traut N, Heuer K, Lemaître G, Beggiato A, Germanaud D, Elmaleh M, Bethegnies A, Bonnasse-Gahot L, Cai W, Chambon S, Cliquet F, Ghriss A, Guigui N, de Pierrefeu A, Wang M, Zantedeschi V, Boucaud A, van den Bossche J, Kegl B, Delorme R, Bourgeron T, Toro R, and Varoquaux G

Subjects
Biomarkers, Brain diagnostic imaging, Humans, Magnetic Resonance Imaging methods, Autism Spectrum Disorder diagnostic imaging, Autistic Disorder diagnostic imaging
Abstract

MRI has been extensively used to identify anatomical and functional differences in Autism Spectrum Disorder (ASD). Yet, many of these findings have proven difficult to replicate because studies rely on small cohorts and are built on many complex, undisclosed, analytic choices. We conducted an international challenge to predict ASD diagnosis from MRI data, where we provided preprocessed anatomical and functional MRI data from > 2,000 individuals. Evaluation of the predictions was rigorously blinded. 146 challengers submitted prediction algorithms, which were evaluated at the end of the challenge using unseen data and an additional acquisition site. On the best algorithms, we studied the importance of MRI modalities, brain regions, and sample size. We found evidence that MRI could predict ASD diagnosis: the 10 best algorithms reliably predicted diagnosis with AUC∼0.80 - far superior to what can be currently obtained using genotyping data in cohorts 20-times larger. We observed that functional MRI was more important for prediction than anatomical MRI, and that increasing sample size steadily increased prediction accuracy, providing an efficient strategy to improve biomarkers. We also observed that despite a strong incentive to generalise to unseen data, model development on a given dataset faces the risk of overfitting: performing well in cross-validation on the data at hand, but not generalising. Finally, we were able to predict ASD diagnosis on an external sample added after the end of the challenge (EU-AIMS), although with a lower prediction accuracy (AUC=0.72). This indicates that despite being based on a large multisite cohort, our challenge still produced biomarkers fragile in the face of dataset shifts., Competing Interests: Competing interests The authors declare that they have no competing interests., (Copyright © 2022. Published by Elsevier Inc.)

Published
2022
Full Text
View/download PDF

48. Comprehensive decoding mental processes from Web repositories of functional brain images.

Author

Menuet R, Meudec R, Dockès J, Varoquaux G, and Thirion B

Subjects
Cognition, Head, Magnetic Resonance Imaging methods, Brain diagnostic imaging, Brain Mapping methods
Abstract

Associating brain systems with mental processes requires statistical analysis of brain activity across many cognitive processes. These analyses typically face a difficult compromise between scope-from domain-specific to system-level analysis-and accuracy. Using all the functional Magnetic Resonance Imaging (fMRI) statistical maps of the largest data repository available, we trained machine-learning models that decode the cognitive concepts probed in unseen studies. For this, we leveraged two comprehensive resources: NeuroVault-an open repository of fMRI statistical maps with unconstrained annotations-and Cognitive Atlas-an ontology of cognition. We labeled NeuroVault images with Cognitive Atlas concepts occurring in their associated metadata. We trained neural networks to predict these cognitive labels on tens of thousands of brain images. Overcoming the heterogeneity, imbalance and noise in the training data, we successfully decoded more than 50 classes of mental processes on a large test set. This success demonstrates that image-based meta-analyses can be undertaken at scale and with minimal manual data curation. It enables broad reverse inferences, that is, concluding on mental processes given the observed brain activity., (© 2022. The Author(s).)

Published
2022
Full Text
View/download PDF

49. Benchmarking missing-values approaches for predictive models on health databases.

Author

Perez-Lebel A, Varoquaux G, Le Morvan M, Josse J, and Poline JB

Subjects
Data Collection, Databases, Factual, Electronic Health Records, Benchmarking, Machine Learning
Abstract

Background: As databases grow larger, it becomes harder to fully control their collection, and they frequently come with missing values. These large databases are well suited to train machine learning models, e.g., for forecasting or to extract biomarkers in biomedical settings. Such predictive approaches can use discriminative-rather than generative-modeling and thus open the door to new missing-values strategies. Yet existing empirical evaluations of strategies to handle missing values have focused on inferential statistics., Results: Here we conduct a systematic benchmark of missing-values strategies in predictive models with a focus on large health databases: 4 electronic health record datasets, 1 population brain imaging database, 1 health survey, and 2 intensive care surveys. Using gradient-boosted trees, we compare native support for missing values with simple and state-of-the-art imputation prior to learning. We investigate prediction accuracy and computational time. For prediction after imputation, we find that adding an indicator to express which values have been imputed is important, suggesting that the data are missing not at random. Elaborate missing-values imputation can improve prediction compared to simple strategies but requires longer computational time on large data. Learning trees that model missing values-with missing incorporated attribute-leads to robust, fast, and well-performing predictive modeling., Conclusions: Native support for missing values in supervised machine learning predicts better than state-of-the-art imputation with much less computational cost. When using imputation, it is important to add indicator columns expressing which values have been imputed., (© The Author(s) 2022. Published by Oxford University Press GigaScience.)

Published
2022
Full Text
View/download PDF

50. Machine learning for medical imaging: methodological failures and recommendations for the future.

Author

Varoquaux G and Cheplygina V

Abstract

Research in computer analysis of medical images bears many promises to improve patients' health. However, a number of systematic challenges are slowing down the progress of the field, from limitations of the data, such as biases, to research incentives, such as optimizing for publication. In this paper we review roadblocks to developing and assessing methods. Building our analysis on evidence from the literature and data challenges, we show that at every step, potential biases can creep in. On a positive note, we also discuss on-going efforts to counteract these problems. Finally we provide recommendations on how to further address these problems in the future., (© 2022. The Author(s).)

Published
2022
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results