Author: "University College of London [London] (UCL)-University College of London [London] (UCL)" / Publication Year Range: Last 10 years - Searchworks@Jio Institute Digital Library Search Results

Your search keyword '"University College of London [London] (UCL)-University College of London [London] (UCL)"' showing total 70 results

Start Over Author "University College of London [London] (UCL)-University College of London [London] (UCL)" Publication Year Range Last 10 years

70 results on '"University College of London [London] (UCL)-University College of London [London] (UCL)"'

1. Spinal cord involvement in multiple sclerosis and neuromyelitis optica spectrum disorders

Author: Eoin P. Flanagan, Benjamin Greenberg, Olga Ciccarelli, Olaf Stüve, Jean-Philippe Ranjeva, Maria Pia Sormani, Regina Schlaerger, Myla D. Goldman, Jeffrey A. Cohen, Emmanuelle Waubant, Franz Fazekas, Ellen M. Mowry, Daniel S. Reich, Frederik Barkhof, Hans Lassmann, Anthony Traboulsee, Sandra Vukusic, Mar Tintoré, Bernard M. J. Uitdehaag, Anke Henning, Daniel Pelletier, Alan J. Thompson, Junqian Xu, Jorge Correale, Burkhard Becher, Stephen Reingold, Nicola De Stefano, Bruce F. Bebo, Izlem Izbudak, Claudia Chien, Stephen C. Reingold, Steven Galetta, Claudia Gandini Wheeler-Kingshott, Sebastien Ourselin, Alex Rovira, Mark S. Freedman, Brenda Banwell, Aaron E. Miller, Xavier Montalban, Cornelia Laule, Claudia F. Lucchinetti, Giancarlo Comi, Peter A. Calabresi, Hans-Peter Hartung, Maria Trojano, Ludwig Kappos, Bernhard Hemmer, Bruce D. Trapp, Brian G. Weinshenker, Kazuo Fujihara, Tanuja Chitnis, Jérôme De Seze, Francois Bethoux, Per Soelberg Sørensen, Fred D. Lublin, Alexander U. Brandt, Carsten Lukas, Wallace J Brownlee, Maria Pia Amato, Jeremy Chatway, David Miller, Friedemann Paul, Maria A. Rocca, Ruth Ann Marrie, Michael J. Levy, University of Pennsylvania, VU University Medical Center [Amsterdam], University College of London [London] (UCL), Institute of Experimental Neurology, Division of Neuroscience, San Raffaele Scientific Institute, Departamento de Neurologia, Instituto de Investigaciones Neurológicas 'Dr. Raul Carrea', Instituto de Investigaciones Neurológicas 'Dr. Raul Carrea', Service de Neurologie [Strasbourg], CHU Strasbourg-Hopital Civil, Dept. of Neurological and Behavioural Sciences, Siena, Department of Neurology [Austria], Medical University Graz, Ottawa Hospital Research Institute [Ottawa] (OHRI), Tohoku University Graduate School of Medicine, Munich Cluster of Systems Neurology (SyNery), Clinical Neuroimmunology, Department of Biomedicine, University of Basel, Basel, Matière et Systèmes Complexes (MSC), Université Paris Diderot - Paris 7 (UPD7)-Centre National de la Recherche Scientifique (CNRS), St. Josef Hospital, Ruhr-University Bochum, Vall Hebron Research Institute (VHIR), Translational imaging group [London] (TIG), Centre for Medical Image Computing (CMIC), University College of London [London] (UCL)-University College of London [London] (UCL)-Department of Medical Physics and Biomedical Engineering (UCL), Max Delbrueck Centre for Molecular Medicine, Departments of Neurology and Immunobiology [Yale], Yale School of Medicine [New Haven, Connecticut] (YSM), Centre de résonance magnétique biologique et médicale (CRMBM), Aix Marseille Université (AMU)-Assistance Publique - Hôpitaux de Marseille (APHM)-Centre National de la Recherche Scientifique (CNRS), Centre d'Exploration Métabolique par Résonance Magnétique [Hôpital de la Timone - APHM] (CEMEREM), Hôpital de la Timone [CHU - APHM] (TIMONE)-Centre de résonance magnétique biologique et médicale (CRMBM), Aix Marseille Université (AMU)-Assistance Publique - Hôpitaux de Marseille (APHM)-Centre National de la Recherche Scientifique (CNRS)-Aix Marseille Université (AMU)-Assistance Publique - Hôpitaux de Marseille (APHM)-Centre National de la Recherche Scientifique (CNRS), Aix Marseille Université (AMU), Centre National de la Recherche Scientifique (CNRS), National Institutes of Health [Bethesda] (NIH), Vall d'Hebron University Hospital [Barcelona], Dept. of Neurology, University Hospital Rigshospitalet, Centre d'Esclerosi Múltiple de Catalunya (CemCat), Division of Geriatric Medicine, University of British Columbia (UBC), Service de Neurologie [Lyon], CHU Lyon, Department of Neurology, University of California [San Francisco] (UC San Francisco), University of California (UC)-University of California (UC), NMR Research Unit [London], Institute of Neurology [London], University College of London [London] (UCL)-University College of London [London] (UCL), Icahn School of Medicine at Mount Sinai [New York] (MSSM), University of Pennsylvania [Philadelphia], Matière et Systèmes Complexes (MSC (UMR_7057)), Centre National de la Recherche Scientifique (CNRS)-Université Paris Diderot - Paris 7 (UPD7), Yale University School of Medicine, Centre d'Exploration Métabolique par Résonance Magnétique [Hôpital de la Timone - AP-HM] (CEMEREM), Aix Marseille Université (AMU)-Assistance Publique - Hôpitaux de Marseille (APHM)-Centre National de la Recherche Scientifique (CNRS)-Aix Marseille Université (AMU)-Assistance Publique - Hôpitaux de Marseille (APHM)-Centre National de la Recherche Scientifique (CNRS)- Hôpital de la Timone [CHU - APHM] (TIMONE), University of California [San Francisco] (UCSF), and University of California-University of California
Subjects: 0301 basic medicine, Pediatrics, medicine.medical_specialty, Multiple Sclerosis, [SDV.IB.IMA]Life Sciences [q-bio]/Bioengineering/Imaging, Spinal cord involvement, 03 medical and health sciences, Myelopathy, 0302 clinical medicine, medicine, Humans, Pathological, ComputingMilieux_MISCELLANEOUS, business.industry, Multiple sclerosis, Neuromyelitis Optica, Clinical course, medicine.disease, Spinal cord, 3. Good health, Clinical trial, 030104 developmental biology, medicine.anatomical_structure, Spinal Cord, Neuromyelitis Optica Spectrum Disorders, Neurology (clinical), business, 030217 neurology & neurosurgery
Abstract: Spinal cord involvement is an important cause of disability in patients with multiple sclerosis or neuromyelitis optica spectrum disorders (NMOSDs). Multiple sclerosis and NMOSDs can be distinguished from other disorders that cause myelopathy by results from laboratory and radiological investigations. However, limitations in the sensitivity and specificity of spinal cord imaging and poor correlation with disability megasures have impeded the understanding of the relationship between spinal cord involvement and clinical manifestations. Nevertheless, studies of the pathological features of multiple sclerosis and NMOSDs have shown that quantitatively different mechanisms lead to differences in clinical course and pattern of accrual of permanent disability in the two disorders. Better understanding of these mechanisms is necessary to develop more informative clinical measures, electrophysiological methods, fluid biomarkers, and imaging techniques to detect and monitor spinal cord involvement in the diagnosis and management of patients with multiple sclerosis or NMOSDs, and as outcome measures in clinical trials.
Published: 2019

2. Reduced acquisition time PET pharmacokinetic modelling using simultaneous ASL–MRI: proof of concept

Author: Scott, Catherine J, Jiao, Jieqing, Melbourne, Andrew, Burgos, Ninon, Cash, David M, De Vita, Enrico, Markiewicz, Pawel J, O'Connor, Antoinette, Thomas, David L, Weston, Philip SJ, Schott, Jonathan M, Hutton, Brian F, Ourselin, Sébastien, Translational imaging group [London] (TIG), Centre for Medical Image Computing (CMIC), University College of London [London] (UCL)-University College of London [London] (UCL)-Department of Medical Physics and Biomedical Engineering (UCL), University College of London [London] (UCL), Algorithms, models and methods for images and signals of the human brain (ARAMIS), Sorbonne Université (SU)-Inria de Paris, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Institut du Cerveau et de la Moëlle Epinière = Brain and Spine Institute (ICM), Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS), Dementia Research Centre [London] (DRC), UCL, Institute of Neurology [London], University College London Hospitals (UCLH), Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Institut du Cerveau = Paris Brain Institute (ICM), Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Sorbonne Université (SU)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS), Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS), Department of Medical Physics and Biomedical Engineering (UCL), and University College of London [London] (UCL)-University College of London [London] (UCL)-Centre for Medical Image Computing (CMIC)
Subjects: Male, Positron emission tomography, Amyloid beta-Peptides, Aniline Compounds, cerebral blood flow, Models, Cardiovascular, Original Articles, Cerebral blood flow, Reduced acquisition time, Proof of Concept Study, Arterial spin labelling, Pharmacokinetic modelling, Alzheimer Disease, Cerebrovascular Circulation, Positron-Emission Tomography, [INFO.INFO-IM]Computer Science [cs]/Medical Imaging, Humans, Ethylene Glycols, Female, pharmacokinetic modelling, reduced acquisition time, arterial spin labelling
Abstract: International audience; Pharmacokinetic modelling on dynamic positron emission tomography (PET) data is a quantitative technique. However, the long acquisition time is prohibitive for routine clinical use. Instead, the semi-quantitative standardised uptake value ratio (SUVR) from a shorter static acquisition is used, despite its sensitivity to blood flow confounding longitudinal analysis. A method has been proposed to reduce the dynamic acquisition time for quantification by incorporating cerebral blood flow (CBF) information from arterial spin labelling (ASL) magnetic resonance imaging (MRI) into the pharmacokinetic modelling. In this work, we optimise and validate this framework for a study of ageing and preclinical Alzheimer's disease. This methodology adapts the simplified reference tissue model (SRTM) for a reduced acquisition time (RT-SRTM) and is applied to [ 18 F]-florbetapir PET data for amyloid-b quantification. Evaluation shows that the optimised RT-SRTM can achieve amyloid burden estimation from a 30-min PET/MR acquisition which is comparable with the gold standard SRTM applied to 60 min of PET data. Conversely, SUVR showed a significantly higher error and bias, and a statistically significant correlation with tracer delivery due to the influence of blood flow. The optimised RT-SRTM produced amyloid burden estimates which were uncorrelated with tracer delivery indicating its suitability for longitudinal studies.
Published: 2018

3. Diagnosis of Alzheimer’s Disease Through Identification of Abnormality Patterns in FDG PET Data

Author: Burgos, Ninon, Samper-González, Jorge, Bertrand, Anne, Habert, Marie-Odile, Ourselin, Sébastien, Durrleman, Stanley, Cardoso, Jorge, Colliot, Olivier, Algorithms, models and methods for images and signals of the human brain (ARAMIS), Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut du Cerveau et de la Moëlle Epinière = Brain and Spine Institute (ICM), Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Inria de Paris, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), Service de Neuroradiologie [CHU Pitié-Salpêtrière], CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP), Laboratoire d'Imagerie Biomédicale (LIB), Centre National de la Recherche Scientifique (CNRS)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Université Pierre et Marie Curie - Paris 6 (UPMC), Translational imaging group [London] (TIG), Centre for Medical Image Computing (CMIC), University College of London [London] (UCL)-University College of London [London] (UCL)-Department of Medical Physics and Biomedical Engineering (UCL), University College of London [London] (UCL), Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Inria de Paris, Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU), Laboratoire d'Imagerie Biomédicale [Paris] (LIB), Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS), ANR-17-CE18-0016,ZINELABEL,Double marquage site-spécifique des protéines au moyen de tétrazines(2017), Burgos, Ninon, Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU), Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Centre National de la Recherche Scientifique (CNRS)-Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Centre National de la Recherche Scientifique (CNRS)-Inria de Paris, Service de neuroradiologie diagnostique et fonctionnelle [CHU Pitié-Salpêtrière], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU), Department of Medical Physics and Biomedical Engineering (UCL), and University College of London [London] (UCL)-University College of London [London] (UCL)-Centre for Medical Image Computing (CMIC)
Subjects: [INFO.INFO-IM] Computer Science [cs]/Medical Imaging, [INFO.INFO-IM]Computer Science [cs]/Medical Imaging, ComputingMilieux_MISCELLANEOUS
Abstract: International audience
Published: 2017

4. Early Diagnosis of Alzheimer’s Disease Using Subject-Specific Models of FDG-PET Data

Author: Burgos, Ninon, Samper-González, Jorge, Cardoso, Jorge, Durrleman, Stanley, Ourselin, Sébastien, Colliot, Olivier, Algorithms, models and methods for images and signals of the human brain (ARAMIS), Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut du Cerveau et de la Moëlle Epinière = Brain and Spine Institute (ICM), Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Inria de Paris, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), Translational imaging group [London] (TIG), Centre for Medical Image Computing (CMIC), University College of London [London] (UCL)-University College of London [London] (UCL)-Department of Medical Physics and Biomedical Engineering (UCL), University College of London [London] (UCL), Burgos, Ninon, Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Inria de Paris, Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Centre National de la Recherche Scientifique (CNRS)-Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Centre National de la Recherche Scientifique (CNRS)-Inria de Paris, Department of Medical Physics and Biomedical Engineering (UCL), and University College of London [London] (UCL)-University College of London [London] (UCL)-Centre for Medical Image Computing (CMIC)
Subjects: [INFO.INFO-IM] Computer Science [cs]/Medical Imaging, [INFO.INFO-IM]Computer Science [cs]/Medical Imaging
Abstract: International audience; Background: In machine learning classification methods developed for dementia studies,neuroimaging features, e.g. glucose consumption extracted from PET images, are often used todraw the border that differentiates normality from abnormality. However, these features areaffected by the anatomical variability present in the population, which acts as a confounding factormaking the task of finding the frontier (i.e. the decision function) between normality andabnormality very challenging.
Published: 2017

5. MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting

Author: Biggs, Felix, Schrab, Antonin, Gretton, Arthur, University College of London [London] (UCL), Gatsby Computational Neuroscience Unit, Department of Computer science [University College of London] (UCL-CS), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: Methodology (stat.ME), FOS: Computer and information sciences, Computer Science - Machine Learning, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, FOS: Mathematics, Machine Learning (stat.ML), Mathematics - Statistics Theory, Statistics Theory (math.ST), Statistics - Methodology, Machine Learning (cs.LG)
Abstract: We propose novel statistics which maximise the power of a two-sample test based on the Maximum Mean Discrepancy (MMD), by adapting over the set of kernels used in defining it. For finite sets, this reduces to combining (normalised) MMD values under each of these kernels via a weighted soft maximum. Exponential concentration bounds are proved for our proposed statistics under the null and alternative. We further show how these kernels can be chosen in a data-dependent but permutation-independent way, in a well-calibrated test, avoiding data splitting. This technique applies more broadly to general permutation-based MMD testing, and includes the use of deep kernels with features learnt using unsupervised models such as auto-encoders. We highlight the applicability of our MMD-FUSE test on both synthetic low-dimensional and real-world high-dimensional data, and compare its performance in terms of power against current state-of-the-art kernel tests., 42 pages, 7 figures, 1 table
Published: 2023

6. Learning via Wasserstein-Based High Probability Generalisation Bounds

Author: Viallard, Paul, Haddouche, Maxime, Şimşekli, Umut, Guedj, Benjamin, Statistical Machine Learning and Parsimony (SIERRA), Département d'informatique - ENS Paris (DI-ENS), École normale supérieure - Paris (ENS-PSL), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-École normale supérieure - Paris (ENS-PSL), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Centre National de la Recherche Scientifique (CNRS)-Inria de Paris, Institut National de Recherche en Informatique et en Automatique (Inria), The Inria London Programme (Inria-London), Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), University College of London [London] (UCL), Grant number EP/R013616/1, ANR-19-P3IA-0001,PRAIRIE,PaRis Artificial Intelligence Research InstitutE(2019), ANR-18-CE40-0016,BEAGLE,Apprentissage PAC-bayésien agnostique(2018), ANR-18-CE23-0015,APRIORI,Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations(2018), and European Project: 101039676,DYNASTY
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Machine Learning (stat.ML), Machine Learning (cs.LG)
Abstract: Minimising upper bounds on the population risk or the generalisation gap has been widely used in structural risk minimisation (SRM) - this is in particular at the core of PAC-Bayesian learning. Despite its successes and unfailing surge of interest in recent years, a limitation of the PAC-Bayesian framework is that most bounds involve a Kullback-Leibler (KL) divergence term (or its variations), which might exhibit erratic behavior and fail to capture the underlying geometric structure of the learning problem - hence restricting its use in practical applications. As a remedy, recent studies have attempted to replace the KL divergence in the PAC-Bayesian bounds with the Wasserstein distance. Even though these bounds alleviated the aforementioned issues to a certain extent, they either hold in expectation, are for bounded losses, or are nontrivial to minimize in an SRM framework. In this work, we contribute to this line of research and prove novel Wasserstein distance-based PAC-Bayesian generalisation bounds for both batch learning with independent and identically distributed (i.i.d.) data, and online learning with potentially non-i.i.d. data. Contrary to previous art, our bounds are stronger in the sense that (i) they hold with high probability, (ii) they apply to unbounded (potentially heavy-tailed) losses, and (iii) they lead to optimizable training objectives that can be used in SRM. As a result we derive novel Wasserstein-based PAC-Bayesian learning algorithms and we illustrate their empirical advantage on a variety of experiments.
Published: 2023

7. Longitudinal multiple sclerosis lesion segmentation: Resource and challenge

Author: Pierre-Louis Bazin, Pierre Maurel, James Nguyen, Amod Jog, Bram Platel, Heinz Handels, Ariel Birenbaum, Aaron Carass, Oskar Maier, Diana M. Sima, Daniel S. Reich, Devrim Unay, Olga Ciccarelli, Abhijith Chunduru, Jennifer L. Cuzzocreo, Saurabh Jain, Dzung L. Pham, Ciprian M. Crainiceanu, Manuel Jorge Cardoso, Leonardo O. Iheme, Adrian Gherman, Tal Arbel, Snehashis Roy, Olivier Commowick, Ramanathan Muthuganapathy, Xavier Tomas-Fernandez, Dirk Smeets, Suthirth Vaidya, Claudia A. M. Wheeler-Kingshott, Andrew Jesson, Carole H. Sudre, Laurence Catanese, Christian Barillot, Hayit Greenspan, Niamh Cawley, Sebastien Ourselin, Hrishikesh Deshpande, Mohsen Ghafoorian, Peter A. Calabresi, Simon K. Warfield, Elizabeth Magrath, Julia Button, Lotta Maria Ellingsen, Ferran Prados, Jerry L. Prince, Ganapathy Krishnamurthi, Department of Computer Science [Baltimore], Johns Hopkins University (JHU), Department of Electrical and Computer Engineering [Johns Hopkins University] (ECE), Center for Neuroscience and Regenerative Medicine [Bethesda] (CNRM), Henry M. Jackson Foundation for the Advancement of Military Medicine (HJM), Department of Radiology and Radiological Science [Baltimore], Johns Hopkins University School of Medicine [Baltimore], NMR Research Unit [London], Institute of Neurology [London], University College of London [London] (UCL)-University College of London [London] (UCL), Translational imaging group [London] (TIG), Centre for Medical Image Computing (CMIC), University College of London [London] (UCL)-University College of London [London] (UCL)-Department of Medical Physics and Biomedical Engineering (UCL), University College of London [London] (UCL), Dementia Research Centre [London] (DRC), Centre national de recherches météorologiques (CNRM), Institut national des sciences de l'Univers (INSU - CNRS)-Observatoire Midi-Pyrénées (OMP), Institut de Recherche pour le Développement (IRD)-Université Toulouse III - Paul Sabatier (UT3), Université de Toulouse (UT)-Université de Toulouse (UT)-Institut national des sciences de l'Univers (INSU - CNRS)-Centre National d'Études Spatiales [Toulouse] (CNES)-Centre National de la Recherche Scientifique (CNRS)-Météo-France -Institut de Recherche pour le Développement (IRD)-Université Toulouse III - Paul Sabatier (UT3), Université de Toulouse (UT)-Université de Toulouse (UT)-Institut national des sciences de l'Univers (INSU - CNRS)-Centre National d'Études Spatiales [Toulouse] (CNES)-Centre National de la Recherche Scientifique (CNRS)-Météo-France -Centre National de la Recherche Scientifique (CNRS), Vision, Action et Gestion d'informations en Santé (VisAGeS), Institut National de la Santé et de la Recherche Médicale (INSERM)-Inria Rennes – Bretagne Atlantique, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-SIGNAUX ET IMAGES NUMÉRIQUES, ROBOTIQUE (IRISA-D5), Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA), Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Computational Radiology Laboratory [Boston] (CRL), Brigham and Women's Hospital [Boston]-Boston Children's Hospital, Department of Engineering Design [Madras], Indian Institute of Technology Madras (IIT Madras), Centre for Intelligent Machines (CIM), McGill University = Université McGill [Montréal, Canada], Institute of Medical Informatics [Lübeck], Universität zu Lübeck = University of Lübeck [Lübeck], Biomedical Engineering [Istanbul], Bahcesehir University [Istanbul], Icometrix [Leuven], Institute for Computing and Information Sciences [Nijmegen] (ICIS), Radboud University [Nijmegen], Diagnostic Image Analysis Group [Nijmegen], Radboud University Medical Center [Nijmegen], Department of Electrical Engineering, Tel Aviv University (TAU), Department of Biomedical Engineering [Tel Aviv] (TAU), Department of neurophysics [Leipzig], Max Planck Institute for Human Cognitive and Brain Sciences [Leipzig] (IMPNSC), Max-Planck-Gesellschaft-Max-Planck-Gesellschaft, Department of Biostatistics [Baltimore], Department of electrical and computer engineering [Reykjavík], University of Iceland [Reykjavik], Translational neuroradiology unit [Bethesda], National Institute of Neurological Disorders and Stroke [Bethesda] (NINDS), National Institutes of Health [Bethesda] (NIH)-National Institutes of Health [Bethesda] (NIH), Météo France-Centre National de la Recherche Scientifique (CNRS), Université de Rennes 1 (UR1), Université de Rennes (UNIV-RENNES)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Rennes 1 (UR1), Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Boston Children's Hospital-Brigham and Women's Hospital [Boston], McGill University, Universität zu Lübeck [Lübeck] - University of Lübeck [Lübeck], Radboud university [Nijmegen], Tel Aviv University [Tel Aviv], Department of Biomedical Engineering [Tel Aviv], Department of Medical Physics and Biomedical Engineering (UCL), University College of London [London] (UCL)-University College of London [London] (UCL)-Centre for Medical Image Computing (CMIC), Institut national des sciences de l'Univers (INSU - CNRS)-Météo France-Centre National de la Recherche Scientifique (CNRS), SIGNAUX ET IMAGES NUMÉRIQUES, ROBOTIQUE (IRISA-D5), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Inria Rennes – Bretagne Atlantique, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de la Santé et de la Recherche Médicale (INSERM), Universität zu Lübeck [Lübeck], Commowick, Olivier, Université Fédérale Toulouse Midi-Pyrénées-Université Fédérale Toulouse Midi-Pyrénées-Institut national des sciences de l'Univers (INSU - CNRS)-Centre National d'Études Spatiales [Toulouse] (CNES)-Centre National de la Recherche Scientifique (CNRS)-Météo-France -Institut de Recherche pour le Développement (IRD)-Université Toulouse III - Paul Sabatier (UT3), Université Fédérale Toulouse Midi-Pyrénées-Université Fédérale Toulouse Midi-Pyrénées-Institut national des sciences de l'Univers (INSU - CNRS)-Centre National d'Études Spatiales [Toulouse] (CNES)-Centre National de la Recherche Scientifique (CNRS)-Météo-France -Centre National de la Recherche Scientifique (CNRS), Université de Bretagne Sud (UBS)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-Institut National de Recherche en Informatique et en Automatique (Inria)-École normale supérieure - Rennes (ENS Rennes)-Centre National de la Recherche Scientifique (CNRS)-Université de Rennes 1 (UR1), Université de Rennes (UNIV-RENNES)-CentraleSupélec-IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Bretagne Sud (UBS)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), and Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-École normale supérieure - Rennes (ENS Rennes)-Centre National de la Recherche Scientifique (CNRS)-Université de Rennes 1 (UR1)
Subjects: Adult, Male, medicine.medical_specialty, Multiple Sclerosis, Cognitive Neuroscience, computer.software_genre, Article, 030218 nuclear medicine & medical imaging, 03 medical and health sciences, Consistency (database systems), 0302 clinical medicine, Resource (project management), Imaging, Three-Dimensional, medicine, Image Processing, Computer-Assisted, Humans, Medical physics, Segmentation, [SDV.NEU] Life Sciences [q-bio]/Neurons and Cognition [q-bio.NC], Longitudinal Studies, Multiple sclerosis lesion, [SDV.IB] Life Sciences [q-bio]/Bioengineering, Observer Variation, Lesion segmentation, Training set, business.industry, Data Science, Middle Aged, Magnetic Resonance Imaging, White Matter, Women's cancers Radboud Institute for Health Sciences [Radboudumc 17], Data set, Neurology, [SDV.IB]Life Sciences [q-bio]/Bioengineering, [SDV.NEU]Life Sciences [q-bio]/Neurons and Cognition [q-bio.NC], Female, Data mining, business, computer, 030217 neurology & neurosurgery, Algorithms, Test data
Abstract: Contains fulltext : 173122.pdf (Publisher’s version ) (Closed access) In conjunction with the ISBI 2015 conference, we organized a longitudinal lesion segmentation challenge providing training and test data to registered participants. The training data consisted of five subjects with a mean of 4.4 time-points, and test data of fourteen subjects with a mean of 4.4 time-points. All 82 data sets had the white matter lesions associated with multiple sclerosis delineated by two human expert raters. Eleven teams submitted results using state-of-the-art lesion segmentation algorithms to the challenge, with ten teams presenting their results at the conference. We present a quantitative evaluation comparing the consistency of the two raters as well as exploring the performance of the eleven submitted results in addition to three other lesion segmentation algorithms. The challenge presented three unique opportunities: (1) the sharing of a rich data set; (2) collaboration and comparison of the various avenues of research being pursued in the community; and (3) a review and refinement of the evaluation metrics currently in use. We report on the performance of the challenge participants, as well as the construction and evaluation of a consensus delineation. The image data and manual delineations will continue to be available for download, through an evaluation website2 as a resource for future researchers in the area. This data resource provides a platform to compare existing methods in a fair and consistent manner to each other and multiple manual raters.
Published: 2016

8. Optimistic Dynamic Regret Bounds

Author: Haddouche, Maxime, Guedj, Benjamin, Wintenberger, Olivier, University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), Institut National de Recherche en Informatique et en Automatique (Inria), Inria Lille - Nord Europe, The Alan Turing Institute, The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, Optimization and Control (math.OC), FOS: Mathematics, Machine Learning (stat.ML), Mathematics - Optimization and Control, Machine Learning (cs.LG)
Abstract: Online Learning (OL) algorithms have originally been developed to guarantee good performances when comparing their output to the best fixed strategy. The question of performance with respect to dynamic strategies remains an active research topic. We develop in this work dynamic adaptations of classical OL algorithms based on the use of experts' advice and the notion of optimism. We also propose a constructivist method to generate those advices and eventually provide both theoretical and experimental guarantees for our procedures.
Published: 2023

9. Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficulty

Author: Biggs, Felix, Guedj, Benjamin, Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL), Institut National de Recherche en Informatique et en Automatique (Inria), Inria Lille - Nord Europe, The Alan Turing Institute, The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, Machine Learning (stat.ML), Machine Learning (cs.LG)
Abstract: We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages information about the relative hardness of data examples to reduce the variance of its empirical counterpart, tightening the bound. We combine this with a new bound for $[-1, 1]$-valued (and potentially non-independent) signed losses, which is more favourable when they empirically have low variance around $0$. The primary new technical tool is a novel result for sequences of interdependent random vectors which may be of independent interest. We empirically evaluate these new bounds on a number of real-world datasets., 22 pages
Published: 2022

10. A PAC-Bayes bound for deterministic classifiers

Author: Clerico, Eugenio, Deligiannidis, George, Guedj, Benjamin, Doucet, Arnaud, University of Oxford, University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH]
Abstract: We establish a disintegrated PAC-Bayesian bound, for classifiers that are trained via continuous-time (non-stochastic) gradient descent. Contrarily to what is standard in the PAC-Bayesian setting, our result applies to a training algorithm that is deterministic, conditioned on a random initialisation, without requiring any $\textit{de-randomisation}$ step. We provide a broad discussion of the main features of the bound that we propose, and we study analytically and empirically its behaviour on linear models, finding promising results.
Published: 2022

11. PAC-Bayes with Unbounded Losses through Supermartingales

Author: Haddouche, Maxime, Guedj, Benjamin, MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), University College of London [London] (UCL), Computer science department [University College London] (UCL-CS), The Alan Turing Institute, The Inria London Programme (Inria-London), and University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria)
Subjects: [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], generalisation bounds, unbounded losses, PAC-Bayes, [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], supermartingales
Abstract: While PAC-Bayes is now an established learning framework for bounded losses, its extension to the case of unbounded losses (as simple as the squared loss on an unbounded space) remains largely uncharted and has attracted a growing interest in recent years. We contribute to this line of work by developing an extention of Markov's inequality for supermartingales, which we use to establish a novel PAC-Bayesian generalisation bound holding for unbounded losses. We show that this bound extends, unifies and even improves on existing PAC-Bayesian bounds.
Published: 2022

12. MMD Aggregated Two-Sample Test

Author: Schrab, Antonin, Kim, Ilmun, Albert, Mélisande, Laurent, Béatrice, Guedj, Benjamin, Gretton, Arthur, MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), University College of London [London] (UCL), Yonsei University, Institut de Mathématiques de Toulouse UMR5219 (IMT), Université Toulouse Capitole (UT Capitole), Université de Toulouse (UT)-Université de Toulouse (UT)-Institut National des Sciences Appliquées - Toulouse (INSA Toulouse), Institut National des Sciences Appliquées (INSA)-Université de Toulouse (UT)-Institut National des Sciences Appliquées (INSA)-Université Toulouse - Jean Jaurès (UT2J), Université de Toulouse (UT)-Université Toulouse III - Paul Sabatier (UT3), Université de Toulouse (UT)-Centre National de la Recherche Scientifique (CNRS), Institut National des Sciences Appliquées - Toulouse (INSA Toulouse), Institut National des Sciences Appliquées (INSA)-Université de Toulouse (UT), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), Gatsby Computational Neuroscience Unit, Partial support from the French National Agency for Research, grants ANR-18-CE40-0016-01 and ANR-18-CE23-0015-02, ANR-19-P3IA-0004,ANITI,Artificial and Natural Intelligence Toulouse Institute(2019), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, Computer science department [University College London] (UCL-CS), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université Toulouse 1 Capitole (UT1), Université Fédérale Toulouse Midi-Pyrénées-Université Fédérale Toulouse Midi-Pyrénées-Université Toulouse - Jean Jaurès (UT2J)-Université Toulouse III - Paul Sabatier (UT3), Université Fédérale Toulouse Midi-Pyrénées-Centre National de la Recherche Scientifique (CNRS), Institut National des Sciences Appliquées (INSA), Laboratoire Paul Painlevé - UMR 8524 (LPP), Université Toulouse 1 Capitole (UT1), Université Fédérale Toulouse Midi-Pyrénées-Université Fédérale Toulouse Midi-Pyrénées-Institut National des Sciences Appliquées - Toulouse (INSA Toulouse), and Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université Toulouse - Jean Jaurès (UT2J)-Université Toulouse III - Paul Sabatier (UT3)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, Kernel methods, Machine Learning (stat.ML), Mathematics - Statistics Theory, Statistics Theory (math.ST), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Minimax adaptivity, Machine Learning (cs.LG), Methodology (stat.ME), [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, Two-sample testing, FOS: Mathematics, Statistics - Methodology
Abstract: We propose two novel nonparametric two-sample kernel tests based on the Maximum Mean Discrepancy (MMD). First, for a fixed kernel, we construct an MMD test using either permutations or a wild bootstrap, two popular numerical procedures to determine the test threshold. We prove that this test controls the probability of type I error non-asymptotically. Hence, it can be used reliably even in settings with small sample sizes as it remains well-calibrated, which differs from previous MMD tests which only guarantee correct test level asymptotically. When the difference in densities lies in a Sobolev ball, we prove minimax optimality of our MMD test with a specific kernel depending on the smoothness parameter of the Sobolev ball. In practice, this parameter is unknown and, hence, the optimal MMD test with this particular kernel cannot be used. To overcome this issue, we construct an aggregated test, called MMDAgg, which is adaptive to the smoothness parameter. The test power is maximised over the collection of kernels used, without requiring held-out data for kernel selection (which results in a loss of test power), or arbitrary kernel choices such as the median heuristic. We prove that MMDAgg still controls the level non-asymptotically, and achieves the minimax rate over Sobolev balls, up to an iterated logarithmic term. Our guarantees are not restricted to a specific type of kernel, but hold for any product of one-dimensional translation invariant characteristic kernels. We provide a user-friendly parameter-free implementation of MMDAgg using an adaptive collection of bandwidths. We demonstrate that MMDAgg significantly outperforms alternative state-of-the-art MMD-based two-sample tests on synthetic data satisfying the Sobolev smoothness assumption, and that, on real-world image data, MMDAgg closely matches the power of tests leveraging the use of models such as neural networks., 80 pages
Published: 2022

13. Discussion of 'Multiscale Fisher's Independence Test for Multivariate Dependence'

Author: Antonin Schrab, Wittawat Jitkrittum, Zoltán Szabó, Dino Sejdinovic, Arthur Gretton, Schrab, Antonin, Gatsby Computational Neuroscience Unit, University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [STAT.ME] Statistics [stat]/Methodology [stat.ME], [STAT.TH] Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (stat.ML), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], Statistics - Applications, Statistics - Computation, [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], Machine Learning (cs.LG), Methodology (stat.ME), [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, Applications (stat.AP), [STAT.ME]Statistics [stat]/Methodology [stat.ME], Computation (stat.CO), Statistics - Methodology
Abstract: We discuss how MultiFIT, the Multiscale Fisher's Independence Test for Multivariate Dependence proposed by Gorsky and Ma (2022), compares to existing linear-time kernel tests based on the Hilbert-Schmidt independence criterion (HSIC). We highlight the fact that the levels of the kernel tests at any finite sample size can be controlled exactly, as it is the case with the level of MultiFIT. In our experiments, we observe some of the performance limitations of MultiFIT in terms of test power., 8 pages
Published: 2022

14. Efficient Aggregated Kernel Tests using Incomplete $U$-statistics

Author: Antonin Schrab, Ilmun Kim, Benjamin Guedj, Arthur Gretton, University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), Institut National de Recherche en Informatique et en Automatique (Inria), Inria Lille - Nord Europe, The Alan Turing Institute, The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), and Guedj, Benjamin
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [STAT.TH] Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (stat.ML), Mathematics - Statistics Theory, Statistics Theory (math.ST), [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], Machine Learning (cs.LG), Methodology (stat.ME), [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, FOS: Mathematics, Statistics - Methodology
Abstract: We propose a series of computationally efficient nonparametric tests for the two-sample, independence, and goodness-of-fit problems, using the Maximum Mean Discrepancy (MMD), Hilbert Schmidt Independence Criterion (HSIC), and Kernel Stein Discrepancy (KSD), respectively. Our test statistics are incomplete $U$-statistics, with a computational cost that interpolates between linear time in the number of samples, and quadratic time, as associated with classical $U$-statistic tests. The three proposed tests aggregate over several kernel bandwidths to detect departures from the null on various scales: we call the resulting tests MMDAggInc, HSICAggInc and KSDAggInc. This procedure provides a solution to the fundamental kernel selection problem as we can aggregate a large number of kernels with several bandwidths without incurring a significant loss of test power. For the test thresholds, we derive a quantile bound for wild bootstrapped incomplete $U$-statistics, which is of independent interest. We derive non-asymptotic uniform separation rates for MMDAggInc and HSICAggInc, and quantify exactly the trade-off between computational efficiency and the attainable rates: this result is novel for tests based on incomplete $U$-statistics, to our knowledge. We further show that in the quadratic-time case, the wild bootstrap incurs no penalty to test power over the more widespread permutation-based approach, since both attain the same minimax optimal rates (which in turn match the rates that use oracle quantiles). We support our claims with numerical experiments on the trade-off between computational efficiency and test power. In all three testing frameworks, the linear-time versions of our proposed tests perform at least as well as the current linear-time state-of-the-art tests., 34 pages, 5 figures
Published: 2022

15. On Margins and Generalisation for Voting Classifiers

Author: Biggs, Felix, Zantedeschi, Valentina, Guedj, Benjamin, University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), Institut National de Recherche en Informatique et en Automatique (Inria), Inria Lille - Nord Europe, The Alan Turing Institute, The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), and Guedj, Benjamin
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [STAT.TH] Statistics [stat]/Statistics Theory [stat.TH], Mathematics - Statistics Theory, Machine Learning (stat.ML), [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], Statistics Theory (math.ST), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], Machine Learning (cs.LG), ComputingMethodologies_PATTERNRECOGNITION, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, FOS: Mathematics
Abstract: We study the generalisation properties of majority voting on finite ensembles of classifiers, proving margin-based generalisation bounds via the PAC-Bayes theory. These provide state-of-the-art guarantees on a number of classification tasks. Our central results leverage the Dirichlet posteriors studied recently by Zantedeschi et al. [2021] for training voting classifiers; in contrast to that work our bounds apply to non-randomised votes via the use of margins. Our contributions add perspective to the debate on the "margins theory" proposed by Schapire et al. [1998] for the generalisation of ensemble classifiers., Comment: 20 pages, 8 figures
Published: 2022

16. MAGMA: Inference and Prediction using Multi-Task Gaussian Processes with Common Mean

Author: Arthur Leroy, Pierre Latouche, Benjamin Guedj, Servane Gey, Department of Computer Science [Sheffield], University of Sheffield [Sheffield], Mathématiques Appliquées Paris 5 (MAP5 - UMR 8145), Institut National des Sciences Mathématiques et de leurs Interactions (INSMI)-Centre National de la Recherche Scientifique (CNRS)-Université Paris Cité (UPCité), Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Benjamin Guedj acknowledges partial support by the U.S. Army Research Laboratory and the U.S. Army Research Office, and by the U.K. Ministry of Defence and the U.K. Engineering and Physical Sciences Research Council (EPSRC) under grant number EP/R013616, ANR-18-CE40-0016,BEAGLE,Apprentissage PAC-bayésien agnostique(2018), and ANR-18-CE23-0015,APRIORI,Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations(2018)
Subjects: Functional data analysis, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Artificial Intelligence, Multi-task learning, Common mean process, Gaussian process, EM algorithm, [STAT.CO]Statistics [stat]/Computation [stat.CO], [STAT.ME]Statistics [stat]/Methodology [stat.ME], Software
Abstract: A novel multi-task Gaussian process (GP) framework is proposed, by using a common mean process for sharing information across tasks. In particular, we investigate the problem of time series forecasting, with the objective to improve multiple-step-ahead predictions. The common mean process is defined as a GP for which the hyper-posterior distribution is tractable. Therefore an EM algorithm is derived for handling both hyper-parameters optimisation and hyper-posterior computation. Unlike previous approaches in the literature, the model fully accounts for uncertainty and can handle irregular grids of observations while maintaining explicit formulations, by modelling the mean process in a unified GP framework. Predictive analytical equations are provided, integrating information shared across tasks through a relevant prior mean. This approach greatly improves the predictive performances, even far from observations, and may reduce significantly the computational complexity compared to traditional multi-task GP models. Our overall algorithm is called Magma (standing for Multi tAsk GPs with common MeAn). The quality of the mean process estimation, predictive performances, and comparisons to alternatives are assessed in various simulated scenarios and on real datasets.
Published: 2022

17. On PAC-Bayesian reconstruction guarantees for VAEs

Author: Cherief-Abdellatif, B-E, Shi, Y, Doucet, A, Guedj, B, University of Oxford, University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), The Alan Turing Institute, The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), and Partial support from the French National Agency for Research, grants ANR-18-CE40-0016-01 and ANR-18-CE23-0015-02
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, FOS: Mathematics, Mathematics - Statistics Theory, Machine Learning (stat.ML), Statistics Theory (math.ST), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (cs.LG)
Abstract: Despite its wide use and empirical successes, the theoretical understanding and study of the behaviour and performance of the variational autoencoder (VAE) have only emerged in the past few years. We contribute to this recent line of work by analysing the VAE's reconstruction ability for unseen test data, leveraging arguments from the PAC-Bayes theory. We provide generalisation bounds on the theoretical reconstruction error, and provide insights on the regularisation effect of VAE objectives. We illustrate our theoretical results with supporting experiments on classical benchmark datasets., Comment: 14 pages
Published: 2022

18. Non-Vacuous Generalisation Bounds for Shallow Neural Networks

Author: Biggs, Felix, Guedj, Benjamin, Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), The Alan Turing Institute, MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Machine Learning (stat.ML), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (cs.LG)
Abstract: We focus on a specific class of shallow neural networks with a single hidden layer, namely those with $L_2$-normalised data and either a sigmoid-shaped Gaussian error function ("erf") activation or a Gaussian Error Linear Unit (GELU) activation. For these networks, we derive new generalisation bounds through the PAC-Bayesian theory; unlike most existing such bounds they apply to neural networks with deterministic rather than randomised parameters. Our bounds are empirically non-vacuous when the network is trained with vanilla stochastic gradient descent on MNIST and Fashion-MNIST., 19 pages, 12 figures
Published: 2022

19. KSD Aggregated Goodness-of-fit Test

Author: Schrab, Antonin, Guedj, Benjamin, Gretton, Arthur, Gatsby Computational Neuroscience Unit, University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), and The Alan Turing Institute
Subjects: Methodology (stat.ME), FOS: Computer and information sciences, Computer Science - Machine Learning, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, FOS: Mathematics, Mathematics - Statistics Theory, Machine Learning (stat.ML), Statistics Theory (math.ST), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Statistics - Methodology, Machine Learning (cs.LG)
Abstract: We investigate properties of goodness-of-fit tests based on the Kernel Stein Discrepancy (KSD). We introduce a strategy to construct a test, called KSDAgg, which aggregates multiple tests with different kernels. KSDAgg avoids splitting the data to perform kernel selection (which leads to a loss in test power), and rather maximises the test power over a collection of kernels. We provide non-asymptotic guarantees on the power of KSDAgg: we show it achieves the smallest uniform separation rate of the collection, up to a logarithmic term. For compactly supported densities with bounded model score function, we derive the rate for KSDAgg over restricted Sobolev balls; this rate corresponds to the minimax optimal rate over unrestricted Sobolev balls, up to an iterated logarithmic term. KSDAgg can be computed exactly in practice as it relies either on a parametric bootstrap or on a wild bootstrap to estimate the quantiles and the level corrections. In particular, for the crucial choice of bandwidth of a fixed kernel, it avoids resorting to arbitrary heuristics (such as median or standard deviation) or to data splitting. We find on both synthetic and real-world data that KSDAgg outperforms other state-of-the-art quadratic-time adaptive KSD-based goodness-of-fit testing procedures., Comment: 27 pages, 3 figures
Published: 2022

20. An end-to-end data-driven optimisation framework for constrained trajectories

Author: Florent Dewez, Benjamin Guedj, Arthur Talpaert, Vincent Vandewalle, MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Institut National de Recherche en Informatique et en Automatique (Inria), Inria Lille - Nord Europe, The Alan Turing Institute, Université de Lille, Laboratoire Paul Painlevé - UMR 8524 (LPP), and Computer science department [University College London] (UCL-CS)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [STAT.AP]Statistics [stat]/Applications [stat.AP], Constrained optimisation Contents, Machine Learning (stat.ML), Statistics - Applications, Functional data, Machine Learning (cs.LG), [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Optimization and Control (math.OC), FOS: Mathematics, Applications (stat.AP), [MATH.MATH-OC]Mathematics [math]/Optimization and Control [math.OC], Statistical modelling, Mathematics - Optimization and Control
Abstract: Many real-world problems require to optimise trajectories under constraints. Classical approaches are based on optimal control methods but require an exact knowledge of the underlying dynamics, which could be challenging or even out of reach. In this paper, we leverage data-driven approaches to design a new end-to-end framework which is dynamics-free for optimised and realistic trajectories. We first decompose the trajectories on function basis, trading the initial infinite dimension problem on a multivariate functional space for a parameter optimisation problem. A maximum \emph{a posteriori} approach which incorporates information from data is used to obtain a new optimisation problem which is regularised. The penalised term focuses the search on a region centered on data and includes estimated linear constraints in the problem. We apply our data-driven approach to two settings in aeronautics and sailing routes optimisation, yielding commanding results. The developed approach has been implemented in the Python library PyRotor., Comment: 28 pages
Published: 2022

21. PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingales

Author: Haddouche, Maxime, Guedj, Benjamin, MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), University College of London [London] (UCL), and The Alan Turing Institute
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, generalisation bounds, Mathematics - Statistics Theory, unbounded losses, Machine Learning (stat.ML), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Statistics Theory (math.ST), Machine Learning (cs.LG), [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, FOS: Mathematics, PAC-Bayes, supermartingales
Abstract: While PAC-Bayes is now an established learning framework for light-tailed losses (\emph{e.g.}, subgaussian or subexponential), its extension to the case of heavy-tailed losses remains largely uncharted and has attracted a growing interest in recent years. We contribute PAC-Bayes generalisation bounds for heavy-tailed losses under the sole assumption of bounded variance of the loss function. Under that assumption, we extend previous results from \citet{kuzborskij2019efron}. Our key technical contribution is exploiting an extention of Markov's inequality for supermartingales. Our proof technique unifies and extends different PAC-Bayesian frameworks by providing bounds for unbounded martingales as well as bounds for batch and online learning with heavy-tailed losses., Comment: New Section 3 on Online PAC-Bayes
Published: 2022
Full Text: View/download PDF

22. Cluster-Specific Predictions with Multi-Task Gaussian Processes

Author: Leroy, Arthur, Latouche, Pierre, Guedj, Benjamin, Gey, Servane, University of Manchester [Manchester], Mathématiques Appliquées Paris 5 (MAP5 - UMR 8145), Institut National des Sciences Mathématiques et de leurs Interactions (INSMI)-Centre National de la Recherche Scientifique (CNRS)-Université Paris Cité (UPCité), University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Engineering and Physical Sciences Research Council (EPSRC), grant number EP/R013616/1, ANR-18-CE40-0016,BEAGLE,Apprentissage PAC-bayésien agnostique(2018), ANR-18-CE23-0015,APRIORI,Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations(2018), Computer science department [University College London] (UCL-CS), Laboratoire Paul Painlevé - UMR 8524 (LPP), Institut National des Sciences Mathématiques et de leurs Interactions (INSMI)-Centre National de la Recherche Scientifique (CNRS)-Université de Paris (UP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-École polytechnique universitaire de Lille (Polytech Lille), and Institut National des Sciences Mathématiques et de leurs Interactions (INSMI)-Centre National de la Recherche Scientifique (CNRS)-Université Paris Cité (UPC)
Subjects: FOS: Computer and information sciences, cluster-specific predictions, Computer Science - Machine Learning, Gaussian processes mixture, multi-task learning, Machine Learning (stat.ML), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], curve clustering, Statistics - Computation, variational EM, Machine Learning (cs.LG), Methodology (stat.ME), [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, [STAT.CO]Statistics [stat]/Computation [stat.CO], Computation (stat.CO), Statistics - Methodology
Abstract: A model involving Gaussian processes (GPs) is introduced to simultaneously handle multi-task learning, clustering, and prediction for multiple functional data. This procedure acts as a model-based clustering method for functional data as well as a learning step for subsequent predictions for new tasks. The model is instantiated as a mixture of multi-task GPs with common mean processes. A variational EM algorithm is derived for dealing with the optimisation of the hyper-parameters along with the hyper-posteriors' estimation of latent variables and processes. We establish explicit formulas for integrating the mean processes and the latent clustering variables within a predictive distribution, accounting for uncertainty on both aspects. This distribution is defined as a mixture of cluster-specific GP predictions, which enhances the performances when dealing with group-structured data. The model handles irregular grid of observations and offers different hypotheses on the covariance structure for sharing additional information across tasks. The performances on both clustering and prediction tasks are assessed through various simulated scenarios and real datasets. The overall algorithm, called MagmaClust, is publicly available as an R package., 47 pages
Published: 2022

23. Reprint: a randomized extrapolation based on principal components for data augmentation

Author: Wei, Jiale, Chen, Qiyuan, Peng, Pai, Guedj, Benjamin, Li, Le, Guedj, Benjamin, Central China Normal University [Wuhan, China], University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria), The Alan Turing Institute, The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), National Natural Science Foundation of China (Grant No. 62107021), and Hubei ProvincialScience and Technology Innovation Base (Platform) Special Project 2020DFH002
Subjects: FOS: Computer and information sciences, History, Computer Science - Computation and Language, Principal components, Polymers and Plastics, Data augmentation, Hidden-space representation, [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], Randomised extrapolation, Industrial and Manufacturing Engineering, [INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL], [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-CL] Computer Science [cs]/Computation and Language [cs.CL], Business and International Management, Computation and Language (cs.CL)
Abstract: Data scarcity and data imbalance have attracted a lot of attention in many fields. Data augmentation, explored as an effective approach to tackle them, can improve the robustness and efficiency of classification models by generating new samples. This paper presents REPRINT, a simple and effective hidden-space data augmentation method for imbalanced data classification. Given hidden-space representations of samples in each class, REPRINT extrapolates, in a randomized fashion, augmented examples for target class by using subspaces spanned by principal components to summarize distribution structure of both source and target class. Consequently, the examples generated would diversify the target while maintaining the original geometry of target distribution. Besides, this method involves a label refinement component which allows to synthesize new soft labels for augmented examples. Compared with different NLP data augmentation approaches under a range of data imbalanced scenarios on four text classification benchmark, REPRINT shows prominent improvements. Moreover, through comprehensive ablation studies, we show that label refinement is better than label-preserving for augmented examples, and that our method suggests stable and consistent improvements in terms of suitable choices of principal components. Moreover, REPRINT is appealing for its easy-to-use since it contains only one hyperparameter determining the dimension of subspace and requires low computational resource.
Published: 2022
Full Text: View/download PDF

24. Controlling Confusion via Generalisation Bounds

Author: Adams, Reuben, Shawe-Taylor, John, Guedj, Benjamin, Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), The Alan Turing Institute, MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, PAC-Bayes theory, Mathematics - Statistics Theory, Machine Learning (stat.ML), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Statistics Theory (math.ST), Classification, Machine Learning (cs.LG), ComputingMethodologies_PATTERNRECOGNITION, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, FOS: Mathematics, Statistical Learning Theory, Generalisation Bounds
Abstract: We establish new generalisation bounds for multiclass classification by abstracting to a more general setting of discretised error types. Extending the PAC-Bayes theory, we are hence able to provide fine-grained bounds on performance for multiclass classification, as well as applications to other learning problems including discretisation of regression losses. Tractable training objectives are derived from the bounds. The bounds are uniform over all weightings of the discretised error types and thus can be used to bound weightings not foreseen at training, including the full confusion matrix in the multiclass classification case., Comment: 31 pages
Published: 2022
Full Text: View/download PDF

25. On change of measure inequalities for $f$-divergences

Author: Picard-Weibel, Antoine, Guedj, Benjamin, Centre International de Recherche Sur l'Eau et l'Environnement [Suez] (CIRSEE), SUEZ ENVIRONNEMENT (FRANCE), Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL), The Alan Turing Institute, The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, Kullback-Leibler divergence, Computer Science - Information Theory, Information Theory (cs.IT), Probability (math.PR), Mathematics - Statistics Theory, Machine Learning (stat.ML), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Statistics Theory (math.ST), Machine Learning (cs.LG), [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, Change of measure, f-divergence, FOS: Mathematics, Legendre transform, PAC-Bayesian theory, Mathematics - Probability
Abstract: We propose new change of measure inequalities based on $f$-divergences (of which the Kullback-Leibler divergence is a particular case). Our strategy relies on combining the Legendre transform of $f$-divergences and the Young-Fenchel inequality. By exploiting these new change of measure inequalities, we derive new PAC-Bayesian generalisation bounds with a complexity involving $f$-divergences, and holding in mostly unchartered settings (such as heavy-tailed losses). We instantiate our results for the most popular $f$-divergences., Comment: 17 pages
Published: 2022
Full Text: View/download PDF

26. On discord in the voter model for complex networks

Author: Vendeville, Antoine, Guedj, Benjamin, Zhou, Shi, University College of London [London] (UCL), The Inria London Programme (Inria-London), Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), and This project was funded by the UK EPSRC grant EP/S022503/1 that supports the Centre for Doctoral Training in Cybersecurity delivered by UCL’s Departments of Computer Science, Security and Crime Science, and Science, Technology, Engineering and Public Policy.
Subjects: Social and Information Networks (cs.SI), FOS: Computer and information sciences, Physics - Physics and Society, [PHYS.PHYS.PHYS-SOC-PH]Physics [physics]/Physics [physics]/Physics and Society [physics.soc-ph], FOS: Physical sciences, Computer Science - Social and Information Networks, Physics and Society (physics.soc-ph), [INFO.INFO-SI]Computer Science [cs]/Social and Information Networks [cs.SI], [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]
Abstract: We introduce a method to calculate the probability of discord between any two agents in the multi-state voter model with and without zealots. Our work applies to any directed, weighted graph with any finite number of possible opinions, and allows for various update rates across agents. Under certain topological conditions, their opinions are independent and the joint distribution can be decoupled. Otherwise, the evolution of discord probabilities is described by a linear system of ordinary differential equations. We prove the existence of a unique equilibrium solution, which can be computed via an iterative algorithm. The classical definition of active links density is generalised to take into account long-range, weighted interactions. We illustrate our findings on several real-life and synthetic networks. In particular, we uncover a rich landscape of varied behaviours in polarised networks., Comment: 6 pages, 3 figures
Published: 2022
Full Text: View/download PDF

27. Intérêt des bornes désintégrées pour la généralisation avec des mesures de complexité

Author: Viallard, Paul, Emonet, Rémi, Germain, Pascal, Habrard, Amaury, Morvant, Emilie, Zantedeschi, Valentina, Viallard, Paul, Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations - - APRIORI2018 - ANR-18-CE23-0015 - AAPG2018 - VALID, Laboratoire Hubert Curien (LHC), Institut d'Optique Graduate School (IOGS)-Université Jean Monnet - Saint-Étienne (UJM)-Centre National de la Recherche Scientifique (CNRS), Université Laval [Québec] (ULaval), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), and ANR-18-CE23-0015,APRIORI,Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations(2018)
Subjects: [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [STAT.ML] Statistics [stat]/Machine Learning [stat.ML]
Abstract: National audience; Dans ce travail, nous explorons un type de bornes en généralisation pour dériver des garanties qui dépendent d'une mesure de complexité arbitraire. Les bornes classiques avec convergence uniforme sont valides pour toutes les hypothèses (où les données sont tirées d'une distribution de probabilité). Ces bornes prennent donc en compte un terme de complexité prédéfini qui caractérise toute la classe d'hypothèses. En revanche, nous dérivons de nouvelles bornes - appelées bornes désintégrées - qui sont valides avec grande probabilité sur les hypothèses et les données. Ceci nous permet d'intégrer des mesures de complexité qui peuvent être adaptées à la classe d'hypothèses et à la tâche. Nous illustrons l'utilité d'un tel résultat sur quatre variantes de l'algorithme de minimisation du risque empirique pour lequel nous dérivons des garanties de cohérence. Enfin, nous montrons les avantages de notre résultat par rapport à certaines bornes en généralisation classiques.
Published: 2022

28. Measuring dissimilarity with diffeomorphism invariance

Author: Cantelobre, Théophile, Ciliberto, Carlo, Guedj, Benjamin, Rudi, Alessandro, Statistical Machine Learning and Parsimony (SIERRA), Département d'informatique - ENS Paris (DI-ENS), École normale supérieure - Paris (ENS-PSL), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-École normale supérieure - Paris (ENS-PSL), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Centre National de la Recherche Scientifique (CNRS)-Inria de Paris, Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), University College of London [London] (UCL), and The Alan Turing Institute
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Machine Learning (stat.ML), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (cs.LG)
Abstract: Measures of similarity (or dissimilarity) are a key ingredient to many machine learning algorithms. We introduce DID, a pairwise dissimilarity measure applicable to a wide range of data spaces, which leverages the data's internal structure to be invariant to diffeomorphisms. We prove that DID enjoys properties which make it relevant for theoretical study and practical use. By representing each datum as a function, DID is defined as the solution to an optimization problem in a Reproducing Kernel Hilbert Space and can be expressed in closed-form. In practice, it can be efficiently approximated via Nystr\"om sampling. Empirical experiments support the merits of DID., Comment: A pre-print
Published: 2022
Full Text: View/download PDF

29. Online PAC-Bayes Learning

Author: Haddouche, Maxime, Guedj, Benjamin, Guedj, Benjamin, University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), Institut National de Recherche en Informatique et en Automatique (Inria), Inria Lille - Nord Europe, The Alan Turing Institute, The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: FOS: Computer and information sciences, Computer Science::Machine Learning, Computer Science - Machine Learning, [STAT.TH] Statistics [stat]/Statistics Theory [stat.TH], TheoryofComputation_GENERAL, Machine Learning (stat.ML), Mathematics - Statistics Theory, Statistics Theory (math.ST), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], Machine Learning (cs.LG), [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, FOS: Mathematics
Abstract: Most PAC-Bayesian bounds hold in the batch learning setting where data is collected at once, prior to inference or prediction. This somewhat departs from many contemporary learning problems where data streams are collected and the algorithms must dynamically adjust. We prove new PAC-Bayesian bounds in this online learning framework, leveraging an updated definition of regret, and we revisit classical PAC-Bayesian results with a batch-to-online conversion, extending their remit to the case of dependent data. Our results hold for bounded losses, potentially \emph{non-convex}, paving the way to promising developments in online learning., 21 pages
Published: 2022

30. Progress in Self-Certified Neural Networks

Author: Perez-Ortiz, Maria, Rivasplata, Omar, Parrado-Hernandez, Emilio, Guedj, Benjamin, Shawe-Taylor, John, University College of London [London] (UCL), Universidad Carlos III de Madrid [Madrid] (UC3M), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, The Inria London Programme (Inria-London), Computer science department [University College London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), This work is also supported by the U.S. Army Research Laboratory and the U. S. Army Research Office, and by the U.K. Ministry of Defence and the U.K. Engineering and Physical Sciences Research Council (EPSRC) under grant number EP/R013616/1., European Project: 820437,H2020-EU.1.2.3. - FET Flagships ,Humane AI (2019), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), and Department of Computer science [University College of London] (UCL-CS)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, ComputingMethodologies_PATTERNRECOGNITION, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, Machine Learning (cs.LG)
Abstract: A learning method is self-certified if it uses all available data to simultaneously learn a predictor and certify its quality with a tight statistical certificate that is valid on unseen data. Recent work has shown that neural network models trained by optimising PAC-Bayes bounds lead not only to accurate predictors, but also to tight risk certificates, bearing promise towards achieving self-certified learning. In this context, learning and certification strategies based on PAC-Bayes bounds are especially attractive due to their ability to leverage all data to learn a posterior and simultaneously certify its risk with a tight numerical certificate. In this paper, we assess the progress towards self-certification in probabilistic neural networks learnt by PAC-Bayes inspired objectives. We empirically compare (on 4 classification datasets) classical test set bounds for deterministic predictors and a PAC-Bayes bound for randomised self-certified predictors. We first show that both of these generalisation bounds are not too far from out-of-sample test set errors. We then show that in data starvation regimes, holding out data for the test set bounds adversely affects generalisation performance, while self-certified strategies based on PAC-Bayes bounds do not suffer from this drawback, proving that they might be a suitable choice for the small data regime. We also find that probabilistic neural networks learnt by PAC-Bayes inspired objectives lead to certificates that can be surprisingly competitive with commonly used test set bounds.
Published: 2021

31. Towards control of opinion diversity by introducing zealots into a polarised social group

Author: Antoine Vendeville, Benjamin Guedj, Shi Zhou, Computer science department [University College London] (UCL-CS), University College of London [London] (UCL), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, Department of Computer science [University College of London] (UCL-CS), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), Laboratoire Paul Painlevé - UMR 8524 (LPP), and Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe
Subjects: Social and Information Networks (cs.SI), FOS: Computer and information sciences, Computer Science - Machine Learning, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], 0103 physical sciences, Computer Science - Social and Information Networks, 16. Peace & justice, 010306 general physics, 01 natural sciences, [INFO.INFO-SI]Computer Science [cs]/Social and Information Networks [cs.SI], Machine Learning (cs.LG), 010305 fluids & plasmas
Abstract: We explore a method to influence or even control the diversity of opinions within a polarised social group. We leverage the voter model in which users hold binary opinions and repeatedly update their beliefs based on others they connect with. Stubborn agents who never change their minds ("zealots") are also disseminated through the network, which is modelled by a connected graph. Building on earlier results, we provide a closed-form expression for the average opinion of the group at equilibrium. This leads us to a strategy to inject zealots into a polarised network in order to shift the average opinion towards any target value. We account for the possible presence of a backfire effect, which may lead the group to react negatively and reinforce its level of polarisation in response. Our results are supported by numerical experiments on synthetic data., 14 pages, 4 figures
Published: 2021

32. Still No Free Lunches: The Price to Pay for Tighter PAC-Bayes Bounds

Author: Louis Pujol, Benjamin Guedj, University College of London [London] (UCL), Computer science department [University College London] (UCL-CS), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Université Paris-Saclay, Department of Computer science [University College of London] (UCL-CS), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), ANR-18-CE23-0015,APRIORI,Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations(2018), and ANR-18-CE40-0016,BEAGLE,Apprentissage PAC-bayésien agnostique(2018)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, no free lunch theorems, Science, QC1-999, Yield (finance), Robust statistics, Probably approximately correct learning, General Physics and Astronomy, PAC-Bayes theory, Machine Learning (stat.ML), Mathematics - Statistics Theory, Statistics Theory (math.ST), 02 engineering and technology, Astrophysics, 01 natural sciences, Article, Machine Learning (cs.LG), 010104 statistics & probability, Bayes' theorem, statistical learning theory, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, 020204 information systems, FOS: Mathematics, 0202 electrical engineering, electronic engineering, information engineering, No free lunch in search and optimization, Applied mathematics, 0101 mathematics, Impossibility, Mathematics, Physics, State (functional analysis), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], QB460-466, Statistical learning theory
Abstract: “No free lunch” results state the impossibility of obtaining meaningful bounds on the error of a learning algorithm without prior assumptions and modelling, which is more or less realistic for a given problem. Some models are “expensive” (strong assumptions, such as sub-Gaussian tails), others are “cheap” (simply finite variance). As it is well known, the more you pay, the more you get: in other words, the most expensive models yield the more interesting bounds. Recent advances in robust statistics have investigated procedures to obtain tight bounds while keeping the cost of assumptions minimal. The present paper explores and exhibits what the limits are for obtaining tight probably approximately correct (PAC)-Bayes bounds in a robust setting for cheap models.
Published: 2021
Full Text: View/download PDF

33. Learning PAC-Bayes Priors for Probabilistic Neural Networks

Author: Pérez-Ortiz, María, Rivasplata, Omar, Guedj, Benjamin, Gleeson, Matthew, Zhang, Jingyu, Shawe-Taylor, John, Bober, Miroslaw, Kittler, Josef, Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey (UNIS), Support and funding from the U.S. Army Research Laboratory and the U. S. Army Research Office, and by the U.K. Ministry of Defence and the U.K. Engineering and Physical Sciences Research Council (EPSRC) under grant number EP/R013616/1., Computer science department [University College London] (UCL-CS), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (cs.LG)
Abstract: Recent works have investigated deep learning models trained by optimising PAC-Bayes bounds, with priors that are learnt on subsets of the data. This combination has been shown to lead not only to accurate classifiers, but also to remarkably tight risk certificates, bearing promise towards self-certified learning (i.e. use all the data to learn a predictor and certify its quality). In this work, we empirically investigate the role of the prior. We experiment on 6 datasets with different strategies and amounts of data to learn data-dependent PAC-Bayes priors, and we compare them in terms of their effect on test performance of the learnt predictors and tightness of their risk certificate. We ask what is the optimal amount of data which should be allocated for building the prior and show that the optimum may be dataset dependent. We demonstrate that using a small percentage of the prior-building data for validation of the prior leads to promising results. We include a comparison of underparameterised and overparameterised models, along with an empirical study of different training objectives and regularisation strategies to learn the prior distribution.
Published: 2021

34. On Margins and Derandomisation in PAC-Bayes

Author: Biggs, Felix, Guedj, Benjamin, University College of London [London] (UCL), Computer science department [University College London] (UCL-CS), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-École polytechnique universitaire de Lille (Polytech Lille), Department of Computer science [University College of London] (UCL-CS), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), Felix Biggs gratefully acknowledges the support of the CDT for Foundational Artificial Intelligence through UKRI grant EP/S021566/1. Benjamin Guedj acknowledges partial support by the U.S. Army Research Laboratory and the U.S. Army Research Office, and by the U.K. Ministry of Defence and the U.K. Engineering and Physical Sciences Research Council (EPSRC) under grant number EP/R013616/1, Benjamin Guedj also acknowledges partial support from the French National Agency for Research, grants ANR18-CE40-0016-01 and ANR- 18-CE23-0015-02., ANR-18-CE40-0016,BEAGLE,Apprentissage PAC-bayésien agnostique(2018), and ANR-18-CE23-0015,APRIORI,Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations(2018)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], FOS: Mathematics, Mathematics - Statistics Theory, Statistics Theory (math.ST), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (cs.LG)
Abstract: We give a general recipe for derandomising PAC-Bayesian bounds using margins, with the critical ingredient being that our randomised predictions concentrate around some value. The tools we develop straightforwardly lead to margin bounds for various classifiers, including linear prediction -- a class that includes boosting and the support vector machine -- single-hidden-layer neural networks with an unusual $\erf$ activation function, and deep ReLU networks. Further, we extend to partially-derandomised predictors where only some of the randomness is removed, letting us extend bounds to cases where the concentration properties of our predictors are otherwise poor., 23 pages
Published: 2021

35. Learning Binary Decision Trees by Argmin Differentiation

Author: Zantedeschi, Valentina, Kusner, Matt J, Niculae, Vlad, The Inria London Programme (Inria-London), Computer science department [University College London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, University College of London [London] (UCL), University of Amsterdam [Amsterdam] (UvA), Department of Computer science [University College of London] (UCL-CS), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), and Zantedeschi, Valentina
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, Artificial Intelligence (cs.AI), [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Computer Science - Artificial Intelligence, Statistics - Machine Learning, self-supervised learning, argmin differentiation, Machine Learning (stat.ML), implicit layer, [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], binary decision trees, Machine Learning (cs.LG)
Abstract: International audience; We address the problem of learning binary decision trees that partition data for some downstream task. We propose to learn discrete parameters(i.e., for tree traversals and node pruning) and continuous parameters (i.e., for tree split functions and prediction functions) simultaneously usingargmin differentiation. We do so by sparsely relaxing a mixed-integer program for the discrete parameters, to allow gradients to pass throughthe program to continuous parameters. We derive customized algorithms to efficiently compute the forward and backward passes. This meansthat our tree learning procedure can be used as an (implicit) layer in arbitrary deep networks, and can be optimized with arbitrary loss functions. We demonstrate that our approach produces binary trees that are competitive with existing single tree and ensemble approaches, in both supervised and unsupervised settings. Further, apart from greedy approaches (which do not have competitive accuracies), our method is faster to train than all other tree-learning baselines we compare with. The code for reproducing the results is available at https://github.com/vzantedeschi/LatentTrees.
Published: 2021

36. Heavy positive ion groups in Titan's ionosphere: Cassini Plasma Spectrometer IBS observations

Author: Anne Wellbrock, Hunter Waite, Véronique Vuitton, Richard Haythornthwaite, Geraint H. Jones, Andrew J. Coates, Panayotis Lavvas, Department of Space and Climate Physics [UCL London], Mullard Space Science Laboratory (MSSL), University College of London [London] (UCL)-University College of London [London] (UCL), University College of London [London] (UCL), Space Science Division [San Antonio], Southwest Research Institute [San Antonio] (SwRI), Institut de Planétologie et d'Astrophysique de Grenoble (IPAG), Centre National d'Études Spatiales [Toulouse] (CNES)-Observatoire des Sciences de l'Univers de Grenoble (OSUG ), and Institut national des sciences de l'Univers (INSU - CNRS)-Université Savoie Mont Blanc (USMB [Université de Savoie] [Université de Chambéry])-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement (INRAE)-Université Grenoble Alpes (UGA)-Institut national des sciences de l'Univers (INSU - CNRS)-Université Savoie Mont Blanc (USMB [Université de Savoie] [Université de Chambéry])-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement (INRAE)-Université Grenoble Alpes (UGA)
Subjects: Physics, symbols.namesake, Spectrometer, 13. Climate action, [SDU.ASTR.EP]Sciences of the Universe [physics]/Astrophysics [astro-ph]/Earth and Planetary Astrophysics [astro-ph.EP], symbols, Astronomy, Plasma, Ionosphere, Titan (rocket family), 7. Clean energy, Ion
Abstract: IntroductionTitan is the largest moon of Saturn and has a thick extended atmosphere along with a large ionosphere. Titan's ionosphere contains a plethora of hydrocarbons and nitrile cations and anions as measured by the Ion Neutral Mass Spectrometer and Cassini Plasma Spectrometer (CAPS) onboard the Cassini spacecraft1.Previous ion composition studies in Titan’s ionosphere by Cassini instruments revealed "families" of ions around particular mass values and a regular spacing of 12 to 14 u/q between mass groups 2. These are thought to be related to a carbon or nitrogen backbone that dominates the ion chemistry2. Previous studies also identified possible heavy ions such as naphthalene, anthracene derivatives and an anthracene dimer at 130, 170 and 335 u/q respectively1. Methodology The CAPS Ion Beam Spectrometer3 is an electrostatic analyser that measures energy/charge ratios of ions. During the Titan flybys Cassini had a high velocity (~6 km/s) relative to the low ion velocities (< 230 m/s) observed in the ionosphere. The ions were also cold, having ion temperatures around 150K. The combination of these factors meant that the ions appeared as a highly-directed supersonic beam in the spacecraft frame. This means the ions appear at kinetic energies associated with the spacecraft velocity and the ion mass, therefore the measured energy spectra (eV/q) can be converted to mass spectra (u/q). Results and ConclusionsPositive ion masses between 170 and 310 u/q are examined with ion mass groups identified between 170 and 275 u/q containing between 14 and 21 heavy (carbon/nitrogen/oxygen) atoms4. These groups are the heaviest positive ion groups reported so far from the available in situ ion data at Titan.The ion group peaks are found to be consistent with masses associated with Polycyclic Aromatic Compounds, including Polycyclic Aromatic Hydrocarbon (PAH) and nitrogen-bearing polycyclic aromatic molecular ions. The ion group peak identifications are compared with previously proposed neutral PAHs5 and are found to be at similar masses, supporting a PAH interpretation. The spacing between the ion group peaks is also investigated, finding a spacing of 12 or 13 u/q indicating the addition of C or CH. Lastly, the occurrence of several ion groups is seen to vary across the five flybys studied, possibly relating to the varying solar radiation conditions observed across the flybys.The discovery of these groups will aid future atmospheric chemical models of Titan through identification of prominent heavy positive ions and further the understanding between the low mass ions and the high mass negative ions, as well as the process of aerosol formation in Titan's atmosphere.References1. Waite et al., The Process of Tholin Formation in Titan’s Upper Atmosphere, Sci., 2007, doi:10.1126/science.11397272. Crary et al., Heavy ions, temperatures and winds in Titan's ionosphere: Combined Cassini CAPS and INMS observations, P&SS, 2009, doi:10.1016/j.pss.2009.09.006.3. Young et al., Cassini Plasma Spectrometer Investigation. Space Sci. Rev., 2004, doi:10.1007/s11214-004-1406-44. Haythornthwaite et al., Heavy Positive Ion Groups in Titan's Ionosphere from Cassini Plasma Spectrometer IBS Observations, eprint arXiv:2009.087495. López-Puertas et al., Large Abundances of Polycyclic Aromatic Hydrocarbons in Titan's Upper Atmosphere, ApJ, 2013, doi:10.1088/0004-637X/770/2/132
Published: 2021

37. RainBench: Towards Global Precipitation Forecasting from Satellite Imagery

Author: de Witt, CS, Tong, C, Zantedeschi, V, De Martini, D, Kalaitzis, A, Chantry, M, Watson-Parris, D, Bilinski, P, Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Zantedeschi, Valentina, Computer science department [University College London] (UCL-CS), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies
Subjects: FOS: Computer and information sciences, [SDU.OCEAN]Sciences of the Universe [physics]/Ocean, Atmosphere, Computer Science - Machine Learning, Physics - Atmospheric and Oceanic Physics, Artificial Intelligence (cs.AI), ComputingMethodologies_PATTERNRECOGNITION, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Computer Science - Artificial Intelligence, [SDU.OCEAN] Sciences of the Universe [physics]/Ocean, Atmosphere, Atmospheric and Oceanic Physics (physics.ao-ph), FOS: Physical sciences, [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], Machine Learning (cs.LG)
Abstract: Extreme precipitation events, such as violent rainfall and hail storms, routinely ravage economies and livelihoods around the developing world. Climate change further aggravates this issue. Data-driven deep learning approaches could widen the access to accurate multi-day forecasts, to mitigate against such events. However, there is currently no benchmark dataset dedicated to the study of global precipitation forecasts. In this paper, we introduce \textbf{RainBench}, a new multi-modal benchmark dataset for data-driven precipitation forecasting. It includes simulated satellite data, a selection of relevant meteorological data from the ERA5 reanalysis product, and IMERG precipitation data. We also release \textbf{PyRain}, a library to process large precipitation datasets efficiently. We present an extensive analysis of our novel dataset and establish baseline results for two benchmark medium-range precipitation forecasting tasks. Finally, we discuss existing data-driven weather forecasting methodologies and suggest future research avenues., Comment: Work completed during the 2020 Frontier Development Lab research accelerator, a private-public partnership with NASA in the US, and ESA in Europe. Accepted as a spotlight/long oral talk at both Climate Change and AI, as well as AI for Earth Sciences Workshops at NeurIPS 2020
Published: 2021

38. Forecasting elections results via the voter model with stubborn nodes

Author: Shi Zhou, Benjamin Guedj, Antoine Vendeville, Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), Computer science department [University College London] (UCL-CS), Laboratoire Paul Painlevé - UMR 8524 (LPP), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, social networks, Computer Networks and Communications, Mean absolute error, Voter model, Model parameters, 02 engineering and technology, Elections, [INFO.INFO-SI]Computer Science [cs]/Social and Information Networks [cs.SI], Machine Learning (cs.LG), voter model, Politics, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], 020204 information systems, Side product, 0202 electrical engineering, electronic engineering, information engineering, Econometrics, Economics, Popular vote, Social and Information Networks (cs.SI), Multidisciplinary, Markov chain, Markov chains, lcsh:T57-57.97, Computer Science - Social and Information Networks, 16. Peace & justice, Computational Mathematics, Work (electrical), lcsh:Applied mathematics. Quantitative methods, 020201 artificial intelligence & image processing, opinion dynamics
Abstract: 15 pages; International audience; We explore a method to influence or even control the diversity of opinions within a polarised social group. We leverage the voter model in which users hold binary opinions and repeatedly update their beliefs based on others they connect with. Stubborn agents who never change their minds (\zealots") are also disseminated through the network, which is modelled by a connected graph. Building on earlier results, we provide a closed-form expression for the average opinion of the group at equilibrium. This leads us to a strategy to inject zealots into a polarised network in order to shift the average opinion towards any target value. We account for the possible presence of a backfire effect, which may lead the group to react negatively and reinforce its level of polarisation in response. Our results are supported by numerical experiments on synthetic data.
Published: 2021

39. Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound

Author: Zantedeschi, Valentina, Viallard, Paul, Morvant, Emilie, Emonet, Rémi, Habrard, Amaury, Germain, Pascal, Guedj, Benjamin, MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Laboratoire Hubert Curien (LHC), Institut d'Optique Graduate School (IOGS)-Université Jean Monnet - Saint-Étienne (UJM)-Centre National de la Recherche Scientifique (CNRS), Université Laval [Québec] (ULaval), University College of London [London] (UCL), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), Grant number EP/R013616/1, NSERC Discovery grant RGPIN-2020-07223, ANR-18-CE23-0015,APRIORI,Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations(2018), ANR-18-CE40-0016,BEAGLE,Apprentissage PAC-bayésien agnostique(2018), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, Computer science department [University College London] (UCL-CS), Université Jean Monnet [Saint-Étienne] (UJM), Viallard, Paul, Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations - - APRIORI2018 - ANR-18-CE23-0015 - AAPG2018 - VALID, and APPEL À PROJETS GÉNÉRIQUE 2018 - Apprentissage PAC-bayésien agnostique - - BEAGLE2018 - ANR-18-CE40-0016 - AAPG2018 - VALID
Subjects: Methodology (stat.ME), FOS: Computer and information sciences, Computer Science - Machine Learning, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Machine Learning (stat.ML), [STAT.ME]Statistics [stat]/Methodology [stat.ME], [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], Statistics - Methodology, Machine Learning (cs.LG)
Abstract: International audience; We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk, which then turns the generalization bound into a tractable training objective. The resulting stochastic majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight generalization bounds, in a series of numerical experiments when compared to competing algorithms which also minimize PAC-Bayes objectives-both with uninformed (data-independent) and informed (data-dependent) priors.
Published: 2021

40. PAC-Bayes unleashed: generalisation bounds with unbounded losses

Author: John Shawe-Taylor, Benjamin Guedj, Maxime Haddouche, Omar Rivasplata, Ecole Normale Supérieure Paris-Saclay (ENS Paris Saclay), The Inria London Programme (Inria-London), Computer science department [University College London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), University College of London [London] (UCL), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, Department of Computer science [University College of London] (UCL-CS), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), ANR-18-CE23-0015,APRIORI,Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations(2018), and ANR-18-CE40-0016,BEAGLE,Apprentissage PAC-bayésien agnostique(2018)
Subjects: Computer Science::Machine Learning, FOS: Computer and information sciences, Computer Science - Machine Learning, Computer science, Science, QC1-999, generalisation bounds, General Physics and Astronomy, Mathematics - Statistics Theory, Machine Learning (stat.ML), Interval (mathematics), Statistics Theory (math.ST), Astrophysics, 01 natural sciences, Article, Machine Learning (cs.LG), 010104 statistics & probability, Bayes' theorem, statistical learning theory, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, FOS: Mathematics, Applied mathematics, 0101 mathematics, Physics, 010102 general mathematics, Supervised learning, TheoryofComputation_GENERAL, Function (mathematics), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], QB460-466, Range (mathematics), ComputingMethodologies_PATTERNRECOGNITION, Bounded function, Statistical learning theory, Relaxation (approximation), PAC-Bayes
Abstract: We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions. This extends the relevance and applicability of the PAC-Bayes learning framework, where most of the existing literature focuses on supervised learning problems with a bounded loss function (typically assumed to take values in the interval [0;1]). In order to relax this assumption, we propose a new notion called HYPE (standing for \emph{HYPothesis-dependent rangE}), which effectively allows the range of the loss to depend on each predictor. Based on this new notion we derive a novel PAC-Bayesian generalisation bound for unbounded loss functions, and we instantiate it on a linear regression problem. To make our theory usable by the largest audience possible, we include discussions on actual computation, practicality and limitations of our assumptions., Comment: 24 pages
Published: 2021

41. Upper and Lower Bounds on the Performance of Kernel PCA

Author: Haddouche, Maxime, Guedj, Benjamin, Rivasplata, Omar, Shawe-Taylor, John, Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria), Computer science department [University College London] (UCL-CS), University College of London [London] (UCL), Ecole Normale Supérieure Paris-Saclay (ENS Paris Saclay), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé - UMR 8524 (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), Department of Computer science [University College of London] (UCL-CS), and Laboratoire Paul Painlevé (LPP)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, dimension reduction, ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION, kernel PCA, Machine Learning (stat.ML), Mathematics - Statistics Theory, Statistics Theory (math.ST), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (cs.LG), Statistical learning theory, ComputingMethodologies_PATTERNRECOGNITION, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Computer Science::Computer Vision and Pattern Recognition, FOS: Mathematics, PAC-Bayes
Abstract: Principal Component Analysis (PCA) is a popular method for dimension reduction and has attracted an unfailing interest for decades. More recently, kernel PCA (KPCA) has emerged as an extension of PCA but, despite its use in practice, a sound theoretical understanding of KPCA is missing. We contribute several lower and upper bounds on the efficiency of KPCA, involving the empirical eigenvalues of the kernel Gram matrix and new quantities involving a notion of variance. These bounds show how much information is captured by KPCA on average and contribute a better theoretical understanding of its efficiency. We demonstrate that fast convergence rates are achievable for a widely used class of kernels and we highlight the importance of some desirable properties of datasets to ensure KPCA efficiency., 16 pages
Published: 2020

42. A PAC-Bayesian Perspective on Structured Prediction with Implicit Loss Embeddings

Author: Cantelobre, Théophile, Guedj, Benjamin, Pérez-Ortiz, María, Shawe-Taylor, John, Mines Paris - PSL (École nationale supérieure des mines de Paris), Université Paris sciences et lettres (PSL), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), University College of London [London] (UCL), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MINES ParisTech - École nationale supérieure des mines de Paris, Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, and Computer science department [University College London] (UCL-CS)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, PAC-Bayes theory, Machine Learning (stat.ML), Mathematics - Statistics Theory, Statistics Theory (math.ST), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (cs.LG), Implicit Loss Embeddings, Statistical learning theory, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Generalization bounds, FOS: Mathematics, Structured output prediction
Abstract: Many practical machine learning tasks can be framed as Structured prediction problems, where several output variables are predicted and considered interdependent. Recent theoretical advances in structured prediction have focused on obtaining fast rates convergence guarantees, especially in the Implicit Loss Embedding (ILE) framework. PAC-Bayes has gained interest recently for its capacity of producing tight risk bounds for predictor distributions. This work proposes a novel PAC-Bayes perspective on the ILE Structured prediction framework. We present two generalization bounds, on the risk and excess risk, which yield insights into the behavior of ILE predictors. Two learning algorithms are derived from these bounds. The algorithms are implemented and their behavior analyzed, with source code available at \url{https://github.com/theophilec/PAC-Bayes-ILE-Structured-Prediction}., 38 pages
Published: 2020

43. PAC-Bayesian Bound for the Conditional Value at Risk

Author: Mhammedi, Zakaria, Guedj, Benjamin, Williamson, Robert C., Australian National University (ANU), University College of London [London] (UCL), Department of Computer science [University College of London] (UCL-CS), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Unknown Labs [Inria], Computer science department [University College London] (UCL-CS), and Laboratoire Paul Painlevé - UMR 8524 (LPP)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Machine Learning (stat.ML), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (cs.LG)
Abstract: International audience; Conditional Value at Risk (CVAR) is a family of "coherent risk measures" which generalize the traditional mathematical expectation. Widely used in mathematical finance, it is garnering increasing interest in machine learning, e.g., as an alternate approach to regularization, and as a means for ensuring fairness. This paper presents a generalization bound for learning algorithms that minimize the CVAR of the empirical loss. The bound is of PAC-Bayesian type and is guaranteed to be small when the empirical CVAR is small. We achieve this by reducing the problem of estimating CVAR to that of merely estimating an expectation. This then enables us, as a by-product, to obtain concentration inequalities for CVAR even when the random variable in question is unbounded.
Published: 2020

44. PAC-Bayesian Contrastive Unsupervised Representation Learning

Author: Nozawa, Kento, Germain, Pascal, Guedj, Benjamin, Tokyo University of Science [Tokyo], RIKEN Center for Advanced Intelligence Project [Tokyo] (RIKEN AIP), RIKEN - Institute of Physical and Chemical Research [Japon] (RIKEN), MOdel for Data Analysis and Learning (MODAL), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, University College of London [London] (UCL), Computer science department [University College London] (UCL-CS), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), and Department of Computer science [University College of London] (UCL-CS)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, FOS: Mathematics, Mathematics - Statistics Theory, Machine Learning (stat.ML), Statistics Theory (math.ST), [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH], Machine Learning (cs.LG)
Abstract: Contrastive unsupervised representation learning (CURL) is the state-of-the-art technique to learn representations (as a set of features) from unlabelled data. While CURL has collected several empirical successes recently, theoretical understanding of its performance was still missing. In a recent work, Arora et al. (2019) provide the first generalisation bounds for CURL, relying on a Rademacher complexity. We extend their framework to the flexible PAC-Bayes setting, allowing us to deal with the non-iid setting. We present PAC-Bayesian generalisation bounds for CURL, which are then used to derive a new representation learning algorithm. Numerical experiments on real-life datasets illustrate that our algorithm achieves competitive accuracy, and yields non-vacuous generalisation bounds., Comment: Published in the proceedings of the Conference on Uncertainty in Artificial Intelligence 2020 (UAI)
Published: 2020

45. MAGMA: Inference and Prediction with Multi-Task Gaussian Processes

Author: Leroy, Arthur, Latouche, Pierre, Guedj, Benjamin, Gey, Servane, Mathématiques Appliquées Paris 5 (MAP5 - UMR 8145), Université Paris Descartes - Paris 5 (UPD5)-Institut National des Sciences Mathématiques et de leurs Interactions (INSMI)-Centre National de la Recherche Scientifique (CNRS), University College of London [London] (UCL), Computer science department [University College London] (UCL-CS), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé - UMR 8524 (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), and Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, Common mean process, Machine Learning (stat.ML), Statistics - Computation, Machine Learning (cs.LG), Methodology (stat.ME), Functional data analysis, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, Multi-task learning, Gaussian process, EM algorithm, [STAT.CO]Statistics [stat]/Computation [stat.CO], [STAT.ME]Statistics [stat]/Methodology [stat.ME], Computation (stat.CO), Statistics - Methodology
Abstract: A novel multi-task Gaussian process (GP) framework is proposed, by using a common mean process for sharing information across tasks. In particular, we investigate the problem of time series forecasting, with the objective to improve multiple-step-ahead predictions. The common mean process is defined as a GP for which the hyper-posterior distribution is tractable. Therefore an EM algorithm is derived for handling both hyper-parameters optimisation and hyper-posterior computation. Unlike previous approaches in the literature, the model fully accounts for uncertainty and can handle irregular grids of observations while maintaining explicit formulations, by modelling the mean process in a unified GP framework. Predictive analytical equations are provided, integrating information shared across tasks through a relevant prior mean. This approach greatly improves the predictive performances, even far from observations, and may reduce significantly the computational complexity compared to traditional multi-task GP models. Our overall algorithm is called \textsc{Magma} (standing for Multi tAsk Gaussian processes with common MeAn). The quality of the mean process estimation, predictive performances, and comparisons to alternatives are assessed in various simulated scenarios and on real datasets.
Published: 2020

46. Revisiting clustering as matrix factorisation on the Stiefel manifold

Author: Benjamin Guedj, Stéphane Chrétien, National Physical Laboratory [Teddington] (NPL), MOdel for Data Analysis and Learning (MODAL), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, Computer science department [University College London] (UCL-CS), University College of London [London] (UCL), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Santé publique : épidémiologie et qualité des soins-EA 2694 (CERIM), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), and Department of Computer science [University College of London] (UCL-CS)
Subjects: FOS: Computer and information sciences, Clustering high-dimensional data, Computer Science - Machine Learning, Rank (linear algebra), Computer science, Machine Learning (stat.ML), Low-rank approximation, Context (language use), 010103 numerical & computational mathematics, 01 natural sciences, Clustering, Machine Learning (cs.LG), Stiefel manifold, 010104 statistics & probability, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, Applied mathematics, 0101 mathematics, Cluster analysis, Series (mathematics), Estimator, non-negative matrix factorisation, concentration inequalities, PAC-Bayes, Gaussian mixtures, optimisation on manifolds
Abstract: This paper studies clustering for possibly high dimensional data (e.g. images, time series, gene expression data, and many other settings), and rephrase it as low rank matrix estimation in the PAC-Bayesian framework. Our approach leverages the well known Burer-Monteiro factorisation strategy from large scale optimisation, in the context of low rank estimation. Moreover, our Burer-Monteiro factors are shown to lie on a Stiefel manifold. We propose a new generalized Bayesian estimator for this problem and prove novel prediction bounds for clustering. We also devise a componentwise Langevin sampler on the Stiefel manifold to compute this estimator., Accepted at the LOD 2020 Conference -- The Sixth International Conference on Machine Learning, Optimization, and Data Science
Published: 2020

47. Non-linear aggregation of filters to improve image denoising

Author: Benjamin Guedj, Juliette Rengot, Department of Computer science [University College of London] (UCL-CS), University College of London [London] (UCL), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), École des Ponts ParisTech (ENPC), Guedj, Benjamin, Computer science department [University College London] (UCL-CS), Laboratoire Paul Painlevé - UMR 8524 (LPP), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Santé publique : épidémiologie et qualité des soins-EA 2694 (CERIM), Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, CNRS, Université de Lille, and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies
Subjects: Scheme (programming language), FOS: Computer and information sciences, Computer Science - Machine Learning, Statistical aggregation, Ensemble methods, Image denoising, Collaborative filtering, Computer science, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, Machine Learning (stat.ML), 02 engineering and technology, 010501 environmental sciences, 01 natural sciences, Machine Learning (cs.LG), [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Statistics - Machine Learning, 0202 electrical engineering, electronic engineering, information engineering, FOS: Electrical engineering, electronic engineering, information engineering, 0105 earth and related environmental sciences, computer.programming_language, [STAT.ME] Statistics [stat]/Methodology [stat.ME], Pixel, Aggregate (data warehouse), Image and Video Processing (eess.IV), [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], Electrical Engineering and Systems Science - Image and Video Processing, Ensemble learning, [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], Nonlinear system, [INFO.INFO-TI] Computer Science [cs]/Image Processing [eess.IV], [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], Metric (mathematics), 020201 artificial intelligence & image processing, computer, Algorithm, [STAT.ME]Statistics [stat]/Methodology [stat.ME]
Abstract: We introduce a novel aggregation method to efficiently perform image denoising. Preliminary filters are aggregated in a non-linear fashion, using a new metric of pixel proximity based on how the pool of filters reaches a consensus. We provide a theoretical bound to support our aggregation scheme, its numerical performance is illustrated and we show that the aggregate significantly outperforms each of the preliminary filters., Comment: To appear at Computing Conference 2020
Published: 2020

48. How opinions crystallise: an analysis of polarisation in the voter model

Author: Vendeville, Antoine, Guedj, Benjamin, Zhou, Shi, Computer science department [University College London] (UCL-CS), University College of London [London] (UCL), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé - UMR 8524 (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [INFO.INFO-SI]Computer Science [cs]/Social and Information Networks [cs.SI]
Abstract: We address the phenomenon of sedimentation of opinions in networks. We investigate how agents who never change their minds ("stubborn") can influence the opinion of a social group and foster the formation of polarised communities. We study the voter model in which users are divided in two camps and repeatedly update their opinions based on others they connect with. Assuming a proportion of the agents are stubborn, the distribution of opinions reaches an equilibrium. We give novel formulas based on Markov Chain analysis to compute the distribution of opinions at any time and speed of convergence to stationary equilibrium. Theoretical results are supported by numerical experiments on synthetic data, and we discuss a strategy to mitigate the polarisation phenomenon.
Published: 2020

49. Kernel-Based Ensemble Learning in Python

Author: Bhargav Srinivasa Desikan, Benjamin Guedj, Computer science department [University College London] (UCL-CS), University College of London [London] (UCL), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Paul Painlevé - UMR 8524 (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)-Université de Lille, Sciences et Technologies, The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), University of Chicago, Department of Computer science [University College of London] (UCL-CS), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), and Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille)
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, Computational complexity theory, Computer science, Binary number, Machine Learning (stat.ML), Cobra, Machine learning, computer.software_genre, Statistics - Computation, 01 natural sciences, kernels, Machine Learning (cs.LG), 010104 statistics & probability, open source software, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistics - Machine Learning, 0502 economics and business, 0101 mathematics, [STAT.CO]Statistics [stat]/Computation [stat.CO], Computation (stat.CO), 050205 econometrics, computer.programming_language, lcsh:T58.5-58.64, business.industry, lcsh:Information technology, 05 social sciences, Estimator, Python (programming language), Ensemble learning, Regression, python, Open source, machine learning, ensemble learning, Artificial intelligence, business, computer, Information Systems
Abstract: We propose a new supervised learning algorithm, for classification and regression problems where two or more preliminary predictors are available. We introduce \texttt{KernelCobra}, a non-linear learning strategy for combining an arbitrary number of initial predictors. \texttt{KernelCobra} builds on the COBRA algorithm introduced by \citet{biau2016cobra}, which combined estimators based on a notion of proximity of predictions on the training data. While the COBRA algorithm used a binary threshold to declare which training data were close and to be used, we generalize this idea by using a kernel to better encapsulate the proximity information. Such a smoothing kernel provides more representative weights to each of the training points which are used to build the aggregate and final predictor, and \texttt{KernelCobra} systematically outperforms the COBRA algorithm. While COBRA is intended for regression, \texttt{KernelCobra} deals with classification and regression. \texttt{KernelCobra} is included as part of the open source Python package \texttt{Pycobra} (0.2.4 and onward), introduced by \citet{guedj2018pycobra}. Numerical experiments assess the performance (in terms of pure prediction and computational complexity) of \texttt{KernelCobra} on real-life and synthetic datasets., 11 pages
Published: 2020

50. A unified view on beamformers for M/EEG source reconstruction

Author: Britta U. Westner, Sarang S. Dalal, Alexandre Gramfort, Vladimir Litvak, John C. Mosher, Robert Oostenveld, Jan-Mathijs Schoffelen, Donders Institute for Brain, Cognition and Behaviour, Radboud university [Nijmegen], Aarhus University [Aarhus], Modelling brain structure, function and variability based on high-field MRI data (PARIETAL), Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Service NEUROSPIN (NEUROSPIN), Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA), NMR Research Unit [London], Institute of Neurology [London], University College of London [London] (UCL)-University College of London [London] (UCL), The University of Texas Health Science Center at Houston (UTHealth), Karolinska Institutet [Stockholm], Radboud University [Nijmegen], Service NEUROSPIN (NEUROSPIN), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Inria Saclay - Ile de France, and Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)
Subjects: Adult, Cognitive Neuroscience, Source reconstruction, Data analysis, Neurosciences. Biological psychiatry. Neuropsychiatry, [STAT.OT]Statistics [stat]/Other Statistics [stat.ML], 150 000 MR Techniques in Brain Function, 050105 experimental psychology, 310 000 MEG Methods, 03 medical and health sciences, 0302 clinical medicine, Beamforming, [INFO.INFO-IM]Computer Science [cs]/Medical Imaging, Humans, 0501 psychology and cognitive sciences, EEG, ComputingMilieux_MISCELLANEOUS, Cerebral Cortex, Brain Mapping, [STAT.AP]Statistics [stat]/Applications [stat.AP], Neuro- en revalidatiepsychologie, MEG, Neuropsychology and rehabilitation psychology, 05 social sciences, Magnetoencephalography, Source imaging, Electroencephalography, 180 000 Predictive Brain, Models, Theoretical, Neurology, Source localization, 030217 neurology & neurosurgery, RC321-571
Abstract: Contains fulltext : 246921.pdf (Publisher’s version ) (Open Access) Beamforming is a popular method for functional source reconstruction using magnetoencephalography (MEG) and electroencephalography (EEG) data. Beamformers, which were first proposed for MEG more than two decades ago, have since been applied in hundreds of studies, demonstrating that they are a versatile and robust tool for neuroscience. However, certain characteristics of beamformers remain somewhat elusive and there currently does not exist a unified documentation of the mathematical underpinnings and computational subtleties of beamformers as implemented in the most widely used academic open source software packages for MEG analysis (Brainstorm, FieldTrip, MNE, and SPM). Here, we provide such documentation that aims at providing the mathematical background of beamforming and unifying the terminology. Beamformer implementations are compared across toolboxes and pitfalls of beamforming analyses are discussed. Specifically, we provide details on handling rank deficient covariance matrices, prewhitening, the rank reduction of forward fields, and on the combination of heterogeneous sensor types, such as magnetometers and gradiometers. The overall aim of this paper is to contribute to contemporary efforts towards higher levels of computational transparency in functional neuroimaging. 11 p.
Published: 2022

Catalog

Books, media, physical & digital resources

See catalog results

Searchworks

Select search scope, currently: Articles Catalog books, media & more in Jio Institute collections Articles journal articles & other e-resources

Search

Search Constraints

Refine your results

Search Limiters

Topic

Publication Year Range

Language

Journal

Database

Publisher

70 results on '"University College of London [London] (UCL)-University College of London [London] (UCL)"'

Search Results

Catalog

Select search scope, currently: Articles

Catalog

books, media & more in Jio Institute collections

Articles

journal articles & other e-resources