Back to Search
Start Over
Auto-Pytorch: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL.
- Source :
- IEEE Transactions on Pattern Analysis & Machine Intelligence; Sep2021, Vol. 43 Issue 9, p3079-3090, 12p
- Publication Year :
- 2021
-
Abstract
- While early AutoML frameworks focused on optimizing traditional ML pipelines and their hyperparameters, a recent trend in AutoML is to focus on neural architecture search. In this paper, we introduce Auto-PyTorch, which brings together the best of these two worlds by jointly and robustly optimizing the network architecture and the training hyperparameters to enable fully automated deep learning (AutoDL). Auto-PyTorch achieves state-of-the-art performance on several tabular benchmarks by combining multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) and common baselines for tabular data. To thoroughly study our assumptions on how to design such an AutoDL system, we additionally introduce a new benchmark on learning curves for DNNs, dubbed LCBench, and run extensive ablation studies of the full Auto-PyTorch on typical AutoML benchmarks, eventually showing that Auto-PyTorch performs better than several state-of-the-art competitors. [ABSTRACT FROM AUTHOR]
- Subjects :
- DEEP learning
COMPUTER architecture
MACHINE learning
TASK analysis
Subjects
Details
- Language :
- English
- ISSN :
- 01628828
- Volume :
- 43
- Issue :
- 9
- Database :
- Complementary Index
- Journal :
- IEEE Transactions on Pattern Analysis & Machine Intelligence
- Publication Type :
- Academic Journal
- Accession number :
- 153127886
- Full Text :
- https://doi.org/10.1109/TPAMI.2021.3067763