Site-specific crop management in heterogeneous fields has emerged as a promising avenue towards increasing agricultural productivity whilst safeguarding the environment. However, successful implementation is hampered by insufficient availability of accurate spatial information on crop growth, vigor, and health status at large scales. Challenges persist particularly in interpreting remote sensing signals within commercial crop production due to the variability in canopy appearance resulting from diverse factors. Recently, high-resolution imagery captured from unmanned aerial vehicles has shown significant potential for calibrating and validating methods for remote sensing signal interpretation. We present a comprehensive multi-scale image dataset encompassing 35,000 high-resolution aerial RGB images, ground-based imagery, and Sentinel-2 satellite data from nine on-farm wheat fields in Switzerland. We provide geo-referenced orthomosaics, digital elevation models, and shapefiles, enabling detailed analysis of field characteristics across the growing season. In combination with rich meta data such as detailed records of crop husbandry, crop phenology, and yield maps, this data set enables key challenges in remote sensing-based trait estimation and precision agriculture to be addressed., (© 2024. The Author(s).)