Search

Mapping and predicting crop yield on a large scale is increasingly important for use cases such as policy-making, risk insurance and precision agriculture applications at farm and field scale. The higher spatial resolution of Sentinel-2 compared to Landsat allows for satellite-based crop yield mapping even in relatively small scaled agricultural settings such as found in Switzerland and other central European regions. In this study, five years (2017–2021) of cereal crop yield data from a combine harvester were used to model crop yield within-field, on a spatial scale corresponding to the Sentinel-2 pixel level. Three established methods from literature using (i-ii) spectral indices and (iii) raw satellite reflectance as well as (iv) a recurrent neural network (RNN) were chosen for analysis. Although the RNN approach did not outperform the other methods, it was more efficient because of the comparatively simple end-to-end training of the model, resulting in much less time spent on data cleaning and feature extraction needed for spectral index time series analysis. The RNN was also able to discriminate cloudy data by itself, reaching similar performance levels as if using pre-processed, cloud-free data. Modelling was performed on individual years, all years combined and on unseen years using leave-one-year-out cross-validation. The models performed best when using data from all years (R2 up to 0.88, relative RMSE up to 10.49 %) and showed poor performance when predicting on unseen data years, especially for years with previously unknown weather patterns. This highlights the importance of yearly model calibration and the need for continuous data collection enabling long time series for future crop yield models., Field Crops Research, 292, ISSN:0378-4290, ISSN:1872-6852

The aim of this paper is to map agricultural crops by classifying satellite image time series. Domain experts in agriculture work with crop type labels that are organised in a hierarchical tree structure, where coarse classes (like orchards) are subdivided into finer ones (like apples, pears, vines, etc.). We develop a crop classification method that exploits this expert knowledge and significantly improves the mapping of rare crop types. The three-level label hierarchy is encoded in a convolutional, recurrent neural network (convRNN), such that for each pixel the model predicts three labels at different level of granularity. This end-to-end trainable, hierarchical network architecture allows the model to learn joint feature representations of rare classes (e.g., apples, pears) at a coarser level (e.g., orchard), thereby boosting classification performance at the fine-grained level. Additionally, labelling at different granularity also makes it possible to adjust the output according to the classification scores; as coarser labels with high confidence are sometimes more useful for agricultural practice than fine-grained but very uncertain labels. We validate the proposed method on a new, large dataset that we make public. ZueriCrop covers an area of 50 km × 48 km in the Swiss cantons of Zurich and Thurgau with a total of 116′000 individual fields spanning 48 crop classes, and 28,000 (multi-temporal) image patches from Sentinel-2. We compare our proposed hierarchical convRNN model with several baselines, including methods designed for imbalanced class distributions. The hierarchical approach performs superior by at least 9.9 percentage points in F1-score., Remote Sensing of Environment, 264, ISSN:0034-4257

The visual world we sense, interpret and interact everyday is a complex composition of interleaved physical entities. Therefore, it is a very challenging task to generate vivid scenes of similar complexity using computers. In this work, we present a scene generation framework based on Generative Adversarial Networks (GANs) to sequentially compose a scene, breaking down the underlying problem into smaller ones. Different than the existing approaches, our framework offers an explicit control over the elements of a scene through separate background and foreground generators. Starting with an initially generated background, foreground objects then populate the scene one-by-one in a sequential manner. Via quantitative and qualitative experiments on a subset of the MS-COCO dataset, we show that our proposed framework produces not only more diverse images but also copes better with affine transformations and occlusion artifacts of foreground objects than its counterparts., This paper was accepted at AAAI 2019

We propose a new STAckable Recurrent cell (STAR) for recurrent neural networks (RNNs), which has fewer parameters than widely used LSTM and GRU while being more robust against vanishing or exploding gradients. Stacking recurrent units into deep architectures suffers from two major limitations: (i) many recurrent cells (e.g., LSTMs) are costly in terms of parameters and computation resources; and (ii) deep RNNs are prone to vanishing or exploding gradients during training. We investigate the training of multi-layer RNNs and examine the magnitude of the gradients as they propagate through the network in the "vertical" direction. We show that, depending on the structure of the basic recurrent unit, the gradients are systematically attenuated or amplified. Based on our analysis we design a new type of gated cell that better preserves gradient magnitude. We validate our design on a large number of sequence modelling tasks and demonstrate that the proposed STAR cell allows to build and train deeper recurrent architectures, ultimately leading to improved performance while being computationally more efficient., Comment: To appear in TPAMI (accepted March 2021)

3D visual understanding of the surrounding environment is vital for successful mobile robotic tasks such as autonomous navigation or general object interaction. However, current systems have limited perceptual capabilities in the sense that they are not very well adaptable to unknown environments. Human operators, on the other hand, are experts in adapting to previously unknown information. Hence, human-robot teaming in which the human helps the robot to adapt to new environments and the robot assists in automated object recognition to efficiently feed the control environment of the operator is advantageous. In this work, we propose an object recognition and localization system for mobile robots, based on deep learning, and we study the adaptation of the resulting robotic perception to a new environment. We propose two methods to teach the robot a new object category: using prior knowledge and using limited operator input. We conducted several experiments to show the feasibility of proposed methods.

Recent studies show that eyebrows can be used as a biometric or soft biometric for recognition. In some scenarios such as partially occluded or covered faces, they can be used for recognition. In this paper, we study eyebrow recognition using texture-based features. We apply features which have not been used before for eyebrow recognition such as 3-patch local binary pattern and WLD (Weber local descriptor) features. Also, we use more conventional features such as uniform LBP (Local binary pattern) and HOG (Histograms of oriented gradients). Methods are tested on both small- and large-sized datasets of images taken from FRGC database. Our experiments show that using some of these texture-based features together increases the performance significantly. We achieved more than 95% recognition accuracy for left and right eyebrows.