Search

The management of natural environments, whether for conservation or production, requires a deep understanding of wildlife. The number, location, and behavior of wild animals are among the main subjects of study in ecology and wildlife research. The use of camera traps offers the opportunity to quickly collect large quantities of photographs that capture wildlife in its natural habitat, avoiding factors that could alter their behavior. In Tierra del Fuego, Argentina, research is being conducted on forest use by different herbivores (guanacos, cows, sheep) to optimize management and protect these natural ecosystems. Although camera traps allow for the collection of millions of images, interpreting such photographs presents a scalability challenge for manual processing. As a result, much of the valuable knowledge stored in these vast data repositories remains untapped. Neural Networks and Deep Learning are areas of study within Artificial Intelligence. Over the past decade, these two disciplines have made significant contributions to image recognition on a global scale. Ecological and wildlife conservation studies can be combined with these new technologies to extract important information from the photographs obtained by camera traps, contributing to the understanding of various natural processes and improving the management of the involved wild areas. Our project aims to develop neural network models to classify animal species in photographs taken with camera traps, addressing large-scale challenges in scientific research., Comment: in Spanish language, XXIV Workshop de Investigadores en Ciencias de la Computaci\'on (WICC 2022, Mendoza)

This paper presents a novel deep learning-based approach named RealDiffFusionNet incorporating Neural Controlled Differential Equations (Neural CDE) - time series models that are robust in handling irregularly sampled data - and multi-head attention to align relevant multimodal context (image data, time invariant data, etc.) at each time point. Long short-term memory (LSTM) models were also used as a baseline. Two different datasets were used: a data from the Open-Source Imaging Consortium (OSIC) containing structured time series data of demographics and lung function with a baseline CT scan of the lungs and the second from the Alzheimer's Disease Neuroimaging Initiative (ADNI) containing a series of MRI scans along with demographics, physical examinations, and cognitive assessment data. An ablation study was performed to understand the role of CDEs, multimodal data, attention fusion, and interpolation strategies on model performance. When the baseline models were evaluated, the use of multimodal data resulted in an improvement in Neural CDE performance, with a lower test RMSE. Additionally, the performance of multimodal Neural CDE was also superior to multimodal LSTM. In the attention-based architectures, fusion through concatenation and rectilinear interpolation were found to improve model performance. The performance of the proposed RealDiffFusionNet was found to be superior (0.2570) to all models. For the ADNI dataset, between the Neural-CDE and LSTM models trained only on the structured data, the test RMSE were comparable (0.471 for LSTM vs. 0.4581 Neural-CDE). Furthermore, the addition of image features from patients' MRI series resulted in an improvement in performance, with a lower test RMSE (0.4372 with multimodal vs 0.4581 with structured data). RealDiffFusionNet has shown promise in utilizing CDEs and multimodal data to accurately predict disease progression.