Search

The integration of deep neural networks into sparse synthetic aperture radar (SAR) imaging is explored to enhance SAR imaging performance and reduce the system's sampling rate. However, the scarcity of training samples and mismatches between the training data and the SAR system pose significant challenges to the method's further development. In this paper, we propose a novel SAR imaging approach based on deep image prior powered by RED (DeepRED), enabling unsupervised SAR imaging without the need for additional training data. Initially, DeepRED is introduced as the regularization technique within the sparse SAR imaging model. Subsequently, variable splitting and the alternating direction method of multipliers (ADMM) are employed to solve the imaging model, alternately updating the magnitude and phase of the SAR image. Additionally, the SAR echo simulation operator is utilized as an observation model to enhance computational efficiency. Through simulations and real data experiments, we demonstrate that our method maintains imaging quality and system downsampling rate on par with deep-neural-network-based sparse SAR imaging but without the requirement for training data. [ABSTRACT FROM AUTHOR]

Atmospheric drag stands out as the predominant non-gravitational force acting on satellites in Low Earth Orbit (LEO), with altitudes below 2000 km. This drag exhibits a strong dependence on the thermospheric mass density (TMD), a parameter of vital significance in the realms of orbit determination, prediction, collision avoidance, and re-entry forecasting. A multitude of empirical TMD models have been developed, incorporating contemporary data sources, including TMD measurements obtained through onboard accelerometers on LEO satellites. This paper delves into three different TMD modelling techniques, specifically, Fourier series, spherical harmonics, and artificial neural networks (ANNs), during periods of geomagnetic quiescence. The TMD data utilised for modelling and evaluation are derived from three distinct LEO satellites: GOCE (at an altitude of approximately 250 km), CHAMP (around 400 km), and GRACE (around 500 km), spanning the years 2002 to 2013. The consistent utilisation of these TMD data sets allows for a clear performance assessment of the different modelling approaches. Subsequent research will shift its focus to TMD modelling during geomagnetic disturbances, while the present work can serve as a foundation for disentangling TMD variations stemming from geomagnetic activity. Furthermore, this study undertakes precise TMD modelling during geomagnetic quiescence using data obtained from the GRACE (at an altitude of approximately 500 km), CHAMP (around 400 km), and GOCE (roughly 250 km) satellites, covering the period from 2002 to 2013. It employs three distinct methods, namely Fourier analysis, spherical harmonics (SH) analysis, and the artificial neural network (ANN) technique, which are subsequently compared to identify the most suitable methodology for TMD modelling. Additionally, various combinations of time and coordinate representations are scrutinised within the context of TMD modelling. Our results show that the precision of low-order Fourier-based models can be enhanced by up to 10 % through the utilisation of geocentric solar magnetic coordinates. Both the Fourier- and SH-based models exhibit limitations in approximating the vertical gradient of TMD. Conversely, the ANN-based model possesses the capacity to capture vertical TMD variability without manifesting sensitivity to variations in time and coordinate inputs. [ABSTRACT FROM AUTHOR]

Structures made of heterogeneous materials, such as composites, often require a multiscale approach when their behavior is simulated using the finite element method. By solving the boundary value problem of the macroscale model, for previously homogenized material properties, the resulting stress maps can be obtained. However, such stress results do not describe the actual behavior of the material and are often significantly different from the actual stresses in the heterogeneous microstructure. Finding high-accuracy stress results for such materials leads to time-consuming analyses in both scales. This paper focuses on the application of machine learning to multiscale analysis of structures made of composite materials, to substantially decrease the time of computations of such localization problems. The presented methodology was validated by a numerical example where a structure made of resin epoxy with randomly distributed short glass fibers was analyzed using a computational multiscale approach. Carefully prepared training data allowed artificial neural networks to learn relationships between two scales and significantly increased the efficiency of the multiscale approach. [ABSTRACT FROM AUTHOR]

Soybean (Glycine max (L.) Merr.) is a major source of oil and protein for human food and animal feed; however, soybean crops face diverse factors causing damage, including pathogen infections, environmental shifts, poor fertilization, and incorrect pesticide use, leading to reduced yields. Identifying the level of leaf damage aids yield projections, pesticide, and fertilizer decisions. Deep learning models (DLMs) and neural networks mastering tasks from abundant data have been used for binary healthy/unhealthy leaf classification. However, no DLM predicts and categorizes soybean leaf damage severity (five levels) for tailored pesticide use and yield forecasts. This paper introduces a novel DLM for accurate damage prediction and classification, trained on 2930 near-field soybean leaf images. The model quantifies damage severity, distinguishing healthy/unhealthy leaves and offering a comprehensive solution. Performance metrics include accuracy, precision, recall, and F1-score. This research presents a robust DLM for soybean damage assessment, supporting informed agricultural decisions based on specific damage levels and enhancing crop management and productivity. [ABSTRACT FROM AUTHOR]

Deep learning has gained widespread interest in the task of building semantic segmentation modelling using remote sensing images; however, neural network models require a large number of training samples to achieve better classification performance, and the models are more sensitive to error patches in the training samples. The training samples obtained in semi-supervised classification methods need less reliable weakly labelled samples, but current semi-supervised classification research puts the generated weak samples directly into the model for applications, with less consideration of the impact of the accuracy and quality improvement of the weak samples on the subsequent model classification. Therefore, to address the problem of generating and optimising the quality of weak samples from training data in deep learning, this paper proposes a semi-supervised building classification framework. Firstly, based on the test results of the remote sensing image segmentation model and the unsupervised classification results of LiDAR point cloud data, this paper quickly generates weak image samples of buildings. Secondly, in order to improve the quality of the spots of the weak samples, an iterative optimisation strategy of the weak samples is proposed to compare and analyse the weak samples with the real samples and extract the accurate samples from the weak samples. Finally, the real samples, the weak samples, and the optimised weak samples are input into the semantic segmentation model of buildings for accuracy evaluation and analysis. The effectiveness of this paper's approach was experimentally verified on two different building datasets, and the optimised weak samples improved by 1.9% and 0.6%, respectively, in the test accuracy mIoU compared to the initial weak samples. The results demonstrate that the semi-supervised classification framework proposed in this paper can be used to alleviate the model's demand for a large number of real-labelled samples while improving the ability to utilise weak samples, and it can be used as an alternative to fully supervised classification methods in deep learning model applications that require a large number of training samples. [ABSTRACT FROM AUTHOR]

Label corruption leads to a significant challenge in supervised learning, particularly in deep neural networks. This paper considers recovering a small corrupted subset of data samples which are typically caused by non-expert sources, such as automatic classifiers. Our aim is to recover the corrupted data samples by exploiting a finite query-testing system as an additional expert. The task involves identifying the corrupted data samples with minimal expert queries and finding them to their true label values. The proposed query-testing system uses a random selection of a subset of data samples and utilizes finite field operations to construct combined responses. In this paper, we demonstrate an information-theoretic lower bound on the minimum number of queries required for recovering corrupted labels. The lower bound can be represented as a function of joint entropy with an imbalanced rate of data samples and mislabeled probability. In addition, we find an upper bound on the error probability using maximum a posteriori decoding. [ABSTRACT FROM AUTHOR]

Simple Summary: Automatic segmentation of organs and other regions of interest is a promising approach for reducing the workload of doctors in radiotherapeutic planning, but it can be hard for doctors and researchers to keep up with current developments. This review evaluates 807 papers and reveals trends, commonalities, and gaps in the existing corpus. A set of recommendations for conducting effective segmentation studies is also provided. This review provides a formal overview of current automatic segmentation studies that use deep learning in radiotherapy. It covers 807 published papers and includes multiple cancer sites, image types (CT/MRI/PET), and segmentation methods. We collect key statistics about the papers to uncover commonalities, trends, and methods, and identify areas where more research might be needed. Moreover, we analyzed the corpus by posing explicit questions aimed at providing high-quality and actionable insights, including: "What should researchers think about when starting a segmentation study?", "How can research practices in medical image segmentation be improved?", "What is missing from the current corpus?", and more. This allowed us to provide practical guidelines on how to conduct a good segmentation study in today's competitive environment that will be useful for future research within the field, regardless of the specific radiotherapeutic subfield. To aid in our analysis, we used the large language model ChatGPT to condense information. [ABSTRACT FROM AUTHOR]

Simple Summary: This paper introduces a novel graph neural network (a hierarchical graph pooling model), SANEpool, to effectively detect core sub-networks of significant genes for predicting the synergy score of drug/drug combinations in cancer. SANEpool successfully addresses the limitations of the un-transparency in the prediction process of previous computational AI models for drug synergy prediction, while providing the superior predictive performance than popular baselines on numerous drug-synergy prediction datasets. The success of SANEpool indicates that significant gene-gene interactions and gene-drug interactions play a crucial role in designing powerful deep learning models to provide accurate prediction and to reveal the mechanism of the synergy (MoS). Synergistic drug combinations provide huge potentials to enhance therapeutic efficacy and to reduce adverse reactions. However, effective and synergistic drug combination prediction remains an open question because of the unknown causal disease signaling pathways. Though various deep learning (AI) models have been proposed to quantitatively predict the synergism of drug combinations, the major limitation of existing deep learning methods is that they are inherently not interpretable, which makes the conclusions of AI models untransparent to human experts, henceforth limiting the robustness of the model conclusion and the implementation ability of these models in real-world human–AI healthcare. In this paper, we develop an interpretable graph neural network (GNN) that reveals the underlying essential therapeutic targets and the mechanism of the synergy (MoS) by mining the sub-molecular network of great importance. The key point of the interpretable GNN prediction model is a novel graph pooling layer, a self-attention-based node and edge pool (henceforth SANEpool), that can compute the attention score (importance) of genes and connections based on the genomic features and topology. As such, the proposed GNN model provides a systematic way to predict and interpret the drug combination synergism based on the detected crucial sub-molecular network. Experiments on various well-adopted drug-synergy-prediction datasets demonstrate that (1) the SANEpool model has superior predictive ability to generate accurate synergy score prediction, and (2) the sub-molecular networks detected by the SANEpool are self-explainable and salient for identifying synergistic drug combinations. [ABSTRACT FROM AUTHOR]

Accurately identifying the boundaries of agricultural land is critical to the effective management of its resources. This includes the determination of property and land rights, the prevention of non-agricultural activities on agricultural land, and the effective management of natural resources. There are various methods for accurate boundary detection, including traditional measurement methods and remote sensing, and the choice of the best method depends on specific objectives and conditions. This paper proposes the use of convolutional neural networks (CNNs) as an efficient and effective tool for the automatic recognition of agricultural land boundaries. The objective of this research paper is to develop an automated method for the recognition of agricultural land boundaries using deep neural networks and Sentinel 2 multispectral imagery. The Buinsky district of the Republic of Tatarstan, Russia, which is known to be an agricultural region, was chosen for this study because of the importance of the accurate detection of its agricultural land boundaries. Linknet, a deep neural network architecture with skip connections between encoder and decoder, was used for semantic segmentation to extract arable land boundaries, and transfer learning using a pre-trained EfficientNetB3 model was used to improve performance. The Linknet + EfficientNetB3 combination for semantic segmentation achieved an accuracy of 86.3% and an f1 measure of 0.924 on the validation sample. The results showed a high degree of agreement between the predicted field boundaries and the expert-validated boundaries. According to the results, the advantages of the method include its speed, scalability, and ability to detect patterns outside the study area. It is planned to improve the method by using different neural network architectures and prior recognized land use classes. [ABSTRACT FROM AUTHOR]

Facial emotion recognition (FER) is a computer vision process aimed at detecting and classifying human emotional expressions. FER systems are currently used in a vast range of applications from areas such as education, healthcare, or public safety; therefore, detection and recognition accuracies are very important. Similar to any computer vision task based on image analyses, FER solutions are also suitable for integration with artificial intelligence solutions represented by different neural network varieties, especially deep neural networks that have shown great potential in the last years due to their feature extraction capabilities and computational efficiency over large datasets. In this context, this paper reviews the latest developments in the FER area, with a focus on recent neural network models that implement specific facial image analysis algorithms to detect and recognize facial emotions. This paper's scope is to present from historical and conceptual perspectives the evolution of the neural network architectures that proved significant results in the FER area. This paper endorses convolutional neural network (CNN)-based architectures against other neural network architectures, such as recurrent neural networks or generative adversarial networks, highlighting the key elements and performance of each architecture, and the advantages and limitations of the proposed models in the analyzed papers. Additionally, this paper presents the available datasets that are currently used for emotion recognition from facial expressions and micro-expressions. The usage of FER systems is also highlighted in various domains such as healthcare, education, security, or social IoT. Finally, open issues and future possible developments in the FER area are identified. [ABSTRACT FROM AUTHOR]

This paper assesses the fault-detection capabilities of modern deep-learning models. It highlights that a naive deep-learning approach optimized for accuracy is unsuitable for learning fault-detection models from time-series data. Consequently, out-of-the-box deep-learning strategies may yield impressive accuracy results but are ill-equipped for real-world applications. The paper introduces a methodology for estimating fault-detection delays when no oracle information on fault occurrence time is available. Moreover, the paper presents a straightforward approach to implicitly achieve the objective of minimizing fault-detection delays. This approach involves using pseudo-multi-objective deep optimization with data windowing, which enables the utilization of standard deep-learning methods for fault detection and expanding their applicability. However, it does introduce an additional hyperparameter that needs careful tuning. The paper employs the Tennessee Eastman Process dataset as a case study to demonstrate its findings. The results effectively highlight the limitations of standard loss functions and emphasize the importance of incorporating fault-detection delays in evaluating and reporting performance. In our study, the pseudo-multi-objective optimization could reach a fault-detection accuracy of 95% in just a fifth of the time it takes the best naive approach to do so. [ABSTRACT FROM AUTHOR]

Forecasting energy consumption models allow for improvements in building performance and reduce energy consumption. Energy efficiency has become a pressing concern in recent years due to the increasing energy demand and concerns over climate change. This paper addresses the energy consumption forecast as a crucial ingredient in the technology to optimize building system operations and identifies energy efficiency upgrades. The work proposes a modified multi-head transformer model focused on multi-variable time series through a learnable weighting feature attention matrix to combine all input variables and forecast building energy consumption properly. The proposed multivariate transformer-based model is compared with two other recurrent neural network models, showing a robust performance while exhibiting a lower mean absolute percentage error. Overall, this paper highlights the superior performance of the modified transformer-based model for the energy consumption forecast in a multivariate step, allowing it to be incorporated in future forecasting tasks, allowing for the tracing of future energy consumption scenarios according to the current building usage, playing a significant role in creating a more sustainable and energy-efficient building usage. [ABSTRACT FROM AUTHOR]

The goal of the present study is to find a method for improving the predictive capabilities of feedforward neural networks in cases where values distant from the input–output sample interval are predicted. This paper proposes an iterative prediction algorithm based on two assumptions. One is that predictions near the statistical sample have much lower error than those distant from the sample. The second is that a neural network can generate additional training samples and use them to train itself in order to get closer to a distant prediction point. This paper presents the results of multiple experiments with different univariate and multivariate functions and compares the predictions made by neural networks before and after their training with the proposed iterative algorithm. The results show that, having passed through the stages of the algorithm, artificial neural networks significantly improve their interpolation performance in long-term forecasting. The present study demonstrates that neural networks are capable of creating additional samples for their own training, thus increasing their approximating efficiency. [ABSTRACT FROM AUTHOR]

Simple Summary: In this paper, we introduce a new technique for enhancing medical image diagnosis through transfer learning (TL). The approach addresses the issue of limited labelled images by pre-training deep learning models on similar medical images and then refining them with a small set of annotated medical images. Our method demonstrated excellent results in classifying the humerus and wrist, surpassing previous methods, and showing greater robustness in various experiments. Furthermore, we demonstrate the adaptability of the approach with a CT case, which showed improvements in the results. Medical image classification poses significant challenges in real-world scenarios. One major obstacle is the scarcity of labelled training data, which hampers the performance of image-classification algorithms and generalisation. Gathering sufficient labelled data is often difficult and time-consuming in the medical domain, but deep learning (DL) has shown remarkable performance, although it typically requires a large amount of labelled data to achieve optimal results. Transfer learning (TL) has played a pivotal role in reducing the time, cost, and need for a large number of labelled images. This paper presents a novel TL approach that aims to overcome the limitations and disadvantages of TL that are characteristic of an ImageNet dataset, which belongs to a different domain. Our proposed TL approach involves training DL models on numerous medical images that are similar to the target dataset. These models were then fine-tuned using a small set of annotated medical images to leverage the knowledge gained from the pre-training phase. We specifically focused on medical X-ray imaging scenarios that involve the humerus and wrist from the musculoskeletal radiographs (MURA) dataset. Both of these tasks face significant challenges regarding accurate classification. The models trained with the proposed TL were used to extract features and were subsequently fused to train several machine learning (ML) classifiers. We combined these diverse features to represent various relevant characteristics in a comprehensive way. Through extensive evaluation, our proposed TL and feature-fusion approach using ML classifiers achieved remarkable results. For the classification of the humerus, we achieved an accuracy of 87.85%, an F1-score of 87.63%, and a Cohen's Kappa coefficient of 75.69%. For wrist classification, our approach achieved an accuracy of 85.58%, an F1-score of 82.70%, and a Cohen's Kappa coefficient of 70.46%. The results demonstrated that the models trained using our proposed TL approach outperformed those trained with ImageNet TL. We employed visualisation techniques to further validate these findings, including a gradient-based class activation heat map (Grad-CAM) and locally interpretable model-independent explanations (LIME). These visualisation tools provided additional evidence to support the superior accuracy of models trained with our proposed TL approach compared to those trained with ImageNet TL. Furthermore, our proposed TL approach exhibited greater robustness in various experiments compared to ImageNet TL. Importantly, the proposed TL approach and the feature-fusion technique are not limited to specific tasks. They can be applied to various medical image applications, thus extending their utility and potential impact. To demonstrate the concept of reusability, a computed tomography (CT) case was adopted. The results obtained from the proposed method showed improvements. [ABSTRACT FROM AUTHOR]

In recent years, there has been a significant surge in discussions surrounding artificial intelligence (AI), along with a corresponding increase in its practical applications in various facets of everyday life, including the medical industry. Notably, even in the highly specialized realm of neurosurgery, AI has been utilized for differential diagnosis, pre-operative evaluation, and improving surgical precision. Many of these applications have begun to mitigate risks of intraoperative and postoperative complications and post-operative care. This article aims to present an overview of the principal published papers on the significant themes of tumor, spine, epilepsy, and vascular issues, wherein AI has been applied to assess its potential applications within neurosurgery. The method involved identifying high-cited seminal papers using PubMed and Google Scholar, conducting a comprehensive review of various study types, and summarizing machine learning applications to enhance understanding among clinicians for future utilization. Recent studies demonstrate that machine learning (ML) holds significant potential in neuro-oncological care, spine surgery, epilepsy management, and other neurosurgical applications. ML techniques have proven effective in tumor identification, surgical outcomes prediction, seizure outcome prediction, aneurysm prediction, and more, highlighting its broad impact and potential in improving patient management and outcomes in neurosurgery. This review will encompass the current state of research, as well as predictions for the future of AI within neurosurgery. [ABSTRACT FROM AUTHOR]

The issue of climate change holds immense significance, affecting various aspects of life, including the environment, the interaction between soil conditions and the atmosphere, and agriculture. Over the past few decades, a range of spatio-temporal and Deep Neural Network (DNN) techniques had been proposed within the field of Machine Learning (ML) for climate forecasting, using spatial and temporal data. The forecasting model in this paper is highly complex, particularly due to the presence of nonlinear data in the residual modeling of General Space-Time Autoregressive Integrated Moving Average (GSTARIMA), which represented nonstationary data with time and location dependencies. This model effectively captured trends and seasonal data with time and location dependencies. On the other hand, DNNs proved reliable for modeling nonlinear data that posed challenges for spatio-temporal approaches. This research presented a comprehensive overview of the integrated approach between the GSTARIMA model and DNNs, following the six-stage Data Analytics Lifecycle methodology. The focus was primarily on previous works conducted between 2013 and 2022. The review showed that the GSTARIMA–DNN integration model was a promising tool for forecasting climate in a specific region in the future. Although spatio-temporal and DNN approaches have been widely employed for predicting the climate and its impact on human life due to their computational efficiency and ability to handle complex problems, the proposed method is expected to be universally accepted for integrating these models, which encompass location and time dependencies. Furthermore, it was found that the GSTARIMA–DNN method, incorporating multivariate variables, locations, and multiple hidden layers, was suitable for short-term climate forecasting. Finally, this paper presented several future directions and recommendations for further research. [ABSTRACT FROM AUTHOR]

The features of the kernel extreme learning machine—efficient processing, improved performance, and less human parameter setting—have allowed it to be effectively used to batch multi-label classification tasks. These classic classification algorithms must at present contend with accuracy and space–time issues as a result of the vast and quick, multi-label, and concept drift features of the developing data streams in the practical application sector. The KELM training procedure still has a difficulty in that it has to be repeated numerous times independently in order to maximize the model's generalization performance or the number of nodes in the hidden layer. In this paper, a kernel extreme learning machine multi-label data classification method based on the butterfly algorithm optimized by particle swarm optimization is proposed. The proposed algorithm, which fully accounts for the optimization of the model generalization ability and the number of hidden layer nodes, can train multiple KELM hidden layer networks at once while maintaining the algorithm's current time complexity and avoiding a significant number of repeated calculations. The simulation results demonstrate that, in comparison to the PSO-KELM, BBA-KELM, and BOA-KELM algorithms, the PSOBOA-KELM algorithm proposed in this paper can more effectively search the kernel extreme learning machine parameters and more effectively balance the global and local performance, resulting in a KELM prediction model with a higher prediction accuracy. [ABSTRACT FROM AUTHOR]

The paper presents an original approach to the synthesis of polycalciumorganyl silsesquioxanes through the reaction of polyorganyl silsesquioxanes [RSiO1.5]n (where R is an ethyl and phenyl radical) with sea urchin skeleton under the conditions of mechanochemical activation. The novelty and practical significance of the present study lies in the use of an available natural raw source as a source of calcium ions to initiate the reaction of calcium silicate formation and create a matrix for the formation of a porous inorganic composite framework. The thermal stability of the introduced silicates, i.e., the ability to maintain a porous structure at high temperatures, is key to the production of an ordered porous material. The reaction scheme was proposed to be based on the interaction of calcium carbonate with the siloxane bond. FTIR, XRD, GPC, and TGA were used to study the composition and structure of the obtained materials. The cross-sectional area of the polymer chain and the volumes of the coherent scattering regions of the polymers obtained were calculated from the XRD data. To prepare the composites, the sea urchin skeleton was further modified with polycalciumorganyl silsesquioxanes in a toluene solution. To remove the sea urchin skeleton, the obtained biomimetic composites were treated with hydrochloric acid. The results of the morphological and surface composition studies are reported. The method proposed in the paper could be of fundamental importance for the possibility of obtaining structured porous composite materials for a wide range of practical applications, including for the purpose of creating a composite that may be a promising carrier for targeted delivery of chemotherapy agents. [ABSTRACT FROM AUTHOR]

This paper presents reported machine learning approaches in the field of Brillouin distributed fiber optic sensors (DFOSs). The increasing popularity of Brillouin DFOSs stems from their capability to continuously monitor temperature and strain along kilometer-long optical fibers, rendering them attractive for industrial applications, such as the structural health monitoring of large civil infrastructures and pipelines. In recent years, machine learning has been integrated into the Brillouin DFOS signal processing, resulting in fast and enhanced temperature, strain, and humidity measurements without increasing the system's cost. Machine learning has also contributed to enhanced spatial resolution in Brillouin optical time domain analysis (BOTDA) systems and shorter measurement times in Brillouin optical frequency domain analysis (BOFDA) systems. This paper provides an overview of the applied machine learning methodologies in Brillouin DFOSs, as well as future perspectives in this area. [ABSTRACT FROM AUTHOR]

In this paper, the authors investigate the possibility of applying artificial intelligence algorithms to the outputs of a low-cost Kalman filter-based navigation solution in order to achieve performance similar to that of high-end MEMS inertial sensors. To further improve the results of the prototype and simultaneously lighten filter requirements, different AI models are compared in this paper to determine their performance in terms of complexity and accuracy. By overcoming some known limitations (e.g., sensitivity on the dimension of input data from inertial sensors) and starting from Kalman filter applications (whose raw noise parameter estimates were obtained from a simple analysis of sensor specifications), such a solution presents an intermediate behavior compared to the current state of the art. It allows the exploitation of the power of AI models. Different Neural Network models have been taken into account and compared in terms of measurement accuracy and a number of model parameters; in particular, Dense, 1-Dimension Convolutional, and Long Short Term Memory Neural networks. As can be excepted, the higher the NN complexity, the higher the measurement accuracy; the models' performance has been assessed by means of the root-mean-square error (RMSE) between the target and predicted values of all the navigation parameters. [ABSTRACT FROM AUTHOR]

This Special Issue contains 2 reviews and 17 research papers related to the following topics: Time series forecasting [[1], [3], [5]]; Image analysis [[6]]; Medical applications [[7]]; Knowledge graph analysis [[9]]; Cybersecurity [[11], [13]]; Traffic analysis [[14]]; Agriculture [[16]]; Environmental data analysis [[17]]. In [[2]], a time series analysis was applied in a different manner: their prediction of the high stock dividend (HSD) was based on a sequence of typical machine learning approaches instead of state-of-the-art methods such as ARIMA or SMA. The authors of [[1]] focused on short time series forecasting in the domain of crime data (thefts, shoplifting, vehicular crimes, and burglaries in Mexico). In this paper, the authors attempt to answer the questions what is air traffic complexity and which air traffic data variables have greater impacts on increases in complexity?. [Extracted from the article]

In this paper, a new method that computes the aesthetics of a melody fragment is proposed, starting from dissonances. While music generated with artificial intelligence applications may be produced considerably more quickly than human-composed music, it has the drawback of not being appreciated like a human composition, being many times perceived by humans as artificial. For achieving supervised machine learning objectives of improving the quality of the great number of generated melodies, it is a challenge to ask humans to grade them. Therefore, it would be preferable if the aesthetics of artificial-intelligence-generated music is calculated by an algorithm. The proposed method in this paper is based on a neural network and a mathematical formula, which has been developed with the help of a study in which 108 students evaluated the aesthetics of several melodies. For evaluation, numerical values generated by this method were compared with ratings provided by human listeners from a second study in which 30 students participated and scores were generated by an existing different method developed by psychologists and three other methods developed by musicians. Our method achieved a Pearson correlation of 0.49 with human aesthetic scores, which is a much better result than other methods obtained. Additionally, our method made a distinction between human-composed melodies and artificial-intelligence-generated scores in the same way that human listeners did. [ABSTRACT FROM AUTHOR]

Digital signature technology is essential for ensuring the authenticity and unforgeability of transactions in a private blockchain framework. In some scenarios, transactions require verification from multiple parties, each of whom needs to authenticate different parts of the transaction. To address this issue, researchers have developed multi-party ECDSA (Elliptic Curve Digital Signature Algorithm) signature schemes. However, these schemes either need to consider the authentication of different parts of the transaction or generate an aggregated signature. This paper proposes a novel solution that combines functional signatures and multi-party ECDSA signatures to create a multi-party functional signature for private blockchains. Compared to previous constructions, the proposed scheme ensures that each part of the transaction is verified. Furthermore, when the aggregate signature of the entire transaction cannot be verified, this scheme identifies the specific part of the transaction for which the signature authentication fails instead of rejecting the entire transaction. This paper uses a smart contract to securely deploy the proposed scheme and authenticate the f in functional signatures. The constructed scheme also provides security under the existential unforgeability of the ECDSA signature, even if n − 1 parties are corrupted, assuming a total of n parties. The scheme of this paper successfully conducted experiments on a personal computer, with three users taking approximately 343 ms, six users taking 552 ms, and nine users taking 791 ms. [ABSTRACT FROM AUTHOR]

This research paper presents a novel digital color image encryption approach that ensures high-level security while remaining simple and efficient. The proposed method utilizes a composite key r and x of 128-bits to create a small in-dimension private key (a chaotic map), which is then resized to match the color matrix dimension. The proposed method is uncomplicated and can be applied to any image without any modification. Image quality, sensitivity analysis, security analysis, correlation analysis, quality analysis, speed analysis, and attack robustness analysis are conducted to prove the efficiency and security aspects of the proposed method. The speed analysis shows that the proposed method improves the performance of image cryptography by minimizing encryption–decryption time and maximizing the throughput of the process of color cryptography. The results demonstrate that the proposed method provides better throughput than existing methods. Overall, this research paper provides a new approach to digital color image encryption that is highly secure, efficient, and applicable to various images. [ABSTRACT FROM AUTHOR]

A fuel cell is an energy conversion device that utilizes hydrogen energy through an electrochemical reaction. Despite their many advantages, such as high efficiency, zero emissions, and fast startup, fuel cells have not yet been fully commercialized due to deficiencies in service life, cost, and performance. Efficient evaluation methods for performance and service life are critical for the design and optimization of fuel cells. The purpose of this paper was to review the application of common machine learning algorithms in fuel cells. The significance and status of machine learning applications in fuel cells are briefly described. Common machine learning algorithms, such as artificial neural networks, support vector machines, and random forests are introduced, and their applications in fuel cell performance prediction and optimization are comprehensively elaborated. The review revealed that machine learning algorithms can be successfully used for performance prediction, service life prediction, and fault diagnosis in fuel cells, with good accuracy in solving nonlinear problems. Combined with optimization algorithms, machine learning models can further carry out the optimization of design and operating parameters to achieve multiple optimization goals with good accuracy and efficiency. It is expected that this review paper could help the reader comprehend the state of the art of machine learning applications in fuel fuels and shed light on further development directions in fuel cell research. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel attack on image encryption for privacy-preserving deep neural networks (DNNs). Although several encryption schemes have been proposed for privacy-preserving DNNs, existing cipher-text-only attacks (COAs) have succeeded in restoring visual information from encrypted images. Image encryption using the Vision Transformer (ViT) is known to be robust against existing COAs due to the operations of block scrambling and pixel shuffling, which permute divided blocks and pixels in an encrypted image. However, the correlation between blocks in the encrypted image can still be exploited for reconstruction. Therefore, in this paper, a novel jigsaw puzzle solver-based attack that utilizes block correlation is proposed to restore visual information from encrypted images. In the experiments, we evaluated the security of image encryption for privacy-preserving deep neural networks using both conventional and proposed COAs. The experimental results demonstrate that the proposed attack is able to restore almost all visual information from images encrypted for being applied to ViTs. [ABSTRACT FROM AUTHOR]

Short-term load forecasting (STLF) is critical for the energy industry. Accurate predictions of future electricity demand are necessary to ensure power systems' reliable and efficient operation. Various STLF models have been proposed in recent years, each with strengths and weaknesses. This paper comprehensively reviews some STLF models, including time series, artificial neural networks (ANNs), regression-based, and hybrid models. It first introduces the fundamental concepts and challenges of STLF, then discusses each model class's main features and assumptions. The paper compares the models in terms of their accuracy, robustness, computational efficiency, scalability, and adaptability and identifies each approach's advantages and limitations. Although this study suggests that ANNs and hybrid models may be the most promising ways to achieve accurate and reliable STLF, additional research is required to handle multiple input features, manage massive data sets, and adjust to shifting energy conditions. [ABSTRACT FROM AUTHOR]

The deep groove ball bearing is one of the most important components of the rotary motion system and is the research subject in this paper. After factory assembly, new ball bearings need to pass quality control. The conventional approach relies on measuring the vibration amplitudes for each unit and sorting them into classes according to the vibration level. In this paper, based on experimental research, models are created to predict the vibration class and analyze the dynamic behavior of new ball bearings. The models are based on artificial neural networks. A feedforward multilayer perceptron (MLP) was applied, and a backpropagation learning algorithm was used. A specific method of training groups of artificial neural networks was applied, where each network provided an answer to the input within the group, and the final answer was the mean value of the answers of all networks in the group. The models achieved a prediction accuracy of over 90%. The main aim of the research was to construct models that are able to predict the vibration class of a new ball bearing based on the geometric parameters of the bearing rings. The models are also applied to analyze the influence of surface roughness of the raceways and the internal radial clearance on bearing vibrations. [ABSTRACT FROM AUTHOR]

The accurate semantic segmentation of point cloud data is the basis for their application in the inspection of extra high-voltage transmission lines (EHVTL). As deep learning evolves, point-wise-based deep neural networks have shown great potential for the semantic segmentation of EHVTL point clouds. However, EHVTL point cloud data are characterized by a large data volume and significant class imbalance. Therefore, the down-sampling method and point cloud feature extraction method used in current point-wise-based deep neural networks hardly meet the needs of computational accuracy and efficiency. In this paper, we proposed a two-step down-sampling method and a point cloud feature extraction method based on local feature aggregation of the point clouds after down-sampling in each layer of the model (LFAPAD). We then established a deep neural network named PowerLine-Net for the semantic segmentation of the EHVTL point clouds. Furthermore, in order to test and analyze the performance of PowerLine-Net, we constructed a point cloud dataset for the EHVTL scenes. Using this dataset and the Semantic3D dataset, we implemented network parameter testing, semantic segmentation, and an accuracy comparison of different networks based on PowerLine-Net. The results illustrate that the semantic segmentation model proposed in this paper has a high computational efficiency and accuracy in the semantic segmentation of EHVTL point clouds. Compared with conventional deep neural networks, including PointCNN, KPConv, SPG, PointNet++, and RandLA-Net, PowerLine-Net also achieves a higher accuracy in the semantic segmentation of EHVTL point clouds. Moreover, based on the results predicted by PowerLine-Net, the risk point detection for EHVTL point clouds has been achieved, which demonstrates the important value of this network in practical applications. In addition, as shown by the results of Semantic3D, PowerLine-Net also achieves a high segmentation accuracy, which proves its powerful capability and wide applicability in semantic segmentation for the point clouds of large-scale scenes. [ABSTRACT FROM AUTHOR]

Energy systems around the world are going through tremendous transformations, mainly driven by carbon footprint reductions and related policy imperatives and low-carbon technological development. These transformations pose unprecedented technical challenges to the energy sector, but they also bring opportunities for energy systems to develop, adapt, and evolve. With rising complexity and increased digitalization, there has been significant growth in the amount of data in the power/energy sector (data ranging from power grid to household levels). Utilization of this large data (or "big data"), along with the use of proper data analytics, will allow for useful insights to be drawn that will help energy systems to deliver an increased amount of technical, operational, economic, and environmental benefits. This paper reviews various categories of data available in the current and future energy systems and the potential benefits of utilizing those data categories in energy system planning and operation. This paper also discusses the Big Data Analytics (BDA) that can be used to process/analyze the data and extract useful information that can be integrated and used in energy systems. More specifically, this paper discusses typical applications of BDA in energy systems, including how BDA can be used to resolve the critical issues faced by the current and future energy network operations and how BDA contributes to the development of smarter and more flexible energy systems. Combining data characterization and analysis methods, this review paper presents BDA as a powerful tool for making electrical energy systems smarter, more responsive, and more resilient to changes in operations. [ABSTRACT FROM AUTHOR]

In this paper, a set of best practice data sharing guidelines for wind turbine fault detection model evaluation is developed, which can help practitioners overcome the main challenges of digitalisation. Digitalisation is one of the key drivers for reducing costs and risks over the whole wind energy project life cycle. One of the largest challenges in successfully implementing digitalisation is the lack of data sharing and collaboration between organisations in the sector. In order to overcome this challenge, a new collaboration framework called WeDoWind was developed in recent work. The main innovation of this framework is the way it creates tangible incentives to motivate and empower different types of people from all over the world to share data and knowledge in practice. In this present paper, the challenges related to comparing and evaluating different SCADA-data-based wind turbine fault detection models are investigated by carrying out a new case study, the "WinJi Gearbox Fault Detection Challenge", based on the WeDoWind framework. A total of six new solutions were submitted to the challenge, and a comparison and evaluation of the results show that, in general, some of the approaches (Particle Swarm Optimisation algorithm for constructing health indicators, performance monitoring using Deep Neural Networks, Combined Ward Hierarchical Clustering and Novelty Detection with Local Outlier Factor and Time-to-failure prediction using Random Forest Regression) appear to exhibit high potential to reach the goals of the Challenge. However, there are a number of concrete things that would have to have been done by the Challenge providers and the Challenge moderators in order to ensure success. This includes enabling access to more details of the different failure types, access to multiple data sets from more wind turbines experiencing gearbox failure, provision of a model or rule relating fault detection times or a remaining useful lifetime to the estimated costs for repairs, replacements and inspections, provision of a clear strategy for training and test periods in advance, as well as provision of a pre-defined template or requirements for the results. These learning outcomes are used directly to define a set of best practice data sharing guidelines for wind turbine fault detection model evaluation. The guidelines can be used by researchers in the sector in order to improve model evaluation and data sharing in the future. [ABSTRACT FROM AUTHOR]

General circulation models (GCMs) run at regional resolution or at a continental scale. Therefore, these results cannot be used directly for local temperatures and precipitation prediction. Downscaling techniques are required to calibrate GCMs. Statistical downscaling models (SDSM) are the most widely used for bias correction of GCMs. However, few studies have compared SDSM with multi-layer perceptron artificial neural networks and in most of these studies, results indicate that SDSM outperform other approaches. This paper investigates an alternative architecture of neural networks, namely the long-short-term memory (LSTM), to forecast two critical climate variables, namely temperature and precipitation, with an application to five climate gauging stations in the Lake Chad Basin. Lake Chad is a data scarce area which has been impacted by severe drought, where water resources have been influenced by climate change and recent agricultural expansion. SDSM was used as the benchmark in this paper for temperature and precipitation downscaling for monthly time–scales weather prediction, using grid resolution GCM output at a 5 degrees latitude × 5 degrees longitude global grid. Three performance indicators were used in this study, namely the root mean square error (RMSE), to measure the sensitivity of the model to outliers, the mean absolute percentage error (MAPE), to estimate the overall performance of the predictions, as well as the Nash Sutcliffe Efficiency (NSE), which is a standard measure used in the field of climate forecasting. Results on the validation set for SDSM and test set for LSTM indicated that LSTM produced better accuracy on average compared to SDSM. For precipitation forecasting, the average RMSE and MAPE for LSTM were 33.21 mm and 24.82% respectively, while the average RMSE and MAPE for SDSM were 53.32 mm and 34.62% respectively. In terms of three year ahead minimum temperature forecasts, LSTM presents an average RMSE of 4.96 degree celsius and an average MAPE of 27.16%, while SDSM presents an average RMSE of 8.58 degree celsius and an average MAPE of 12.83%. For maximum temperatures forecast, LSTM presents an average RMSE of 4.27 degree celsius and an average MAPE of 11.09 percent, while SDSM presents an average RMSE of 9.93 degree celsius and an average RMSE of 12.07%. Given the results, LSTM may be a suitable alternative approach to downscale global climate simulation models' output, to improve water management and long-term temperature and precipitations forecasting at local level. [ABSTRACT FROM AUTHOR]

This paper presents a model for determining fuzzy evaluations of partial indicators of the availability of continuous systems at coal open pits using a neuro-fuzzy inference system. The system itself is a combination of fuzzy logic and artificial neural networks. The system availability is divided into partial indicators. By combining the fuzzy logic and artificial neural networks, a model is obtained that has the ability to learn and uses expert judgment for that learning. This paper deals with the ECC system (bucket wheel excavator-conveyor-crushing plant) of the open pit Drmno-Kostolac, which operates within the Electric Power Company of Serbia. The advantage of a model of this type is that it does not rely on the historical experiences of experts and usual predicted values for the fuzzy evaluation of partial indicators, which are based on the assumption that similar systems affect availability in a similar way. The fuzzy evaluation of partial indicators is based on historical data for the specific system for which the model was created. As such, it can more accurately predict continuous systems availability on the basis of expert evaluations in the appropriate time period. Another advantage of this model is that the availability is estimated on a quarterly basis, which gives a more accurate view because it uses a smaller time period with more similar characteristics and, thus, includes certain external influences which are related to the quarterly meteorological conditions. [ABSTRACT FROM AUTHOR]

Long series time forecasting has become a popular research direction in recent years, due to the ability to predict weather changes, traffic conditions and so on. This paper provides a comprehensive discussion of long series time forecasting techniques and their applications, using the Informer algorithm model as a framework. Specifically, we examine sequential time prediction models published in the last two years, including the tightly coupled convolutional transformer (TCCT) algorithm, Autoformer algorithm, FEDformer algorithm, Pyraformer algorithm, and Triformer algorithm. Researchers have made significant improvements to the attention mechanism and Informer algorithm model architecture in these different neural network models, resulting in recent approaches such as wavelet enhancement structure, auto-correlation mechanism, and depth decomposition architecture. In addition to the above, attention algorithms and many models show potential and possibility in mechanical vibration prediction. In recent state-of-the-art studies, researchers have used the Informer algorithm model as an experimental control, and it can be seen that the algorithm model itself has research value. The informer algorithm model performs relatively well on various data sets and has become a more typical algorithm model for time series forecasting, and its model value is worthy of in-depth exploration and research. This paper discusses the structures and innovations of five representative models, including Informer, and reviews the performance of different neural network structures. The advantages and disadvantages of each model are discussed and compared, and finally, the future research direction of long series time forecasting is discussed. [ABSTRACT FROM AUTHOR]

Simple Summary: This paper proposes a deep learning-based skin cancer classification network (DSCC_Net) that is based on a convolutional neural network (CNN) and implemented on three publicly available benchmark datasets (ISIC 2020, HAM10000, and DermIS). The proposed DSCC_Net obtained a 99.43% AUC, along with a 94.17% accuracy, a recall of 93.76%, a precision of 94.28%, and an F1-score of 93.93% in categorizing the four distinct types of skin cancer diseases. The accuracies of ResNet-152, Vgg-19, MobileNet, and Vgg-16, EfficientNet-B0, and Inception-V3 are 89.68%, 92.51%, 91.46%, 89.12%, 89.46%, and 91.82%, respectively. The results showed that the proposed DSCC_Net model performs better as compared to baseline models, thus offering significant support to dermatologists and health experts to diagnose skin cancer. Skin cancer is one of the most lethal kinds of human illness. In the present state of the health care system, skin cancer identification is a time-consuming procedure and if it is not diagnosed initially then it can be threatening to human life. To attain a high prospect of complete recovery, early detection of skin cancer is crucial. In the last several years, the application of deep learning (DL) algorithms for the detection of skin cancer has grown in popularity. Based on a DL model, this work intended to build a multi-classification technique for diagnosing skin cancers such as melanoma (MEL), basal cell carcinoma (BCC), squamous cell carcinoma (SCC), and melanocytic nevi (MN). In this paper, we have proposed a novel model, a deep learning-based skin cancer classification network (DSCC_Net) that is based on a convolutional neural network (CNN), and evaluated it on three publicly available benchmark datasets (i.e., ISIC 2020, HAM10000, and DermIS). For the skin cancer diagnosis, the classification performance of the proposed DSCC_Net model is compared with six baseline deep networks, including ResNet-152, Vgg-16, Vgg-19, Inception-V3, EfficientNet-B0, and MobileNet. In addition, we used SMOTE Tomek to handle the minority classes issue that exists in this dataset. The proposed DSCC_Net obtained a 99.43% AUC, along with a 94.17%, accuracy, a recall of 93.76%, a precision of 94.28%, and an F1-score of 93.93% in categorizing the four distinct types of skin cancer diseases. The rates of accuracy for ResNet-152, Vgg-19, MobileNet, Vgg-16, EfficientNet-B0, and Inception-V3 are 89.32%, 91.68%, 92.51%, 91.12%, 89.46% and 91.82%, respectively. The results showed that our proposed DSCC_Net model performs better as compared to baseline models, thus offering significant support to dermatologists and health experts to diagnose skin cancer. [ABSTRACT FROM AUTHOR]

The author comments on the studies on neural networks and sensors. The author wonders on the possibility of the study to be applied in addressing medical applications such as breast cancer. He knows several applications of neural networks in breast cancer in such an instance when a developed website was created that offer predictions of survival probability among cancer patients. The author hopes for an artificial neural network (ANN) expert to investigate on ductal carcinoma in-situ (DCIS).

This paper proposes fault diagnosis methods aimed at proactively preventing potential safety issues in robot systems, particularly human coexistence robots (HCRs) used in industrial environments. The data were collected from durability tests of the driving module for HCRs, gathering time-series vibration data until the module failed. In this study, to apply classification methods in the absence of post-failure data, the initial 50% of the collected data were designated as the normal section, and the data from the 10 h immediately preceding the failure were selected as the fault section. To generate additional data for the limited fault dataset, the Wasserstein generative adversarial networks with gradient penalty (WGAN-GP) model was utilized and residual connections were added to the generator to maintain the basic structure while preventing the loss of key features of the data. Considering that the performance of image encoding techniques varies depending on the dataset type, this study applied and compared five image encoding methods and four CNN models to facilitate the selection of the most suitable algorithm. The time-series data were converted into image data using image encoding techniques including recurrence plot, Gramian angular field, Markov transition field, spectrogram, and scalogram. These images were then applied to CNN models, including VGGNet, GoogleNet, ResNet, and DenseNet, to calculate the accuracy of fault diagnosis and compare the performance of each model. The experimental results demonstrated significant improvements in diagnostic accuracy when employing the WGAN-GP model to generate fault data, and among the image encoding techniques and convolutional neural network models, spectrogram and DenseNet exhibited superior performance, respectively. [ABSTRACT FROM AUTHOR]

The presented paper investigates the problem of endoscopic bleeding detection in endoscopic videos in the form of a binary image classification task. A set of definitions of high-level visual features of endoscopic bleeding is introduced, which incorporates domain knowledge from the field. The high-level features are coupled with respective feature descriptors, enabling automatic capture of the features using image processing methods. Each of the proposed feature descriptors outputs a feature activation map in the form of a grayscale image. Acquired feature maps can be appended in a straightforward way to the original color channels of the input image and passed to the input of a convolutional neural network during the training and inference steps. An experimental evaluation is conducted to compare the classification ROC AUC of feature-extended convolutional neural network models with baseline models using regular color image inputs. The advantage of feature-extended models is demonstrated for the Resnet and VGG convolutional neural network architectures. [ABSTRACT FROM AUTHOR]

This paper presents an approach to estimate demand in the Polish Power System (PPS) using the historical electricity usage of 27 thousand commercial customers, observed between 2016 and 2020. The customer data were clustered and samples as well as features were created to build neural network models. The goal of this research is to analyze if the clustering of customers can help to explain demand in the PPS. Additionally, considering that the datasets available for commercial customers are typically much smaller, it was analyzed what a minimal sample size drawn from the clusters would have to be in order to accurately estimate demand in the PPS. The evaluation and experiments were conducted for each year separately; the results proved that, considering adjusted R2 and mean absolute percentage error, our clustering-based method can deliver a high accuracy in the load estimation. [ABSTRACT FROM AUTHOR]

Continual learning has recently become a primary paradigm for deep neural network models in modern artificial intelligence services, where streaming data patterns frequently and irregularly change over time in dynamic environments. Unfortunately, there is still a lack of studies on computing cluster management for the processing of continual learning tasks, particularly in terms of the timeliness of model updates and associated energy consumption. In this paper, we propose a novel timeliness-aware continual learning management (TA-CLM) system aimed at ensuring timely deep neural network model updates for continual learning tasks while minimizing the energy consumption of computing worker nodes in clusters. We introduce novel penalty cost functions to penalize quantitatively deep neural network model update latency and present the associated optimization formulation to ensure the best task allocation. Additionally, we design a simulated annealing-based optimizer, which is a meta-heuristic technique and easy to implement, to solve the non-convex and non-linear optimization problem. We demonstrate that the proposed TA-CLM system improves both latency and energy performance over its competitors by an average of 51.3% and 51.6%, respectively, based on experimental results using raw data from well-known deep neural network models on an NVIDIA GPU-based testbed and a large-scale simulation environment. [ABSTRACT FROM AUTHOR]

With the widespread application and functional complexity of deep neural networks (DNNs), the demand for training samples is increasing. This elevated requirement also extends to DNN-based SAR object detection. Most public SAR object detection datasets are oriented to marine targets such as ships, while data sets oriented to land targets are relatively rare, though they are an effective way to improve the land object detection capability of deep models through SAR sample generation. In this paper, a synthesis generation collaborative SAR sample augmentation framework is proposed to achieve flexible and diverse high-quality sample augmentation. First, a semantic-layout-guided image synthesis strategy is proposed to generate diverse detection samples. The issues of object location rationality and object layout diversity are also addressed. Meanwhile, a pix2pixGAN network guided by layout maps is utilized to achieve diverse background augmentation. Second, a progressive training strategy of diffusion models is proposed to achieve semantically controllable SAR sample generation to further improve the diversity of scene clutter. Finally, a sample cleaning method considering distribution migration and network filtering is employed to further improve the quality of detection samples. The experimental results show that this semantic synthesis generation method can outperform existing sample augmentation methods, leading to a comprehensive improvement in the accuracy metrics of classical detection networks. [ABSTRACT FROM AUTHOR]

Foundation models trained on vast quantities of data have demonstrated impressive performance in capturing complex nonlinear relationships and accurately predicting neuronal responses. Due to the fact that deep learning neural networks depend on massive amounts of data samples and high energy consumption, foundation models based on spiking neural networks (SNNs) have the potential to significantly reduce calculation costs by training on neuromorphic hardware. In this paper, a visually inspired computational model composed of an SNN and echo state network (ESN) is proposed for the recognition of optic flow. The visually inspired SNN model serves as a foundation model that is trained using spike-timing-dependent plasticity (STDP) for extracting core features. The ESN model makes readout decisions for recognition tasks using the linear regression method. The results show that STDP can perform similar functions as non-negative matrix decomposition (NMF), i.e., generating sparse and linear superimposed readouts based on basis flow fields. Once the foundation model is fully trained from enough input samples, it can considerably reduce the training samples required for ESN readout learning. Our proposed SNN-based foundation model facilitates efficient and cost-effective task learning and could also be adapted to new stimuli that are not included in the training of the foundation model. Moreover, compared with the NMF algorithm, the foundation model trained using STDP does not need to be retrained during the testing procedure, contributing to a more efficient computational performance. [ABSTRACT FROM AUTHOR]

Data envelopment analysis (DEA) is a leading approach in performance analysis and discovering newer benchmarks, and the traditional DEA models cannot forecast the future efficiency of decision-making units (DMUs). Machine learning, such as the artificial neural networks (ANNs), support vector machine/regression (SVM/SVR), projection pursuit regression (PPR), etc., have been viewed as beneficial for managers in predicting system behaviors. PPR is especially suitable for small and non-normal distribution samples, the usual cases in DEA analysis. This paper integrates DEA and PPR to cover the shortcomings we faced while using DEA and DEA-BPNN, DEA-SVR, etc. This study explores the advantages of combining these complementary methods into an integrated performance measurement and prediction model. Firstly, the DEA approach is used to evaluate and rank the efficiency of DMUs. Secondly, we establish two DEA-PPR combined models to describe the DEA efficiency scores (also called the production function) and the DEA-efficient frontier function. The first combined model's input variables are input–output indicators in the DEA model, and the output variable is the DEA efficiency. In the second model, its input variables are input or output indicators in the DEA model, and the output variable is the optimal input indicator for input-oriented DEA or the output indicator for output-oriented DEA. We conducted positive research on two examples with actual data and virtual small, medium-sized, and large samples. Compared with the DEA-BPNN and DEA-SVR models, the results show that the DEA-PPR combined model has more vital global optimization ability, better convergence, higher accuracy, and a simple topology. The DEA-PPR model can obtain robust results for both small and large cases. The DEA-BPNN and DEA-SVR models cannot obtain robust results for small and medium-sized samples due to overfitting. For large samples, the DEA-PPR model outperforms DEA-BPNN, DEA-SVR, etc. The DEA-PPR combined model possesses better suitability, applicability, and reliability than the DEA-BPNN model, the DEA-SVR model, etc. [ABSTRACT FROM AUTHOR]

Interactions between clouds, aerosol, and precipitation are crucial aspects of weather and climate. The simple Koren–Feingold conceptual model is important for providing deeper insight into the complex aerosol–cloud–precipitation system. Recently, artificial neural networks (ANNs) and physics-informed neural networks (PINNs) have been used to study multiple dynamic systems. However, the Koren–Feingold model for aerosol–cloud–precipitation interactions has not yet been studied with either ANNs or PINNs. It is challenging for pure data-driven models, such as ANNs, to accurately predict and reconstruct time series in a small data regime. The pure data-driven approach results in the ANN becoming a "black box" that limits physical interpretability. We demonstrate how these challenges can be overcome by combining a simple ANN with physical laws into a PINN model (not purely data-driven, good for the small data regime, and interpretable). This paper is the first to use PINNs to learn about the original and modified Koren–Feingold models in a small data regime, including external forcings such as wildfire-induced aerosols or the diurnal cycle of clouds. By adding external forcing, we investigate the effects of environmental phenomena on the aerosol–cloud–precipitation system. In addition to predicting the system's future, we also use PINN to reconstruct the system's past: a nontrivial task because of time delay. So far, most research has focused on using PINNs to predict the future of dynamic systems. We demonstrate the PINN's ability to reconstruct the past with limited data for a dynamic system with nonlinear delayed differential equations, such as the Koren–Feingold model, which remains underexplored in the literature. The main reason that this is possible is that the model is non-diffusive. We also demonstrate for the first time that PINNs have significant advantages over traditional ANNs in predicting the future and reconstructing the past of the original and modified Koren–Feingold models containing external forcings in the small data regime. We also show that the accuracy of the PINN is not sensitive to the value of the regularization factor ( λ ), a key parameter for the PINN that controls the weight for the physics loss relative to the data loss, for a broad range (from λ = 1 × 10 3 to λ = 1 × 10 5 ). [ABSTRACT FROM AUTHOR]

The direct current (DC) microgrid is one of the key research areas for our advancement toward carbon-free energy production. In this paper, a two-step controller is designed for the DC microgrid using a combination of the deep neural network (DNN) and exponential reaching law-based global terminal sliding mode control (ERL-GTSMC). The DC microgrid under consideration involves multiple renewable sources (wind, PV) and an energy storage unit (ESU) connected to a 700 V DC bus and a 4–12 kW residential load. The proposed control method eliminates the chattering phenomenon and offers quick reaching time by utilizing the exponential reaching law (ERL). In the two-step control configuration, first, DNNs are used to find maximum power point tracking (MPPT) reference values, and then ERL-based GTSMC is utilized to track the reference values. The real dynamics of energy sources and the DC bus are mathematically modeled, which increases the system's complexity. Through the use of Lyapunov stability criteria, the stability of the control system is examined. The effectiveness of the suggested hybrid control algorithm has been examined using MATLAB simulations. The proposed framework has been compared to traditional sliding mode control and terminal sliding mode control to showcase its superiority and robustness. Experimental tests based on the hardware-in-the-loop (HIL) setup are then conducted using 32-bit TMS320F28379D microcontrollers. Both MATLAB and HIL results show strong performance under a range of environmental circumstances and system uncertainties. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) using wearable sensors enables continuous monitoring for healthcare applications. However, the conventional centralised training of deep learning models on sensor data poses challenges related to privacy, communication costs, and on-device efficiency. This paper proposes a federated learning framework integrating spiking neural networks (SNNs) with long short-term memory (LSTM) networks for energy-efficient and privacy-preserving HAR. The hybrid spiking-LSTM (S-LSTM) model synergistically combines the event-driven efficiency of SNNs and the sequence modelling capability of LSTMs. The model is trained using surrogate gradient learning and backpropagation through time, enabling fully supervised end-to-end learning. Extensive evaluations of two public datasets demonstrate that the proposed approach outperforms LSTM, CNN, and S-CNN models in accuracy and energy efficiency. For instance, the proposed S-LSTM achieved an accuracy of 97.36% and 89.69% for indoor and outdoor scenarios, respectively. Furthermore, the results also showed a significant improvement in energy efficiency of 32.30%, compared to simple LSTM. Additionally, we highlight the significance of personalisation in HAR, where fine-tuning with local data enhances model accuracy by up to 9% for individual users. [ABSTRACT FROM AUTHOR]

Wireless electroencephalography (EEG) has emerged as a critical interface between human cognitive processes and machine learning technologies in the burgeoning field of sensor communications. This paper presents a comprehensive review of advancements in wireless EEG communication and analysis, with an emphasis on their role in next-generation green wireless networks and industrial IoT. The review explores the efficacy of modulation techniques, such as amplitude-shift keying (ASK) and frequency-shift keying (FSK) in EEG data transmission, and emphasizes the transformative role of deep learning in the joint transmission and restoration of EEG signals. In addition, we propose a novel, energy-efficient approach to deep learning-based EEG analytics, designed to enhance wireless information transfer for industrial IoT applications. By applying an autoencoder to sample the EEG data and incorporating a hidden layer to simulate a noisy communication channel, we assessed the energy efficiency and reliability of the transmission. Our results demonstrate that the chosen network topology and parameters significantly affect not only data fidelity but also energy consumption, thus providing valuable insights for the development of sustainable and efficient wireless EEG systems in industrial IoT environments. A key aspect of our study is related to symmetry. Our results demonstrate that the chosen network topology and parameters significantly impact not only data fidelity but also energy fidelity and energy consumption, thus providing valuable insights for the development of sustainable and efficient wireless EEG systems in industrial IoT environments. Furthermore, we realized that the EEG data showed mildly marked symmetry. Neural networks must also exhibit asymmetric behavior for better performance. [ABSTRACT FROM AUTHOR]

In this paper, we propose using quiver representations as a tool for understanding artificial neural network algorithms. Specifically, we construct these algorithms by utilizing the group algebra of a finite cyclic group as vertices and convolution transformations as maps. We will demonstrate the neural network using convolution operation in the group algebra. The convolution operation in the group algebra that is formed by a finite cyclic group can be seen as a circulant matrix. We will represent a circulant matrix as a map from a cycle permutation matrix to a polynomial function. Using the permutation matrix, we will see some properties of the circulant matrix. Furthermore, we will examine some properties of circulant matrices using representations of finite symmetric groups as permutation matrices. Using the properties, we also examine the properties of moduli spaces formed by the actions of the change of basis group on the set of quiver representations. Through this analysis, we can compute the dimension of the moduli spaces. [ABSTRACT FROM AUTHOR]

This paper investigates the application of the YOLOv7 object detection model combined with knowledge distillation techniques in forest fire detection. As an advanced object detection model, YOLOv7 boasts efficient real-time detection capabilities. However, its performance may be constrained in resource-limited environments. To address this challenge, this research proposes a novel approach: considering that deep neural networks undergo multi-layer mapping from the input to the output space, we define the knowledge propagation between layers by evaluating the dot product of features extracted from two different layers. To this end, we utilize the Flow of Solution Procedure (FSP) matrix based on the Gram matrix and redesign the distillation loss using the Pearson correlation coefficient, presenting a new knowledge distillation method termed ILKDG (Intermediate Layer Knowledge Distillation with Gram Matrix-based Feature Flow). Compared with the classical knowledge distillation algorithm, KD, ILKDG achieved a significant performance improvement on a self-created forest fire detection dataset. Specifically, without altering the student network's parameters or network layers, mAP@0.5 improved by 2.9%, and mAP@0.5:0.95 increased by 2.7%. These results indicate that the proposed ILKDG method effectively enhances the accuracy and performance of forest fire detection without introducing additional parameters. The ILKDG method, based on the Gram matrix and Pearson correlation coefficient, presents a novel knowledge distillation approach, providing a fresh avenue for future research. Researchers can further optimize and refine this method to achieve superior results in fire detection. [ABSTRACT FROM AUTHOR]

This paper provides an in-depth overview of Deep Neural Networks and their application in the segmentation and analysis of lung Magnetic Resonance Imaging (MRI) scans, specifically focusing on hyperpolarized gas MRI and the quantification of lung ventilation defects. An in-depth understanding of Deep Neural Networks is presented, laying the groundwork for the exploration of their use in hyperpolarized gas MRI and the quantification of lung ventilation defects. Five distinct studies are examined, each leveraging unique deep learning architectures and data augmentation techniques to optimize model performance. These studies encompass a range of approaches, including the use of 3D Convolutional Neural Networks, cascaded U-Net models, Generative Adversarial Networks, and nnU-net for hyperpolarized gas MRI segmentation. The findings highlight the potential of deep learning methods in the segmentation and analysis of lung MRI scans, emphasizing the need for consensus on lung ventilation segmentation methods. [ABSTRACT FROM AUTHOR]