Search

The dual-channel assumption of the cognitive theory of multimedia learning suggests that importing a large amount of information through a single (visual or audio) channel overloads that channel, causing partial loss of information, while importing it simultaneously through multiple channels relieves the burden on them and leads to the registration of a larger amount of information. In light of such knowledge, this study investigates the possibility of reinforcing visual stimuli with audio for supporting e-learners in memorization tasks. Specifically, we consider three kinds of learning material and two kinds of audio stimuli and partially reinforce each kind of material with each kind of stimuli in an arbitrary way. In a series of experiments, we determine the particular type of audio, which offers the highest improvement for each kind of material. Our work stands out as being the first study investigating the differences in memory performance in relation to different combinations of learning content and stimulus. Our key findings from the experiments are: (i) E-learning is more effective in refreshing memory rather than studying from scratch, (ii) Non-informative audio is more suited to verbal content, whereas informative audio is better for numerical content, (iii) Constant audio triggering degrades learning performance and thus audio triggering should be handled with care. Based on these findings, we develop an ANN-based estimator to determine the proper moment for triggering audio (i.e. when memory performance is estimated to be declining) and carry out follow-up experiments for testing the integrated framework. Our contributions involve (i) determination of the most effective audio for each content type, (ii) estimation of memory deterioration based on learners' interaction logs, and (iii) the proposal of improvement of memory registration through auditory reinforcement. We believe that such findings constitute encouraging evidence the memory registration of e

A spaceborne synthetic aperture radar (SAR) can provide accurate images of the ocean surface roughness day-or-night in nearly all-weather conditions, being a unique asset for many geophysical applications. Considering the huge amount of data daily acquired by satellites, automated techniques for physical features extraction are needed. Even if supervised deep learning methods attain state-of-the-art results, they require a great amount of labeled data, which are difficult and excessively expensive to acquire for ocean SAR imagery. To this end, we use the subaperture decomposition (SD) algorithm to enhance the unsupervised learning retrieval on the ocean surface, empowering ocean researchers to search into large ocean databases. We empirically prove that SD improves the retrieval precision with over 20% for an unsupervised transformer autoencoder network. Moreover, we show that SD brings an important performance boost when Doppler centroid images are used as input data, leading the way to new unsupervised physics-guided retrieval algorithms.

Optimization fabrics are a geometric approach to real-time local motion generation, where motions are designed by the composition of several differential equations that exhibit a desired motion behavior. We generalize this framework to dynamic scenarios and nonholonomic robots and prove that fundamental properties can be conserved. We show that convergence to desired trajectories and avoidance of moving obstacles can be guaranteed using simple construction rules of the components. In addition, we present the first quantitative comparisons between optimization fabrics and model predictive control and show that optimization fabrics can generate similar trajectories with better scalability, and thus, much higher replanning frequency (up to 500 Hz with a 7 degrees of freedom robotic arm). Finally, we present empirical results on several robots, including a nonholonomic mobile manipulator with 10 degrees of freedom and avoidance of a moving human, supporting the theoretical findings., Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public., Learning & Autonomous Control, Robot Dynamics

Most of the existing path planning methods of automated guided vehicles (AGVs) are static. This paper proposes a new methodology for the path planning of a fleet of AGVs to improve the flexibility, robustness, and scalability of the AGV system. We mathematically describe the transport process as a dynamical system using an ad hoc mixed logical dynamical (MLD) model. Based on our MLD model, model predictive control is proposed to determine the collision paths dynamically, and the corresponding optimization problem is formulated as 0-1 integer linear programming. An alternating direction method of multipliers (ADMM)-based decomposition technique is then developed to coordinate the AGVs and reduce the computational burden, aiming for real-time decisions. The proposed methodology is tested on industrial scenarios, and results from numerical experiments show that the proposed method can obtain high transport productivity of the multi-AGV system at a low computational burden and deal with uncertainties resulting from the industrial environment., Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public., Transport Engineering and Logistics

Instead of predicting just one emotion for one activity (e.g., video watching), fine-grained emotion recognition enables more temporally precise recognition. Previous works on fine-grained emotion recognition require segment-by-segment, fine-grained emotion labels to train the recognition algorithm. However, experiments to collect these labels are costly and time-consuming compared with only collecting one emotion label after the user watched that stimulus (i.e., the post-stimuli emotion labels). To recognize emotions at a finer granularity level when trained with only post-stimuli labels, we propose an emotion recognition algorithm based on Deep Multiple Instance Learning (EDMIL) using physiological signals. EDMIL recognizes fine-grained valence and arousal (V-A) labels by identifying which instances represent the post-stimuli V-A annotated by users after watching the videos. Instead of fully-supervised training, the instances are weakly-supervised by the post-stimuli labels in the training stage. The V-A of instances are estimated by the instance gains, which indicate the probability of instances to predict the post-stimuli labels. We tested EDMIL on three different datasets, CASE, MERCA and CEAP-360VR, collected in three different environments: desktop, mobile and HMD-based Virtual Reality, respectively. Recognition results validated with the fine-grained V-A self-reports show that for subject-independent 3-class classification (high/neutral/low), EDMIL obtains promising recognition accuracies: 75.63% and 79.73% for V-A on CASE, 70.51% and 67.62% for V-A on MERCA and 65.04% and 67.05% for V-A on CEAP-360VR. Our ablation study shows that all components of EDMIL contribute to both the classification and regression tasks. Our experiments also show that (1) compared with fully-supervised learning, weakly-supervised learning can reduce the problem of overfitting caused by the temporal mismatch between fine-grained annotations and physiological signals, (2) instance, Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public., Multimedia Computing, Intelligent Systems

One weakness of machine-learning algorithms is the need to train the models for a new task. This presents a specific challenge for biometric recognition due to the dynamic nature of databases and, in some instances, the reliance on subject collaboration for data collection. In this paper, we investigate the behavior of deep representations in widely used CNN models under extreme data scarcity for One-Shot periocular recognition, a biometric recognition task. We analyze the outputs of CNN layers as identity-representing feature vectors. We examine the impact of Domain Adaptation on the network layers’ output for unseen data and evaluate the method’s robustness concerning data normalization and generalization of the best-performing layer. We improved state-of-the-art results that made use of networks trained with biometric datasets with millions of images and fine-tuned for the target periocular dataset by utilizing out-of-the-box CNNs trained for the ImageNet Recognition Challenge and standard computer vision algorithms. For example, for the Cross-Eyed dataset, we could reduce the EER by 67% and 79% (from 1.70%and 3.41% to 0.56% and 0.71%) in the Close-World and Open-World protocols, respectively, for the periocular case. We also demonstrate that traditional algorithms like SIFT can outperform CNNs in situations with limited data or scenarios where the network has not been trained with the test classes like the Open-World mode. SIFT alone was able to reduce the EER by 64% and 71.6% (from 1.7% and 3.41% to 0.6% and 0.97%) for Cross-Eyed in the Close-World and Open-World protocols, respectively, and a reduction of 4.6% (from 3.94% to 3.76%) in the PolyU database for the Open-World and single biometric case.

Continuously analyzing medical time series as new classes emerge is meaningful for health monitoring and medical decision-making. Few-shot class-incremental learning (FSCIL) explores the classification of few-shot new classes without forgetting old classes. However, little of the existing research on FSCIL focuses on medical time series classification, which is more challenging to learn due to its large intra-class variability. In this paper, we propose a framework, the Meta self-Attention Prototype Incrementer (MAPIC) to address these problems. MAPIC contains three main modules: an embedding encoder for feature extraction, a prototype enhancement module for increasing inter-class variation, and a distance-based classifier for reducing intra-class variation. To mitigate catastrophic forgetting, MAPIC adopts a parameter protection strategy in which the parameters of the embedding encoder module are frozen at incremental stages after being trained in the base stage. The prototype enhancement module is proposed to enhance the expressiveness of prototypes by perceiving inter-class relations using a self-attention mechanism. We design a composite loss function containing the sample classification loss, the prototype non-overlapping loss, and the knowledge distillation loss, which work together to reduce intra-class variations and resist catastrophic forgetting. Experimental results on three different time series datasets show that MAPIC significantly outperforms state-of-the-art approaches by 27.99%, 18.4%, and 3.95%, respectively. IEEE

Robots are more capable of achieving manipulation tasks for everyday activities than before. However, the safety of manipulation skills that robots employ is still an open problem. Considering all possible failures during skill learning increases the complexity of the process and restrains learning an optimal policy. Nonetheless, safety-focused modularity in the acquisition of skills has not been adequately addressed in previous works. For that purpose, we reformulate skills as base and failure prevention skills, where base skills aim at completing tasks and failure prevention skills aim at reducing the risk of failures to occur. Then, we propose a modular and hierarchical method for safe robot manipulation by augmenting base skills by learning failure prevention skills with reinforcement learning and forming a skill library to address different safety risks. Furthermore, a skill selection policy that considers estimated risks is used for the robot to select the best control policy for safe manipulation. Our experiments show that the proposed method achieves the given goal while ensuring safety by preventing failures. We also show that with the proposed method, skill learning is feasible and our safe manipulation tools can be transferred to the real environment © 2023 IEEE, This work was supported by the Scientific and Technological Research Council of Turkey (TUBITAK) under Grant 119E-436.

Intelligent traffic incident detection provides benefits such as minimizing traffic accidents and fuel consumption, reducing congestion, and enhancing transportation safety. Hence, traffic incident detection has been an active research area in customer-centric intelligent transportation systems (ITS). Given that a driver’s negative emotions (e.g. anger, nervousness) are often a main cause of traffic incidents, we argue there is a close relationship between traffic incident detection and driver emotion recognition. We propose a Dual channel Dual attention Graph Attention neTworks, termed DDGAT. Specifically, the traffic channel builds a sequential-based graph, where words are nodes and their co-occurrences are edges. In contrast, the emotion channel builds a syntactic-based graph with words as nodes and semantic dependencies as edges. The first attention mechanism automatically learns the importance of neighbors in different layers for different tasks. The second attention produces the attentive graph representation for both tasks. Experiments on two benchmarking datasets including GIIE and Twitter, show the effectiveness of the proposed model over state-of-the-art baselines in terms of micro F1 and H@1, with significant improvements of 3.5%, 3.2%, 2.0%, and 1.7%. © 2023 IEEE

One weakness of machine-learning algorithms is the need to train the models for a new task. This presents a specific challenge for biometric recognition due to the dynamic nature of databases and, in some instances, the reliance on subject collaboration for data collection. In this paper, we investigate the behavior of deep representations in widely used CNN models under extreme data scarcity for One-Shot periocular recognition, a biometric recognition task. We analyze the outputs of CNN layers as identity-representing feature vectors. We examine the impact of Domain Adaptation on the network layers’ output for unseen data and evaluate the method’s robustness concerning data normalization and generalization of the best-performing layer. We improved state-of-the-art results that made use of networks trained with biometric datasets with millions of images and fine-tuned for the target periocular dataset by utilizing out-of-the-box CNNs trained for the ImageNet Recognition Challenge and standard computer vision algorithms. For example, for the Cross-Eyed dataset, we could reduce the EER by 67% and 79% (from 1.70%and 3.41% to 0.56% and 0.71%) in the Close-World and Open-World protocols, respectively, for the periocular case. We also demonstrate that traditional algorithms like SIFT can outperform CNNs in situations with limited data or scenarios where the network has not been trained with the test classes like the Open-World mode. SIFT alone was able to reduce the EER by 64% and 71.6% (from 1.7% and 3.41% to 0.6% and 0.97%) for Cross-Eyed in the Close-World and Open-World protocols, respectively, and a reduction of 4.6% (from 3.94% to 3.76%) in the PolyU database for the Open-World and single biometric case.

Measuring interoception (‘perceiving internal bodily states’) has diagnostic and wellbeing implications. Since heartbeats are distinct and frequent, various methods aim at measuring cardiac interoceptive accuracy (CIAcc). However, the role of exteroceptive modalities for representing heart rate (HR) across screen-based and Virtual Reality (VR) environments remains unclear. Using a PolarH10 HR monitor, we develop a modality-dependent cardiac recognition task that modifies displayed HR. In a mixed-factorial design (N=50), we investigate how task environment (Screen, VR), modality (Audio, Visual, Audio-Visual), and real-time HR modifications ($\pm$15%, $\pm$30%, None) influence CIAcc, interoceptive awareness, mind-body measures, VR presence, and post-experience responses. Findings showed that participants confused their HR with underestimates up to 30%; environment did not affect CIAcc but influenced mind-related measures; modality did not influence CIAcc, however including audio increased interoceptive awareness; and VR presence inversely correlated with CIAcc. We contribute a lightweight and extensible cardiac interoception measurement method, and implications for biofeedback displays.

Recently, neural network compression schemes like channel pruning have been widely used to reduce the model size and computational complexity of deep neural networks (DNNs) for applications in power-constrained scenarios, such as embedded systems. Reinforcement learning (RL)-based auto-pruning has been further proposed to automate the DNN pruning process to avoid expensive hand-crafted work. However, the RL-based pruner involves a time-consuming training process, and pruning and evaluating each network comes at high computational expense. These problems have greatly restricted the real-world application of RL-based auto-pruning. Thus, we propose an efficient auto-pruning framework that solves this problem by taking advantage of the historical data from the previous auto-pruning process. In our framework, we first boost the convergence of the RL-pruner by transfer learning. Then, an augmented transfer learning scheme is proposed to further speed up the training process by improving the transferability. Finally, an assistant learning process is proposed to improve the sample efficiency of the RL agent. The experiments show that our framework can accelerate the auto-pruning process by 1.5x 2.5x for ResNet20, and 1.81x 2.375x for other neural networks, such as ResNet56, ResNet18, and MobileNet v1. IEEE

With the development of artificial intelligence (AI) techniques and the increasing popularity of camera-equipped devices, many edge video analytics applications are emerging, calling for the deployment of computation-intensive AI models at the network edge. Edge inference is a promising solution to move computation-intensive workloads from low-end devices to a powerful edge server for video analytics, but device-server communications will remain a bottleneck due to limited bandwidth. This paper proposes a task-oriented communication framework for edge video analytics, where multiple devices collect the visual sensory data and transmit the informative features to an edge server for processing. To enable low-latency inference, this framework removes video redundancy in spatial and temporal domains and transmits minimal information that is essential for the downstream task, rather than reconstructing the videos on the edge server. Specifically, it extracts compact task-relevant features based on the deterministic information bottleneck (IB) principle, which characterizes a tradeoff between the informativeness of the features and the communication cost. As the features of consecutive frames are temporally correlated, we propose a temporal entropy model (TEM) to reduce the bitrate by taking the previous features as side information in feature encoding. To further improve the inference performance, we build a spatial-temporal fusion module on the server to integrate features of the current and previous frames for joint inference. Extensive experiments on video analytics tasks evidence that the proposed framework effectively encodes task-relevant information of video data and achieves a better rate-performance tradeoff than existing methods.

Vehicles in platoons need to process many tasks to support various real-time vehicular applications. When a task arrives at a vehicle, the vehicle may not process the task due to its limited computation resource. In this case, it usually requests to offload the task to other vehicles in the platoon for processing. However, when the computation resources of all the vehicles in the platoon are insufficient, the task cannot be processed in time through offloading to the other vehicles in the platoon. Vehicular fog computing (VFC)-assisted platoon can solve this problem through offloading the task to the VFC which is formed by the vehicles driving near the platoon. Offloading delay is an important performance metric, which is impacted by both the offloading strategy for deciding where the task is offloaded and the number of the allocated vehicles in VFC to process the task. Thus, it is critical to propose an offloading strategy to minimize the offloading delay. In the VFC-assisted platoon system, vehicles usually adopt the IEEE 802.11p distributed coordination function (DCF) mechanism while having various computation resources. Moreover, when vehicles arrive and depart the VFC randomly, their tasks also arrive at and depart the system randomly. In this paper, we propose a semi-Markov decision process (SMDP) based offloading strategy while considering these factors to obtain the maximal long-term reward reflecting the offloading delay. Our research provides a robust strategy for task offloading in VFC systems, its effectiveness is demonstrated through simulation experiments and comparison with benchmark strategies. IEEE

The vision for 6G networks is to offer pervasive intelligence and internet of intelligence, in which the networks natively support artificial intelligence (AI), empower smart applications and scenarios in various fields, and create a "ubiquitous-intelligence" world. In this vision, the traditional session-oriented architecture cannot achieve flexible per-user customization, ultimate performance, security and reliability required by future AI services. In addition, users’ requirements for personalized AI services may become a key feature in the near future. However, the traditional AI deployment based on cloud/mobile edge computing (MEC) has limitations such as low throughput, long delay, poor privacy, and high carbon emissions, resulting in the inability to provide personalized quality of experience (QoE) assurance. By integrating AI in the network, the network AI has more advantages than cloud/MEC AI, such as better QoS assurance, lower latency, less transmission and computing overhead, and stronger security and privacy. Therefore, this article proposes the task-oriented native-AI network architecture (TONA), to natively support the network AI. By introducing task control and quality of AI services (QoAIS) assurance mechanisms at the control layer of 6G, the TONA can achieve the finest service granularity at the task level for guaranteeing every user’s personalized QoE.

Brain tumor segmentation is a fundamental task and existing approaches usually rely on multi-modality magnetic resonance imaging (MRI) images for accurate segmentation. However, the common problem of missing/incomplete modalities in clinical practice would severely degrade their segmentation performance, and existing fusion strategies for incomplete multi-modality brain tumor segmentation are far from ideal. In this work, we propose a novel framework named M2 FTrans to explore and fuse cross-modality features through modality-masked fusion transformers under various incomplete multi-modality settings. Considering vanilla self-attention is sensitive to missing tokens/inputs, both learnable fusion tokens and masked self-attention are introduced to stably build long-range dependency across modalities while being more flexible to learn from incomplete modalities. In addition, to avoid being biased toward certain dominant modalities, modality-specific features are further re-weighted through spatial weight attention and channel- wise fusion transformers for feature redundancy reduction and modality re-balancing. In this way, the fusion strategy in M2 FTrans is more robust to missing modalities. Experimental results on the widely-used BraTS2018, BraTS2020, and BraTS2021 datasets demonstrate the effectiveness of M2 FTrans, outperforming the state-of-the-art approaches with large margins under various incomplete modalities for brain tumor segmentation. Code is available at https://github.com/Jun-Jie-Shi/M2FTrans .

Image segmentation is essential to medical image analysis as it provides the labeled regions of interest for the subsequent diagnosis and treatment. However, fully-supervised segmentation methods require high-quality annotations produced by experts, which is laborious and expensive. In addition, when performing segmentation on another unlabeled image modality, the segmentation performance will be adversely affected due to the domain shift. Unsupervised domain adaptation (UDA) is an effective way to tackle these problems, but the performance of the existing methods is still desired to improve. Also, despite the effectiveness of recent Transformer-based methods in medical image segmentation, the adaptability of Transformers is rarely investigated. In this paper, we present a novel UDA framework using a Transformer for building a cross-modality segmentation method with the advantages of learning long-range dependencies and transferring attentive information. To fully utilize the attention learned by the Transformer in UDA, we propose Meta Attention (MA) and use it to perform a fully attention-based alignment scheme, which can learn the hierarchical consistencies of attention and transfer more discriminative information between two modalities. We have conducted extensive experiments on cross-modality segmentation using three datasets, including a whole heart segmentation dataset (MMWHS), an abdominal organ segmentation dataset, and a brain tumor segmentation dataset. The promising results show that our method can significantly improve performance compared with the state-of-the-art UDA methods. IEEE

Hybrid transformer-based segmentation approaches have shown great promise in medical image analysis. However, they typically require considerable computational power and resources during both training and inference stages, posing a challenge for resource-limited medical applications common in the field. To address this issue, we present an innovative framework called Slim UNETR, designed to achieve a balance between accuracy and efficiency by leveraging the advantages of both convolutional neural networks and transformers. Our method features the Slim UNETR Block as a core component, which effectively enables information exchange through self-attention mechanism decomposition and cost-effective representation aggregation. Additionally, we utilize the throughput metric as an efficiency indicator to provide feedback on model resource consumption. Our experiments demonstrate that Slim UNETR outperforms state-of-the-art models in terms of accuracy, model size, and efficiency when deployed on resource-constrained devices. Remarkably, Slim UNETR achieves 92.44% dice accuracy on BraTS2021 while being 34.4x smaller and 13.4x faster during inference compared to Swin UNETR. Code: https://github.com/aigzhusmart/Slim-UNETR. IEEE

Cross-domain recommendation, as a cutting-edge technology to settle data sparsity and cold start problems, is gaining increasingly popular. Existing research paradigms primarily focus on leveraging the representation of overlapping entities, such as representation aggregation or cross-domain consistency constraints, to facilitate knowledge transfer and enhance the performance of single-domain or dual-domain recommender systems. Even though these approaches bring significant promotions, they still suffer from optimization bottlenecks when faced with sparse overlapping users, which often occurs in reality. Unlocking the full potential of overlapping user information and exploring novel sources of cross-domain knowledge are pivotal in addressing this challenge effectively. On account of this, this paper proposes an innovative cross-domain recommendation framework, namely SEAGULL, to promote dual-target recommendation performance in line with these two perspectives. We bolster the utilization of overlapping user knowledge and extract non-overlapping user interests by refining the message passing mechanism in a unified heterogeneous cross-domain graph and facilitating the transfer of latent structural relationships among users. Specifically, we first construct the interaction of two domains as a unified cross-domain heterogeneous graph and design a novel attention mechanism to incorporate cross-domain collaboration signals between users and items. Second, we perform user structure alignment from global and local levels to extend semantic transfer and information augmentation. Finally, unlike previous work that directly incorporates mixed cross-domain knowledge, we employ a gentle and progressive cross-domain transfer strategy to reduce empirical risk loss. Extensive experiments on five tasks derived from three data sets fully demonstrate the effectiveness of SEAGULL. IEEE

Brain-Machine Interfaces (BMIs) assist paralyzed people to brain control (BC) the neuro-prosthesis continuously moving in space. During the BC process, the subject imagines the movement of the real limb and adapts the brain activity according to the sensory feedback. The neural adaptation in the closed-loop control results in complex and changing brain signals. Simultaneously, the decoder interprets the time-varying functional mapping between neural activity and continuous trajectory. It is crucial and challenging to accurately and adaptively track the mapping to help the subject accomplish the BC task with a stable performance. Existing Kalman Filter (KF) based decoders achieve continuous trajectory control by linearly interpreting neural firing observations into self-evolving prosthetic states. However, the linear neural-state mapping might not accurately reflect the movement intention of the subject. In this paper, we propose a novel method that allows subjects to achieve continuous brain control efficiently and stably. The proposed method incorporates a kernel reinforcement learning method into a state-observation model to decode the nonlinearly neural observation into a continuous trajectory state. The state transition function ensures the continuity of the prosthetic state. The kernel reinforcement learning allows the quick adaptation of the nonlinear neural-movement mapping during the BC process. The proposed method is tested in an online brain control reaching task for rats. Compared with KF, our method achieved more successful trials, faster response time, shorter inter-trial time, and remained stable over days. These results demonstrate that the proposed method is an efficient tool to assist subjects in brain control tasks. Author

Cross-modal Retrieval (CMR) is formulated for the scenarios where the queries and retrieval results are of different modalities. Existing CMR studies mainly focus on the common contextualized information between text transcripts and images, and the synchronized event information in audio-visual recordings. Different from all previous works, in this paper, we investigate the geometric correspondence between images and speech recordings captured in the same space and formulate a novel CMR task, called Spatial Image-Acoustic Retrieval (SIAR). To this end, we first design a novel speech encoder that consists of convolution neural networks and transformer layers, to learn space-aware speech representations. Then, to eliminate the crossmodal inherent discrepancy, we propose the Contrastive Speech Image Retrieval (CSIR) method which uses supervised contrastive learning to attract the same-space cross-modal features while repelling the ones from different spaces. Finally, image and speech features are directly compared and we predict the SIAR result with the maximum similarity. Extensive experiments demonstrate that our proposed speech encoder can recognize space from human speeches with superior performance over the other prevailing networks. It also sets our penultimate goal of speech-to-speech retrieval. Furthermore, our CSIR proposal can successfully perform bi-directional SIAR between spatial images and reverberant speeches with promising results. Code and data will be available. IEEE

Secure cloud-assisted data publish/subscribe (Pub/Sub) service provides an asynchronous method for publishers and subscribers to non-interactively exchange encrypted messages. Besides performing conjunctive subscription policy, numerous data Pub/Sub systems have recently been proposed to provide dynamic access control enforced from the publisher side to the subscriber side. However, these solutions fail to consider the following properties: (i) bilateral access control for both publishers and subscribers; (ii) the anonymity of the publisher; (iii) high matching time cost between publication and subscription. Therefore, we present P/S-BiAC, a secure and boolean cloud-assisted data Pub/Sub system with attribute-based bilateral access control that achieves authenticity and anonymity of publishers. In particular, P/S-BiAC enables cloud-based brokers to use the subscriber's trapdoor to match published data with sub-linear time complexity. Technically, we introduce a "BiAC-and-Hidden" technique to refine publication tuples and trapdoor in classic searchable symmetric encryption solutions. Moreover, we implement P/S-BiAC and evaluate its practical performance based on Enron dataset in real cloud environment. To deal with a conjunctive subscription policy, P/S-BiAC runs $27.8\textbackslash{}times $ faster for matching time cost (with $s$ -term=10) compared to state-of-the-art solutions, which demonstrates its feasibility in practical data Pub/Sub services with strong security properties.

The development of digitized humanity information provides a new perspective on data-oriented studies of history. Many previous studies have ignored uncertainty in the exploration of historical figures and events, which has limited the capability of researchers to capture complex processes associated with historical phenomena. We propose a visual reasoning system to support visual reasoning of uncertainty associated with spatio-temporal events of historical figures based on data from the China Biographical Database Project. We build a knowledge graph of entities extracted from a historical database to capture uncertainty generated by missing data and error. The proposed system uses an overview of chronology, a map view, and an interpersonal relation matrix to describe and analyse heterogeneous information of events. The system also includes uncertainty visualization to identify uncertain events with missing or imprecise spatio-temporal information. Results from case studies and expert evaluations suggest that the visual reasoning system is able to quantify and reduce uncertainty generated by the data.

The COVID-19 pandemic has forced many organizations to make an unprecedented shift to remote work, with virtual teams becoming an increasingly important tool for organizations to configure and manage team-based work. The way teams are configured has major implications for how individuals, teams, and organizations collect and contribute knowledge in order to innovate. By looking at team member diversity, this research paper examines the impact of different forms of diversity on the knowledge sharing processes and innovation output of virtual teams. Responses from 103 virtual team members based in the U.K. are used to assess the role of diversity inputs, on knowledge sharing processes and innovation output. Partial least squares structural equation modeling was employed to assess and validate the input-process-output model with statistically significant empirical results presented. Furthermore, the empirical results infer that knowledge sharing plays a critical role in mediating the role between diversity and innovation output in virtual team settings. Findings are discussed and implications are presented.

Head pose estimation plays a vital role in biometric systems related to facial and human behavior analysis. Typically, neural networks are trained on head pose datasets. Unfortunately, manual or sensor-based annotation of head pose is impractical. A solution is synthetic training data generated from 3D face models, which can provide an infinite number of perfect labels. However, computer generated images only provide an approximation of real-world images, leading to a performance gap between training and application domain. Therefore, there is a need for strategies that allow simultaneous learning on labeled synthetic data and unlabeled real-world data to overcome the domain gap. In this work we propose relative pose consistency, a semi-supervised learning strategy for head pose estimation based on consistency regularization. Consistency regularization enforces consistent network predictions under random image augmentations, including pose-preserving and pose-altering augmentations. We propose a strategy to exploit the relative pose introduced by pose-altering augmentations between augmented image pairs, to allow the network to benefit from relative pose labels during training on unlabeled data. We evaluate our approach in a domain-adaptation scenario and in a commonly used cross-dataset scenario. Furthermore, we reproduce related works to enforce consistent evaluation protocols and show that for both scenarios we outperform SOTA.

Infrastructure-enabled autonomous driving systems have been increasingly applied in confined environments. Automated valet parking (AVP) in smart parking garages is one of the notable applications that require high-precision indoor positioning services. However, the performances of the existing wireless indoor positioning techniques, including Wi-Fi, Bluetooth, and ultra-wideband (UWB), tend to decline substantially as the working time increases and the building environment varies. In this paper, we propose a lifelong framework using crowdsourced data of fully instrumented autonomous vehicles(e.g. vehicles equipping with LiDAR), to maintain the availability and precision of indoor positioning systems. We establish a map-aided deep learning positioning correction model based on continuous data sequences, which utilizes convolution and long short-term memory (LSTM) modules to extract the spatial and temporal features of positioning errors. A local grid map generator is designed and embedded into the correction model to learn the influencing factors of errors from the building environment and facility. A deep-learning-based anomaly detector is designed to keep the lifelong stability of our framework. Based on the proposed method, we develop a lifelong UWB positioning correction system and apply it for the path tracking of AVP in a real underground parking garage. The test results show that the system can maintain positioning correction precision in the environment of varying sensor errors and reduce the positioning error by 60% and the tracking error by 40%. The study showcases an innovative infrastructure-enabled application that can accelerate the widespread use of autonomous driving systems.

The image-based 3D object detection task expects that the predicted 3D bounding box has a “tightness” projection (also referred to as cuboid) to facilitate 2D-based training, which fits the object contour well on the image while remaining reasonable on the 3D space. These requirements bring significant challenges to the annotation. Projecting the Lidar-labeled 3D boxes to the image leads to non-trivial misalignment, while directly drawing a cuboid on the image cannot access the original 3D information. In this work, we propose a learning-based 3D box adaptation approach that automatically adjusts minimum parameters of the 360$^{\circ }$ Lidar 3D bounding box to fit the image appearance of panoramic cameras perfectly. With only a few 2D boxes annotation as guidance during the training phase, our network can produce accurate image-level cuboid annotations with 3D properties from Lidar boxes. We call our method “you only label once”, which means labeling on the point cloud once and automatically adapting to all surrounding cameras. Our refinement balances the accuracy and efficiency well and dramatically reduces the labeling effort for accurate cuboid annotation. Extensive experiments on the public Waymo and NuScenes datasets show that our method can produce human-level cuboid annotation on the image without manual adjustment and can accelerate monocular-3D training tasks. IEEE

Traffic safety, reliability, and resilience are significantly influenced by environmental factors such as visibility, road surface, and weather conditions. Yet, current monitoring methods, including weather stations and onboard environmental sensors, often fall short due to their high costs, significant latency, and limited dissemination. This paper presents the Edge-based Multi-task Safety-oriented Environmental (Edge-MuSE) sensing system, designed to address these traffic safety challenges associated with environmental factors. Edge-MuSE departs from traditional single-task sensing methods by performing multidimensional traffic environment perception tasks. It estimates key safety-related environmental factors exclusively through camera inputs and incorporates four innovative sensing tasks: visibility estimation, image dehazing, road segmentation, and road surface condition classification. The system is tailored to edge devices to transition computational loads from central servers to distributed nodes, thereby enhancing privacy and reducing latency. Additionally, Edge-MuSE integrates communication functions based on TCP/IP and Wi-Fi protocols, enabling rapid dissemination of sensing results and warning messages to local road users. System structures and data streaming have been optimized to accommodate the constraints of edge devices, ensuring high-efficiency edge computing. Field testing of Edge-MuSE in multiple testbeds in Bellevue (WA, US) and Oslo (Norway) has demonstrated its reliable and precise performance in perception tasks (92.15% accuracy in visibility estimation and 92.25% in road surface condition classification) as well as an impressive processing speed of 21.3 FPS. As such, Edge-MuSE presents a promising solution for enhancing roadway safety, efficiency, and resilience. IEEE

With the proliferation of mobile devices (e.g., vehicles and smartphones), rich media content services from massive users lead to high network resource consumption and energy usage. How to effectively allocate heterogeneous network resources and achieve green content delivery is a major challenge to be addressed in urgency, especially when vehicular users are involved and the spatiotemporal distribution of the content requests can change drastically. In this paper, we propose a new deep reinforcement learning (DRL)-aided task offloading and resource allocation scheme, named TORA-DRL, to minimize power consumption in cloud-edge cooperation environments, where in-network caching and request aggregation are incorporated to reduce replicated transmissions of network contents. Based on the history of content requests and available network resources in the system, TORA-DRL jointly optimizes the decisions about task offloading, as well as computing, caching and communication resource allocation, adapting to the changes in network states and user requirements. Simulation results demonstrate that the proposed TORA-DRL solution converges fast and has much higher energy efficiency than the existing popular cloud-edge cooperation schemes under different network environments. IEEE

Blockchain has been leveraged to secure transactions for the m-commerce. However, the intensive computation in the mining process restricts the participation of mobile devices. Currently, some studies have deployed edge computing services to support the mining process, where edge servers managed by one Service Provider (SP) are considered. This paper investigates a more practical scenario with multiple SPs, where servers managed by different SPs have distinct capacities and prices, making miners’ offloading decisions rather complicated. To tackle the above challenges, we consider task offloading, block propagation and miner mobility comprehensively to maximize utilities of miners. Specifically, we first formulate a Markov game, and then design a learning-based offloading algorithm for off-chain computation, where a novel learning model is constructed by integrating Deep Reinforcement Learning (DRL) and Mean Field Theory (MFT) to guarantee a Nash equilibrium. Different from existing studies, each miner merely needs to respond to the average effect from others in our system, insteading of knowing policies of others. Finally, both theoritical and performance results show that our designed algorithm has superiority on average miner utilities and algorithm convergence time compared with other representative algorithms. IEEE

The visual feature pyramid has shown its superiority in both effectiveness and efficiency in a variety of applications. However, current methods overly focus on inter-layer feature interactions while disregarding the importance of intra-layer feature regulation. Despite some attempts to learn a compact intra-layer feature representation with the use of attention mechanisms or vision transformers, they overlook the crucial corner regions that are essential for dense prediction tasks. To address this problem, we propose a Centralized Feature Pyramid (CFP) network for object detection, which is based on a globally explicit centralized feature regulation. Specifically, we first propose a spatial explicit visual center scheme, where a lightweight MLP is used to capture the globally long-range dependencies, and a parallel learnable visual center mechanism is used to capture the local corner regions of the input images. Based on this, we then propose a globally centralized regulation for the commonly-used feature pyramid in a top-down fashion, where the explicit visual center information obtained from the deepest intra-layer feature is used to regulate frontal shallow features. Compared to the existing feature pyramids, CFP not only has the ability to capture the global long-range dependencies but also efficiently obtain an all-round yet discriminative feature representation. Experimental results on the challenging MS-COCO validate that our proposed CFP can achieve consistent performance gains on the state-of-the-art YOLOv5 and YOLOX object detection baselines.

Location privacy protection (LPP) has become a key concern during mobile crowdsourcing (MCS) task allocation. Existing LPP mechanisms for MCS applications mainly focus on two-dimensional (2D) plane scenarios or directly apply 2D techniques into three-dimensional (3D) space scenarios, leaving the height dimension of 3D geolocation vulnerable to privacy breaches. To facilitate the LPP in 3D MCS, we propose a learning-based geo-perturbation mechanism using 3D geo-indistinguishability (3D-GI). In this mechanism, we first define an optimization objective to balance location privacy and MCS server profit, making it adaptable to different types of MCS applications. Then, we adopt the Asynchronous Advantage Actor-Critic (A3C) algorithm to design a reinforcement learning (RL)-based approach without knowing the accurate system and attack models. This approach enables us to derive the optimal perturbation policy in continuous policy space and accelerates the learning speed using asynchronous multi-thread training. Simulation results demonstrate that the proposed mechanism can better balance location privacy and server profit in 3D MCS applications compared to existing benchmarks.

Document-level Relation Extraction (RE) is a promising task aiming at identifying relations of multiple entity pairs in a document. Compared with the sentence-level counterpart, it has raised two significant challenges: a) In most cases, a relational fact can be adequately expressed via a small subset of sentences from the document, namely evidence. But the traditional method cannot model such strong semantic correlations between evidence sentences that collaborate to describe a specific relation; b) The data of this task is extremely long-tail in terms of too many NA instances and imbalanced relational types. Such data can mislead the tail prediction bias to the head categories in the RE model. In this paper, we present a novel E vidence reasoning and C urriculum learning method for D oc RE (DRE-EC) to address these challenges. Particularly, we first formulate evidence extraction as a sequential decision problem through a crafted reinforcement learning mechanism with an efficient path searching strategy to reduce the action space. Providing the evidence for each entity pair as a customized-filtered document in advance helps infer the relations better. To address the long-tail issue, we further develop a hybrid curriculum learning method at the NA-level (NC) and relation-level (RC) with our customized difficulty measure score. In NC, the NA samples are scheduled in an easy-to-hard scheme and gradually added, resulting in the data distribution from ideal and balanced to real and unbalanced. In RC, the scheme is switched into hard-to-easy to enhance the hard and tail samples. In addition, we propose a new Equalization adaptive Focal Loss(EFLoss) that can adjust to the changing data distribution and focus more on the tail categories. We conduct various experiments on two document-level RE benchmarks and achieve a remarkable improvement over previous competitive baselines. Furthermore, we provide detailed analyses of the advantages and effectiveness of our method.

One weakness of machine-learning algorithms is the need to train the models for a new task. This presents a specific challenge for biometric recognition due to the dynamic nature of databases and, in some instances, the reliance on subject collaboration for data collection. In this paper, we investigate the behavior of deep representations in widely used CNN models under extreme data scarcity for One-Shot periocular recognition, a biometric recognition task. We analyze the outputs of CNN layers as identity-representing feature vectors. We examine the impact of Domain Adaptation on the network layers’ output for unseen data and evaluate the method’s robustness concerning data normalization and generalization of the best-performing layer. We improved state-of-the-art results that made use of networks trained with biometric datasets with millions of images and fine-tuned for the target periocular dataset by utilizing out-of-the-box CNNs trained for the ImageNet Recognition Challenge and standard computer vision algorithms. For example, for the Cross-Eyed dataset, we could reduce the EER by 67% and 79% (from 1.70%and 3.41% to 0.56% and 0.71%) in the Close-World and Open-World protocols, respectively, for the periocular case. We also demonstrate that traditional algorithms like SIFT can outperform CNNs in situations with limited data or scenarios where the network has not been trained with the test classes like the Open-World mode. SIFT alone was able to reduce the EER by 64% and 71.6% (from 1.7% and 3.41% to 0.6% and 0.97%) for Cross-Eyed in the Close-World and Open-World protocols, respectively, and a reduction of 4.6% (from 3.94% to 3.76%) in the PolyU database for the Open-World and single biometric case.

Self-supervised learning enables networks to learn discriminative features from massive data itself. Most state-of-the-art methods maximize the similarity between two augmentations of one image based on contrastive learning. By utilizing the consistency of two augmentations, the burden of manual annotations can be freed. Contrastive learning exploits instance-level information to learn robust features. However, the learned information is probably confined to different views of the same instance. In this paper, we attempt to leverage the similarity between two distinct images to boost representation in self-supervised learning. In contrast to instance-level information, the similarity between two distinct images may provide more useful information. Besides, we analyze the relation between similarity loss and feature-level cross-entropy loss. These two losses are essential for most deep learning methods. However, the relation between these two losses is not clear. Similarity loss helps obtain instance-level representation, while feature-level cross-entropy loss helps mine the similarity between two distinct images. We provide theoretical analyses and experiments to show that a suitable combination of these two losses can get state-of-the-art results.

This paper studies a sequential task offloading problem for a multiuser mobile edge computing (MEC) system. While most of the existing works consider static one-shot offloading optimization with fixed wireless channel conditions and fixed computational tasks, we consider a dynamic optimization approach, which embraces wireless channel fluctuations and random deep neural network (DNN) task arrivals over an infinite horizon. Specifically, we introduce a local CPU workload queue (WD-QSI) and an MEC server workload queue (MEC-QSI) to model the dynamic workload of DNN tasks at each WD and the MEC server, respectively. The transmit power and the partitioning of the local DNN task at each WD are dynamically determined based on the instantaneous channel conditions (to capture the transmission opportunities) and the instantaneous WD-QSI and MEC-QSI (to capture the dynamic urgency of the tasks) to minimize the average latency of the DNN tasks. The joint optimization can be formulated as an ergodic Markov decision process (MDP), in which the optimality condition is characterized by a centralized Bellman equation. However, the brute force solution of the MDP is not viable due to the curse of dimensionality as well as the requirement for knowledge of the global state information. To overcome these issues, we first decompose the MDP into multiple lower dimensional sub-MDPs, each of which can be associated with a WD or the MEC server. Next, we further develop a parametric online Q-learning algorithm, so that each sub-MDP is solved locally at its associated WD or the MEC server. The proposed solution is completely decentralized in the sense that the transmit power for sequential offloading and the DNN task partitioning can be determined based on the local channel state information (CSI) and the local WD-QSI at the WD only. Additionally, no prior knowledge of the distribution of the DNN task arrivals or the channel statistics will be needed for the MEC server. The proposed solution

To alleviate the local computation demands from the ever-increasing computation-intensive mobile applications, Mobile Edge Computing (MEC) has proved promising. Especially, by opportunistically offloading these computation tasks to the MEC server, the delay of computing could be significantly improved through communication. In this paper, we develop an analytical framework for joint communication and computation resources allocation for multi-user MEC systems. Specifically, to retrieve the combined effect of communication and computation capabilities, we establish a dual queue system, including a data queue sub-system and a computation queue sub-system. To address the associated stochastic resource optimization problem, we propose a low-complexity resource allocation algorithm by Lyapunov optimization to stabilize all the sub-queue systems. As the practical buffers are finite, the conventional delay analysis of Lyapunov optimization becomes inaccurate. Alternatively, we model the stochastic queue lengthes as discrete time controlled random walk processes, which are transformed to continuous time Stochastic Differential Equations (SDEs) with reflections by strong approximation. According to the steady state analysis on the SDEs, we derive closed-form steady state distributions of the queue lengths, and then obtain the average delay performance with finite buffers. Finally, the accuracy of the proposed delay analysis is verified through simulation. IEEE

Traffic lights are important components of traffic systems, and perceptual tasks on traffic lights are crucial for intelligent agents on the road. Auxiliary countdown timers, providing the remaining time of the current traffic phase, improve the safety and smoothness of the entire traffic system. This work proposes a state estimation framework for countdown timer traffic lights. Time-domain information is adequately integrated into a variable transition Hidden Markov Model (VT-HMM), and our system provides optimal estimates of traffic light colors and countdown numbers based on noisy detection inputs. A dynamic state transition matrix is designed based on a 1-step transition logic and a probability of the number of transitions related to the current state sojourn duration. A recursive decoding method based on the Viterbi algorithm is proposed to update all the state candidates and select the optimal state chain. Extensive experiments evaluate the robustness and effectiveness of the proposed work. The performance boundaries of this system are also found under various input noise levels. The source code is available here: https://github.com/ShuyangUni/countdown-timer-traffic-light-estimation

In this paper, we present a novel end-to-end group collaborative learning network, termed GCoNet+, which can effectively and efficiently (250 fps) identify co-salient objects in natural scenes. The proposed GCoNet+ achieves the new state-of-the-art performance for co-salient object detection (CoSOD) through mining consensus representations based on the following two essential criteria: 1) intra-group compactness to better formulate the consistency among co-salient objects by capturing their inherent shared attributes using our novel group affinity module (GAM); 2) inter-group separability to effectively suppress the influence of noisy objects on the output by introducing our new group collaborating module (GCM) conditioning on the inconsistent consensus. To further improve the accuracy, we design a series of simple yet effective components as follows: i) a recurrent auxiliary classification module (RACM) promoting model learning at the semantic level; ii) a confidence enhancement module (CEM) assisting the model in improving the quality of the final predictions; and iii) a group-based symmetric triplet (GST) loss guiding the model to learn more discriminative features. Extensive experiments on three challenging benchmarks, i.e., CoCA, CoSOD3k, and CoSal2015, demonstrate that our GCoNet+ outperforms the existing 12 cutting-edge models. Code has been released at https://github.com/ZhengPeng7/GCoNet_plus. IEEE

The rapid proliferation of learning systems in an arbitrarily changing environment mandates the need to manage tensions between exploration and exploitation. This work proposes a quantum-inspired bandit learning approach for the learning-and-adapting-based offloading problem where a client observes and learns the costs of each task offloaded to the candidate resource providers, e.g., fog nodes. In this approach, a new action update strategy and novel probabilistic action selection are adopted, provoked by the amplitude amplification and collapse postulate in quantum computation theory. We devise a locally linear mapping between a quantum-mechanical phase in a quantum domain, e.g., Grover-type search algorithm, and a distilled probability-magnitude in a value-based decision-making domain, e.g., adversarial multi-armed bandit algorithm. The proposed algorithm is generalized, via the devised mapping, for better learning weight adjustments on favorable/unfavorable actions, and its effectiveness is verified via simulation.

The Scene Graph Generation (SGG) task aims to detect all the objects and their pairwise visual relationships in a given image. Although SGG has achieved remarkable progress over the last few years, almost all existing SGG models follow the same training paradigm: they treat both object and predicate classification in SGG as a single-label classification problem, and the ground-truths are one-hot target labels. However, this prevalent training paradigm has overlooked two characteristics of current SGG datasets: 1) For positive samples, some specific subject-object instances may have multiple reasonable predicates. 2) For negative samples, there are numerous missing annotations. Regardless of the two characteristics, SGG models are easy to be confused and make wrong predictions. To this end, we propose a novel model-agnostic Label Semantic Knowledge Distillation (LS-KD) for unbiased SGG. Specifically, LS-KD dynamically generates a "soft" label for each subject-object instance by fusing a predicted Label Semantic Distribution (LSD) with its original one-hot target label. LSD reflects the correlations between this instance and multiple predicate categories. Meanwhile, we propose two different strategies to predict LSD: iterative self-KD and synchronous self-KD. Extensive ablations and results on three SGG tasks have attested to the superiority and generality of our proposed LS-KD, which can consistently achieve decent trade-off performance between different predicate categories.

With the popularity of video/audio streaming applications in recent years, the wide spread of Autonomous Sensory Meridian Response (ASMR) erotica content is becoming a serious issue in social networks. Due to the subtle nature of ASMR erotica and its relative rareness in real scenario, detecting ASMR erotica contents is a challenging task. In this paper, we propose a novel neural framework for ASMR erotica content moderation. The proposed framework consists of a pipeline of novel strategies to tackle challenges unique in ASMR Erotica Contents such as data scarcity and imbalanced data. Based on large-scale industrial data, the proposed framework demonstrates high moderation accuracy in quantitative analysis and significantly outperforming the existing counterparts.

Object detection in aerial images is a critical and essential task in the fields of geoscience and remote sensing. Despite the popularity of computer vision methods in detecting objects, these methods have been faced with significant limitations of aerial images such as appearance occlusion and variable object sizes. In this letter, we explore the limitations of conventional neck networks in object detection by analyzing information bottlenecks. We propose an enhanced neck network to address the information deficiency issue in current neck networks. Our proposed neck network, which serves as a bridge between the backbone network and the head network, comprises a global semantic network (GSNet) and a feature fusion refinement module (FRM). The GSNet is designed to perceive contextual surroundings and propagate discriminative knowledge through a bidirectional global pattern. The FRM is developed to exploit different levels of features to capture comprehensive location information. We validate the efficacy and efficiency of our approach through experiments conducted on two challenging datasets, DOTA and HRSC2016. Our method outperforms existing approaches in terms of accuracy and complexity, demonstrating the superiority of our proposed method.

Motion planning is important to the automatic operation of the manipulator. It is difficult for traditional motion planning algorithms to achieve efficient online motion planning in a rapidly changing environment and high-dimensional planning space. The neural motion planning (NMP) algorithm based on reinforcement learning provides a new way to solve the above-mentioned task. Aiming to overcome the difficulty of training the neural network in high-accuracy planning tasks, this article proposes to combine the artificial potential field (APF) method and reinforcement learning. The neural motion planner can avoid obstacles in a wide range; meanwhile, the APF method is exploited to adjust the partial position. Considering that the action space of the manipulator is high-dimensional and continuous, the soft-actor-critic (SAC) algorithm is adopted to train the neural motion planner. By training and testing with different accuracy values in a simulation engine, it is verified that, in the high-accuracy planning tasks, the success rate of the proposed hybrid method is better than using the two algorithms alone. Finally, the feasibility of directly transferring the learned neural network to the real manipulator is verified by a dynamic obstacle-avoidance task.

Bluetooth Low Energy (BLE) beacon network is one of the essential infrastructures for many IoT and smart city applications that involve a plethora of sensing tasks. However, the BLE beacon network usually suffers from poor reliability and high maintenance costs due to the short-lived battery lifetime. Multiple works have attempted to extend the lifetime via energy harvesting hardware, adaptive advertising interval by user existence-aware operation, and energy-efficient routing schemes. However, few attempts were made to reduce the energy consumption related to sensing tasks. In light of this shortcoming, a sensing-aware framework is proposed to adjust the sensing task interval adaptively based on the predicted portion of changes of the sensor measurements. Furthermore, to estimate the impact of varying sensing task intervals on the amount of sensed information, a model that correlates energy and amount of information is proposed. The sensor portion of changes is predicted with a novel neural network, coined oracle-interpreter network, that significantly reduces the energy consumption while upkeeping a good prediction accuracy by leveraging two independent neural networks tailored for feature extraction and prediction tasks. The effectiveness of the proposed framework is verified by comprehensive simulations based on real-life data. The results demonstrate that the proposed framework can effectively reduce the energy consumption involved in sensing tasks up to 30%, machine learning tasks up to 60%, and finally, extend the lifetime up to 75%. IEEE

Weakly supervised temporal action localization (WTAL) aims to classify and localize temporal boundaries of actions for the video, given only video-level category labels in the training datasets. Due to the lack of boundary information during training, existing approaches formulate WTAL as a classification problem, i.e., generating the temporal class activation map (T-CAM) for localization. However, with only classification loss, the model would be suboptimized, i.e., the action-related scenes are enough to distinguish different class labels. Regarding other actions in the action-related scene (i.e., the scene same as positive actions) as co-scene actions, this suboptimized model would misclassify the co-scene actions as positive actions. To address this misclassification, we propose a simple yet efficient method, named bidirectional semantic consistency constraint (Bi-SCC), to discriminate the positive actions from co-scene actions. The proposed Bi-SCC first adopts a temporal context augmentation to generate an augmented video that breaks the correlation between positive actions and their co-scene actions in the inter-video. Then, a semantic consistency constraint (SCC) is used to enforce the predictions of the original video and augmented video to be consistent, hence suppressing the co-scene actions. However, we find that this augmented video would destroy the original temporal context. Simply applying the consistency constraint would affect the completeness of localized positive actions. Hence, we boost the SCC in a bidirectional way to suppress co-scene actions while ensuring the integrity of positive actions, by cross-supervising the original and augmented videos. Finally, our proposed Bi-SCC can be applied to current WTAL approaches and improve their performance. Experimental results show that our approach outperforms the state-of-the-art methods on THUMOS14 and ActivityNet. The code is available at https://github.com/lgzlIlIlI/BiSCC.

Outstanding users (OUs) denote the influential, 'core' or 'bridge' users in online social networks. How to accurately detect and rank them is an important problem for third-party online service providers and researchers. Conventional efforts, ranging from early graph-based algorithms to recent machine learning-based approaches, typically rely on an entire social network's information. However, for privacy-conscious users or newly-registered users, such information is not easily accessible. To address this issue, we present DeepPick, a novel framework that considers both the generalization and specialization in the detection task of OUs. For generalization, we introduce deep neural networks to capture dynamic features of the users. For specialization, we leverage the traditional descriptive features to make use of public information about users. Extensive experiments based on real-world datasets demonstrate that our approach achieves a high efficacy of detection performance against the state-of-the-art., Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public., Information and Communication Technology

Repetitive control (RC) can perfectly attenuate disturbances that are periodic in the time domain. The aim of this article is to develop an RC approach that compensates for disturbances that are time-domain nonperiodic but are repeating in the position domain. The developed position-domain buffer consists of a Gaussian process (GP), which is learned using appropriate dynamic filters and nonequidistant data. This approach estimates position-domain disturbances resulting in perfect compensation. The method is successfully applied to a substrate carrier system, demonstrating performance robustness against time-domain nonperiodic disturbances that are amplified by traditional RC.

This quasi-experimental study examined the dissociability of thought constraint from task relatedness (i.e., whether thoughts are related to the task at hand) and the rank-order stability of thought constraint. The study also tested predictions (Christoff et al., 2016) about the relationship between thought constraint and psychological symptoms and the influence of thought constraint on task performance. Like previous research, thought constraint was dissociable from task relatedness (Alperin et al., 2021; Kam et al., 2021; Mills et al., 2018; Mills et al., 2021; O’Neill et al., 2020; Smith et al., 2018). Thought constraint demonstrated rank-order stability across two measures of sustained attention. Higher reports of depressive symptoms, rumination, state anxiety, and trait anxiety were associated with less constrained thoughts, which is contrary to the predictions of Christoff et al. (2016) and Mills et al. (2021). Consistent with Christoff et al. (2016) and Alperin et al. (2021), higher reports of overall ADHD symptoms (inattention and hyperactivity combined) were associated with less constrained thought. As expected, symptoms of inattention and hyperactivity alone were positively associated with less constrained thought. Thought constraint also predicted unique variance in response accuracy via the Sustained Attention to Response Task (Robertson et al., 1997; as in Smith et al., 2022) and response time variability via the Metronome Response Task (Seli et al., 2013a). There was an interaction between thought constraint and task relatedness in the model predicting SART response time variability, indicating that response time was more variable when thoughts were off-task and less constrained than usual. The study suggests that thought constraint is a promising individual differences variable, but further research is needed to fully understand this phenomenon.

Inference carried out on pretrained deep neural networks (DNNs) is particularly effective as it does not require retraining and entails no loss in accuracy. Unfortunately, resource-constrained devices such as those in the Internet of Things may need to offload the related computation to more powerful servers, particularly, at the network edge. However, edge servers have limited resources compared to those in the cloud; therefore, inference offloading generally requires dividing the original DNN into different pieces that are then assigned to multiple edge servers. Related approaches in the state-of-the-art either make strong assumptions on the system model or fail to provide strict performance guarantees. This article specifically addresses these limitations by applying distributed assignment to DNN inference at the edge. In particular, it devises a detailed model of DNN-based inference, suitable for realistic scenarios involving edge computing. Optimal inference offloading with load balancing is also defined as a multiple assignment problem that maximizes proportional fairness. Moreover, a distributed algorithm for DNN inference offloading is introduced to solve such a problem in polynomial time with strong optimality guarantees. Finally, extensive simulations employing different data sets and DNN architectures establish that the proposed solution significantly improves upon the state-of-the-art in terms of inference time (1.14 to 2.62 times faster), load balance (with Jain's fairness index of 0.9), and convergence (one order of magnitude less iterations)., QC 20230524