Search

We present a solution to egocentric 3D body pose estimation from monocular images captured from downward looking fish-eye cameras installed on the rim of a head mounted VR device. This unusual viewpoint leads to images with unique visual appearance, with severe self-occlusions and perspective distortions that result in drastic differences in resolution between lower and upper body. We propose an encoder-decoder architecture with a novel multi-branch decoder designed to account for the varying uncertainty in 2D predictions. The quantitative evaluation, on synthetic and real-world datasets, shows that our strategy leads to substantial improvements in accuracy over state of the art egocentric approaches. To tackle the lack of labelled data we also introduced a large photo-realistic synthetic dataset. xR-EgoPose offers high quality renderings of people with diverse skintones, body shapes and clothing, performing a range of actions. Our experiments show that the high variability in our new synthetic training corpus leads to good generalization to real world footage and to state of theart results on real world datasets with ground truth. Moreover, an evaluation on the Human3.6M benchmark shows that the performance of our method is on par with top performing approaches on the more classic problem of 3D human pose from a third person viewpoint., 14 pages. arXiv admin note: substantial text overlap with arXiv:1907.10045

In this paper, we propose to tackle egocentric action recognition by suppressing background distractors and enhancing action-relevant interactions. The existing approaches usually utilize two independent branches to recognize egocentric actions, i.e., a verb branch and a noun branch. However, the mechanism to suppress distracting objects and exploit local human-object correlations is missing. To this end, we introduce two extra sources of information, i.e., the candidate objects' spatial location and their discriminative features, to enable concentration on the occurring interactions. We design a Symbiotic Attention withObject-centric featureAlignmentframework (SAOA) to provide meticulous reasoning between the actor and the environment. First, we introduce an object-centric feature alignment method to inject the local object features to the verb branch and noun branch. Second, we propose a symbiotic attention mechanism to encourage the mutual interaction between the two branches and select the most action-relevant candidates for classification. The framework benefits from the communication among the verb branch, the noun branch, and the local object information. Experiments based on different backbones and modalities demonstrate the effectiveness of our method. Notably, our framework achieves the state-of-the-art on the largest egocentric video dataset.

We present JRDB, a novel egocentric dataset collected from our social mobile manipulator JackRabbot. The dataset includes 64 minutes of annotated multimodal sensor data including stereo cylindrical 360$^\circ$ RGB video at 15 fps, 3D point clouds from two Velodyne 16 Lidars, line 3D point clouds from two Sick Lidars, audio signal, RGB-D video at 30 fps, 360$^\circ$ spherical image from a fisheye camera and encoder values from the robot's wheels. Our dataset incorporates data from traditionally underrepresented scenes such as indoor environments and pedestrian areas, all from the ego-perspective of the robot, both stationary and navigating. The dataset has been annotated with over 2.3 million bounding boxes spread over 5 individual cameras and 1.8 million associated 3D cuboids around all people in the scenes totaling over 3500 time consistent trajectories. Together with our dataset and the annotations, we launch a benchmark and metrics for 2D and 3D person detection and tracking. With this dataset, which we plan on extending with further types of annotation in the future, we hope to provide a new source of data and a test-bench for research in the areas of egocentric robot vision, autonomous navigation, and all perceptual tasks around social robotics in human environments.

In this work, we consider transferring the structure information from large networks to compact ones for dense prediction tasks in computer vision. Previous knowledge distillation strategies used for dense prediction tasks often directly borrow the distillation scheme for image classification and perform knowledge distillation for each pixel separately, leading to sub-optimal performance. Here we propose to distill structured knowledge from large networks to compact networks, taking into account the fact that dense prediction is a structured prediction problem. Specifically, we study two structured distillation schemes: i) pair-wise distillation that distills the pair-wise similarities by building a static graph; and ii) holistic distillation that uses adversarial training to distill holistic knowledge. The effectiveness of our knowledge distillation approaches is demonstrated by experiments on three dense prediction tasks: semantic segmentation, depth estimation and object detection. Code is available at: https://git.io/StructKD, Comment: v1:10 pages cvpr2019 accepted; v2:15 pages for a journal version; Code is available at: https://github.com/irfanICMLL/structure_knowledge_distillation; fix typos

We propose a novel logic, called Frame Logic (FL), that extends first-order logic (with recursive definitions) using a construct Sp(.) that captures the implicit supports of formulas -- the precise subset of the universe upon which their meaning depends. Using such supports, we formulate proof rules that facilitate frame reasoning elegantly when the underlying model undergoes change. We show that the logic is expressive by capturing several data-structures and also exhibit a translation from a precise fragment of separation logic to frame logic. Finally, we design a program logic based on frame logic for reasoning with programs that dynamically update heaps that facilitates local specifications and frame reasoning. This program logic consists of both localized proof rules as well as rules that derive the weakest tightest preconditions in FL., This manuscript is an extended and revised version of the publication with the same title that appeared at ESOP 2022 (https://doi.org/10.1007/978-3-030-44914-8_19). It is currently under review

Social touch is essential to human development and communication. Mediated social touch is suggested as a solution for circumstances where distance prevents skin-to-skin contact. However, past research aimed at demonstrating efficacy of mediated touch in reducing stress and promoting helping have produced mixed findings. These inconsistent findings could possibly be due to insufficient control of contextual factors combined with unnatural interaction scenarios. For example, touch occurs less frequently among strangers and is often accompanied with nonverbal visual cues. We investigated how visual presentation of touch, and interpersonal relationship to the sender influence perception, affective experiences, and autonomic responses the touch evoke. Fifty couples of mixed gender were recruited. A mediated touch was repeatedly applied by either the male partner or male confederate to female participants. The latter witnessed through a webcam as the sender caressed a rubber hand or touchpad to send the touch. Following our hypotheses, touch sent by one's partner was perceived softer and more comforting than stranger touch. The partner's touch also resulted in weaker skin conductance responses, particularly when sent by touching a touchpad. In sum, how a mediated touch is experienced depends both on who is touching, and on how the touch is visually represented.

In this article, we investigate the problem of social link inference in a target location-aware social network (LSN), which aims at predicting the unobserved links between users within the network. This problem is critical for downstream applications, including network completion and friend recommendation. In addition to the network structures commonly used in general link prediction, the studies tailored for social link inference in an LSN leverage user trajectories from the spatial aspect. However, the temporal factor lying in user trajectories is largely overlooked by most of the prior studies, limiting the capabilities of capturing the temporal relevance between users. Moreover, effective user matching by fusing different views, i.e., social, spatial, and temporal factors, remains unresolved, which hinders the potential improvement of link inference. To this end, this article devises a novel multiview matching network (MVMN) by regarding each of the three factors as one view of any target user pair. MVMN enjoys the flexibility and completeness of modeling each factor by developing its suitable matching module: 1) location matching module; 2) time-series matching module; and 3) relation matching module. Each module learns a view-specific representation for matching, and MVMN fuses them for final link inference. Extensive experiments on two real-world data sets demonstrate the superiority of our approach against several competitive baselines for link prediction and sequence matching, validating the contribution of its key components.

Temporal point process is widely used for sequential data modeling. In this paper, we focus on the problem of modeling sequential event propagation in graph, such as retweeting by social network users, news transmitting between websites, etc. Given a collection of event propagation sequences, conventional point process model consider only the event history, i.e. embed event history into a vector, not the latent graph structure. We propose a Graph Biased Temporal Point Process (GBTPP) leveraging the structural information from graph representation learning, where the direct influence between nodes and indirect influence from event history is modeled respectively. Moreover, the learned node embedding vector is also integrated into the embedded event history as side information. Experiments on a synthetic dataset and two real-world datasets show the efficacy of our model compared to conventional methods and state-of-the-art., Comment: 9 pages, 6 figures, 2 tables

The adaptive neurofuzzy inference system (ANFIS) is a structured multioutput learning machine that has been successfully adopted in learning problems without noise or outliers. However, it does not work well for learning problems with noise or outliers. High-accuracy real-time forecasting of traffic flow is extremely difficult due to the effect of noise or outliers from complex traffic conditions. In this study, a novel probabilistic learning system, probabilistic regularized extreme learning machine combined with ANFIS (probabilistic R-ELANFIS), is proposed to capture the correlations among traffic flow data and, thereby, improve the accuracy of traffic flow forecasting. The new learning system adopts a fantastic objective function that minimizes both the mean and the variance of the model bias. The results from an experiment based on real-world traffic flow data showed that, compared with some kernel-based approaches, neural network approaches, and conventional ANFIS learning systems, the proposed probabilistic R-ELANFIS achieves competitive performance in terms of forecasting ability and generalizability.

Unit square visibility graphs (USV) are described by axis-parallel visibility between unit squares placed in the plane. If the squares are required to be placed on integer grid coordinates, then USV become unit square grid visibility graphs (USGV), an alternative characterisation of the well-known rectilinear graphs. We extend known combinatorial results for USGV and we show that, in the weak case (i.e., visibilities do not necessarily translate into edges of the represented combinatorial graph), the area minimisation variant of their recognition problem is $${{\,\mathrm{{\textsf{N}}{\textsf{P}}}\,}}$$ N P -hard. We also provide combinatorial insights with respect to USV, and as our main result, we prove their recognition problem to be $${{\,\mathrm{{\textsf{N}}{\textsf{P}}}\,}}$$ N P -hard, which settles an open question.

For cloud network performance profiling, network tomography based on end-to-end measurement is often used in deducing the network performance for its efficiency. However, most tomography problems are under-constrained, which require additional assumptions or probing monitors planted among network switches, which are often unavailable in software-defined networking (SDN) environment. On the other hand, SDN based flow mirroring could provide accurate flow information, but the cost of both gathering and analysing the packet traces is tremendous that it is impossible to cover the whole network. We propose ScoutFlow, a method combining SDN flow measurement and end-to-end performance tomography, to achieve accurate performance profiling for cloud network while keeping low monitoring overhead. We evaluate ScoutFlow in our campus data center cloud, the experiment shows good scalability and accuracy.

TruthSift is a cloud-based platform that logically combines members' contributions into a collective intelligence. Members add statements and directed connectors to diagrams. TruthSift monitors which statements have been logically established by demonstrations for which every challenge has beenrefuted by an established refutation, and the complement: which statements have been refuted by established refutations. When members run out of rational objections the result is a converged diagramsuccinctly representing the state of knowledge about a topic, including plausible challenges and how they were refuted. Previous computer systems for collaborative intelligence did not have a qualitatively better solution for combining contributions than voting, and are subject to group think, interest group capture, and inability to follow a multi-step logical argument. They did not settle issues automatically point by point and propagate the consequences up. I review indications that many practically important statements most people believe to be firmly established will be revealed to be firmly refuted upon computer assisted scrutiny. TruthSift also supports construction of powerful probabilistic models over networks of causes, implications, tests, and necessary factors.

Multi-view classification with limited sample size and data augmentation is a very common machine learning (ML) problem in medicine. With limited data, a triplet network approach for two-stage representation learning has been proposed. However, effective training and verifying the features from the representation network for their suitability in subsequent classifiers are still unsolved problems. Although typical distance-based metrics for the training capture the overall class separability of the features, the performance according to these metrics does not always lead to an optimal classification. Consequently, an exhaustive tuning with all feature-classifier combinations is required to search for the best end result. To overcome this challenge, we developed a novel nearest-neighbor (NN) validation strategy based on the triplet metric. This strategy is supported by a theoretical foundation to provide the best selection of the features with a lower bound of the highest end performance. The proposed strategy is a transparent approach to identify whether to improve the features or the classifier. This avoids the need for repeated tuning. Our evaluations on real-world medical imaging tasks (i.e., radiation therapy delivery error prediction and sarcoma survival prediction) show that our strategy is superior to other common deep representation learning baselines [i.e., autoencoder (AE) and softmax]. The strategy addresses the issue of feature's interpretability which enables more holistic feature creation such that the medical experts can focus on specifying relevant data as opposed to tedious feature engineering.

With the rapid progress of wireless communication technologies along with their digital revolutions, the quantity of Internet of Things (IoT) has been increased by manifolds resulting in a huge increase in data volume and network traffic. It became easier for an intruder to pretend as a valid service provider, and generate different types of network attacks. This becomes even more severe when the service involves digital financial transactions. This paper proposes an intrusion detection system (IDS) based on a stacked autoencoder (AE) and a deep neural network (DNN). The stacked AE learns the features of the input network record in an unsupervised manner to decrease the feature width. Then, the DNN is trained in a supervised manner to extract deep-learned features for the classifier. In the proposed system, the stacked AE has two latent layers and the DNN has two or three layers, where each layer has a fully-connected layer, a batch normalization, and a dropout. The system was evaluated on three publicly available datasets: KDDCup99, NSL-KDD, and AWID. Experimental results exhibited that the proposed IDS achieved 94.2%, 99.7%, 99.9% accuracy, respectively, for multiclass classification.

This paper aims to predict a set of apps a user will open on her mobile device in the next time slot. Such an information is essential for many smartphone operations, e.g., app pre-loading and content pre-caching, to improve user experience. However, it is hard to build an explicit model that accurately captures the complex environment context and predicts a set of apps at one time. This paper presents a deep reinforcement learning framework, named as DeepAPP, which learns a model-free predictive neural network from historical app usage data. Meanwhile, an online updating strategy is designed to adapt the predictive network to the time-varying app usage behavior. To transform DeepAPP into a practical deep reinforcement learning system, several challenges are addressed by developing a context representation method for complex contextual environment, a general agent for overcoming data sparsity and a lightweight personalized agent for minimizing the prediction time. Extensive experiments on a large-scale anonymized app usage dataset reveal that DeepAPP provides high accuracy (precision 70.6% and recall of 62.4%) and reduces the prediction time of the state-of-the-art by 6.58 times. A field experiment of 29 participants demonstrates DeepAPP can effectively reduce launch time of apps.

The assets in Industry 4.0 are categorised into physical, virtual and human. The innovation and popularisation of ubiquitous computing enhance the usage of smart devices: RFID tags, QR codes, LoRa tags, etc. for assets identification and tracking. The generated data from Industrial Internet of Things (IIoT) eases information visibility and process automation in Industry 4.0. Virtual assets include the data produced from IIoT. One of the applications of the industrial big data is to predict the failure of manufacturing equipment. Predictive maintenance enables the business owner to decide such as repairing or replacing the component before an actual failure which affects the whole production line. Therefore, Industry 4.0 requires an effective asset management to optimise the tasks distributions and predictive maintenance model. This paper presents the Genetic Algorithm (GA) based resource management integrating with machine learning for predictive maintenance in fog computing. The time, cost and energy performance of GA along with MinMin, MaxMin, FCFS, RoundRobin are simulated in the FogWorkflowsim. The predictive maintenance model is built in two-class logistic regression using real-time datasets. The results demonstrate that the proposed technique outperforms MinMin, MaxMin, FCFS, RoundRobin in execution time, cost and energy usage. The execution time is 0.48% faster, 5.43% lower cost and energy usage is 28.10% lower in comparison with second-best results. The training and testing accuracy of the prediction model is 95.1% and 94.5%, respectively.

Issuing a disaster certificate, which is used to decide the contents of a victim’s support, requires accuracy and rapidity. However, in Japan at large, issuing of damage certificates has taken a long time in past earthquake disasters. Hence, the government needs a more efficient mechanism for issuing damage certificates. This study developed an estimation system of roof-damaged buildings to obtain an overview of earthquake damage based on aero-photo images using deep learning. To provide speedy estimation, this system utilized the trimming algorithm, which automatically generates roof image data using the location information of building polygons on GIS (Geographic Information System). Consequently, the proposed system can estimate, if a house is covered with a blue sheet with 97.57 % accuracy and also detect whether a house is damaged, with 93.51 % accuracy. It would therefore be worth considering the development of an image recognition model and a method of collecting aero-photo data to operate this system during a real earthquake.

The future communication technologies like 6G are capable to provide higher mobility and better quality of service requirements to Internet of Things (IoT). To ensure mobility, the 6G technologies need more reliable and scalable solutions, which are capable to integrate large scale heterogeneous IoT networks. In a heterogeneous environment, seamless mobility along with the demands of IP addresses requires a proxy mobile IPv6 (PMIPv6) protocol that provides cost-effective solutions in next generation IoT networks. The PMIPv6 has been exploited for resource efficiency in IoT-enabled next generation networks. In this paper, we have proposed a demand-based resource-efficient location-aware PMIPv6 extension for seamless mobility in the next generation IoT networks. The proposed approach efficiently utilizes the network resources using location information and received signal strength (RSS). This solution enhances the performance of PMIPv6 protocol in terms of signaling cost, and load on network entities. Furthermore, mathematical models are derived in terms of signaling cost, load distribution. The proposed solution is compared with the existing RSS-based PMIPv6 extension protocols. The results show that the proposed scheme enhances the performance and is a resource-friendly for the next generation large scale IoT networks.

One of the combinatorial optimization problems is Knapsack problem, which aims to maximize the benefit of objects whose weight not exceeding the capacity of knapsack. This paper introduces artificial bee colony algorithm to select a subset of project and represented by knapsack problem to put the best investment plan which achieve the highest profits within a determined costs, this plan is one of the applications of the financial field. The result from the proposed algorithm implemented by matlab (8.3) show the ability to find best solution with precisely and rapidity compared to genetic algorithm http://dx.doi.org/10.25130/tjps.23.2018.039

The rapid growth of GPS technology and mobile devices has led to a massive accumulation of location data, bringing considerable benefits to individuals and society. One of the major usages of such data is travel time prediction, a typical service provided by GPS navigation devices and apps. Meanwhile, the constant collection and analysis of the individual location data also pose unprecedented privacy threats. We leverage the notion of geo-indistinguishability, an extension of differential privacy to the location privacy setting, and propose a procedure for privacy-preserving travel time prediction without collecting actual individual GPS trace data. We propose new concepts to examine the impact of geo-indistinguishability-based sanitization on the usefulness of GPS traces and provide analytical and experimental utility analysis for privacy-preserving travel time prediction. We also propose new metrics to measure the adversary error in learning individual GPS traces from the collected sanitized data. Our experiment results suggest that the proposed procedure provides travel time prediction with satisfactory accuracy at reasonably small privacy costs.

Unconstrained handwritten text recognition remains challenging for computer vision systems. Paragraph text recognition is traditionally achieved by two models: the first one for line segmentation and the second one for text line recognition. We propose a unified end-to-end model using hybrid attention to tackle this task. This model is designed to iteratively process a paragraph image line by line. It can be split into three modules. An encoder generates feature maps from the whole paragraph image. Then, an attention module recurrently generates a vertical weighted mask enabling to focus on the current text line features. This way, it performs a kind of implicit line segmentation. For each text line features, a decoder module recognizes the character sequence associated, leading to the recognition of a whole paragraph. We achieve state-of-the-art character error rate at paragraph level on three popular datasets: 1.91% for RIMES, 4.45% for IAM and 3.59% for READ 2016. Our code and trained model weights are available at https://github.com/FactoDeepLearning/VerticalAttentionOCR.

Given the prevalence of depression worldwide and its major impact on society, several studies employed artificial intelligence modelling to automatically detect and assess depression. However, interpretation of these models and cues are rarely discussed in detail in the AI community, but have received increased attention lately. In this study, we aim to analyse the commonly selected features using a proposed framework of several feature selection methods and their effect on the classification results, which will provide an interpretation of the depression detection model. The developed framework aggregates and selects the most promising features for modelling depression detection from 38 feature selection algorithms of different categories. Using three real-world depression datasets, 902 behavioural cues were extracted from speech behaviour, speech prosody, eye movement and head pose. To verify the generalisability of the proposed framework, we applied the entire process to depression datasets individually and when combined. The results from the proposed framework showed that speech behaviour features (e.g. pauses) are the most distinctive features of the depression detection model. From the speech prosody modality, the strongest feature groups were F0, HNR, formants, and MFCC, while for the eye activity modality they were left-right eye movement and gaze direction, and for the head modality it was yaw head movement. Modelling depression detection using the selected features (even though there are only 9 features) outperformed using all features in all the individual and combined datasets. Our feature selection framework did not only provide an interpretation of the model, but was also able to produce a higher accuracy of depression detection with a small number of features in varied datasets. This could help to reduce the processing time needed to extract features and creating the model.

The suffix array, perhaps the most important data structure in modern string processing, is often augmented with the longest common prefix (LCP) array which stores the lengths of the LCPs for lexicographically adjacent suffixes of a string. Together the two arrays are roughly equivalent to the suffix tree with the LCP array representing the tree shape. In order to better understand the combinatorics of LCP arrays, we consider the problem of inferring a string from an LCP array, i.e., determining whether a given array of integers is a valid LCP array, and if it is, reconstructing some string or all strings with that LCP array. There are recent studies of inferring a string from a suffix tree shape but using significantly more information (in the form of suffix links) than is available in the LCP array. We provide two main results. (1) We describe two algorithms for inferring strings from an LCP array when we allow a generalized form of LCP array defined for a multiset of cyclic strings: a linear time algorithm for binary alphabet and a general algorithm with polynomial time complexity for a constant alphabet size. (2) We prove that determining whether a given integer array is a valid LCP array is NP-complete when we require more restricted forms of LCP array defined for a single cyclic or non-cyclic string or a multiset of non-cyclic strings. The result holds whether or not the alphabet is restricted to be binary. In combination, the two results show that the generalized form of LCP array for a multiset of cyclic strings is fundamentally different from the other more restricted forms.

Methods to measure work stress generally rely on subjective measures from questionnaires or require dedicated sensors that are cumbersome to wear and interfere with the task. To address this problem, we propose a method to detect stress unobtrusively using commodity devices (keyboards, mice) instrumented with pressure sensors. We propose a minimalist design that can be easily replicated by other researchers using off-the-shelf and low-cost hardware. We validate the design in a laboratory experiment that simulates office tasks and mild stressors while avoiding methodological limitations of previous studies. We compare stress-detection performance when using conventional features reported in the literature (keystroke dynamics, mouse trajectories) augmented with information from pressure sensors. Our results indicate that pressure provides additional information for stress discrimination; adding pressure information to keystroke dynamics and mouse trajectories improves classification performance by 6% and 3%, respectively. These results show how devices that are already part of the modern workplace may be used and enhanced to automatically and unobtrusively detect stress.

This paper addresses the problem of anomaly detection for high-dimensional sensing data. The one-class support vector machine (OCSVM) is one of the most popular unsupervised methods for anomaly detection. When data are high dimensional and large scale, however, the efficiency of OCSVM-based methods in anomaly detection suffers. Although dimensionality-reduction tools, such as deep belief networks, can be applied to compress the high-dimensional data to alleviate the problem, the accuracy and timely detection are still hard to improve due to the inherent features of OCSVM. In this paper, we propose a new form of OCSVM model based on the structure of the compressed data and the characteristics of OCSVM. Based on the new model, we design both optimal and approximate methods for model training and testing. We evaluate the performance of our methods with extensive experiments on four real-world datasets. The experimental results demonstrate that our new methods, both optimal and approximate ones, not only significantly outperform the state-of-the-art in accuracy and efficiency, but also achieve the good performance without the need of manual parameter tuning. In addition, our approximate training and testing mechanism can reduce the computing time by three orders of magnitude with a negligible loss in accuracy.

We propose a simple and effective method to inject word-level information into character-aware neural language models. Unlike previous approaches which usually inject word-level information at the input of a long short-term memory (LSTM) network, we inject it into the softmax function. The resultant model can be seen as a combination of character-aware language model and simple word-level language model. Our injection method can also be used together with previous methods. Through the experiments on 14 typologically diverse languages, we empirically show that our injection method, when used together with the previous methods, works better than the previous methods, including a gating mechanism, averaging, and concatenation of word vectors. We also provide a comprehensive comparison of these injection methods.

The degree distributions of complex networks are usually considered to be power law. However, it is not the case for a large number of them. We thus propose a new model able to build random growing networks with (almost) any wanted degree distribution. The degree distribution can either be theoretical or extracted from a real-world network. The main idea is to invert the recurrence equation commonly used to compute the degree distribution in order to find a convenient attachment function for node connections - commonly chosen as linear. We compute this attachment function for some classical distributions, as the power-law, broken power-law, geometric and Poisson distributions. We also use the model on an undirected version of the Twitter network, for which the degree distribution has an unusual shape. We finally show that the divergence of chosen attachment functions is heavily links to the heavy-tailed property of the obtained degree distributions., Comment: 23 pages, 3 figures

International audience; Internet of Things (IoT) applications often require the processing of data streams generated by devices dispersed over a large geographical area. Traditionally, these data streams are forwarded to a distant cloud for processing, thus resulting in high application end-to-end latency. Recent work explores the combination of resources located in clouds and at the edges of the Internet, called cloud-edge infrastructure, for deploying Data Stream Processing (DSP) applications. Most previous work, however, fails to scale to very large IoT settings. This paper introduces deployment strategies for the placement of DSP applications on to cloud-edge infrastructure. The strategies split an application graph into regions and consider regions with stringent time requirements for edge placement. The proposed Aggregate End-to-End Latency Strategy with Region Patterns and Latency Awareness (AELS+RP+LA) decreases the number of evaluated resources when computing an operator’s placement by considering the communication overhead across computing resources. Simulation results show that, unlike the state-of-the-art, AELS+RP+LA scales to environments with more than 100k resources with negligible impact on the application end-to-end latency.

Emotion recognition based on electroencephalography (EEG) is a significant task in the brain-computer interface field. Recently, many deep learning-based emotion recognition methods are demonstrated to outperform traditional methods. However, it remains challenging to extract discriminative features for EEG emotion recognition, and most methods ignore useful information in channel and time. This paper proposes an attention-based convolutional recurrent neural network (ACRNN) to extract more discriminative features from EEG signals and improve the accuracy of emotion recognition. First, the proposed ACRNN adopts a channel-wise attention mechanism to adaptively assign the weights of different channels, and a CNN is employed to extract the spatial information of encoded EEG signals. Then, to explore the temporal information of EEG signals, extended self-attention is integrated into an RNN to recode the importance based on intrinsic similarity in EEG signals. We conducted extensive experiments on the DEAP and DREAMER databases. The experimental results demonstrate that the proposed ACRNN outperforms state-of-the-art methods.

Physiological studies have shown that there are some differences in speech and facial activities between depressive and healthy individuals. Based on this fact, we propose a novel Spatio-Temporal Attention (STA) network and a Multimodal Attention Feature Fusion (MAFF) strategy to obtain the multimodal representation of depression cues for predicting the individual depression level. Specifically, we firstly divide the speech amplitude spectrum/video into fixed-length segments and input these segments into the STA network, which not only integrates the spatial and temporal information through attention mechanism, but also emphasizes the audio/video frames related to depression detection. The audio/video segment-level feature is obtained from the output of the last full connection layer of the STA network. Secondly, this paper employs the eigen evolution pooling method to summarize the changes of each dimension of the audio/video segment-level features to aggregate them into the audio/video level feature. Thirdly, the multimodal representation with modal complementary information is generated using the MAFF and inputs into the support vector regression predictor for estimating depression severity. Experimental results on the AVEC2013 and AVEC2014 depression databases illustrate the effectiveness of our method.

The Micro-expressions (MEs) are very important nonverbal communication clues. However, due to their local and short nature, spotting them is challenging. In this paper, we address this problem by using a dedicated local and temporal pattern (LTP) of facial movement. This pattern has a specific shape (S-pattern) when ME are displayed. Thus, by using a classical classification algorithm (SVM), MEs are distinguished from other facial movements. We also propose a global final fusion analysis on the whole face to improve the distinction between ME (local) and head (global) movements. However, the learning of S-patterns is limited by the small number of ME databases and the low volume of ME samples. Hammerstein models (HMs) are known to be a good approximation of muscle movements. By approximating each S-pattern with a HM, we can both filter outliers and generate new similar S-patterns. By this way, we perform a data augmentation for S-pattern training dataset and improve the ability to differentiate micro-expressions from other facial movements. The spotting results, performed on CASME and CASME, show that our proposed LTP outperforms the most popular spotting method in terms of F1-score. Adding the fusion process and data augmentation improve even more the spotting performance.

Textual emotion detection is an attractive task while previous studies mainly focused on polarity or single-emotion classification. However, human expressions are complex, and multiple emotions often occur simultaneously with non-negligible emotion correlations. In this paper, a Multi-label Emotion Detection Architecture (MEDA) is proposed to detect all associated emotions expressed in a given piece of text. MEDA is mainly composed of two modules: Multi-Channel Emotion-Specified Feature Extractor (MC-ESFE) and Emotion Correlation Learner (ECorL). MEDA captures underlying emotion-specified features through MC-ESFE module in advance. MC-ESFE is composed of multiple channel-wise ESFE networks. Each channel is devoted to the feature extraction of a specified emotion from sentence-level to context-level through a hierarchical structure. Based on obtained features, emotion correlation learning is implemented through an emotion sequence predictor in ECorL. During model training, we define a new loss function, which is called multi-label focal loss. With this loss function, the model can focus more on misclassified positive-negative emotion pairs and improve the overall performance by balancing the prediction of positive and negative emotions. The evaluation of proposed MEDA architecture is carried out on emotional corpus: RenCECps and NLPCC2018 datasets. The experimental results indicate that the proposed method can achieve better performance than state-of-the-art methods in this task.

This paper presents a new large multiview dataset called HUMBI for human body expressions with natural clothing. The goal of HUMBI is to facilitate modeling view-specific appearance and geometry of five primary body signals including gaze, face, hand, body, and garment from assorted people. 107 synchronized HD cameras are used to capture 772 distinctive subjects across gender, ethnicity, age, and style. With the multiview image streams, we reconstruct high fidelity body expressions using 3D mesh models, which allows representing view-specific appearance. We demonstrate that HUMBI is highly effective in learning and reconstructing a complete human model and is complementary to the existing datasets of human body expressions with limited views and subjects such as MPII-Gaze, Multi-PIE, Human3.6M, and Panoptic Studio datasets. Based on HUMBI, we formulate a new benchmark challenge of a pose-guided appearance rendering task that aims to substantially extend photorealism in modeling diverse human expressions in 3D, which is the key enabling factor of authentic social tele-presence. HUMBI is publicly available at http://humbi-data.net, Comment: 18 pages; Accepted to TPAMI

In this paper, we study detectability problem of Networked Discrete Event Systems (NDES) where the communication between the plant and the agent is carried out through a network that can have several channels, so that, communication delays can cause changes in the order of the observations, and, also, loss of observations. We deploy an approach previously presented in the literature to construct an untimed nondeterministic automaton that models the behavior of a NDES, and, based on this model, we present a methodology for the state estimation of NDES, and new networked D-detectability definitions. Finally, we show that the NDES is networked D-detectable if, and only if, the equivalent untimed nondeterministic model is D-detectable.

We prove the following theorem. Given a planar graph $G$ and an integer $k$, it is possible in polynomial time to randomly sample a subset $A$ of vertices of $G$ with the following properties: (i) $A$ induces a subgraph of $G$ of treewidth $\mathcal{O}(\sqrt{k}\log k)$, and (ii) for every connected subgraph $H$ of $G$ on at most $k$ vertices, the probability that $A$ covers the whole vertex set of $H$ is at least $(2^{\mathcal{O}(\sqrt{k}\log^2 k)}\cdot n^{\mathcal{O}(1)})^{-1}$, where $n$ is the number of vertices of $G$. Together with standard dynamic programming techniques for graphs of bounded treewidth, this result gives a versatile technique for obtaining (randomized) subexponential parameterized algorithms for problems on planar graphs, usually with running time bound $2^{\mathcal{O}(\sqrt{k} \log^2 k)} n^{\mathcal{O}(1)}$. The technique can be applied to problems expressible as searching for a small, connected pattern with a prescribed property in a large host graph, examples of such problems include Directed $k$-Path, Weighted $k$-Path, Vertex Cover Local Search, and Subgraph Isomorphism, among others. Up to this point, it was open whether these problems can be solved in subexponential parameterized time on planar graphs, because they are not amenable to the classic technique of bidimensionality. Furthermore, all our results hold in fact on any class of graphs that exclude a fixed apex graph as a minor, in particular on graphs embeddable in any fixed surface.

Oblivious RAM (ORAM), first introduced in the ground-breaking work of Goldreich and Ostrovsky (STOC ’87 and J. ACM ’96) is a technique for provably obfuscating programs’ access patterns, such that the access patterns leak no information about the programs’ secret inputs. To compile a general program to an oblivious counterpart, it is well-known that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varOmega (\log N)$$\end{document}Ω(logN) amortized blowup is necessary, where N is the size of the logical memory. This was shown in Goldreich and Ostrovksy’s original ORAM work for statistical security and in a somewhat restricted model (the so called balls-and-bins model), and recently by Larsen and Nielsen (CRYPTO ’18) for computational security. A long standing open question is whether there exists an optimal ORAM construction that matches the aforementioned logarithmic lower bounds (without making large memory word assumptions, and assuming a constant number of CPU registers). In this paper, we resolve this problem and present the first secure ORAM with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(\log N)$$\end{document}O(logN) amortized blowup, assuming one-way functions. Our result is inspired by and non-trivially improves on the recent beautiful work of Patel et al. (FOCS ’18) who gave a construction with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(\log N\cdot \log \log N)$$\end{document}O(logN·loglogN) amortized blowup, assuming one-way functions. One of our building blocks of independent interest is a linear-time deterministic oblivious algorithm for tight compaction: Given an array of n elements where some elements are marked, we permute the elements in the array so that all marked elements end up in the front of the array. Our O(n) algorithm improves the previously best known deterministic or randomized algorithms whose running time is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(n \cdot \log n)$$\end{document}O(n·logn) or \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(n \cdot \log \log n)$$\end{document}O(n·loglogn), respectively.

Contextual information, such as the co-occurrence of objects and the spatial and relative size among objects, provides rich and complex information about digital scenes. It also plays an important role in improving object detection and determining out-of-context objects. In this work, we present contextual models that leverage contextual information (16 contextual relationships are applied in this paper) to enhance the performance of two of the state-of-the-art object detectors (i.e., Faster RCNN and YOLO), which are applied as a post-processing process for most of the existing detectors, especially for refining the confidences and associated categorical labels, without refining bounding boxes. We experimentally demonstrate that our models lead to enhancement in detection performance using the most common dataset used in this field (MSCOCO), where in some experiments PASCAL2012 is also used.We also show that iterating the process of applying our contextual models also enhances the detection performance further.

We present a simple and effective approach for non-blind image deblurring, combining classical techniques and deep learning. In contrast to existing methods that deblur the image directly in the standard image space, we propose to perform an explicit deconvolution process in a feature space by integrating a classical Wiener deconvolution framework with learned deep features. A multi-scale cascaded feature refinement module then predicts the deblurred image from the deconvolved deep features, progressively recovering detail and small-scale structures. The proposed model is trained in an end-to-end manner and evaluated on scenarios with simulated Gaussian noise, saturated pixels, or JPEG compression artifacts as well as real-world images. Moreover, we present detailed analyses of the benefit of the feature-based Wiener deconvolution and of the multi-scale cascaded feature refinement as well as the robustness of the proposed approach. Our extensive experimental results show that the proposed deep Wiener deconvolution network facilitates deblurred results with visibly fewer artifacts and quantitatively outperforms state-of-the-art non-blind image deblurring methods by a wide margin.

In unsupervised domain adaptation (UDA), a classifier for the target domain is trained with massive true-label data from the source domain and unlabeled data from the target domain. However, collecting fully-true-label data in the source domain is high-cost and sometimes impossible. Compared to the true labels, a complementary label specifies a class that a pattern does not belong to, hence collecting complementary labels would be less laborious than collecting true labels. Thus, in this paper, we propose a novel setting that the source domain is composed of complementary-label data, and a theoretical bound for it is first proved. We consider two cases of this setting, one is that the source domain only contains complementary-label data (completely complementary unsupervised domain adaptation, CC-UDA), and the other is that the source domain has plenty of complementary-label data and a small amount of true-label data (partly complementary unsupervised domain adaptation, PC-UDA). To this end, a complementary label adversarial network} (CLARINET) is proposed to solve CC-UDA and PC-UDA problems. CLARINET maintains two deep networks simultaneously, where one focuses on classifying complementary-label source data and the other takes care of source-to-target distributional adaptation. Experiments show that CLARINET significantly outperforms a series of competent baselines on handwritten-digits-recognition and objects-recognition tasks., arXiv admin note: text overlap with arXiv:2007.14612

Night beats with alternating current (AC) illumination. By passively sensing this beat, we reveal new scene information which includes: the type of bulbs in the scene, the phases of the electric grid up to city scale, and the light transport matrix. This information yields unmixing of reflections and semi-reflections, nocturnal high dynamic range, and scene rendering with bulbs not observed during acquisition. The latter is facilitated by a dataset of bulb response functions for a range of sources, which we collected and provide. To do all this, we built a novel coded-exposure high-dynamic-range imaging technique, specifically designed to operate on the grid's AC lighting.

In this paper, a non‐negative matrix factorization feature expansion (NMFFE) approach was proposed to overcome the feature‐sparsity issue when expanding features of short‐text. First, we took the internal relationships of short texts and words into account when segmenting words from texts and constructing their relationship matrix. Second, we utilized the Dual regularization non‐negative matrix tri‐factorization (DNMTF) algorithm to obtain the words clustering indicator matrix, which was used to get the feature space by dimensionality reduction methods. Thirdly, words with close relationship were selected out from the feature space and added into the short‐text to solve the sparsity issue. The experimental results showed that the accuracy of short text classification of our NMFFE algorithm increased 25.77%, 10.89%, and 1.79% on three data sets: Web snippets, Twitter sports, and AGnews, respectively compared with the Word2Vec algorithm and Char‐CNN algorithm. It indicated that the NMFFE algorithm was better than the BOW algorithm and the Char‐CNN algorithm in terms of classification accuracy and algorithm robustness.

In plenty of real-life tasks, strongly supervised information is hard to obtain, such that there is not sufficient high-quality supervision to make traditional learning approaches succeed. Therefore, weakly supervised learning has drawn considerable attention recently. In this paper, we consider the problem of learning from incomplete and inaccurate supervision, where only a limited subset of training data is labeled but potentially with noise. This setting is challenging and of great importance but rarely studied in the literature. We notice that in many applications, the limited labeled data are usually with one-sided noise. For instance, considering the bug detection task in the software system, the identified buggy codes are indeed with defects whereas the codes that have been checked many times or newly fixed may still have other flaws due to the complexity of the system. We propose a novel method which is able to effectively alleviate the negative influence of one-sided label noise with the help of a vast number of unlabeled data. Excess risk analysis is provided as theoretical justifications on the usefulness of incomplete and one-sided inaccurate supervision. We conduct experiments on synthetic, benchmark datasets, and real-life tasks to validate the effectiveness of the proposed approach.

This paper proposes a semantic framework for Business Model evaluation and its application to a real case study in the context of smart energy and sustainable mobility. It presents an ontology based representation of an original business model and examples of inferential rules for knowledge extraction and automatic population of the ontology. The real case study belongs to the GreenCharge European Project, that in these last years is proposing some original business models to promote sustainable e-mobility plans. An original OWL Ontology contains all relevant Business Model concepts referring to GreenCharge’s domain, including a semantic description of TestCards, survey results and inferential rules.

We study the notion of consistency between a 3D shape and a 2D observation and propose a differentiable formulation which allows computing gradients of the 3D shape given an observation from an arbitrary view. We do so by reformulating view consistency using a differentiable ray consistency (DRC) term. We show that this formulation can be incorporated in a learning framework to leverage different types of multi-view observations e.g. foreground masks, depth, color images, semantics etc. as supervision for learning single-view 3D prediction. We present empirical analysis of our technique in a controlled setting. We also show that this approach allows us to improve over existing techniques for single-view reconstruction of objects from the PASCAL VOC dataset., Comment: To appear at CVPR 2017. Project webpage : https://shubhtuls.github.io/drc/

In this paper, we study undirected multiple graphs of any natural multiplicity k > 1. There are edges of three types: ordinary edges, multiple edges and multi-edges. Each edge of the last two types is a union of k linked edges, which connect 2 or (k + 1) vertices correspondingly. The linked edges should be used simultaneously. If a vertex is incident to a multiple edge, it can be also incident to other multiple edges and it can be the common end of k linked edges of some multi-edge. If a vertex is the common end of some multi-edge, it cannot be the common end of another multi-edge. A multiple tree is a connected multiple graph with no cycles. Unlike ordinary trees, the number of edges in a multiple tree is not fixed. The problem of finding the spanning tree can be set for a multiple graph. Complete spanning trees form a special class of spanning trees of a multiple graph. Their peculiarity is that a multiple path joining any two selected vertices exists in the tree if and only if such a path exists in the initial graph. If the multiple graph is weighted, the minimum spanning tree problem and the minimum complete spanning tree problem can be set. Also we can formulate the problems of recognition of the spanning tree and complete spanning tree of the limited weight. The main result of this article is the proof of NPcompleteness of such recognition problems for arbitrary multiple graphs as well as for divisible multiple graphs in the case when multiplicity k ≥ 3. The corresponding optimization problems are NP-hard.

We consider Bandits with Knapsacks (henceforth, BwK), a general model for multi-armed bandits under supply/budget constraints. In particular, a bandit algorithm needs to solve a well-known knapsack problem: find an optimal packing of items into a limited-size knapsack. The BwK problem is a common generalization of numerous motivating examples, which range from dynamic pricing to repeated auctions to dynamic ad allocation to network routing and scheduling. While the prior work on BwK focused on the stochastic version, we pioneer the other extreme in which the outcomes can be chosen adversarially. This is a considerably harder problem, compared to both the stochastic version and the "classic" adversarial bandits, in that regret minimization is no longer feasible. Instead, the objective is to minimize the competitive ratio: the ratio of the benchmark reward to the algorithm's reward. We design an algorithm with competitive ratio O(log T) relative to the best fixed distribution over actions, where T is the time horizon; we also prove a matching lower bound. The key conceptual contribution is a new perspective on the stochastic version of the problem. We suggest a new algorithm for the stochastic version, which builds on the framework of regret minimization in repeated games and admits a substantially simpler analysis compared to prior work. We then analyze this algorithm for the adversarial version and use it as a subroutine to solve the latter., The extended abstract appeared in FOCS 2019. The definitive version was published in JACM '22. V8 is the latest version with all technical changes. Subsequent versions fixes minor LATEX presentation issues

The proposed work aims to solve data sparsity problem in the recommendation system. It handles two-level pre-processing techniques to reduce the data size at the item level. Additional resources like items genre, tag, and time are added to learn and analyse the behaviour of the user preferences in-depth. The advantage of the proposed method is to recommend the item, based on user interest pattern and avoid recommending the outdated items. User information are grouped based on similar item genre and tag feature. This effectively handle overlapping conditions that exist on item’s genre, as it has more than one genre at initial level. Further, based on time, it analyses the user non-static interest. Overall it reduces the dimensions which is an initial way to prepare data, to analyse hidden pattern. To enhance the performance, the proposed method utilized Apache’s spark Mllib FP-Growth and association rule mining approach in a distributed environment. To reduce the computation cost of constructing tree in FP-Growth, the candidate data set is stored in matrix form. The experiments were conducted using MovieLens data set. The observed results shows that the proposed method achieves 4% increase in accuracy when compared to earlier methods.