Search

In this paper, we propose a Hierarchical Multimodal Variational Encoder-Decoder (HMMVED) to predict the popularity of micro-videos by comprehensively leveraging the user information and micro-video content in a hierarchical fashion. In particular, the multimodal variational encoder-decoder framework encodes the input modalities to a lower dimensional stochastic variable, from which the popularity of micro-videos can be decoded. Considering the important role of users social influence in social media for information dissemination, a user encoderdecoder is designed to learn the user Gaussian embedding from the user information, which is informative about the coarsegrained popularity for the micro-videos uploaded by the user. The learned user embedding then acts as the prior of the micro-video latent embeddings in the micro-video encoder-decoder, where the refined posterior embedding of the micro-video is obtained by taking the multimodal content into consideration. The finegrained popularity of each micro-video is generated from the posterior embedding of the micro-video. Based on the multimodal extension of variational information bottleneck theory, we show that the learned latent embeddings of micro-videos are maximally expressive about the popularity whilst maximally compressing the information from input modalities. Extensive experiments conducted on two real-world datasets demonstrate the effectiveness of the proposed method. Codes and datasets are available at: https://github.com/JennyXieJiayi/HMMVED.

Multiview clustering (MVC) has recently been the focus of much attention due to its ability to partition data from multiple views via view correlations. However, most MVC methods only learn either interfeature correlations or intercluster correlations, which may lead to unsatisfactory clustering performance. To address this issue, we propose a novel dual-correlated multivariate information bottleneck (DMIB) method for MVC. DMIB is able to explore both interfeature correlations (the relationship among multiple distinct feature representations from different views) and intercluster correlations (the close agreement among clustering results obtained from individual views). For the former, we integrate both view-shared feature correlations discovered by learning a shared discriminative feature subspace and view-specific feature information to fully explore the interfeature correlation. This allows us to attain multiple reliable local clustering results of different views. Following this, we explore the intercluster correlations by learning the shared mutual information over different local clusterings for an improved global partition. By integrating both correlations, we formulate the problem as a unified information maximization function and further design a two-step method for optimization. Moreover, we theoretically prove the convergence of the proposed algorithm, and discuss the relationships between our method and several existing clustering paradigms. The experimental results on multiple datasets demonstrate the superiority of DMIB compared to several state-of-the-art clustering methods.

Deep neural networks are a key component of behavior prediction and motion generation for self-driving cars. One of their main drawbacks is a lack of transparency: they should provide easy to interpret rationales for what triggers certain behaviors. We propose an architecture called Attentional Bottleneck with the goal of improving transparency. Our key idea is to combine visual attention, which identifies what aspects of the input the model is using, with an information bottleneck that enables the model to only use aspects of the input which are important. This not only provides sparse and interpretable attention maps (e.g. focusing only on specific vehicles in the scene), but it adds this transparency at no cost to model accuracy. In fact, we find improvements in accuracy when applying Attentional Bottleneck to the ChauffeurNet model, whereas we find that the accuracy deteriorates with a traditional visual attention model.

Freeway bottleneck identification is an essential component in the process of deploying mitigation strategies to reduce congestion at freeway bottlenecks. Most previous studies on bottleneck identification focus on recurrent bottlenecks, and limited work has been conducted to identify the locations of non-recurrent bottlenecks. Therefore, in this study, we propose a new travel time reliability (TTR) measurement and develop a freeway bottleneck identification method based on this measurement, which can identify with high probability not only recurrent bottlenecks but also the locations of non-recurrent bottlenecks. The TTR measurement is developed based on statistical distance between travel time distributions. Three statistical distance measurements, Jensen–Shannon divergence, Wasserstein distance, and Hellinger distance, are applied in the TTR measurement. The bottleneck identification method is evaluated in a case study on I-15 freeway corridor in Salt Lake City, Utah. The three statistical distance measurements show good consistency in ranking locations by the impacts of recurrent and non-recurrent congestion, especially for extreme cases with very high or low variation between travel time distributions. The recurrent bottlenecks identified in this study show their clustering characteristics, which is similar to the generating and dismissing process of recurrent congestion. The locations with high probability of non-recurrent bottlenecks scatter both spatially and temporally, which agrees with the random characteristic of non-recurrent congestion.

With the arrival of the industrial big data era, it offers unprecedented opportunities for machine learning to intelligently uncover hidden tasks and restore the entire underlying process for business process modelling. While recent studies, e.g., process mining and ontologies, have advanced the research agenda of business process modelling and management, identifying a bottleneck task automatically needs more in-depth research. In this paper, a text mining-based bottleneck task identification approach is proposed. Firstly, to extract tasks from documents in different lengths, a dynamic sliding window is introduced to the biterm topic model. The sliding window size is adjusted according to document length during biterm selection process to ensure the two words in biterm comes from a context. Secondly, a fusion-based clustering algorithm is studied to uncover business tasks. The improved biterm topic model and the Doc2vec model are used to train two document vectors and then calculate two distances. The linear fusion of these two distances is used as the metric of clustering. Thirdly, the temporal frequency of each task at different periods is calculated to show the timeline and abnormal occurrence of tasks to identify bottleneck tasks. The proposed approach is evaluated using a data set containing the execution of a multi-year multidisciplinary student design project. The experiment results show the approach can effectively identify bottleneck tasks without manual intervention.

Traffic bottleneck refers to the road section or node that often causes the propagation or spread of traffic congestion in the road network, which is the source of the whole road network congestion. In this paper, the author analyzes the bottleneck prediction of urban road network based on improved PSO algorithms and fuzzy control. This paper analyzes the factors and characteristics of the main road of the system, proposes the traffic coordination control of the main road based on the delay model, and carries on the statistical simulation to the actual traffic data, develops the basic theory of the traffic coordination control which is more effective than the traditional timing control strategy. Compared with the traditional model, the algorithm considers the waiting time of the red light at the intersection. For the congested road section, it can better calculate the travel time of the vehicle, making the results more accurate and more applicable. The results of this study can provide a strong theoretical basis and prediction scheme for the traffic management and control of the road network in the target area.

Cloud servers are highly prone to resource bottleneck failures. In this work, we propose an ensemble learning model to build LSTM-based multiclass classifier for resource bottleneck identification. The workload at cloud servers is highly dynamic in nature. To support continuous online learning of resource bottleneck identification models, we propose relevance feedback based online learning of proposed ensemble model. Here we propose to analyse, catastrophe forgetting and incremental architectural evolution as two fundamental challenges associated with online adaptation of LSTM-based multiclass classifier models. To avoid catastrophic forgetting, we propose a combination of distillation loss and the standard crossentropy loss. For architectural evolution, we propose and analyse three different alternatives to update the architecture of the bottleneck identification model on the fly. We evaluate the proposed approaches on a real world dataset collected in an industrial case study and on a dataset collected in a virtual environment setup using Docker containers. The experimental results show that the proposed approaches outperform existing state-of-the-art methods for bottleneck identification.

Private Ethereum blockchain-based systems are demanded in many industry sectors. However, the throughput performance of these systems does not meet their expectations. Many researchers have analyzed the performance of private blockchains, but their studies have failed to analyze root causes. In this paper, we perform a deep function-level bottleneck analysis for the private Ethereum blockchain. As the Ethereum client application is developed with golang, we leverage pprof , which is a resource-profiling tool for golang, and custom golang functions to measure the time taken by functions. To easily configure parameters and conduct our test, we code a shell script that automates the building process of a private Ethereum blockchain with docker containers. We conducted a series of experiments and identified the bottleneck function that is called every time a transaction arrives at an Ethereum node. In addition, we also found that the multi-threading is not well utilized, meaning that there is much room for improvement.

An important factor for designing the interaction of automated vehicles (AVs) with surrounding road users is the capability of the human communication partner, especially the perception, processing and response. As these processes need time, there is a latest moment the AV has to define its communication strategy. The present work analyzes the situation when an AV negotiates the right of way to a simultaneously oncoming human driver in a road bottleneck scenario. We modeled the sequence of human tasks when perceiving and processing the AV’s information and the time needed to respond to it. The interaction model results in a latest moment after which a comfortable interaction can no longer be initiated. Comparing the model with the data of an underlying study indicates the appropriate modeling of the time required for communication. A later start of communication leads to inefficient traffic flow and worse subjective ratings of the AV’s communication.

Cloud computing became widespread on IT industry, saving costs of acquisition and maintenance for companies of all sizes, and enabling fair management of resources according to the demand. Stochastic models can enable performance and dependability evaluation of cloud computing systems efficiently, what is needed for proper capacity planning. Distinct models may be combined in a hierarchy to address the huge number of components and levels of interaction among the system parts. Identification of bottlenecks in such composite models might be hard yet, due to the huge amount of input factors and variables which may interfere with the results. This paper proposes a method for bottleneck detection of computational systems represented with hierarchical models, that is remarkably applied in cloud computing systems. This is achieved through the composition of indices computed from lower level models in equations and solution methods of the top level model, for computing the sensitivity indices of all parameters with respect to a global system measure. A unified sensitivity ranking, comprising the composite indices, indicates the parameters with highest impact on output metrics. A case study supports the demonstration of accuracy and utility of our methodology. The study addresses a web service running on a private cloud with auto scaling mechanisms. The methods and algorithms presented here are helpful for decision-making when designing and managing cloud computing infrastructures, regarding incremental and architectural improvements.

The widespread adoption of low-cost cameras generates massive amounts of videos recorded from different viewpoints every day. To cope with this vast amount of unlabeled and heterogeneous data, a new multi-task information bottleneck co-clustering (MIBC) approach is proposed to automatically categorize human actions in collections of unlabeled cross-view videos. Our motivation is that, if a learning action category from each view is seen as a single task, it is reasonable to assume that the tasks of learning action patterns from the videos recorded by multiple cameras are dependent and inter-related, since the actions of the same subjects synchronously recorded from different camera viewpoints are complementary to each other. MIBC aims to transfer the shared view knowledge across multiple tasks (i.e., camera viewpoints) to boost the performance of each task. Specifically, MIBC involves the following two parts: (1) extracting action categories for each task by independently maintaining its own relevant information, and (2) allowing the feature representations of all tasks to be compressed into a common feature space, which is utilized to capture the relatedness of multiple tasks and transfer the shared knowledge across different camera viewpoints. These two parts of MIBC work simultaneously and can be solved in a novel co-clustering mechanism. Our experimental evaluation on several cross-view action collections shows that the MIBC algorithm outperforms the existing state-of-the-art baselines.

With the rapid development of e-commerce, online consumption has become a mainstream form of consumption in recent years. Text sentiment analysis for a large number of customer reviews on the e-commerce platform can dramatically improve the customers' consumption experience. Although the sentiment analysis approaches based on deep neural network can achieve higher accuracy without human-design features compared with traditional sentiment analysis methods, the accuracy still cannot meet the demand and the training suffers from the issues of over-fitting, vanishing gradient, etc. In this paper, a novel sentiment analysis model named MBGCV is designed to alleviate these problems and improve the accuracy, MBGCV employs a multichannel paradigm and integrates Bidirectional Gated Recurrent Unit (BiGRU), Convolutional Neural Network (CNN) and Variational Information Bottleneck (VIB). The multichannel is exploited to extract multi-grained sentiment features. In each channel, BiGRU is utilized to extract context information, and then CNN is adopted to extract local features. Furthermore, the model combines the advantages of VIB and Maxout activation function to overcome shortcomings such as over-fitting, vanishing gradient in existing sentiment analysis models. By using real review datasets, we carry out extensive experiments to demonstrate that our proposed model can achieve superior accuracy and improve the performance of text sentiment analysis.

As Deep Neural Networks (DNNs) have achieved state-of- the-art performance in various scientific fields and applica- tions, the memory and computational complexity of DNNs have increased concurrently. The increased complexity re- quired by DNNs prohibits them from running on platforms with limited computational resources. This has sparked a re- newed interest in parameter pruning. We propose to replace the standard cross-entropy objective – typically used in clas- sification problems – with the Nonlinear Information Bottle- neck (NIB) objective to improve the accuracy of a pruned net- work. We demonstrate, that our proposal outperforms cross- entropy combined with global magnitude pruning for high compression rates on VGG-nets trained on CIFAR10. With approximately 97% of the parameters pruned, we obtain an accuracy of 87.63% and 88.22% for VGG-16 and VGG-19, respectively, where the baseline accuracy is 91.5% for the un- pruned networks. We observe that the majority of biases are pruned completely, and pruning parameters globally outper- forms layer-wise pruning.

Multi-view logo classification is a challenging task due to the cross-view misalignment of logo image varies under different viewpoints, large intra-classes and small inter-classes variation of logo appearance. Cross-view data can represent objects from different views and thus provide complementary information for data analysis. However, most existing multi-view algorithms usually maximize the correlation between different views for consistency. Those methods ignore the interaction among different views and may cause semantic bias during the process of common feature learning. In this paper, we investigate the information bottleneck (IB) to the multi-view learning for extracting the different view common features of one category, named Dual-View Information Bottleneck representation (Dual-view IB). To the best of our knowledge, this is the first cross-view learning method for logo classification. Specifically, we maximize the mutual information between the representations of the two views to achieve the preservation of key features in the classification task, while eliminating the redundant information that is not shared between the two views. In addition, due to the unbalance of samples and limited computing resources, we further introduce a novel Pair Batch Data Augmentation (PB) algorithm for Dual-view IB model, which applies augmentations from a learned policy based on replicates instances of two samples within the same batch. Comprehensive experiments on three existing benchmark datasets, which demonstrate the effectiveness of the proposed method that outperforms the methods in the state of the art. The proposed method is expected to further the development of cross-view representation learning.

In this study, the information bottleneck method is proposed as an optimisation method for steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI). The information bottleneck is an information-theoretic optimisation method for solving problems with a trade-off between preserving meaningful information and compression. Its main practical application in machine learning is in representation learning or feature extraction. In this study, we use the information bottleneck to find optimal classification rule for a BCI. This is a novel application for the information bottleneck. This approach is particularly suitable for BCIs since the information bottleneck optimises the amount of information transferred by the BCI. Steady-state visual evoked potential-based BCIs often classify targets using very simple rules like choosing the class corresponding to the largest feature value. We call this classifier the arg max classifier. It is unlikely that this approach is optimal, and in this study, we propose a classification method specifically designed to optimise the performance measure of BCIs. This approach gives an advantage over standard machine learning methods, which aim to optimise different measures. The performance of the proposed algorithm is tested on two publicly available datasets in offline experiments. We use the standard power spectral density analysis (PSDA) and canonical correlation analysis (CCA) feature extraction methods on one dataset and show that the current approach outperforms most of the related studies on this dataset. On the second dataset, we use the task-related component analysis (TRCA) method and demonstrate that the proposed method outperforms the standard argmax classification rule in terms of information transfer rate when using a small number of classes. To our knowledge, this is the first time the information bottleneck is used in the context of SSVEP-based BCIs. The approach is unique in the sense that optimisation is done over the space of classification functions. It potentially improves the performance of BCIs and makes it easier to calibrate the system for different subjects.

Despite its significant effectiveness in adversarial training approaches to multidomain task-oriented dialogue systems, adversarial inverse reinforcement learning of the dialogue policy frequently fails to balance the performance of the reward estimator and policy generator. During the optimization process, the reward estimator frequently overwhelms the policy generator, resulting in excessively uninformative gradients. We propose the variational reward estimator bottleneck (VRB), which is a novel and effective regularization strategy that aims to constrain unproductive information flows between inputs and the reward estimator. The VRB focuses on capturing discriminative features by exploiting information bottleneck on mutual information. Quantitative analysis on a multidomain task-oriented dialogue dataset demonstrates that the VRB significantly outperforms previous studies.

The current big data era routinely requires the processing of large-scale data on massive distributed computing clusters. In these applications, data sets are often so large that they cannot be housed in the memory and/or the disk of any one computer. Thus, the data and the processing are typically distributed across multiple nodes. Distributed computation is thus a necessity rather than a luxury. The widespread use of such clusters presents several opportunities and advantages over traditional computing paradigms. However, it also presents newer challenges where coding-theoretic ideas have recently had a significant impact. Large-scale clusters (which can be heterogeneous in nature) suffer from the problem of stragglers, which are slow or failed worker nodes in the system. Thus, the overall speed of a computation is typically dominated by the slowest node in the absence of a sophisticated assignment of tasks to the worker nodes.

Initialization plays a critical role in the training of deep neural networks (DNN). Existing initialization strategies mainly focus on stabilizing the training process to mitigate gradient vanish/explosion problems. However, these initialization methods are lacking in consideration about how to enhance generalization ability. The Information Bottleneck (IB) theory is a well-known understanding framework to provide an explanation about the generalization of DNN. Guided by the insights provided by IB theory, we design two criteria for better initializing DNN. And we further design a neuron campaign initialization algorithm to efficiently select a good initialization for a neural network on a given dataset. The experiments on MNIST dataset show that our method can lead to a better generalization performance with faster convergence., 5 pages, Accepted by CIKM'21

Recently, the cross-modal analysis has drawn much attention due to the rapid growth and widespread emergence of multimodal data. It integrates multiple modalities to improve learning and generalization performance. However, most previous methods just focus on learning a common shared feature space for all modalities and ignore the private information hidden in each individual modality. To address this problem, we propose a novel shared-private information bottleneck (SPIB) method for cross-modal clustering. First, we devise a hybrid words model and a consensus clustering model to construct the shared information of multiple modalities, which capture the statistical correlation of low-level features and the semantic relations of the high-level clustering partitions, respectively. Second, the shared information of multiple modalities and the private information of individual modalities are maximally preserved through a unified information maximization function. Finally, the optimization of SPIB function is performed by a sequential “draw-and-merge” procedure, which guarantees the function converges to a local maximum. Besides, to solve the lack of tags in cross-modal social images, we also investigate the use of structured prior knowledge in the form of knowledge graph to enrich the information in semantic modality and design a novel semantic similarity measurement for social images. The experimental results on four types of cross-modal datasets demonstrate that our method outperforms the state-of-the-art approaches.

Cross-speaker style transfer is crucial to the applications of multi-style and expressive speech synthesis at scale. It does not require the target speakers to be experts in expressing all styles and to collect corresponding recordings for model training. However, the performances of existing style transfer methods are still far behind real application needs. The root causes are mainly twofold. Firstly, the style embedding extracted from single reference speech can hardly provide fine-grained and appropriate prosody information for arbitrary text to synthesize. Secondly, in these models the content/text, prosody, and speaker timbre are usually highly entangled, it's therefore not realistic to expect a satisfied result when freely combining these components, such as to transfer speaking style between speakers. In this paper, we propose a cross-speaker style transfer text-to-speech (TTS) model with explicit prosody bottleneck. The prosody bottleneck builds up the kernels accounting for speaking style robustly, and disentangles the prosody from content and speaker timbre, therefore guarantees high quality cross-speaker style transfer. Evaluation result shows the proposed method even achieves on-par performance with source speaker's speaker-dependent (SD) model in objective measurement of prosody, and significantly outperforms the cycle consistency and GMVAE-based baselines in objective and subjective evaluations., in Proceedings of INTERSPEECH 2021

Deep learning has become the most powerful machine learning tool in the last decade. However, how to efficiently train deep neural networks remains to be thoroughly solved. The widely used minibatch stochastic gradient descent (SGD) still needs to be accelerated. As a promising tool to better understand the learning dynamic of minibatch SGD, the information bottleneck (IB) theory claims that the optimization process consists of an initial fitting phase and the following compression phase. Based on this principle, we further study typicality sampling, an efficient data selection method, and propose a new explanation of how it helps accelerate the training process of the deep networks. We show that the fitting phase depicted in the IB theory will be boosted with a high signal-to-noise ratio of gradient approximation if the typicality sampling is appropriately adopted. Furthermore, this finding also implies that the prior information of the training set is critical to the optimization process and the better use of the most important data can help the information flow through the bottleneck faster. Both theoretical analysis and experimental results on synthetic and real-world datasets demonstrate our conclusions., Comment: 11 pages, to be published in IEEE Transactions on Neural Networks and Learning Systems

We present 'embo', a Python package to analyze empirical data using the Information Bottleneck (IB) method and its variants, such as the Deterministic Information Bottleneck (DIB). Given two random variables 'X' and 'Y', the IB finds the stochastic mapping 'M' of 'X' that encodes the most information about 'Y', subject to a constraint on the information that 'M' is allowed to retain about 'X'. Despite the popularity of the IB, an accessible implementation of the reference algorithm oriented towards ease of use on empirical data was missing. Embo is optimized for the common case of discrete, low-dimensional data. Embo is fast, provides a standard data-processing pipeline, offers a parallel implementation of key computational steps, and includes reasonable defaults for the method parameters. Embo is broadly applicable to different problem domains, as it can be employed with any dataset consisting in joint observations of two discrete variables. It is available from the Python Package Index (PyPI), Zenodo and GitLab.

Cross-lingual voice conversion (CLVC) is quite challenging since the source and target speakers speak different languages. It is essential for various applications such as developing mixed-language speech synthesis systems, customization of speaking devices, etc. This paper proposes a deep neural network (DNN)-based approach utilizing bottleneck features for CLVC. In the proposed method, the speaker-independent information present in the speech signals from different languages is represented by using the bottleneck features extracted from a deep auto-encoder. A DNN model is trained to learn the mapping between bottleneck features and the corresponding spectral features of the target speaker. The proposed approach can capture speaker-specific characteristics of a target speaker, and requires no speech data from the source speaker during training. The performance of the proposed method is evaluated using data from three Indian languages: Telugu, Tamil and Malayalam. The experimental results show that the proposed method can effectively convert the source speaker voice to target speaker voice in a cross-lingual scenario.

How to effectively mitigate the bias of feedback in recommender systems is an important research topic. In this paper, we first describe the generation process of the biased and unbiased feedback in recommender systems via two respective causal diagrams, where the difference between them can be regarded as the source of bias. We then define this difference as a confounding bias, which can be regarded as a collection of some specific biases that have previously been studied. For the case with biased feedback alone, we derive the conditions that need to be satisfied to obtain a debiased representation from the causal diagrams. Based on information theory, we propose a novel method called debiased information bottleneck (DIB) to optimize these conditions and then find a tractable solution for it. In particular, the proposed method constrains the model to learn a biased embedding vector with independent biased and unbiased components in the training phase, and uses only the unbiased component in the test phase to deliver more accurate recommendations. Finally, we conduct extensive experiments on a public dataset and a real product dataset to verify the effectiveness of the proposed method and discuss its properties.

The selective visual attention mechanism in the human visual system (HVS) restricts the amount of information to reach visual awareness for perceiving natural scenes, allowing near real-time information processing with limited computational capacity [Koch and Ullman, 1987]. This kind of selectivity acts as an 'Information Bottleneck (IB)', which seeks a trade-off between information compression and predictive accuracy. However, such information constraints are rarely explored in the attention mechanism for deep neural networks (DNNs). In this paper, we propose an IB-inspired spatial attention module for DNN structures built for visual recognition. The module takes as input an intermediate representation of the input image, and outputs a variational 2D attention map that minimizes the mutual information (MI) between the attention-modulated representation and the input, while maximizing the MI between the attention-modulated representation and the task label. To further restrict the information bypassed by the attention map, we quantize the continuous attention scores to a set of learnable anchor values during training. Extensive experiments show that the proposed IB-inspired spatial attention mechanism can yield attention maps that neatly highlight the regions of interest while suppressing backgrounds, and bootstrap standard DNN structures for visual recognition tasks (e.g., image classification, fine-grained recognition, cross-domain classification). The attention maps are interpretable for the decision making of the DNNs as verified in the experiments. Our code is available at https://github.com/ashleylqx/AIB.git., To appear in IJCAI 2021; with supplymentary

Train delays occur often in daily railway operations due to a variety of initiating incidents. On a heavily loaded mainline railway, a single train delay may lead to a series of secondary delays across the network. In this study, the authors describe a peer-to-peer system to solve train rescheduling problems in railway network bottlenecks. A designed Genetic Algorithm is chosen as the local search algorithm on each side. Based on the local search algorithm, different negotiation protocols are raised to find globally feasible solutions. The proposed approach is tested in a railway bottleneck section in the UK, and the computational result is compared with a centralised method to show its performance in terms of computation time and optimality.

International audience; Point cloud acquisition techniques are an essential tool for the digitization of industrial plants, yet the bulk of a designer's work remains manual. A first step to automatize drawing generation is to extract the semantics of the point cloud. Towards this goal, we investigate the use of deep learning to semantically segment oil and gas industrial scenes. We focus on domain characteristics such as high variation of object size, increased concavity and lack of annotated data, which hampers the use of conventional approaches. To address these issues, we advocate the use of synthetic data, adaptive downsampling and context sharing.

Low-rate Distributed Denial of Service (LDDoS) attacks degrade the quality of TCP links with low average traffic by exploiting vulnerabilities in the TCP/IP protocol and routing queue management mechanisms. The ideal transmission rate of the attack pulse in the most classical RTO-based LDDoS attacks (aka: Shrew attacks) is greater than or equal to the bandwidth of the target bottleneck link, and it is assumed that the attacker is able to set a suitably large transmission rate. However, the attacker does not always know the bandwidth of the target bottleneck link in real-world attack scenarios. In this paper, we propose a strategy for executing a low-rate shrew DDoS attack against a bottleneck link of unknown bandwidth and buffer size under a scenario targeting a residential network. The proposed attack strategy degrades downstream traffic from a specific service to the target network to the desired quality while keeping the attack traffic stealthy by exploratively increasing the pulse rate while estimating the attack effect by building bot nodes in the residential network.

Recognizing user location with WiFi fingerprints is a popular approach for accurate indoor positioning problems. In this work, our goal is to interpret WiFi fingerprints into actual user locations. However, WiFi fingerprint data can be very high dimensional in some cases, we need to find a good representation of the input data for the learning task first. Otherwise, using neural networks will suffer from severe overfitting. In this work, we solve this issue by combining the Information Bottleneck method and Variational Inference. Based on these two approaches, we propose a Variational Information Bottleneck model for accurate indoor positioning. The proposed model consists of an encoder structure and a predictor structure. The encoder is to find a good representation in the input data for the learning task. The predictor is to use the latent representation to predict the final output. To enhance the generalization of our model, we also adopt the Dropout technique for each hidden layer of the decoder. We conduct the validation experiments on a real-world dataset. We also compare the proposed model to other existing methods so as to quantify the performances of our method.

A bottleneck usually is a sub-process in the main process which delays the process. The performance of a process can be increased by eliminating the bottlenecks. To this end, opportunities to analyze and mitigate bottlenecks by using process mining techniques can be an interesting direction to utilize. This paper aims to classify literature on process mining bottleneck analysis techniques and propose a model for operational support regarding bottleneck analysis utilizing process mining. To this end, we first propose a model for classifying bottleneck analysis techniques. Then, we conduct a systematic literature review to identify existing papers that address bottleneck analysis by utilizing process mining techniques. The results indicate that many researchers are focusing on detecting bottlenecks, while limited attention is paid to predicting bottlenecks or recommending actions on what to do with bottlenecks. The proposed classification model is validated through a demonstration, showing how process mining bottleneck analysis techniques can be applied to a logistics case study.

Gesture is a typical human-machine interaction manner, accurate and robust gesture recognition can assist to achieve more natural interaction and understanding. Multi-modal gesture recognition can improve the recognition performance with the help of complex multi-modal relationship. However, it still faces the challenge of how to effectively balance the correlation and redundancy among different modalities, so as to guarantee the accuracy and robustness of the recognition. Hence, in this paper, a collaborative multi-modal learning method based on Random Variational Information Bottleneck (RVIB) is proposed. With random local information selection strategy, some information is compressed by information bottleneck, and the rest is retained directly, so as to make full use of effective redundant information while eliminating invalid redundant information. Experiments on open dataset show that the proposed method can achieve 95.77% recognition accuracy for 21 dynamic gestures, and can guarantee the recognition accuracy when some modality is missing.

The maximum output of manufacturing systems depends on the production capacity of bottleneck. So it is very important to predict abottleneck unit before actual production. However, the bottleneck often shifts from one place to another, it is difficult to predict accurately. To solve this problem, an intelligent prediction model based bottleneck (IPMFB) method is proposed by combining petri nets, object-oriented technology and cloud manufacturing ideas. With the proposed method, process and rules are designed, and a simulation model is established by the Plant Simulation software. Then all production data are input into the simulation model to intelligently predict the bottleneck by cloud computing and simulation running. In addition, the IPMFB method gives priority to production resources scheduling for bottleneck, which can satisfy a demand of bottleneck firstly by dynamic programming and a fuzzy algorithm. Finally, the validity and feasibility of the IPMFB method are verified by empirical research.

12th International Workshop on Machine Learning in Medical Imaging, MLMI 2021, held in conjunction with 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 -- 27 September 2021 through 27 September 2021 -- -- 266089, Visual explanation methods have an important role in the prognosis of the patients where the annotated data is limited or unavailable. There have been several attempts to use gradient-based attribution methods to localize pathology from medical scans without using segmentation labels. This research direction has been impeded by the lack of robustness and reliability. These methods are highly sensitive to the network parameters. In this study, we introduce a robust visual explanation method to address this problem for medical applications. We provide an innovative visual explanation algorithm for general purpose and as an example application we demonstrate its effectiveness for quantifying lesions in the lungs caused by the Covid-19 with high accuracy and robustness without using dense segmentation labels. This approach overcomes the drawbacks of commonly used Grad-CAM and its extended versions. The premise behind our proposed strategy is that the information flow is minimized while ensuring the classifier prediction stays similar. Our findings indicate that the bottleneck condition provides a more stable severity estimation than the similar attribution methods. The source code will be publicly available upon publication. © 2021, Springer Nature Switzerland AG.

This paper chooses car travel and bus travel as the research objects, establishes a dual-mode equilibrium model based on the bottleneck model, and compares the travel characteristics of the no-toll and fine-toll schemes. We find that the fine-toll scheme can eliminate the queuing time at the bottleneck, but it also increases the congestion risk cost of bus travel. In order to eliminate the queuing time at the bottleneck and reduce the congestion risk cost of bus travel without increasing the car travel cost and bus travel cost, we propose an optimization scheme of fine toll and bus departure quantity and analyze its travel characteristics theoretically. Through the numerical example, we calculate and analyze the equilibrium results of no-toll scheme, fine-toll scheme, and optimization scheme of fine toll and bus departure quantity. The results indicate that the optimization scheme of fine toll and bus departure quantity can help travelers to choose a reasonable travel mode and travel time to travel in the rush hour.

This paper presents a DNN bottleneck reinforcement scheme to alleviate the vulnerability of Deep Neural Networks (DNN) against adversarial attacks. Typical DNN classifiers encode the input image into a compressed latent representation more suitable for inference. This information bottleneck makes a trade-off between the image-specific structure and class-specific information in an image. By reinforcing the former while maintaining the latter, any redundant information, be it adversarial or not, should be removed from the latent representation. Hence, this paper proposes to jointly train an auto-encoder (AE) sharing the same encoding weights with the visual classifier. In order to reinforce the information bottleneck, we introduce the multi-scale low-pass objective and multi-scale high-frequency communication for better frequency steering in the network. Unlike existing approaches, our scheme is the first reforming defense per se which keeps the classifier structure untouched without appending any pre-processing head and is trained with clean images only. Extensive experiments on MNIST, CIFAR-10 and ImageNet demonstrate the strong defense of our method against various adversarial attacks., Comment: ACM MM 2020 - Full Paper

Urban traffic scenario is a combination of different traffic elements and their interactions. Road, flow, and management are three subsystems which are considered in the present scenario. Traffic problems such as traffic congestion are mainly due to the uncoordinated behavior of these three systems. In this study, three different bottlenecks are analyzed for Human capital hour loss, Delay, and Occupancy for the bottleneck. Based on the result obtained different models like ANN and SVM are developed using MATLAB and R-Studios, These Machine learning models are developed to analyze the bottleneck condition and predict the delay, human capital hour loss, and percent occupancy. The utility equations were developed with the SVM model so that it can predict the delay, human capital hour losses, and percent occupancies. Based on the performance measures from the models it can concluded that the ANN model gives high accuracy when compared to the SVM model.

Deep neural networks are a key component of behavior prediction and motion generation for self-driving cars. One of their main drawbacks is a lack of transparency: they should provide easy to interpret rationales for what triggers certain behaviors. We propose an architecture called Attentional Bottleneck with the goal of improving transparency. Our key idea is to combine visual attention, which identifies what aspects of the input the model is using, with an information bottleneck that enables the model to only use aspects of the input which are important. This not only provides sparse and interpretable attention maps (e.g. focusing only on specific vehicles in the scene), but it adds this transparency at no cost to model accuracy. In fact, we find slight improvements in accuracy when applying Attentional Bottleneck to the ChauffeurNet model, whereas we find that the accuracy deteriorates with a traditional visual attention model.

Estimating the effects of an intervention from high-dimensional observational data is a challenging problem due to the existence of confounding. The task is often further complicated in healthcare applications where a set of observations may be entirely missing for certain patients at test time, thereby prohibiting accurate inference. In this paper, we address this issue using an approach based on the information bottleneck to reason about the effects of interventions. To this end, we first train an information bottleneck to perform a low-dimensional compression of covariates by explicitly considering the relevance of information for treatment effects. As a second step, we subsequently use the compressed covariates to perform a transfer of relevant information to cases where data are missing during testing. In doing so, we can reliably and accurately estimate treatment effects even in the absence of a full set of covariate information at test time. Our results on two causal inference benchmarks and a real application for treating sepsis show that our method achieves state-of-the-art performance, without compromising interpretability.

Are simultaneous interpreters subject to the central processing bottleneck, which can postpone the reaction time and impair the performance of another concurrent task, during word production? Moreover, is there any difference between interpreters, bilinguals and monolinguals in the word production bottleneck? In this study, professional simultaneous interpreters, proficient bilinguals and monolinguals performed a dual task consisting of a picture naming task in sentence context (Task 1) and a pitch tone discrimination task (Task 2). The results show that interpreters are also subject to the central processing bottleneck during word production, and there is no significant difference between the three groups in the duration of the word production bottleneck. Unexpected differences in performance were found between English–Asian language and English–European language pairs within the interpreter group, but not within the bilingual group, showing that European-language interpreters were as fast as monolinguals in lexical access, and faster than Asian-language interpreters and bilinguals.

While Radio Frequency Machine Learning (RFML) is expected to be a key enabler of future wireless standards, a significant challenge to the widespread adoption of RFML techniques is the lack of explainability in deep learning models. This work investigates the use of Concept Bottleneck (CB) models as a means to provide inherent decision explanations in the context of deep learning (DL)-based Automatic Modulation Classification (AMC). Results show that the proposed approach not only meets the performance of single-network DL-based AMC algorithms, but provides the desired model explainability and shows potential for classifying modulation schemes not seen during training (i.e. zero-shot learning).

Context-aware neural machine translation (NMT) remains challenging due to the lack of large-scale document-level parallel corpora. To break the corpus bottleneck, in this paper we aim to improve context-aware NMT by taking the advantage of the availability of both large-scale sentence-level parallel dataset and source-side monolingual documents. To this end, we propose two pre-training tasks. One learns to translate a sentence from source language to target language on the sentence-level parallel dataset while the other learns to translate a document from deliberately noised to original on the monolingual documents. Importantly, the two pre-training tasks are jointly and simultaneously learned via the same model, thereafter fine-tuned on scale-limited parallel documents from both sentence-level and document-level perspectives. Experimental results on four translation tasks show that our approach significantly improves translation performance. One nice property of our approach is that the fine-tuned model can be used to translate both sentences and documents.

In the domain of big data analytics, the bottleneck of converting storage-focused file formats to in-memory data structures has shifted from the bandwidth of storage to the performance of decoding and decompression software. Two widely used formats for big data storage and in-memory data are Apache Parquet and Apache Arrow, respectively. In order to improve the speed at which data can be loaded from disk to memory, we propose an FPGA accelerator design that converts Parquet files to Arrow in-memory data structures. We describe an extensible, publicly available, free and open-source implementation of the proposed converter that supports various Parquet file configurations. The performance of the converter is measured on an AWS EC2 F1 system and on a POWER9 system using the recently released OpenCAPI interface. A single instance of the converter can reach between 6 and 12 GB/s of end-to-end throughput, and shows up to a threefold improvement over the fastest single-thread CPU implementation. It has a low resource utilization (less than 5% for all types of FPGA resources). This allows scaling out the design to match the bandwidth of the coming generation of accelerator interfaces. The proposed design and implementation can be extended to support more of the many possible Parquet file configurations.

This work deals with identification of the three main Scandinavian languages (Swedish, Danish and Norwegian) from spoken data. For this purpose, various state-of-the-art approaches are adopted, compared and combined, including i-vectors, deep neural networks (DNNs), bottleneck features (BTNs) as well as x-vectors. The best resulting approaches take advantage of multilingual BTNs and allow us to identify the target languages in speech segments lasting 5 s with a very low error rate around 1%. Therefore, they have many practical applications, such as in systems for transcription of Scandinavian TV and radio programs, where different persons speaking any of the target languages may occur. Within identification of Norwegian, we also focus on an unexplored sub-task of distinguishing between Bokmal and Nynorsk. Our results show that this problem is much harder to solve since these two language variants are acoustically very similar to each other: the best error rate achieved in this case is around 20%.

Large-scale deep neural networks (DNNs) such as convolutional neural networks (CNNs) have achieved impressive performance in audio classification for their powerful capacity and strong generalization ability. However, when training a DNN model on low-resource tasks, it is usually prone to overfitting the small data and learning too much redundant information. To address this issue, we propose to use variational information bottleneck (VIB) to mitigate overfitting and suppress irrelevant information. In this work, we conduct experiments ona 4-layer CNN. However, the VIB framework is ready-to-use and could be easily utilized with many other state-of-the-art network architectures. Evaluation on a few audio datasets shows that our approach significantly outperforms baseline methods, yielding more than 5.0% improvement in terms of classification accuracy in some low-source settings., Comment: Accepted by InterSpeech 2021

Many unsupervised hashing methods are implicitly established on the idea of reconstructing the input data, which basically encourages the hashing codes to retain as much information of original data as possible. However, this requirement may force the models spending lots of their effort on reconstructing the unuseful background information, while ignoring to preserve the discriminative semantic information that is more important for the hashing task. To tackle this problem, inspired by the recent success of contrastive learning in learning continuous representations, we propose to adapt this framework to learn binary hashing codes. Specifically, we first propose to modify the objective function to meet the specific requirement of hashing and then introduce a probabilistic binary representation layer into the model to facilitate end-to-end training of the entire model. We further prove the strong connection between the proposed contrastive-learning-based hashing method and the mutual information, and show that the proposed model can be considered under the broader framework of the information bottleneck (IB). Under this perspective, a more general hashing model is naturally obtained. Extensive experimental results on three benchmark image datasets demonstrate that the proposed hashing method significantly outperforms existing baselines., Comment: IJCAI 2021

In this position paper we argue that the best way to overcome the notorious knowledge bottleneck in AI is using lifelong learning by social intelligent agents. Keys to this capability are deep language understanding, dialog interaction, sufficiently broad-coverage and fine-grain knowledge bases to bootstrap the learning process, and the agent’s operation within a comprehensive cognitive architecture.

Forecasting short-term, nonrecurring traffic dynamics caused by incidents is an essential capability in the Intelligent Transportation Systems. This research proposes a prediction framework in which Conditional Deep Convolutional Generative Adversarial Nets (C-DCGAN) is trained to predict the traffic spillbacks patterns associated with freeway incidents at merging bottleneck. Speed tensors, which depict the spatiotemporal incident-induced impacts for multiple neighboring routes, is a suitable object for the GAN model to understand and predict. Further, we demonstrated how to use the mesoscopic Dynamic Traffic Assignment (DTA) model DynusT to generate a large number of training data, thus speeding up the model training. The developed model achieves both statistical and spatial similarities between predicted speed tensors and actual tensors, to 83.84%. To the best of our knowledge, this line of work is one of the first attempts in the literature to train the Machine Learning model to predict speed tensor representation of multi-location incident-induced spatiotemporal impact at merging bottleneck and speeding up the training via simulation.

Word Sense Disambiguation (WSD) is the task of identifying the meaning of a word in a given context. It lies at the base of Natural Language Processing as it provides semantic information for words. In the last decade, great strides have been made in this field and much effort has been devoted to mitigate the knowledge acquisition bottleneck problem, i.e., the problem of semantically annotating texts at a large scale and in different languages. This issue is ubiquitous in WSD as it hinders the creation of both multilingual knowledge bases and manually-curated training sets. In this work, we first introduce the reader to the task of WSD through a short historical digression and then take the stock of the advancements to alleviate the knowledge acquisition bottleneck problem. In that, we survey the literature on manual, semi-automatic and automatic approaches to create English and multilingual corpora tagged with sense annotations and present a clear overview over supervised models for WSD. Finally, we provide our view over the future directions that we foresee for the field.

The variety of event categories and event boundary information have resulted in limited success for acoustic event detection systems. To deal with this, we propose to utilize the long contextual information, low-dimensional discriminant global bottleneck features and category-specific bottleneck features. By concatenating several adjacent frames together, the use of contextual information makes it easier to cope with acoustic signals with long duration. Global and category-specific bottleneck features can extract the prior knowledge of the event category and boundary, which is ideally matched by the task of an event detection system. Evaluations on the UPC-TALP and ITC-IRST databases of highly variable acoustic events demonstrate the effectiveness of the proposed approaches by achieving a 5.30% and 4.44% absolute error rate improvement respectively compared to the state of art technique.