Search

Neural network quantization has shown to be an effective way for network compression and acceleration. However, existing binary or ternary quantization methods suffer from two major issues. First, low bit-width input/activation quantization easily results in severe prediction accuracy degradation. Second, network training and quantization are always treated as two non-related tasks, leading to accumulated parameter training error and quantization error. In this work, we introduce a novel scheme, named Residual Quantization, to train a neural network with both weights and inputs constrained to low bit-width, e.g., binary or ternary values. On one hand, by recursively performing residual quantization, the resulting binary/ternary network is guaranteed to approximate the full-precision network with much smaller errors. On the other hand, we mathematically re-formulate the network training scheme in an EM-like manner, which iteratively performs network quantization and parameter optimization. During expectation, the low bit-width network is encouraged to approximate the full-precision network. During maximization, the low bit-width network is further tuned to gain better representation capability. Extensive experiments well demonstrate that the proposed quantization scheme outperforms previous low bit-width methods and achieves much closer performance to the full-precision counterpart. [ABSTRACT FROM AUTHOR]

Given a language query, temporal grounding task is to localize temporal boundaries of the described event in an untrimmed video. There is a long-standing challenge that multiple moments may be associated with one same video-query pair, termed label uncertainty. However, existing methods struggle to localize diverse moments due to the lack of multi-label annotations. In this paper, we propose a novel Diverse Temporal Grounding framework (DTG) to achieve diverse moment localization with only single-label annotations. By delving into the label uncertainty, we find the diverse moments retrieved tend to involve similar actions/objects, driving us to perceive these interest moments. Specifically, we construct soft multi-label through semantic similarity of multiple video-query pairs. These soft labels reveal whether multiple moments in the intra-videos contain similar verbs/nouns, thereby guiding interest moment generation. Meanwhile, we put forward a diverse moment regression network (DMRNet) to achieve multiple predictions in a single pass, where plausible moments are dynamically picked out from the interest moments for joint optimization. Moreover, we introduce new metrics that better reveal multi-output performance. Extensive experiments conducted on Charades-STA and ActivityNet Captions show that our method achieves state-of-the-art performance in terms of both standard and new metrics. [ABSTRACT FROM AUTHOR]

Mobile services are becoming more diverse, making them have different demands on communications, computing, and caching (3C) resources in mobile systems. Unlike the traditional work that considers only one type of service, this paper designs a unified framework to characterize the different kinds of services, and jointly optimizes the 3C resources of the base station (BS) and mobile devices to provide differentiated quality of service (QoS) for diverse services. In the proposed framework, we model the task required by the mobile device to be generated at the BS, the mobile device, or both of them, which means the requested tasks are served through different paths, consuming different bandwidth, computing and caching resources. Since diverse services have different QoS, we formulate a multi-objective programming (MOP) to optimize the allocation of the 3C resources for minimizing the total delay while maximizing the number of executed tasks requested by the mobile devices. We transform the MOP problem as a multi-objective Markov decision process (MO-MDP) and design a multi-objective proximal policy optimization (MO-PPO) algorithm to solve the MO-MDP. The proposed MO-PPO first trains two sub-policies separately for the two objectives, and then combines them to search for Pareto dominating solutions. By alternately perform the separate training and the combination, we can finally obtain a set of Pareto optimal solutions and the corresponding Pareto front. Simulation results show that the proposed MO-PPO outperforms traditional methods in finding a higher-quality set of Pareto optimal solutions and can more appropriately allocate 3C resources to different types of services. [ABSTRACT FROM AUTHOR]

Mobile edge computing (MEC) which enables communication, computation and storage capabilities at the network edge, is an emerging paradigm for 5G and beyond 5G networks. Resource-limited mobile devices can offload their tasks to the MEC server to reduce computation latency and save energy consumption. However, limited wireless spectrum resources make it hard for MEC to guarantee low latency when a large number of users offload the computation tasks or download the computed results. Full-duplex (FD) communication that allows simultaneous transmission and reception of signals over the same frequency band is a promising solution to the shortage of spectrum resources. In this paper, we study a multi-user FD-MEC system involving all the phases of uploading, computing, and downloading. With the help of an FD-capable base station (BS), two half-duplex (HD) users can form an FD pair and share the same time-frequency resource for task uploading and result downloading. We formulate a latency minimization problem considering power control, time scheduling, offloading decision, and user pairing strategy, which is a mixed-integer non-linear programming (MINLP) problem. The problem can be decomposed into two subproblems. The first subproblem is delay minimization for each possible FD pair, and the second one is to decide the offloading and user pairing strategy, and the user scheduling order. We analyze the problem in two cases: (1) the computing delay at the edge server is negligible and (2) the computing delay at the edge server is non-negligible. For both cases, the first subproblem is solved using a bisection algorithm. Then, we solve the second subproblem in the two cases by a relaxing-based algorithm and a concave-convex procedure (CCCP)-based algorithm, respectively. Finally, numerical results demonstrate that with our proposed strategies, the overall latency can be significantly reduced by integrating FD communication with the MEC system. [ABSTRACT FROM AUTHOR]

In this paper, we investigate a joint device activity detection (DAD), channel estimation (CE), and data decoding (DD) algorithm for multiple-input multiple-output (MIMO) massive unsourced random access (URA). Different from the state-of-the-art slotted transmission scheme, the data in the proposed framework is split into only two parts. A portion of the data is coded by compressed sensing (CS) and the rest is low-density-parity-check (LDPC) coded. In addition to being part of the data, information bits in the CS phase also undertake the task of interleaving pattern design and CE. The principle of interleave-division multiple access (IDMA) is exploited to reduce the interference among devices in the LDPC phase. Based on the belief propagation (BP) algorithm, a low-complexity iterative message passing (MP) algorithm is utilized to decode the data embedded in these two phases separately. Moreover, combined with successive interference cancellation (SIC), the proposed joint DAD-CE-DD algorithm is performed to further improve performance by utilizing the belief of each other. Additionally, based on the energy detection (ED) and sliding window protocol (SWP), we develop a collision resolution protocol to handle the codeword collision, a common issue in the URA system. In addition to the complexity reduction, the proposed algorithm exhibits a substantial performance enhancement compared to the state-of-the-art in terms of efficiency and accuracy. [ABSTRACT FROM AUTHOR]

Advances in sensing and communication technologies have enabled emerging applications in ocean research. The stringent resource limitations, however, hinder local task processing on maritime devices. Cloud-based task offloading is also not feasible due to the unaffordable costs induced by satellite links. In this work, we propose a visionary architecture on software-defined maritime fog computing (SD-MFC), which adopts fog computing to provide localized services and uses software-defined networking technology to provide the abstraction of underlying resources. In SD-MFC, coastal infrastructures and large-size vessels are turned into maritime fog nodes to process the tasks generated by surrounding maritime devices. By exploration of a real-world vessel trajectory dataset, the feasibility and effectiveness of the proposed SD-MFC are verified. [ABSTRACT FROM AUTHOR]

Learning to generate continuous linguistic descriptions for multi-subject interactive videos in great details has particular applications in team sports auto-narrative. In contrast to traditional video caption, this task is more challenging as it requires simultaneous modeling of fine-grained individual actions, uncovering of spatio-temporal dependency structures of frequent group interactions, and then accurate mapping of these complex interaction details into long and detailed commentary. To explicitly address these challenges, we propose a novel framework Graph-based Learning for Multi-Granularity Interaction Representation (GLMGIR) for fine-grained team sports auto-narrative task. A multi-granular interaction modeling module is proposed to extract among-subjects’ interactive actions in a progressive way for encoding both intra- and inter-team interactions. Based on the above multi-granular representations, a multi-granular attention module is developed to consider action/event descriptions of multiple spatio-temporal resolutions. Both modules are integrated seamlessly and work in a collaborative way to generate the final narrative. In the meantime, to facilitate reproducible research, we collect a new video dataset from YouTube.com called Sports Video Narrative dataset (SVN). It is a novel direction as it contains $6K$ 6 K team sports videos (i.e., NBA basketball games) with $10K$ 10 K ground-truth narratives(e.g., sentences). Furthermore, as previous metrics such as METEOR (i.e., used in coarse-grained video caption task) DO NOT cope with fine-grained sports narrative task well, we hence develop a novel evaluation metric named Fine-grained Captioning Evaluation (FCE), which measures how accurate the generated linguistic description reflects fine-grained action details as well as the overall spatio-temporal interactional structure. Extensive experiments on our SVN dataset have demonstrated the effectiveness of the proposed framework for fine-grained team sports video auto-narrative. [ABSTRACT FROM AUTHOR]

5G and edge computing have brought great changes to video industry. Interactive video is becoming an emerging application form of multimedia service, which provides attraction beyond typical scenarios like cloud gaming and remote virtual reality (VR), and puts forward great challenges in resource capacity, response latency, and function flexibility to its service system. In this paper, we propose an elastic system architecture with low latency features to accommodate generic interactive video applications on near user edges. To increase system flexibility, we firstly design a dynamic Directed Acyclic Graph (dDAG) model for efficient task representation. Secondly, based on the model, we present the elastic architecture together with its scalable workflow pipeline. Thirdly, we propose a set of novel latency measurement metrics to analyze and optimize the performance of an interactive video system. Based on the proposed approaches, we disassemble a real world free-viewpoint synthesis application and benchmark its performance with the metrics. Extensive experimental results show the flexibility of our system to handle the stochastic human interactions during a video service session, with less than 5 ms additional scheduling latency introduced. End to end latency is kept within 43 ms for complex functions, and 28 ms for simpler scenarios, which satisfies the restrictions of most interactive video applications provided by an edge. Client of the architecture serves as a pure video player, which is also friendly to power limited terminals such as 5G phones. Efficiency and stability analyses of the system show superiorities over existing work, and also reveal potential optimization directions for future research. [ABSTRACT FROM AUTHOR]

Scale variation of pedestrians in a crowd image presents a significant challenge for vision-based people counting systems. Such variations are mainly caused by perspective-related distortions due to the camera pose relative to the ground plane. Following the density-based counting paradigm, we postulate that generating density values adaptive to object scales plays a critical role in the accuracy of the final counting results. Motivated by this, we distill the underlying information from depth cues to obtain scale-aware representations that can respond to object scales considering the fact that the scale is inversely proportional to the object depth. Specifically, we propose a depth embedding module as add-ons into existing networks. This module exploits essential depth cues to spatially re-calibrate the magnitude of the original features. In this way, the objects, although in the same class, will attain distinct representations according to their scales, which directly benefits the estimation of scale-aware density values. We conduct a comprehensive analysis of the effects of the depth embedding module and validate that exploiting depth cues to perceive object scale variations in convolutional neural networks improves crowd counting performances. Our experiments demonstrate the effectiveness of the proposed approach on four popular benchmark datasets. [ABSTRACT FROM AUTHOR]