Search

Robust and imperceptible adversarial video attack is challenging due to the spatial and temporal characteristics of videos. The existing video adversarial attack methods mainly take a gradient-based approach and generate adversarial videos with noticeable perturbations. In this paper, we propose a novel Sparse Adversarial Video Attack via Spatio-Temporal Invertible Neural Networks (SVASTIN) to generate adversarial videos through spatio-temporal feature space information exchanging. It consists of a Guided Target Video Learning (GTVL) module to balance the perturbation budget and optimization speed and a Spatio-Temporal Invertible Neural Network (STIN) module to perform spatio-temporal feature space information exchanging between a source video and the target feature tensor learned by GTVL module. Extensive experiments on UCF-101 and Kinetics-400 demonstrate that our proposed SVASTIN can generate adversarial examples with higher imperceptibility than the state-of-the-art methods with the higher fooling rate. Code is available at \href{https://github.com/Brittany-Chen/SVASTIN}{https://github.com/Brittany-Chen/SVASTIN}.

A vertex transitive graph $\Gamma$ is said to be $2$-distance transitive if for each vertex $u$, the group of automorphisms of $\Gamma$ fixing the vertex $u$ acts transitively on the set of vertices at distance $1$ and $2$ from $u$, while $\Gamma$ is said to be $2$-arc transitive if its automorphism group is transitive on the set of $2$-arcs. Then $2$-arc transitive graphs are $2$-distance transitive. The classification of $2$-arc transitive Cayley graphs on dihedral groups was given by Du, Malni\v{c} and Maru\v{s}i\v{c} in [Classification of 2-arc-transitive dihedrants, J. Combin. Theory Ser. B 98 (2008), 1349--1372]. In this paper, it is shown that a connected 2-distance transitive Cayley graph on the dihedral group of order $2n$ is either $2$-arc transitive, or isomorphic to the complete multipartite graph $K_{m[b]}$ for some $m\geq3$ and $b\geq2$ with $mb=2n$., Comment: 21

The existing image steganography methods either sequentially conceal secret images or conceal a concatenation of multiple images. In such ways, the interference of information among multiple images will become increasingly severe when the number of secret images becomes larger, thus restrict the development of very large capacity image steganography. In this paper, we propose an Invertible Mosaic Image Hiding Network (InvMIHNet) which realizes very large capacity image steganography with high quality by concealing a single mosaic secret image. InvMIHNet consists of an Invertible Image Rescaling (IIR) module and an Invertible Image Hiding (IIH) module. The IIR module works for downscaling the single mosaic secret image form by spatially splicing the multiple secret images, and the IIH module then conceal this mosaic image under the cover image. The proposed InvMIHNet successfully conceal and reveal up to 16 secret images with a small number of parameters and memory consumption. Extensive experiments on ImageNet-1K, COCO and DIV2K show InvMIHNet outperforms state-of-the-art methods in terms of both the imperceptibility of stego image and recover accuracy of secret image.

Finite Rate of Innovation (FRI) sampling theory enables reconstruction of classes of continuous non-bandlimited signals that have a small number of free parameters from their low-rate discrete samples. This task is often translated into a spectral estimation problem that is solved using methods involving estimating signal subspaces, which tend to break down at a certain peak signal-to-noise ratio (PSNR). To avoid this breakdown, we consider alternative approaches that make use of information from labelled data. We propose two model-based learning methods, including deep unfolding the denoising process in spectral estimation, and constructing an encoder-decoder deep neural network that models the acquisition process. Simulation results of both learning algorithms indicate significant improvements of the breakdown PSNR over classical subspace-based methods. While the deep unfolded network achieves similar performance as the classical FRI techniques and outperforms the encoder-decoder network in the low noise regimes, the latter allows to reconstruct the FRI signal even when the sampling kernel is unknown. We also achieve competitive results in detecting pulses from in vivo calcium imaging data in terms of true positive and false positive rate while providing more precise estimations., Comment: Accepted version; Code available at: https://github.com/vchleung/LearningBasedFRI

Macro X-ray Fluorescence (MA-XRF) scanning is increasingly widely used by researchers in heritage science to analyse easel paintings as one of a suite of non-invasive imaging techniques. The task of processing the resulting MA-XRF datacube generated in order to produce individual chemical element maps is called MA-XRF deconvolution. While there are several existing methods that have been proposed for MA-XRF deconvolution, they require a degree of manual intervention from the user that can affect the final results. The state-of-the-art AFRID approach can automatically deconvolute the datacube without user input, but it has a long processing time and does not exploit spatial dependency. In this paper, we propose two versions of a fast automatic deconvolution (FAD) method for MA-XRF datacubes collected from easel paintings with ADMM (alternating direction method of multipliers) and FISTA (fast iterative shrinkage-thresholding algorithm). The proposed FAD method not only automatically analyses the datacube and produces element distribution maps of high-quality with spatial dependency considered, but also significantly reduces the running time. The results generated on the MA-XRF datacubes collected from two easel paintings from the National Gallery, London, verify the performance of the proposed FAD method., Comment: 13 pages, 13 figures

A vertex triple $(u,v,w)$ of a graph is called a $2$-geodesic if $v$ is adjacent to both $u$ and $w$ and $u$ is not adjacent to $w$. A graph is said to be $2$-geodesic transitive if its automorphism group is transitive on the set of $2$-geodesics. In this paper, a complete classification of $2$-geodesic transitive graphs of order $p^n$ is given for each prime $p$ and $n\leq 3$. It turns out that all such graphs consist of three small graphs: the complete bipartite graph $K_{4,4}$ of order $8$, the Schl\"{a}fli graph of order $27$ and its complement, and fourteen infinite families: the cycles $C_p, C_{p^2}$ and $C_{p^3}$, the complete graphs $K_p, K_{p^2}$ and $K_{p^3}$, the complete multipartite graphs $K_{p[p]}$, $K_{p[p^2]}$ and $K_{p^2[p]}$, the Hamming graph $H(2,p)$ and its complement, the Hamming graph $H(3,p)$, and two infinite families of normal Cayley graphs on extraspecial group of order $p^3$ and exponent $p$., Comment: 27 pages

Single image reflection removal problem aims to divide a reflection-contaminated image into a transmission image and a reflection image. It is a canonical blind source separation problem and is highly ill-posed. In this paper, we present a novel deep architecture called deep unfolded single image reflection removal network (DURRNet) which makes an attempt to combine the best features from model-based and learning-based paradigms and therefore leads to a more interpretable deep architecture. Specifically, we first propose a model-based optimization with transform-based exclusion prior and then design an iterative algorithm with simple closed-form solutions for solving each sub-problems. With the deep unrolling technique, we build the DURRNet with ProxNets to model natural image priors and ProxInvNets which are constructed with invertible networks to impose the exclusion prior. Comprehensive experimental results on commonly used datasets demonstrate that the proposed DURRNet achieves state-of-the-art results both visually and quantitatively.

In this paper, we focus on X-ray images of paintings with concealed sub-surface designs (e.g., deriving from reuse of the painting support or revision of a composition by the artist), which include contributions from both the surface painting and the concealed features. In particular, we propose a self-supervised deep learning-based image separation approach that can be applied to the X-ray images from such paintings to separate them into two hypothetical X-ray images. One of these reconstructed images is related to the X-ray image of the concealed painting, while the second one contains only information related to the X-ray of the visible painting. The proposed separation network consists of two components: the analysis and the synthesis sub-networks. The analysis sub-network is based on learned coupled iterative shrinkage thresholding algorithms (LCISTA) designed using algorithm unrolling techniques, and the synthesis sub-network consists of several linear mappings. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The proposed method is demonstrated on a real painting with concealed content, Do\~na Isabel de Porcel by Francisco de Goya, to show its effectiveness.

Image denoising aims to restore a clean image from an observed noisy image. The model-based image denoising approaches can achieve good generalization ability over different noise levels and are with high interpretability. Learning-based approaches are able to achieve better results, but usually with weaker generalization ability and interpretability. In this paper, we propose a wavelet-inspired invertible network (WINNet) to combine the merits of the wavelet-based approaches and learningbased approaches. The proposed WINNet consists of K-scale of lifting inspired invertible neural networks (LINNs) and sparsity-driven denoising networks together with a noise estimation network. The network architecture of LINNs is inspired by the lifting scheme in wavelets. LINNs are used to learn a non-linear redundant transform with perfect reconstruction property to facilitate noise removal. The denoising network implements a sparse coding process for denoising. The noise estimation network estimates the noise level from the input image which will be used to adaptively adjust the soft-thresholds in LINNs. The forward transform of LINNs produce a redundant multi-scale representation for denoising. The denoised image is reconstructed using the inverse transform of LINNs with the denoised detail channels and the original coarse channel. The simulation results show that the proposed WINNet method is highly interpretable and has strong generalization ability to unseen noise levels. It also achieves competitive results in the non-blind/blind image denoising and in image deblurring.

Intelligent video summarization algorithms allow to quickly convey the most relevant information in videos through the identification of the most essential and explanatory content while removing redundant video frames. In this paper, we introduce the 3DST-UNet-RL framework for video summarization. A 3D spatio-temporal U-Net is used to efficiently encode spatio-temporal information of the input videos for downstream reinforcement learning (RL). An RL agent learns from spatio-temporal latent scores and predicts actions for keeping or rejecting a video frame in a video summary. We investigate if real/inflated 3D spatio-temporal CNN features are better suited to learn representations from videos than commonly used 2D image features. Our framework can operate in both, a fully unsupervised mode and a supervised training mode. We analyse the impact of prescribed summary lengths and show experimental evidence for the effectiveness of 3DST-UNet-RL on two commonly used general video summarization benchmarks. We also applied our method on a medical video summarization task. The proposed video summarization method has the potential to save storage costs of ultrasound screening videos as well as to increase efficiency when browsing patient video data during retrospective analysis or audit without loosing essential information

In this paper, we propose an invertible neural network for image denoising (DnINN) inspired by the transform-based denoising framework. The proposed DnINN consists of an invertible neural network called LINN whose architecture is inspired by the lifting scheme in wavelet theory and a sparsity-driven denoising network which is used to remove noise from the transform coefficients. The denoising operation is performed with a single soft-thresholding operation or with a learned iterative shrinkage thresholding network. The forward pass of LINN produces an over-complete representation which is more suitable for denoising. The denoised image is reconstructed using the backward pass of LINN using the output of the denoising network. The simulation results show that the proposed DnINN method achieves results comparable to the DnCNN method while only requiring 1/4 of learnable parameters., Comment: Accepted by the 29th European Signal Processing Conference, EUSIPCO 2021

In this paper, we construct an infinite family of normal Cayley graphs, which are $2$-distance-transitive but neither distance-transitive nor $2$-arc-transitive. This answers a question raised by Chen, Jin and Li in 2019 and corrects a claim in a literature given by Pan, Huang and Liu in 2015., Comment: 9 pages

The lattice Boltzmann method (LBM) has gained increasing popularity in incompressible viscous flow simulations, but it uses many more variables than necessary. This defect was overcome by a recent approach that solves the more actual macroscopic equations obtained through Taylor series expansion analysis of the lattice Boltzmann equations [Lu et al., J. Comp. Phys., 415, 109546 (2020)]. The key is to keep some small additional terms (SATs) to stabilize the numerical solution of the weakly compressible Navier-Stokes equations. However, there are many SATs that complicate the implementation of their method. Based on some analyses and numerous tests, we ultimately pinpoint two essential ingredients for stable simulations: (1) suitable density (pressure) diffusion added to the continuity equation; (2) proper numerical dissipation related to the velocity divergence added to the momentum equations. Then, we propose a simplified method that is not only easier to implement but noticeably faster than the original method and the LBM. It contains much simpler SATs that only involve the density (pressure) derivatives and it requires no intermediate steps or variables. Besides, it is extended for two-phase flows with uniform density and viscosity. Several test cases, including some two-phase problems under two dimensional, axisymmetric and three dimensional geometries, are presented to demonstrate its capability. This work may help pave the way for the simplest simulation of incompressible viscous flows on collocated grids based on the artificial compressibility methodology.

The general pressure equation (GPE) is a new method proposed recently by Toutant (J. Comput. Phys., 374:822-842 (2018)) for incompressible flow simulation. It circumvents the Poisson equation for the pressure and performs better than the classical artificial compressibility method. Here it is generalized for two-phase incompressible viscous flows with variable density and viscosity. First, the pressure evolution equation is modified to account for the density variation. Second, customized discretizations are proposed to deal with the viscous stress terms with variable viscosity. Besides, additional terms related to the bulk viscosity are included to stabilize the simulation. The interface evolution and surface tension effects are handled by a phase-field model coupled with the GPE-based flow equations. The pressure and momentum equations are discretized on a stagger grid using the second order centered scheme and marched in time using the third order total variation diminishing Runge-Kutta scheme. Several unsteady two-phase problems in two dimensional, axisymmetric and three dimensional geometries at intermediate density and viscosity ratios were simulated and the results agreed well with those obtained by other incompressible solvers and/or the lattice-Boltzmann method (LBM) simulations. Similar to the LBM, the proposed GPE-based method is fully explicit and easy to be parallelized. Although slower than the LBM, it requires much less memory than the LBM. Thus, it can be a good alternative to simulate two-phase flows with limited memory resource.

In this paper, we propose a novel solution for non-convex problems of multiple variables, especially for those typically solved by an alternating minimization (AM) strategy that splits the original optimization problem into a set of sub-problems corresponding to each variable, and then iteratively optimize each sub-problem using a fixed updating rule. However, due to the intrinsic non-convexity of the original optimization problem, the optimization can usually be trapped into spurious local minimum even when each sub-problem can be optimally solved at each iteration. Meanwhile, learning-based approaches, such as deep unfolding algorithms, are highly limited by the lack of labelled data and restricted explainability. To tackle these issues, we propose a meta-learning based alternating minimization (MLAM) method, which aims to minimize a partial of the global losses over iterations instead of carrying minimization on each sub-problem, and it tends to learn an adaptive strategy to replace the handcrafted counterpart resulting in advance on superior performance. Meanwhile, the proposed MLAM still maintains the original algorithmic principle, which contributes to a better interpretability. We evaluate the proposed method on two representative problems, namely, bi-linear inverse problem: matrix completion, and non-linear problem: Gaussian mixture models. The experimental results validate that our proposed approach outperforms AM-based methods in standard settings, and is able to achieve effective optimization in challenging cases while other comparing methods would typically fail.

In this paper, we introduce a Deep Convolutional Analysis Dictionary Model (DeepCAM) by learning convolutional dictionaries instead of unstructured dictionaries as in the case of deep analysis dictionary model introduced in the companion paper. Convolutional dictionaries are more suitable for processing high-dimensional signals like for example images and have only a small number of free parameters. By exploiting the properties of a convolutional dictionary, we present an efficient convolutional analysis dictionary learning approach. A L-layer DeepCAM consists of L layers of convolutional analysis dictionary and element-wise soft-thresholding pairs and a single layer of convolutional synthesis dictionary. Similar to DeepAM, each convolutional analysis dictionary is composed of a convolutional Information Preserving Analysis Dictionary (IPAD) and a convolutional Clustering Analysis Dictionary (CAD). The IPAD and the CAD are learned using variations of the proposed learning algorithm. We demonstrate that DeepCAM is an effective multilayer convolutional model and, on single image super-resolution, achieves performance comparable with other methods while also showing good generalization capabilities.

Inspired by the recent success of deep neural networks and the recent efforts to develop multi-layer dictionary models, we propose a Deep Analysis dictionary Model (DeepAM) which is optimized to address a specific regression task known as single image super-resolution. Contrary to other multi-layer dictionary models, our architecture contains L layers of analysis dictionary and soft-thresholding operators to gradually extract high-level features and a layer of synthesis dictionary which is designed to optimize the regression task at hand. In our approach, each analysis dictionary is partitioned into two sub-dictionaries: an Information Preserving Analysis Dictionary (IPAD) and a Clustering Analysis Dictionary (CAD). The IPAD together with the corresponding soft-thresholds is designed to pass the key information from the previous layer to the next layer, while the CAD together with the corresponding soft-thresholding operator is designed to produce a sparse feature representation of its input data that facilitates discrimination of key features. DeepAM uses both supervised and unsupervised setup. Simulation results show that the proposed deep analysis dictionary model achieves better performance compared to a deep neural network that has the same structure and is optimized using back-propagation when training datasets are small., Comment: Accepted by IEEE Transactions on Signal Processing

Pedestrian detection methods have been significantly improved with the development of deep convolutional neural networks. Nevertheless, detecting small-scaled pedestrians and occluded pedestrians remains a challenging problem. In this paper, we propose a pedestrian detection method with a couple-network to simultaneously address these two issues. One of the sub-networks, the gated multi-layer feature extraction sub-network, aims to adaptively generate discriminative features for pedestrian candidates in order to robustly detect pedestrians with large variations on scales. The second sub-network targets in handling the occlusion problem of pedestrian detection by using deformable regional RoI-pooling. We investigate two different gate units for the gated sub-network, namely, the channel-wise gate unit and the spatio-wise gate unit, which can enhance the representation ability of the regional convolutional features among the channel dimensions or across the spatial domain, repetitively. Ablation studies have validated the effectiveness of both the proposed gated multi-layer feature extraction sub-network and the deformable occlusion handling sub-network. With the coupled framework, our proposed pedestrian detector achieves state-of-the-art results on the Caltech and the CityPersons pedestrian detection benchmarks.

Pedestrian detection methods have been significantly improved with the development of deep convolutional neural networks. Nevertheless, robustly detecting pedestrians with a large variant on sizes and with occlusions remains a challenging problem. In this paper, we propose a gated multi-layer convolutional feature extraction method which can adaptively generate discriminative features for candidate pedestrian regions. The proposed gated feature extraction framework consists of squeeze units, gate units and a concatenation layer which perform feature dimension squeezing, feature elements manipulation and convolutional features combination from multiple CNN layers, respectively. We proposed two different gate models which can manipulate the regional feature maps in a channel-wise selection manner and a spatial-wise selection manner, respectively. Experiments on the challenging CityPersons dataset demonstrate the effectiveness of the proposed method, especially on detecting those small-size and occluded pedestrians.

Finite Rate of Innovation (FRI) theory considers sampling and reconstruction of classes of non-bandlimited continuous signals that have a small number of free parameters, such as a stream of Diracs. The task of reconstructing FRI signals from discrete samples is often transformed into a spectral estimation problem and solved using Prony's method and matrix pencil method which involve estimating signal subspaces. They achieve an optimal performance given by the Cram\'{e}r-Rao bound yet break down at a certain peak signal-to-noise ratio (PSNR). This is probably due to the so-called subspace swap event. In this paper, we aim to alleviate the subspace swap problem and investigate alternative approaches including directly estimating FRI parameters using deep neural networks and utilising deep neural networks as denoisers to reduce the noise in the samples. Simulations show significant improvements on the breakdown PSNR over existing FRI methods, which still outperform learning-based approaches in medium to high PSNR regimes., Comment: 5 pages, 7 figures, 2020 International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2020)

In this paper, we propose an alternative approach to implement the contact angle boundary condition on immersed surfaces for phase-field simulations of two-phase flows using the Cahn-Hilliard equation on a Cartesian mesh. This simple and effective method was inspired by previous works on the geometric formulation of the wetting boundary condition. In two dimensions, by making full use of the hyperbolic tangent profile of the order parameter, we were able to obtain its unknown value at a ghost point from the information at only one point in the fluid. This is in contrast with previous approaches using interpolations involving several points. The special feature allows this method to be easily implemented on immersed surfaces (including curved ones) that cut through the grid lines. It is verified through the study of two examples: (1) the shape of a drop on a circular cylinder with different contact angles; (2) the spreading of a drop on an embedded inclined wall with a given contact angle.

We studied several wetting boundary conditions (WBCs) in the simulation of binary fluids based on phase-field theory. Five WBCs, three belonging to the surface energy (SE) formulation using the linear, cubic and sine functions (denoted as LinSE, CubSE and SinSE), the fourth using a geometric formulation (Geom), and the fifth using a characteristic interpolation (CI), were compared with each other through the study of several problems: (1) the static contact angle of a drop; (2) a Poiseuille flow-driven liquid column; (3) a wettability gradient (WG)-driven liquid column; (4) drop dewetting. It was found that while all WBCs can predict the static contact angle fairly accurately, they may affect the simulation outcomes of dynamic problems differently, depending on the driving mechanism. For the flow-driven problem, to use different WBCs had almost no effect on the flow characteristics over a large scale. But for other capillarity-driven problems, the WBC had some noticeable effects. For the WG-driven liquid column, Geom gave the most consistent prediction between the drop velocity and dynamic contact angles, and LinSE delivered the poorest prediction in this aspect. Except for Geom, the dynamic contact angle differed from the prescribed (static) one when other WBCs were used. For drop dewetting, Geom led to the most violent drop motion whereas CubSE caused the weakest motion; the initial contact line velocity was also found to be dependent on the WBC. For several problems, CubSE and SinSE gave almost the same results, and those by Geom and CI were close as well, possibly due to similar consideration in their design. Besides various comparisons, a new implementation that may be used for all WBCs was proposed to mimic the wall energy relaxation and control the degree of slip. This new procedure made it possible to allow the simulations to match experimental measurements well.

In this work, the motion of a 2-D drop on a surface with given wettability gradient is studied numerically by a hybrid lattice-Boltzmann finite-difference method using the multiple-relaxation-time collision model. We incorporate the geometric wetting boundary condition that allows accurate implementation of a contact angle hysteresis model. The method is first validated through three benchmark tests, including the layered Poiseuille flow with a viscosity contrast, the motion of a liquid column in a channel with specified wettability gradient and the force balance for a static drop attached to a surface with hysteresis subject to a body force. Then, simulations of a drop on a wall with given wettability gradient are performed under different conditions. The effects of the Reynolds number, the viscosity ratio, the wettability gradient, as well as the contact angle hysteresis on the drop motion are investigated in detail. It is found that the capillary number of the drop in steady state is significantly affected by the viscosity ratio, the magnitudes of the wettability gradient and the contact angle hysteresis, whereas it only shows very weak dependence on the Reynolds number.