Search

Electronic monitoring (EM) is increasingly used to monitor catch and bycatch in wild capture fisheries. EM video data are still manually reviewed and adds to ongoing management costs. Computer vision, machine learning, and artificial intelligence-based systems are seen to be the next step in automating EM data workflows. Here we show some of the obstacles we have confronted and approaches taken as we develop a system to automatically identify and count target and bycatch species using cameras deployed to an industry vessel. A Convolutional Neural Network was trained to detect and classify target and bycatch species groups, and a visual tracking system was developed to produce counts. The multiclass detector achieved a mean average precision of 53.42%. Based on the detection results, the visual tracking system provided automatic fish counts for the test video data. Automatic counts were within two standard deviations of the manual counts for the target species and most times for the bycatch species. Unlike other recent attempts, weather and lighting conditions were largely controlled by mounting cameras under cover.

Audio-visual recognition systems rely on efficient feature extraction. Many spatio-temporal interest point detectors for visual feature extraction are either too sparse, leading to loss of information, or too dense resulting in noisy and redundant information. Furthermore, interest point detectors designed for a controlled environment can be affected by camera motion. In this paper, a salient spatio-temporal interest point detector is proposed based on a low-rank and group-sparse matrix approximation. The detector handles the camera motion through a short-window video stabilization. The multimodal audio-visual features from multiple descriptors are represented by a super descriptor, from which a compact set of features is extracted through a tensor decomposition and feature selection. This tensor decomposition retains the spatiotemporal structure among features obtained from multiple descriptors. Experimental validation is conducted using two benchmark human interaction recognition datasets: TVHID and Parliament. Experimental results are presented which show that the proposed approach outperforms many state-ofthe-art methods, achieving classification rates of 74.7% and 88.5% on the TVHID and Parliament datasets, respectively.

This article presents a new method for violent scene detection using super descriptor tensor decomposition. Multi-modal local features comprising auditory and visual features are extracted from Mel-frequency cepstral coefficients (including first and second order derivatives) and refined dense trajectories. There is usually a large number of dense trajectories extracted from a video sequence; some of these trajectories are unnecessary and can affect the accuracy. We propose to refine the dense trajectories by selecting only discriminative trajectories in the region of interest. Visual descriptors consisting of oriented gradient and motion boundary histograms are computed along the refined dense trajectories. In traditional bag-of-visual-words techniques, the feature descriptors are concatenated to form a single large feature vector for classification. This destroys the spatio-temporal interactions among features extracted from multi-modal data. To address this problem, a super descriptor tensor decomposition is proposed. The extracted feature descriptors are first encoded using super descriptor vector method. Then the encoded features are arranged as tensors so as to retain the spatio-temporal structure of the features. To obtain a compact set of features for classification, the TUCKER-3 decomposition is applied to the super descriptor tensors, followed by feature selection using Fisher feature ranking. The obtained features are fed to a support vector machine classifier. Experimental evaluation is performed on violence detection benchmark dataset, MediaEval VSD2014. The proposed method outperforms most of the state-of-the-art methods, achieving MAP2014 scores of 60.2% and 67.8% on two subsets of the dataset.

This article presents a new method for crowd behavior recognition, using dynamic features extracted from dense trajectories. The histogram of oriented gradient and motion boundary histogram descriptors are computed at dense points along motion trajectories, and tracked using median filtering and displacement information obtained from a dense optical flow field. Then a global representation of the scene is obtained using a bag-of-words model of the extracted features. The locality-constrained linear encoding with sum pooling and L2 plus power normalization are employed in the bag-of-words model. Finally, a support vector machine classifier is trained to recognize the crowd behavior in a short video sequence. The proposed method is tested on two benchmark datasets, and its performance is compared with those of some existing methods. Experimental results show that the proposed approach can achieve a classification rate of 93.8% on PETS2009 S3 and area under the curve score of 0.985 on UMN datasets respectively.

A fuzzy logic based active contour model for medical image segmentation is proposed. Image local features are incorporated in active contour model. Fuzzy logic is used to assign weights to pixels. Higher weights are assigned to pixels having less entropy and local variance whereas Lower weights are assigned to pixels having high entropy and local variance. Simulation results show the improvement in segmentation results in terms of efficiency and accuracy.

A non-blind color image watermarking scheme using principle component analysis, discrete wavelet transform and singular value decomposition is proposed. The color components are uncorrelated using principle component analysis. The watermark is embedded into the singular values of discrete wavelet transformed sub-band associated with principle component containing most of the color information. The scheme was tested against various attacks (including histogram equalization, rotation, Gaussian noise, scaling, cropping, Y-shearing, X-shearing, median filtering, affine transformation, translation, salt & pepper, sharpening), to check the robustness. The results of proposed scheme are compared with state-of-the-art existing color watermarking schemes using normalized correlation coefficient and peak signal to noise ratio. The simulation results show that proposed scheme is robust and imperceptible.

This work deals with the graph cuts based image segmentation of gray scale, color and texture images. Multilevel graph partitioning approach is used along with the normalized cuts framework. From the input image, an optimized graph is constructed using intensity, color and texture profiles of the image simultaneously. Based on nature of the image, a fuzzy rule based system is designed for weighted average of image features during graph development. Multilevel graph cuts algorithm is then applied to this graph in which graph is first coarsened to multiple levels in order to reduce the size of graph, coarsened graph is iteratively bi-partitioned through normalized cuts framework to get optimum partitions and then these partitions are projected back towards the original graph during uncoarsening phase to get the segmented image. Berkeley segmentation database is used to test our algorithm and segmentation results are evaluated through probabilistic rand index and global consistency error methods. It is shown that the presented segmentation method provides effective results for most type of images in terms of both accuracy and computational efficiency.

This work deals with segmentation of the gray scale, color and texture images using graph cuts. From the input image, a graph is constructed using intensity, color and texture profiles of the image simultaneously. Based on nature of the image, a fuzzy rule based system is designed to find the weight that should be given to a specific image feature during graph development. The graph obtained from fuzzy rule based weighted average of different image features is further used in normalized graph cuts framework. The graph is iteratively bi-partitioned through the normalized graph cuts to get optimum partitions resulting in segmented image. The Berkeley segmentation database is used to test our algorithm and segmentation results are evaluated through probabilistic rand index and global consistency error methods along with binary classifiers based approaches like sensitivity, positive predictive value and Dice similarity coefficient. It is shown that the presented segmentation method provides effective results for most type of the images.

This work deals with the segmentation of gray scale, color and texture images using graph cuts. From input image, a graph is constructed using intensity, color and texture profiles of the image simultaneously. Based on the nature of image, a fuzzy rule based system is designed to find the weight that should be given to a specific image feature during graph development. The graph obtained from the fuzzy rule based weighted average of different image features is further used in normalized graph cuts framework. Graph is iteratively bi-partitioned through the normalized graph cuts algorithm to get optimum partitions resulting in the segmented image. Berkeley segmentation database is used to test our algorithm and the segmentation results are evaluated through probabilistic rand index, global consistency error, sensitivity, positive predictive value and Dice similarity coefficient. It is shown that the presented segmentation method provides effective results for most types of images.