Search

In film making, shot has a profound influence on how the movie content is delivered and how the audiences are echoed, where different emotions and contents can be delivered through well-designed camera movements or shot editing. Therefore, in pursuit of a high-level understanding of long videos, accurate shot detection from untrimmed videos should be considered as the first and the most fundamental step. Existing approaches address this problem based on the visual difference and content transitions between consecutive frames, while ignoring intrinsic shot attributes, viz., camera movements, scales and viewing angles, which essentially reveals how each shot is created. In this work, we propose a new learning framework (SCTSNet) for shot boundary detection by jointly recognizing the attributes and composition of shots in videos. To facilitate the analysis of shots and the evaluation of shot detection models, we collect a large-scale shot boundary dataset MovieShots2, which contains 15K shots from 282 movie clips. It is richly annotated with the temporal boundary between consecutive shots and its shot attributes, including camera movements, scales and viewing angles, which are the three most distinct shot attributes. Our experiments show that the joint learning framework can significantly boost the boundary detection performance, surpassing the previous scores by a large margin. SCTSNet improves shot boundary detection AP from 0.65 to 0.77, pushing the performance to a new level.

Scene understanding in 3-D point clouds requires to annotate points manually at the model training stage. To reduce manual efforts in labeling points, this letter focuses on proposing an efficient method to implement semiautomatic segmentation of 3-D objects in 3-D point clouds. Specifically, to handle point clouds with high point density, supervoxels are treated as basic operating units during the object segmentation procedure. To obtain the valuable boundaries for guiding 3-D object segmentation, we propose to filter meaningless boundaries obtained by a traditional boundary detection method. Once valuable boundaries are obtained, we propose a boundary-aware Markov random field (MRF) model to consider the object-boundary constraint into generating the boundary-preserved segmentation results. Extensive experiments on two data sets show the effectiveness of our proposed framework on segmenting 3-D objects from point cloud scenes.

Background:: In the internet era, there is a prime need to access and manage the huge volume of multimedia data in an effective manner. Shot is a sequence of frames captured by a single camera in an uninterrupted space and time. Shot detection is suitable for various applications such that video browsing, video indexing, content based video retrieval and video summarization. Objective:: To detect the shot transitions in the video within a short duration. It compares the visual features of frames like correlation, histogram and texture features only in the candidate region frames instead of comparing the full frames in the video file. Methods: This paper analyses candidate frames by searching the values of frame features which matches with the abrupt detector followed by the correct cut transition frame with in the datacube recursively until it detects the correct transition frame. If they are matched with the gradual detector, then it will give the gradual transition ranges, otherwise the algorithm will compare the frames within the next datacube to detect shot transition. Results:: The total average detection rates of all transitions computed in the proposed Data-cube Search Based Shot Boundary Detection technique are 92.06 for precision, 96.92 for recall and 93.94 for f1 measure and the maximum accurate detection rate. Conclusion:: Proposed method for shot transitions uses correlation value for searching procedure with less computation time than the existing methods which compares every single frame and uses multi features such as color, edge, motion and texture features in wavelet domain.

The paper proposes a novel video segmentation system with maiden application of CIEDE2000 colour-difference and mean luminace pattern. CIEDE2000 colour-difference uses Lab colour space which is a stable and efficient colour space. The main advantage of Lab colour space model is that it can approximate all the available colours perceived by our human eye. CIEDE2000 colour difference is used for detecting abrupt transitions in the video. The novel contribution of the paper is the maiden use of the mean luminance pattern, increasing and decreasing patterns of the mean value of frame luminance, for detecting the gradual transition. The approach is validated on standard databases TRECVid 2001 and 2007 test video database. The performance of the proposed technique is compared with recently reported techniques and found to be superior as compared to other techniques. The accuracy achieved with the proposed method on the standard databases is 95.9% for cut transition, 78.6% for gradual transition and 92.1% overall.

Breast cancer is class of cancer that sets off when the breast cells grow out of proportion and control. The radiologist recognizes the sign of breast cancer by performing a kind of X-ray called screening mammography. During analysis of mammography the biggest problem arise because of the presence of pectoral muscle. The mass of tissue on which the breast is rest called the Pectoral muscle. The primary problem is that pectoral muscle area density is almost similar to the tumour cell and this condition generates confusion to recognize the tumour cell. For analysis the Medio-Lateral oblique (MLO) views of mammograms is being taken so that the complete breast image can be viewed. Some part of the pectoral muscle also gets visible along with the breast in the MLO view which must be segmented from the mammogram. Pectoral muscle involvement can lead to false positive or false negatives. The workforce shortages of Radiologist with respect to growing demands and to declare the result in a very short time have also increased the pressure. Consequently, a radiologist is sometimes unable to detect an anomaly. This is the time where CAD system can help radiologists to detect breast cancer at an earlier stage. Numerous strategies for the selection of the pectoral muscle have been suggested and developed so far. This article reviews the different segmentation techniques for pectoral muscle removal in mammograms through digital images..

Brightness and color are two basic visual features of the human visual system. In the retina, color-sensitive cells are sensitive to brightness and color information and exhibit salient direction selectivity. However, evidence from neurobiology indicates that apart from color features, the rod and cone cells of the retina are also sensitive to high or low luminance, respectively termed the light and dark adaptation mechanisms. Classical single-opponent and color-opponent contour detection model frameworks include the computational processes of single red (R), green (G), blue (B) and yellow (Y) channels and opponent R–GB–Y channels, respectively. Thus, to combine luminance cues and traditional color cues to improve boundary detection in natural scenes, we propose the use of a dark and a light channel to simulate light and dark adaptation mechanisms. The results of the proposed model considering three datasets (BSDS300, BSDS500, NYUD) demonstrate an improvement compared with current bio-inspired contour detection models.

Neural circuits can be reconstructed from brain images acquired by serial section electron microscopy. Image analysis has been performed by manual labor for half a century, and efforts at automation date back almost as far. Convolutional nets were first applied to neuronal boundary detection a dozen years ago, and have now achieved impressive accuracy on clean images. Robust handling of image defects is a major outstanding challenge. Convolutional nets are also being employed for other tasks in neural circuit reconstruction: finding synapses and identifying synaptic partners, extending or pruning neuronal reconstructions, and aligning serial section images to create a 3D image stack. Computational systems are being engineered to handle petavoxel images of cubic millimeter brain volumes.

International audience; HyperSpectral (HS) images have been successfully used for brain tumor boundary detection during resection operations. Nowadays, these classification maps coexist with other technologies such as MRI or IOUS that improve a neurosurgeon's action, with their incorporation being a neurosurgeon's task. The project in which this work is framed generates an unified and more accurate 3D immersive model using HS, MRI, and IOUS information. To do so, the HS images need to include 3D information and it needs to be generated in real-time operating room conditions, around a few seconds. This work presents Graph cuts Reference depth estimation in GPU (GoRG), a GPU-accelerated multiview depth estimation tool for HS images also able to process YUV images in less than 5.5 s on average. Compared to a high-quality SoA algorithm, MPEG DERS, GoRG YUV obtain quality losses of -0.93 dB, -0.6 dB, and -1.96% for WS-PSNR, IV-PSNR, and VMAF, respectively, using a video synthesis processing chain. For HS test images, GoRG obtains an average RMSE of 7.5 cm, with most of its errors in the background, needing around 850 ms to process one frame and view. These results demonstrate the feasibility of using GoRG during a tumor resection operation.

Scenes play a crucial role in breaking the storyline of movies and TV episodes into semantically cohesive parts. However, given their complex temporal structure, finding scene boundaries can be a challenging task requiring large amounts of labeled training data. To address this challenge, we present a self-supervised shot contrastive learning approach (ShotCoL) to learn a shot representation that maximizes the similarity between nearby shots compared to randomly selected shots. We show how to apply our learned shot representation for the task of scene boundary detection to offer state-of-the-art performance on the MovieNet dataset while requiring only ~25% of the training labels, using 9x fewer model parameters and offering 7x faster runtime. To assess the effectiveness of ShotCoL on novel applications of scene boundary detection, we take on the problem of finding timestamps in movies and TV episodes where video-ads can be inserted while offering a minimally disruptive viewing experience. To this end, we collected a new dataset called AdCuepoints with 3,975 movies and TV episodes, 2.2 million shots and 19,119 minimally disruptive ad cue-point labels. We present a thorough empirical analysis on this dataset demonstrating the effectiveness of ShotCoL for ad cue-points detection., Accepted to CVPR 2021

Social online video has emerged as one of the most popular application, where “bullet screen comment” is one of the favorite features of Asian users. User behavior report finds that most people are used to quickly navigate and locate his concerned video clip according to its corresponding video labels. Traditional scene segmentation algorithms are mostly based on the analysis of frames, which cannot automatically generate labels. Since time-synchronized comments can reflect the episode of current moment, this paper proposed an unsupervised video episode boundary detection model (VEBD) for bullet screen comment video. It could not only automatically identify each episode boundary, but also detect the topic for video tagging. Specifically, a Joint Episode-Topic model is first constructed to detect the hidden topic in initial partitioned time slices. Then, based on the detected topics, temporal and semantic relevancy between adjacent time slices are measured to refine the boundary detection accuracy. Experiments based on real data show that our model outperforms the existing algorithms in both boundary detection and semantic tagging quality.

Recently, significant progress has been made in single-view depth estimation thanks to increasingly large and diverse depth datasets. However, these datasets are largely limited to specific application domains (e.g. indoor, autonomous driving) or static in-the-wild scenes due to hardware constraints or technical limitations of 3D reconstruction. In this paper, we introduce the first depth dataset DynOcc consisting of dynamic in-the-wild scenes. Our approach leverages the occlusion cues in these dynamic scenes to infer depth relationships between points of selected video frames. To achieve accurate occlusion detection and depth order estimation, we employ a novel occlusion boundary detection, filtering and thinning scheme followed by a robust foreground/background classification method. In total our DynOcc dataset contains 22M depth pairs out of 91K frames from a diverse set of videos. Using our dataset we achieved state-of-the-art results measured in weighted human disagreement rate (WHDR). We also show that the inferred depth maps trained with DynOcc can preserve sharper depth boundaries., 3DV 2020

Benefiting from the advantages of flexibility and timeliness, Unmanned Aerial Vehicles (UAVs) play an important role in crop growth monitoring, precision agriculture and intelligent agriculture. This paper focuses on the farmland boundary detection in UAV images. Traditional farmland boundary detection methods have problems such as over-segmentation and discontinuous boundary. To address these problems, we propose a Deep Boundaries Combination (DBC) algorithm for the detection of farmland plots boundaries in UAV remote sensing images. DBC uses deep convolutional networks to obtain edge probability map of farmland images, and then applies Oriented Watershed Transform (OWT) and Ultrametric Contour Map (UCM) to convert edge probability map into closed boundary hierarchy tree, which layers the boundaries by edge probability. We perform experiments on two farmland images acquired by UAV. Experimental results show that our method can extract more accurate farmland boundaries than other methods.

Thermal face image analysis is favorable for certain circumstances. For example, illumination-sensitive applications, like nighttime surveillance; and privacy-preserving demanded access control. However, the inadequate study on thermal face image analysis calls for attention in responding to the industry requirements. Detecting facial landmark points are important for many face analysis tasks, such as face recognition, 3D face reconstruction, and face expression recognition. In this paper, we propose a robust neural network enabled facial landmark detection, namely Deep Multi-Spectral Learning (DMSL). Briefly, DMSL consists of two sub-models, i.e. face boundary detection, and landmark coordinates detection. Such an architecture demonstrates the capability of detecting the facial landmarks on both visible and thermal images. Particularly, the proposed DMSL model is robust in facial landmark detection where the face is partially occluded, or facing different directions. The experiment conducted on Eurecom's visible and thermal paired database shows the superior performance of DMSL over the state-of-the-art for thermal facial landmark detection. In addition to that, we have annotated a thermal face dataset with their respective facial landmark for the purpose of experimentation.

Providing closed and well-connected boundaries of coronary artery is essential to assist cardiologists in the diagnosis of coronary artery disease (CAD). Recently, several deep learning-based methods have been proposed for boundary detection and segmentation in a medical image. However, when applied to coronary wall detection, they tend to produce disconnected and inaccurate boundaries. In this paper, we propose a novel boundary detection method for coronary arteries that focuses on the continuity and connectivity of the boundaries. In order to model the spatial continuity of consecutive images, our hybrid architecture takes a volume (i.e., a segment of the coronary artery) as input and detects the boundary of the target slice (i.e., the central slice of the segment). Then, to ensure closed boundaries, we propose a contour-constrained weighted Hausdorff distance loss. We evaluate our method on a dataset of 34 patients of coronary CT angiography scans with curved planar reconstruction (CCTA-CPR) of the arteries (i.e., cross-sections). Experiment results show that our method can produce smooth closed boundaries outperforming the state-of-the-art accuracy., 5 pages, 2 figures, accepted by ISBI 2020

Road boundary detection is an important part of the perception of the autonomous driving. It is difficult to detect road boundaries of unstructured roads because there are no curbs. There are no clear boundaries on mine roads to distinguish areas within the road boundary line and areas outside the road boundary line. This paper proposes a real-time road boundary detection and tracking method by a 3D-LIDAR sensor. The road boundary points are extracted from the detected elevated point clouds above the ground point cloud according to the spatial distance characteristics and the angular features. Road tracking is to predict and update the boundary point information in real-time, in order to prevent false and missed detection. The experimental verification of mine road data shows the accuracy and robustness of the proposed algorithm.

Satellite images have a very high resolution, which make their automatic processing computationally costly, and they suffer from artifacts making their processing difficult. This paper describes a method for the effective semantic segmentation of satellite images, and compares different object classifiers in terms of accuracy and execution time. In the paper, the image spectrum is used to reduce the computational cost during the segmentation and classification steps. Firstly, artifacts are corrected from the satellite images for facilitating the feature extraction process. After this, semantic representation is used to gather the semantic regions of downscaled images. As the images are very large, this scaling down significantly reduces the computing time with little degradation in the coarse object detection results. A deep neural forest classifier finds potential regions before executing the pixel-based segmentation. Finally, in our experiments, boundary detection and several classifiers are evaluated to find the objects associated with these regions. The paper details the set-up for our tree-based convolutional neural network. The results indicate that this tree-based convolutional neural network outperforms the other surveyed techniques in the literature.

In this article, we present a procedure that uses low-resolution cameras to define the boundaries of point clouds quickly. Using the same procedure, we get almost identical results to high- resolution images. The emphasis is not so much on the method presented, but on the fact that applying this method on images of low-resolution cameras will made the process being much quicker. This result is remarkable because the segmentation and clustering methods, when applied to big sized and high-resolution images, are sometimes overwhelmingly slow. If you use low-resolution images in order to detect boundaries of larger objects then the process accelerates remarkably. This approach, which is described here, attempts to provide a possible solution to recognize the desired objects.

This paper presents a system for improving the robustness of LiDAR lateral localisation systems. This is made possible by including detections of road boundaries which are invisible to the sensor (due to occlusion, e.g. traffic) but can be located by our Occluded Road Boundary Inference Deep Neural Network. We show an example application in which fusion of a camera stream is used to initialise the lateral localisation. We demonstrate over four driven forays through central Oxford - totalling 40 km of driving - a gain in performance that inferring of occluded road boundaries brings., Comment: accepted for publication at the IEEE/ION Position, Location and Navigation Symposium (PLANS) 2020

Video shot boundary detection (SBD) or video cut detection is one of the fundamental processes of video-processing with respect to semantic understanding, contextual information accumulation, labeling, content-based information retrieval and many more applications, such as video surveillance and monitoring. In this work, the authors have proposed a generative-model based framework for detecting shot boundaries in between the frames of a video segment. To generate a model of shot-boundaries, the authors have applied the concepts of Support Vector Machine to estimate the distance between any two images, and then, have generated a Gaussian Mixture Model from the estimated distances. Next, a Bayesian Estimation process checks the presence of boundaries in between the images by exploiting the Gaussian Mixture-based boundary model. Further, the authors have used the principles of Compressive Sensing to reduce the overhead of boundary detection process without losing of important information.

Comparison of feature detectors and evaluation of their performance is very important in computer vision. A new algorithm is proposed in this paper to compare the performance of four corner feature detectors based on abrupt shot boundary detection. The proposed algorithm consists of two stages: feature vectors generation where corner detector for all video frames is computed to obtain the descriptor feature vectors, and features matching where the number of matching features between two successive frames is calculated. The corner feature detectors used in this paper are BRISK, Harries, MinEigen, and FAST. Experimental results indicate that the proposed algorithm using MinEigen features detector provides better performance than other features detectors where the average value of recall, precision, and F measure is 0.99083, 0.98808, and 0.98875 for selected testing videos respectively. The results also show that the FAST is superior to others feature detectors when considering execution time.

This paper presents a feature-based method for road boundary detection in both straight and curved lane lines. The Sobel edge detection is performed followed by Inverse Perspective Mapping (IPM) to extract feature points from left and right lane markings. The polynomial regression is then applied to approximate both lane lines from two datasets of points. Finally, the slopes of both lines are exploited for identifying the direction of road lane to help drivers for decision making while driving. Experiments are conducted on 2-D road images downloaded from the Internet. The results are demonstrated the effectiveness and robustness under various situations.

In this paper, the traditional algorithm of shot boundary detection based on histogram difference is introduced briefly. Based on this, a video shot boundary detection algorithm based on HSV color model is proposed. The method firstly detects the shot hard cut based on the weighted color histogram difference curve, then performs the shot gradual transition detection according to the image local average pixel value characteristic curve. Finally, two partial detection results are fused, and all the shot transitions of the video can be obtained. Experimental result shows that the method has a higher recall and precision.

Medical image segmentation has been widely used in clinical practice. It is an important basis for medical experts to diagnose the disease. However, weak edges and intensity inhomogeneity (Niu et al. in 2nd IEEE international conference on computational intelligence and applications (ICCIA). https://doi.org/10.1109/ciapp.2017.816722 , 2017) in medical images may hinder the accuracy of any traditional active contour segmentation methods. In this paper, we propose an improved active contour method by embedding a boundary constraint factor and adding the local information of images to the energy function of Chan–Vese model. Then graph cuts method is used to optimize the new energy function of the improved active contour method in this paper. There are several salient features: (1) we can obtain more accurate boundaries by embedding a constraint factor and adding local information of images. (2) The energy function is not easily to fall into local optimum. (3) Finally, our method only need to adjust one parameter. The evaluation results on Magnetic Resonance Imaging and Computed Tomography, blood vessel images, and mammograms masses show that the proposed method leads to more accurate boundary detection results than the state-of-the-art edge-based and region-based active contour segmentation methods.

The amount of video data generated and made publicly available has been tremendously increased in today's digital era. Analyzing these huge video repositories require effective and efficient content-based video analysis systems. Shot boundary detection and Keyframe extraction are the two major tasks in video analysis. In this direction, a method for detecting abrupt shot boundaries and extracting representative keyframe from each video shot is proposed. These objectives are achieved by incorporating the concepts of fuzzy sets and intuitionistic fuzzy sets. Shot boundaries are detected using coefficient of correlation on fuzzified frames. Further, probabilistic entropy measures are computed to extract the keyframe within fuzzified frames of a shot. The keyframe representative of a shot is the frame with highest entropy value. To show the efficacy of the proposed methods two benchmark datasets are used (TRECVID and Open Video Project). The proposed methods outperform when compared with some of state-of-the-art shot boundary detection and keyframe extraction methods.

The advancement of multimedia technology has contributed to a large number of videos, so it is important to know how to retrieve information from video, especially for crime prevention and forensics. For the convenience of retrieving video data, content-based video retrieval (CBVR) has got great publicity. Aiming at improving the retrieval performance, we focus on the two key technologies: shot boundary detection and keyframe extraction. After being compared with pixel analysis and chi-square histogram, histogram-based method is chosen in this paper. Then we combine it with adaptive threshold method and use HSV color space to get the histogram. For keyframe extraction, four methods are analyzed and four evaluation criteria are summarized, both objective and subjective, so the opinion is finally given that different types of keyframe extraction methods can be used for varied types of videos. Then the retrieval can be based on keyframes, simplifying the process of video investigation, and helping criminal investigation personnel to improve work efficiency.

Sign Language (SL) is a three dimensional language used for communication by deaf people. The recognition system for SL is always an apprehensive task which is handled by vision collaboration and technology. Basically, detection of edges is deliberated to be the precursor for detection of objects, as the edges are the outline of the objects. Detecting continuous edges in real time images is a hard problem, especially in Tamil Sign Language (TSL) recognition system. This paper proposes an algorithm which finds optimal threshold values (L and H) based on Synergistic Fibroblast Optimization (SFO) for detection of continuous, smooth and thin edges of TSL hand pose images. A novel SFO algorithm is proposed with sphere objective function and two constraints for reducing the ruined edges. The efficiency of the algorithm is compared experimentally with conventional Canny, Classical PSO and variant based PSO on TSL Consonants images. Experimental results suggested that the novel SFO based canny operator detects edges more accurately, and the edges detected are smoother and thinner when compared to other analyzed algorithms. Article DOI: https://dx.doi.org/10.20319/mijst.2017.32.193208 This work is licensed under the Creative Commons Attribution-Non-commercial 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

This paper presents field programmable logic array (FPGA) based hardware accelerators for iris localization, which can be used to accelerate the iris localization task in reliable and affordable embedded iris recognition systems. This work uses edge-map generation and circular Hough transform (CHT) based algorithm to localize irises in the images captured under near infrared (NIR) illumination. The proposed hardware accelerators for iris localization are: 1) Edge-map generation hardware for pupillary boundary detection; 2) Edge-map generation hardware for limbic boundary detection; and 3) CHT hardware for pupillary and limbic boundary detection. These hardware accelerators have processing time of 390.46 μsec, 393.67 μsec and 3.46 msec (average) respectively for an image of 320 × 240 pixels and achieve the iris localization accuracy of 96.52%. The proposed CHT and median filter hardware implementations show better results than the previous work.