Search

This paper considers the area of digital forensics (DF). One of the problem in DF is the issue of identification of digital cameras based on images. This aspect has been attractive in recent years due to popularity of social media platforms like Facebook, Twitter etc., where lots of photographs are shared. Although many algorithms and methods for digital camera identification have been proposed, there is lack of research about their robustness. Therefore, in this paper the robustness of digital camera identification with the use of convolutional neural network is discussed. It is assumed that images may be of poor quality, for example, degraded by Poisson noise, Gaussian blur, random noise or removing pixels’ least significant bit. Experimental evaluation conducted on two large image datasets (including Dresden Image Database) confirms usefulness of proposed method, where noised images are recognized with almost the same high accuracy as normal images.

Extreme learning machine (ELM) is a training algorithm for single hidden layer feedforward neural network (SLFN), which converges much faster than traditional methods and yields promising performance. In this paper, we hope to present a comprehensive review on ELM. Firstly, we will focus on the theoretical analysis including universal approximation theory and generalization. Then, the various improvements are listed, which help ELM works better in terms of stability, efficiency, and accuracy. Because of its outstanding performance, ELM has been successfully applied in many real-time learning tasks for classification, clustering, and regression. Besides, we report the applications of ELM in medical imaging: MRI, CT, and mammogram. The controversies of ELM were also discussed in this paper. We aim to report these advances and find some future perspectives.

Due to the automatic feature extraction procedure via multi-layer nonlinear transformations, the deep learning-based visual trackers have recently achieved a great success in challenging scenarios for visual tracking purposes. Although many of those trackers utilize the feature maps from pre-trained convolutional neural networks (CNNs), the effects of selecting different models and exploiting various combinations of their feature maps are still not compared completely. To the best of our knowledge, all those methods use a fixed number of convolutional feature maps without considering the scene attributes (e.g., occlusion, deformation, and fast motion) that might occur during tracking. As a pre-requisition, this paper proposes adaptive discriminative correlation filters (DCF) based on the methods that can exploit CNN models with different topologies. First, the paper provides a comprehensive analysis of four commonly used CNN models to determine the best feature maps of each model. Second, with the aid of analysis results as attribute dictionaries, an adaptive exploitation of deep features is proposed to improve the accuracy and robustness of visual trackers regarding video characteristics. Third, the generalization of proposed method is validated on various tracking datasets as well as CNN models with similar architectures. Finally, extensive experimental results demonstrate the effectiveness of proposed adaptive method compared with the state-of-the-art visual tracking methods.

In the COVID-19 pandemic, telehealth plays a significant role in the e-healthcare. E-health security risks have also risen significantly with the rise in the use of telehealth. This paper addresses one of e-health’s key concerns, namely security. Secret sharing is a cryptographic method to ensure reliable and secure access to information. To eliminate the constraint that in the existing secret sharing schemes, this paper presents Tree Parity Machine (TPM) guided patients’ privileged based secure sharing. This is a new secret sharing technique that generates the shares using a simple mask based operation. This work considers addressing the challenges presents in the original secret sharing scheme. This proposed technique enhances the security of the existing scheme. This research introduces a concept of privileged share in which among k number of shares one share should come from a specific recipient (patient) to whom a special privilege is given to recreate the original information. In the absence of this privileged share, the original information cannot be reconstructed. This technique also offers TPM based exchange of secret shares to prevent Man-In-The-Middle-Attack (MITM). Here, two neural networks receive common inputs and exchange their outputs. In some steps, it leads to full synchronization by setting the discrete weights according to the specific rule of learning. This synchronized weight is used as a common secret session key for transmitting the secret shares. The proposed method has been found to produce attractive results that show that the scheme achieves a great degree of protection, reliability, and efficiency and also comparable to the existing secret sharing scheme.

In this paper, we recognize serial numbers on banknotes using deep learning. The samples used in this paper are digital images which have been preprocessed with data labelling and data augmentation, e.g., scaling transformation, etc. The algorithms based on deep learning are proposed and have the stability for serial number recognition with complex backgrounds. In this paper, a pipeline of deep neural networks is established for the recognition of banknote serial numbers. Because high reliability is more important than accuracy in financial applications, DenseNet is set forth as the primary classifier, the scaling transformation of SegLink is put forward to locate the characters, the detection rate is up to 95.80%. A convolutional neural network with residual attention model is proposed for serial number recognition, the precision is up to 97.09%.

In this paper we introduce a cavity reconstructing algorithm for 3D surface scans (CRASS) developed for filling cavities in point clouds representing human body surfaces. The presented method uses Bezier patches to reconstruct missing data. The source of input data for the algorithm was an 8-directional structured light scanning system for the human body. Typical 3D scan representing human body consists of about 1 million points with average sampling density of 1 mm. The paper describes the complete scan processing pipeline: data pre-processing, boundary selection, cavity extraction and reconstruction, and a post-processing step to smooth and resample resulting geometry. The developed algorithm was tested on simulated and scanned 3D input data. Quality assessment was made based on simulated cavities, reconstructed using presented method and compared to original 3D geometry. Additionally, comparison to the state-of-the-art screened Poisson method is presented. Values’ ranges of parameters influencing result of described method were estimated for sample scans and comprehensively discussed. The results of the quantitative assessment of the reconstruction were lower than 0,5 of average sampling density.

In this paper, we propose a new method for improving human activity recognition (HAR) datasets in order to increase their classification accuracy when trained with a certain classifier like a Neural Network. In this paper a novel training/testing process for building/testing a classification model for human activity recognition (HAR) is proposed. Traditionally, HAR is done by a classifier that learns what activities a person is doing by training with skeletal data obtained from a motion sensor such as Microsoft Kinect or accelerometer sensors. These skeletal data are the spatial coordinates (x, y, z) of different parts of the human body. In addition to the spatial features that describe current positions in the skeletal data, new features called Shadow Features are used to improve the supervised learning efficiency and accuracy of Neural Network classifiers. Shadow Features are inferred from the dynamics of body movements, thereby modelling the underlying momentum of the performed activities. They provide extra dimensions of information for characterizing activities in the classification process and thus significantly improving the accuracy. These Shadow Features are generated based on the existing features obtained from sensor datasets. In this paper we show that the accuracy of a neural network classifier can be significantly improved by the addition of Shadow Features and we also show that the generation of Shadow Features can be achieved with little time cost, on the fly, with the NVIDIA GPU technology and the CUDA programming model, this way we can improve the Neural Network accuracy at almost no time cost. GPUs are particularly suitable for generating Shadow Features, since they possess multiple cores which can be taken advantage of, in order to generate Shadow Features for multiple data columns in parallel, therefore reducing a lot of processing time, especially when dealing with huge datasets.

Biometric technologies have become the main focus in the design of state-of-the-art border security solutions. While respective research in the field of multimedia vision has been centred around improving quality and accuracy of identity recognition, the impact of such technologies upon society and legal regulations still remains a topic unaddressed, specifically within the engineering community. Research in technology can and in some respect must include collaboration with social sciences and social practice. Building on participation in the EU funded research project PERSONA [18] (Privacy, Ethical, Regulatory and SOcial No-gate crossing point solutions Acceptance), authors of this paper look at the challenges associated with biometrics-based solutions in no-gate border crossing point scenarios. This included the procedures needed for the assessment of their social, ethical, privacy and regulatory acceptance, particularly in view of the impact on both, the passengers and border control authorities as well as the potential pitfalls of biometric technology due to fraudulent activities. In consultation with the collaborating border control authorities, the paper reports on the formal assessment of biometric technologies for real-world acceptance to cope with the increasing demand of global travellers crossing state borders.

A deterioration level estimation method via neural network maximizing category-based ordinally supervised multi-view canonical correlation is presented in this paper. This paper focuses on real world data such as industrial applications and has two contributions. First, a novel neural network handling multi-modal features transforms original features into features effectively representing deterioration levels in transmission towers, which are one of the infrastructures, with consideration of only correlation maximization. It can be realized by setting projection matrices maximizing correlations between multiple features into weights of hidden layers. That is, since the proposed network has only a few hidden layers, it can be trained from a small amount of training data. Second, since there exist diverse characteristics and an ordinal scale in deterioration levels, the proposed method newly derives category-based ordinally supervised multi-view canonical correlation analysis (Co-sMVCCA). Co-sMVCCA enables estimation of effective projection considering both within-class divergence and the ordinal scale between classes. Experimental results showed that the proposed method realizes accurate deterioration level estimation.

Movie genre classification is a challenging task that has increasingly attracted the attention of researchers. The number of movie consumers interested in taking advantage of automatic movie genre classification is overgrowing, thanks to media streaming service providers’ popularization. In this paper, we addressed the multi-label classification of movie genres in a multimodal way. To this end, we created a dataset composed of trailer video clips, subtitles, synopses, and movie posters from 152,622 movie titles of the Movie Database (TMDb). Such a large dataset was carefully curated, organized, and made available as a contribution of this work. We labeled each movie of the dataset according to a set of eighteen genre labels. In the experimental evaluation performed in this paper, we computed different kinds of descriptors, such as Mel Frequency Cepstral Coefficients (MFCCs), Statistical Spectrum Descriptor (SSD), Local Binary Pattern (LBP) from spectrograms, Long-Short Term Memory (LSTM), and Convolutional Neural Networks (CNN). With these descriptors, we trained different monolithic classifiers using BinaryRelevance and ML-kNN techniques. Besides, we also explored the combination of classifiers/features using a late fusion strategy. The fusion of a LSTM trained on synopses and another LSTM trained on the movie subtitles provided our best results in F-Score (0.674) and AUC-PR (0.725) metrics. These results corroborate the existence of complementarity among classifiers trained on different sources of information in this field of application. As far as we know, this is the most comprehensive study developed in terms of diversity of multimedia sources of information to perform movie genre classification.

Accessibility is a key requirement for any multimedia tool and application. With the current trend towards immersive experiences, such as Virtual Reality (VR) and 360o video, it becomes key that these environments are adapted to be fully accessible. However, until recently the focus has been mostly on adapting the existing techniques to fit immersive displays, rather than considering new approaches for accessibility designed specifically for these increasingly relevant media experiences.This paper surveys a wide range of 360o video players and examines the features they include for dealing with accessibility, such as Subtitles, Audio Description, Sign Language, User Interfaces and other interaction features, like voice control and support for multi-screen scenarios. These features have been chosen based on guidelines from standardization contributions, like in the World Wide Web Consortium (W3C) and the International Communication Union (ITU), and from research contributions for making 360° video consumption experiences accessible. The in-depth analysis has been part of a research effort towards the development of a fully inclusive and accessible 360° video player. The paper concludes by discussing how the newly developed player has gone above and beyond the existing solutions and guidelines, by providing accessibility features that meet the expectations for a widely used immersive medium, like 360° video.

International audience; Lightweight cryptography (LWC) is an interesting research area in the field of information security. Some limitations like: increased components usage, time consumption, power consumption and memory requirement mandate the need for lightweight cryptography. One of the proposed algorithms in this field is Speck which was designed by the National Security Agency (NSA) in June 2013. In this paper, we propose a new ultra-lightweight cryptographic algorithm based on Speck known as Speck-R. Speck-R is a hybrid cipher, combining ARX architecture with a dynamic substitution layer. The novelty in this paper resides in adding a key-dynamic substitution layer that changes according to a dynamic key. With this modification, the number of rounds can be reduced from 26 (in Speck) to 7 (in Speck-R). Thus, the main contribution of this paper consists in reducing the execution time of Speck by at least 18% on limited devices to reach a reduction of 77% while keeping a high level of security. To backbone Speck-R’s security, different security and statistical tests are exerted on Speck-R. In addition, a real hardware implementation on three different famous IoT devices is also presented where Speck-R outperformed Speck in terms of execution times. Finally, extensive tests show that Speck-R possesses the necessary criteria to be considered as a good cipher scheme that is suitable for lightweight devices.

With the growth of data mining and machine learning approaches in recent years, many efforts have been made to generalize these sciences so that researchers from any field can easily utilize these sciences. One of the most important of these efforts is the development of data mining tools that try to hide the complexities from researchers so that they can achieve a professional output with any level of knowledge. This paper is focused on reviewing and comparing data mining and machine learning tools including WEKA, KNIME, Keel, Orange, Azure, IBM SPSS Modeler, R and Scikit-Learn to show what approach each of these methods has taken in the face of the complexities and problems of different scenarios of generalization of data mining and machine learning. In addition, for a more detailed review, this paper examines the challenge of network intrusion detection in two tools, Knime with graphical interface and Scikit-Learn with coding environment.

Social media platform like Twitter is one of the primary sources for sharing real-time information at the time of events such as disasters, political events, etc. Detecting the resource tweets during a disaster is an essential task because tweets contain different types of information such as infrastructure damage, resources, opinions and sympathies of disaster events, etc. Tweets are posted related to Need and Availability of Resources (NAR) by humanitarian organizations and victims. Hence, reliable methodologies are required for detecting the NAR tweets during a disaster. The existing works don't focus well on NAR tweets detection and also had poor performance. Hence, this paper focus on detection of NAR tweets during a disaster. Existing works often use features and appropriate machine learning algorithms on several Natural Language Processing (NLP) tasks. Recently, there is a wide use of Convolutional Neural Networks (CNN) in text classification problems. However, it requires a large amount of manual labeled data. There is no such large labeled data is available for NAR tweets during a disaster. To overcome this problem, stacking of Convolutional Neural Networks with traditional feature based classifiers is proposed for detecting the NAR tweets. In our approach, we propose several informative features such as aid, need, food, packets, earthquake, etc. are used in the classifier and CNN. The learned features (output of CNN and classifier with informative features) are utilized in another classifier (meta-classifier) for detection of NAR tweets. The classifiers such as SVM, KNN, Decision tree, and Naive Bayes are used in the proposed model. From the experiments, we found that the usage of KNN (base classifier) and SVM (meta classifier) with the combination of CNN in the proposed model outperform the other algorithms. This paper uses 2015 and 2016 Nepal and Italy earthquake datasets for experimentation. The experimental results proved that the proposed model achieves the best accuracy compared to baseline methods.

IBM Watson is one of the representative tools for speech recognition system which can automatically generate not only speech-to-text information but also speaker ID and timing information, which is called as Informatized Caption. However, if there is some noise in the voice signal to the IBM Watson API, the recognition performance is significantly decreased. It can be easily found in movies with background music and special sound effects. This paper aims to improve the inaccuracy problem of current Informatized Captions in noisy environments. In this paper, a method of modifying incorrectly recognized words and a method of enhancing timing accuracy while updating database in real time are suggested based on the original caption and Informatized Caption information. Experimental results shows that the proposed method can give 81.09% timing accuracy for the case of 10 representative animation, horror and action movies.

Human activity recognition (HAR) systems attempt to automatically identify and analyze human activities using acquired information from various types of sensors. Although several extensive review papers have already been published in the general HAR topics, the growing technologies in the field as well as the multi-disciplinary nature of HAR prompt the need for constant updates in the field. In this respect, this paper attempts to review and summarize the progress of HAR systems from the computer vision perspective. Indeed, most computer vision applications such as human computer interaction, virtual reality, security, video surveillance and home monitoring are highly correlated to HAR tasks. This establishes new trend and milestone in the development cycle of HAR systems. Therefore, the current survey aims to provide the reader with an up to date analysis of vision-based HAR related literature and recent progress in the field. At the same time, it will highlight the main challenges and future directions.

Gaming video streaming services are growing rapidly due to new services such as passive video streaming of gaming content, e.g. Twitch.tv, as well as cloud gaming, e.g. Nvidia GeForce NOW and Google Stadia. In contrast to traditional video content, gaming content has special characteristics such as extremely high and special motion patterns, synthetic content and repetitive content, which poses new opportunities for the design of machine learning-based models to outperform the state-of-the-art video and image quality approaches for this special computer generated content. In this paper, we train a Convolutional Neural Network (CNN) based on an objective quality model, VMAF, as ground truth and fine-tuned it based on subjective image quality ratings. In addition, we propose a new temporal pooling method to predict gaming video quality based on frame-level predictions. Finally, the paper also describes how an appropriate CNN architecture can be chosen and how well the model performs on different contents. Our result shows that among four popular network architectures that we investigated, DenseNet performs best for image quality assessment based on the training dataset. By training the last 57 convolutional layers of DenseNet based on VMAF values, we obtained a high performance model to predict VMAF of distorted frames of video games with a Spearman’s Rank correlation (SRCC) of 0.945 and Root Mean Score Error (RMSE) of 7.07 on the image level, while achieving a higher performance on the video level leading to a SRCC of 0.967 and RMSE of 5.47 for the KUGVD dataset. Furthermore, we fine-tuned the model based on subjective quality ratings of images from gaming content which resulted in a SRCC of 0.93 and RMSE of 0.46 using one-hold-out cross validation. Finally, on the video level, using the proposed pooling method, the model achieves a very good performance indicated by a SRCC of 0.968 and RMSE of 0.30 for the used gaming video dataset.

In this paper visualization techniques for modern closed circuit television (CCTV) smart city services are discussed with application to prevention of threats. Unconventional approaches to the intelligent visual data processing are proposed in order to support video surveillance operators, thus to make their work less exhaustive and more effective. Although registration of a huge amount of video data requires development of intelligent and automatic signal processing information extraction techniques, improvement of visualization methods for operators is also a very important task, because of the crucial role the human factor plays and should always play in the decision making, e.g. in the operator reactions to various crisis situations, which can never be fully eliminated by artificial intelligence. Four software based mechanisms connected with a standard or with a slightly extended hardware are proposed as options for the CCTV operators. They utilize rather known ideas but are implemented with new extensions to original algorithms, as well as with additional, innovative modifications and solutions (not presented in the literature). With them they become reliable and efficient tools for the CCTV systems. First, generation of cylindrical panoramas is suggested in order to make long-time video content analysis of a defined area easier and faster. Using panoramas it is possible to reduce the time that is required to watch the video by a factor of hundreds or even thousands and perform an efficient compression of the video stream for the long-time storage. Second, the controlled stereovision option is discussed for quicker and more precise extraction of relevant information from the observed scene. Third, the thermo-vision is analyzed for faultless detection of pedestrians at night. Finally, a novel high dynamic range (HDR) technique is proposed, dedicated to the CCTV systems, in contrast to other typical entertainment oriented HDR approaches, for clear visualization of important and meaningful image details, otherwise invisible. We validated usefulness of the proposed techniques with many experiments presented in this paper.

Salient object detection models mimic the behavior of human beings and capture the most salient region/object from the images or scenes, this field contains many important applications in both computer vision and pattern recognition tasks. Despite hundreds of models that have been proposed in this field, but still, it requires a large room for research. This paper demonstrates a detailed overview of the recent progress of saliency detection models in terms of heuristic-based techniques and deep learning-based techniques. we have discussed and reviewed its co-related fields, such as Eye-fixation-prediction, RGBD salient-object-detection, co-saliency object detection, and video-saliency-detection models. We have reviewed the key issues of the current saliency models and discussed future trends and recommendations. The broadly utilized datasets and assessment strategies are additionally investigated in this paper.

In this paper we deal with the problem of calculating Automatic Exposure (AE) in digital cameras. The main problem that often occurs when taking pictures is correct exposure setting. Typically, smartphones with built-in cameras, as well as “cheap” compact digital cameras do not offer possibility of manual exposure setting. The reason is that users do not have knowledge how to set the optimal exposure, or just simply do not want to do this. Therefore, it forces that user has to rely on automatic exposure algorithms implemented in the camera. Unfortunately, these algorithms often do not perform well what causes improperly exposed images. In this paper, new algorithms for automatic exposure are proposed with the special focus on minimizing overexposed areas in the images. We have implemented proposed algorithms and conducted experiments for their efficiency, comparing with some modern cameras or smartphones. Experimental verification (enhanced by statistical analysis) shows that proposed algorithms give statistically less overexposed areas than comparative AEs.

Sentiment analysis consists in the identification of the sentiment polarity associated with a target object, such as a book, a movie or a phone. Sentiments reflect feelings and attitudes, while emotions provide a finer characterization of the sentiments involved. With the huge number of comments generated daily on the Internet, besides sentiment analysis, emotion identification has drawn keen interest from different researchers, businessmen and politicians for polling public opinions and attitudes. This paper reviews and discusses existing emotion categorization models for emotion analysis and proposes methods that enhance existing emotion research. We carried out emotion analysis by inviting experts from different research areas to produce comprehensive results. Moreover, a computational emotion sensing model is proposed, and future improvements are discussed in this paper.

In this digital era, one thing that still holds the convention is a printed archive. Printed documents find their use in many critical domains such as contract papers, legal tenders and proof of identity documents. As more advanced printing, scanning and image editing techniques are becoming available, forgeries on these legal tenders pose a serious threat. Ability to easily and reliably identify source printer of a printed document can help a lot in reducing this menace. During printing procedure, printer hardware introduces certain distortions in printed characters' locations and shapes which are invisible to naked eyes. These distortions are referred as geometric distortions, their profile (or signature) is generally unique for each printer and can be used for printer classification purpose. This paper proposes a set of features for characterizing text-line-level geometric distortions, referred as geometric distortion signatures and presents a novel system to use them for identification of the origin of a printed document. Detailed experiments performed on a set of thirteen printers demonstrate that the proposed system achieves state of the art performance and gives much higher accuracy under small training size constraint. For four training and six test pages of three different fonts, the proposed method gives 99\% classification accuracy., Comment: 20 pages

In this paper we deal with the problem of digital camera identification by photographs. Identifying camera is possible by analyzing camera’s sensor artifacts that occur during the process of photo processing. The problem of digital camera identification has been popular for a long time. Recently many effective and robust algorithms for solving this problem have been proposed. However, almost all solutions are based on state-of-the-art algorithm, proposed by Lukás et al. in 2006. Core of this algorithm is to calculate the so-called sensor pattern noise based on denoising images with wavelet-based denoising filter. Such technique is very efficient, but very time consuming. In this paper we consider tracing cameras by analyzing defects of their optical systems, like vignetting and lens distortion. We show that analysis of vignetting defect allows for recognizing brand of the camera. Lens distortion can be used to distinguish images from different cameras. Experimental evaluation was carried out on 60 devices (compact cameras and smartphones) for a total number of 12 051 images, with support of the Dresden Image Database. Proposed methods do not require denoising images with wavelet-based denoising filter what has a significant influence for speed of image processing, compared with state-of-the-art algorithm.

To avoid delays arising from a need to decrypt a video prior to transcoding and then re-encrypt it afterwards, this paper assesses a selective encryption (SE) content protection scheme. The scheme is suited to both recent standardized codecs, namely H.264/Advanced Video Coding (AVC) and High Efficiency Video Coding (HEVC). Specifically, the paper outlines a joint crypto-transcoding scheme for secure transrating of a video bitstream. That is to say it generates new video bitrates, possibly as part of an HTTP Adaptive Streaming (HAS) content delivery network. The scheme will reduce the bitrate to one or more lower desired bit-rate without consuming time in the encryption/decryption process, which would be the case when full encryption is used. In addition, the decryption key no longer needs to be exposed at intermediate middleboxes, including when transrating is performed in a cloud datacenter. The effectiveness of the scheme is variously evaluated: by examination of the SE generated visual distortion; by the extent of computational and bitrate overheads; and by choice of cipher when encrypting the selected elements within the bitstream. Results indicate that there remains: a content; quantization level (after transrating of an encrypted video); and codec-type dependency to any distortion introduced. A further recommendation is that the Advanced Encryption Standard (AES) is preferred for SE to lightweight XOR encryption, despite it being taken up elsewhere as a real-time encryption method., Revision-3 Version of Multimedia Tools and Application (Springer)

Crowd stampede has attracted significant attention of emergency management researchers in recent years. Early-warning of crowd stampede in metro station commercial area is discussed in this paper under the context of Internet of Things (IoT). Metro station commercial area is one of the entity carriers of E-commerce. IOT is a new concept of realizing intelligent sense, monitoring, tracking and management, which can be used in early-warning analysis of crowd stampede in metro station. Stampede risk early-warning in commercial area plays an important role in ensuring the operation of e-commerce online. Firstly, the laws and characteristics of the crowd movement in the commercial area of metro station are studied, which include the laeuna effect, block effect and aggravation effect. Secondly, the early-warning paradigm is constructed from four dimensions, ie. function, modules, principle and process. And then, under the IOT environment, the AHPsort II is applied to integrate the early-warning information and classify the stampede risk level. Finally, the paper takes the commercial area of Wuhan A metro station as an example to verify the practicability and effectiveness of the AHPsort II application to early-warning of crowd stampede in metro station commercial area. Funding Agencies|National Social Science Foundation of China [15AGL021]

The field of biometrics research encompasses the need to associate an identity to an individual based on the persons physiological or behaviour traits. While the use of intrusive techniques such as retina scans and finger print identification has resulted in highly accurate systems, the scalability of such systems in real-world applications such as surveillance and border security has been limited. As a branch of biometrics research, the origin of soft biometrics could be traced back to need for non-intrusive solutions for extracting physiological traits of a person. Following high number of research outcomes reported in the literature on soft biometrics, this paper aims to consolidate the scope of soft biometrics research across four thematic schemes (i) a detailed review of soft biometrics research data sets, their annotation strategies and building a largest novel collection of soft traits; (ii) the assessment of metrics that affect the performance of soft biometrics system; (iii) a comparative analysis on feature and modality level fusion reported in the literature for enhancing the system performance; and (iv) a performance analysis of hybrid soft biometrics recognition system using multi-scale criterion. The paper also presents a detailed analysis on the global traits associated to person identity such as gender, age and ethnicity. The contribution of the paper is to provide a comprehensive review of scientific literature, identify open challenges and offer insights on new research directions in the filed.

The development of unmanned aerial vehicles has been identified as a potential source of a weapon for causing operational disruptions against critical infrastructures. To mitigate and neutralise the threat posed by the misuse of drones against malicious and terrorist activity, this paper presents a holistic design of a long-range autonomous drone detection platform. The novelty of the proposed system lies in the confluence between the design of hardware and software components to effective and efficient localisation of the intruder objects. The research presented in the paper proposes the design and validation of a situation awareness component which is interfaced with the hardware component for controlling the focal length of the camera. The continuous stream of media data obtained from the region of vulnerability is processed using the object detection that is built on region based fully connected neural network. The novelty of the proposed system relies on the processing of multi-threaded dual-media input streams that are evaluated to mitigate the latency of the system. Upon the successful detection of malicious drones, the system logs the occurrence of intruders that consists of both event description and the associated media evidence for the deployment of the mitigation strategy. The analytics platform that controls the signalling of the low-cost sensing equipment contains the NVIDIA GeForce GTX 1080 for detecting drones. The experimental testbeds developed for the validation of the proposed system has been constructed to include environments and situations that are commonly faced by critical infrastructure operators such as the area of protection, drone flight path, tradeoff between the angle of coverage against the distance of coverage. The validation of the proposed system has resulted in yielding a range of intruder drone detection by 250m with an accuracy of 95.5%.

3D modelling of man-made objects is widely used in the cultural heritage sector, among others. It is relevant for its documentation, dissemination and preservation. Related to historical fabrics, weaves and weaving techniques are still mostly represented in forms of 2D graphics and textual descriptions. However, complex geometries are difficult to represent in such forms, hindering the way this legacy is transmitted to new generations. In this paper, we present the design and implementation of SILKNOW’s Virtual Loom, an interactive tool aimed to document, preserve and represent in interactive 3D forms historical weaves and weaving techniques of silk fabrics, dating from the 15th to the 19th centuries. To that end, our tool only requires an image of a historical fabric. Departing from this image, the tool automatically subtracts the design, and allows the user to apply different weaves and weaving techniques. In its current version, the tool embeds five traditional weaving techniques, 39 weaves and six types of yarns, which have been defined thanks to close collaboration of experts in computer graphics, art history and historical fabrics. Additionally, users can change the color of yarns and produce different 3D representations for a given fabric, which are interactive in real time. In this paper, we bring the details of the design and implementation of this tool, focusing on the input data, the strategy to process images, the 3D modelling of yarns, the definition of weaves and weaving techniques and the graphical user interface. In the results section, we show some examples of image analysis in order to subtract the design of historical fabrics, and then we provide 3D representations for all the considered weaving techniques, combining different types of yarns.

Augmented Reality applications are set to revolutionize the smartphone industry due to the integration of RGB-D sensors into mobile devices. Given the large number of smartphone users, efficient storage and transmission of RGB-D data is of paramount interest to the research community. While there exist Video Coding Standards such as HEVC and H.264/AVC for compression of RGB/texture component, the coding of depth data is still an area of active research. This paper presents a method for coding depth videos, captured from mobile RGB-D sensors, by planar segmentation. The segmentation algorithm is based on Markov Random Field assumptions on depth data and solved using Graph Cuts. While all prior works based on this approach remain restricted to images only and under noise-free conditions, this paper presents an efficient solution to planar segmentation in noisy depth videos. Also presented is a unique method to encode depth based on its segmented planar representation. Experiments on depth captured from a noisy sensor (Microsoft Kinect) shows superior Rate-Distortion performance over the 3D extension of HEVC codec.

For realizing quick and accurate access to desired information and ef- fective advertisements or election campaigns, personalized tweet recommendation is highly demanded. Since multimedia contents including tweets are tools for users to convey their sentiment, users’ interest in tweets is strongly influenced by sen- timent factors. Therefore, successful personalized tweet recommendation can be realized if sentiment in tweets can be estimated. However, sentiment factors were not taken into account in previous works and the performance of previous methods may be limited. To overcome the limitation, a method for sentiment-aware per- sonalized tweet recommendation through multimodal Field-aware Factorization Machines (FFM) is newly proposed in this paper. Successful personalized tweet recommendation becomes feasible through the following three contributions: (i) sentiment factors are newly introduced into personalized tweet recommendation, (ii) users’ interest is modeled by deriving multimodal FFM that enables collabora- tive use of multiple factors in a tweet, i.e., publisher, topic and sentiment factors, and (iii) the effectiveness of using sentiment factors as well as publisher and topic factors is clarified from results of experiments using real-world datasets related to worldwide hot topics, “#trump”, “#hillaryclinton” and “#ladygaga”. In addition to showing the effectiveness of the proposed method, the applicability of the pro- posed method to other tasks such as advertisement and social analysis is discussed as a conclusion and future work of this paper.

This paper describes a system for performing alignment of subtitles to audio on multigenre broadcasts using a lightly supervised approach. Accurate alignment of subtitles plays a substantial role in the daily work of media companies and currently still requires large human effort. Here, a comprehensive approach to performing this task in an automated way using lightly supervised alignment is proposed. The paper explores the different alternatives to speech segmentation, lightly supervised speech recognition and alignment of text streams. The proposed system uses lightly supervised decoding to improve the alignment accuracy by performing language model adaptation using the target subtitles. The system thus built achieves the third best reported result in the alignment of broadcast subtitles in the Multi–Genre Broadcast (MGB) challenge, with an F1 score of 88.8%. This system is available for research and other non–commercial purposes through webASR, the University of Sheffield’s cloud–based speech technology web service. Taking as inputs an audio file and untimed subtitles, webASR can produce timed subtitles in multiple formats, including TTML, WebVTT and SRT.

It is of great interest in spectral-spatial features classification for hyperspectral images (HSI) with high spatial resolution. This paper presents a novel Spectral-spatial classification method for improving hyperspectral image classification accuracy. Specifically, a new texture feature extraction algorithm exploits spatial texture feature from spectrum is proposed. It employs local binary patterns (LBPs) in order to extract the image texture feature with respect to spectrum information diversity (SID) to measure the differences of spectrum information. The classifier adopted in this work is support vector machine (SVM) because of its outstanding classification performances. In this paper, two real hyperspectral image datasets are used for testing the performance of the proposed method. Our experimental results from real hyperspectral images indicate that the proposed framework can enhance the classification accuracy compare to traditional alternatives.

In recent years, the use of mobile and cloud Computing is rapidly growing. In other word, mobile and cloud Computing play an important role in everybody’s life today. It carries rich computational resources to mobile users, network operators, as well as cloud computing providers. Moreover, the explosion of multimedia big data (image, video, 3D, etc.) in mobile and cloud computing have created unprecedented opportunities and fundamental security and privacy challenges as they are not just big in volume, but also unstructured and multi-modal. Therefore, the papers of this special issue address variety of security and privacy issues of multimedia big data in mobile and cloud computing and security of other aspects of mobile and cloud networks and also emphasizes many open questions. We anticipate that papers of this special issue will open new entrance for further research and technology improvements in this important area.

Wireless visual sensor networks (WVSNs) are composed of a large number of visual sensor nodes covering a specific geographical region. This paper addresses the target detection problem within WVSNs where visual sensor nodes are left unattended for long-term deployment. As battery energy is a critical issue it is always challenging to maximize the network’s lifetime. In order to reduce energy consumption, nodes undergo cycles of active-sleep periods that save their battery energy by switching sensor nodes ON and OFF, according to predefined duty cycles. Moreover, adaptive compressive sensing is expected to dynamically reduce the size of transmitted data through the wireless channel, saving communication bandwidth and consequently saving energy. This paper derives for the first time an analytical framework for selecting node’s duty cycles and dynamically choosing the appropriate compression rates for the captured images and videos based on their sparsity nature. This reduces energy waste by reaching the maximum compression rate for each dataset without compromising the probability of detection. Experiments were conducted on different standard datasets resembling different scenes; indoor and outdoor, for single and multiple targets detection. Moreover, datasets were chosen with different sparsity levels to investigate the effect of sparsity on the compression rates. Results showed that by selecting duty cycles and dynamically choosing the appropriate compression rates, the desired performance of detection can be achieved with adaptive CS and at the same time saving energy, where the proposed framework results in an 70% on average energy saving as compared to transmitting the captured image without CS.

An elementary visual unit – the viseme is concerned in the paper in the context of preparing the feature vector as a main visual input component of Audio-Visual Speech Recognition systems. The aim of the presented research is a review of various approaches to the problem, the implementation of algorithms proposed in the literature and a comparative research on their effectiveness. In the course of the study an optimal feature vector construction and an appropriate selection of the classifier were sought. The experimental research was conducted on the basis of a spoken corpus in which speech was represented both acoustically and visually. The extracted features represented three types: geometrical, textural and mixed ones. The features were processed employing the classification algorithms based on Hidden Markov Models and Sequential Minimal Optimization. Tests were carried out employing the processed video material recorded with English native speakers who read specially prepared list of commands. The obtained results are discussed in the paper.

A novel scheme for retrieving users’ desired contents, i.e., contents with topics in which users are interested, from multiple social media platforms is pre- sented in this paper. In existing retrieval schemes, users first select a particular platform and then input a query into the search engine. If users do not specify suitable platforms for their information needs and do not input suitable queries corresponding to the desired contents, it becomes difficult for users to retrieve the desired contents. The proposed scheme extracts the hierarchical structure of content groups (sets of contents with similar topics) from different social media platforms, and it thus becomes feasible to retrieve desired contents even if users do not specify suitable platforms and do not input suitable queries. This paper has two contributions: (1) A new feature extraction method, Locality Preserving Canoni- cal Correlation Analysis with multiple social metadata (LPCCA-MSM) that can detect content groups without the boundaries of different social media platforms is presented in this paper. LPCCA-MSM uses multiple social metadata as auxiliary information unlike conventional methods that only use content-based information such as textual or visual features. (2) The proposed novel retrieval scheme can re- alize hierarchical content structuralization from different social media platforms. The extracted hierarchical structure shows various abstraction levels of content groups and their hierarchical relationships, which can help users select topics re- lated to the input query. To the best of our knowledge, an intensive study on such an application has not been conducted; therefore, this paper has strong nov- elty. To verify the effectiveness of the above contributions, extensive experiments for real-world datasets containing YouTube videos and Wikipedia articles were conducted.

Contrast enhancement is a very important issue in image processing, pattern recognition and computer vision. Fuzzy logic based techniques perform enhancement using more detailed information of grayness of an image. However, these methods do not perform well on images taken in uncontrolled environment which pose different challenges such as illumination variation, perspective distortion and viewpoint variation. In this paper, we have worked to devise a more robust image enhancement method using fuzzy logic. We propose a novel multi scale entropy based measurement performed using fuzzy logic image processing and utilize it to define and enhance the contrast. For this purpose, we present a mathematical formula to calculate contrast using an adaptive amplification constant. Our approach uses both the local and global entropy information. We have experimented our algorithm on images from Crowd Counting UCF dataset, which contains very dense crowds and complex texture that stands in line with the challenges targeted in this paper. The results show an improved quality than original dataset images and prove that our method enhances the images with a more dynamic ranged contrast as well as better visual results.

This paper presents the Dynamic Adaptive Streaming over CoAP (DASCo), a solution for adaptive media streaming in the Internet of Things (IoT) environment. DASCo combines DASH (Dynamic Adaptive Streaming over HTTP), the widespread open standard for HTTP-compliant streaming, with Constrained Application Protocol (CoAP), the vendor-independent web transfer protocol designed for resource-constrained devices. The proposed solution uses DASH formats and mechanisms to make media segments available for consumers, and exploits CoAP to deliver media segments to consumers’ applications. Consequently, the DASCo player offers native interoperability with IoT devices that are accessed via CoAP, thus it allows easy access to data collected by different sensors in order to enrich the multimedia services. In the paper we present an overview of constraints of default CoAP implementation with respect to media streaming, and propose guidelines for development of adaptive streaming service over CoAP. Moreover, we discuss the features of CoAP that can be investigated when designing an efficient adaptive algorithm for DASCo. Presented experimental results show poor performance of DASCo when default values of CoAP transmission parameters have been used. However, adjusting the parameters according to the network conditions considerably improves DASCo efficiency. Furthermore, in bad network conditions the enhanced DASCo is characterized by a more stable download rate compared to DASH. This feature is important in the context of dynamic media adaptation, since it allows an adaptation algorithm to better fit media bit rate with download conditions.

The automatic analysis of the surgical process, from videos recorded during surgeries, could be very useful to surgeons, both for training and for acquiring new techniques. The training process could be optimized by automatically providing some targeted recommendations or warnings, similar to the expert surgeon’s guidance. In this paper, we propose to reuse videos recorded and stored during cataract surgeries to perform the analysis. The proposed system allows to automatically recognize, in real time, what the surgeon is doing: what surgical phase or, more precisely, what surgical step he or she is performing. This recognition relies on the inference of a multilevel statistical model which uses 1) the conditional relations between levels of description (steps and phases) and 2) the temporal relations among steps and among phases. The model accepts two types of inputs: 1) the presence of surgical tools, manually provided by the surgeons, or 2) motion in videos, automatically analyzed through the Content Based Video retrieval (CBVR) paradigm. Different data-driven statistical models are evaluated in this paper. For this project, a dataset of 30 cataract surgery videos was collected at Brest University hospital. The system was evaluated in terms of area under the ROC curve. Promising results were obtained using either the presence of surgical tools (A z = 0.983) or motion analysis (A z = 0.759). The generality of the method allows to adapt it to other kinds of surgeries. The proposed solution could be used in a computer assisted surgery tool to support surgeons during the surgery.

Lip synchronization of 3D face model is now being used in a multitude of important fields. It brings a more human, social and dramatic reality to computer games, films and interactive multimedia, and is growing in use and importance. High level of realism can be used in demanding applications such as computer games and cinema. Authoring lip syncing with complex and subtle expressions is still difficult and fraught with problems in terms of realism. This research proposed a lip syncing method of realistic expressive 3D face model. Animated lips requires a 3D face model capable of representing the myriad shapes the human face experiences during speech and a method to produce the correct lip shape at the correct time. The paper presented a 3D face model designed to support lip syncing that align with input audio file. It deforms using Raised Cosine Deformation (RCD) function that is grafted onto the input facial geometry. The face model was based on MPEG-4 Facial Animation (FA) Standard. This paper proposed a method to animate the 3D face model over time to create animated lip syncing using a canonical set of visemes for all pairwise combinations of a reduced phoneme set called ProPhone. The proposed research integrated emotions by the consideration of Ekman model and Plutchik’s wheel with emotive eye movements by implementing Emotional Eye Movements Markup Language (EEMML) to produce realistic 3D face model. © 2017 Springer Science+Business Media New York

Interactive live streaming is a key feature of applications and platforms in which the actions of the viewers affect the content of the stream. In those, a minimal capture-display delay is critical. Though recent technological advances have certainly made it possible to provide web-based interactive live-streaming, little research is available that compares the real-world performance of the different web-based schemes. In this paper we use educational remote laboratories as a case study. We analyze the restrictions that web-based interactive live-streaming applications have, such as a low delay. We also consider additional characteristics that are often sought in production systems, such as universality and deployability behind institutional firewalls. The paper describes and experimentally compares the most relevant approaches for the study. With the provided descriptions and real-world experimental results, researchers, designers and developers can: a) select among the interactive live-streaming approaches which are available for their real-world systems, b) decide which one is most appropriate for their purpose, and c) know what performance and results they can expect.

In the digital right management value chain, digital watermarking technology plays a very important role in digital product’s security, especially on its usage tracking and copyrights infringement authentication. However, watermark procedures can only effectively support copyright protection processes if they are applied as part of an appropriate watermark protocol. In this regard, a number of watermark protocols have been proposed in the literature and have been shown to facilitate the use of digital watermarking technology as copyright protection. One example of such protocols is the anonymous buyer-seller watermarking protocol. Although there are a number of protocols that have been proposed in the literature and provide suitable solutions, they are mainly designed as a watermarking protocol for the first-hand market and are unsuitable for second-hand transactions. As the complexity of online transaction increases, so does the size of the digital second-hand market. In this paper, we present a new buyer-seller watermark protocol that addresses the needs of customer’s rights problem in the digital secondary market. The proposed protocol consists of five sub-protocols that cover the registration process, watermarking process for the first, second and third-hand transactions as well as the identification & arbitration processes. This paper provides analysis that compares the proposed protocols with existing state-of-the-arts and shows that it has met not only all the buyer’s and seller’s requirements in the traditional sense but also accommodates the same requirements in the secondary market.

Design Patterns (DPs) are acknowledged as powerful conceptual tools to improve design quality and to reduce time and cost of the development process by effect of the reuse of "good" design solutions. In many fields (e.g., software engineering, web engineering, interface design) patterns are widely used by practitioners and are also investigated from a research perspective. Still, they have been seldom explored in the arena of Recommender Systems (RSs). RSs provide suggestions ("recommendations") for items that are likely to be appropriate for the user profile, and are increasingly adopted in content-intensive multimedia applications to complement traditional forms of search in large information spaces. This paper explores RSs through the lens of User Interface (UI) Design Patterns. We have performed a systematic analysis of 54 recommendation-empowered content-intensive multimedia applications, in order to: (i) discover the occurrences of existing domain independent UI patterns; (ii) identify frequently adopted UI solutions that are not modelled by existing patterns, and define a set of new UI patterns, some of which are specific of the interfaces for recommendation features while others can be useful also in a broader context. The results of our inspection have been discussed with and evaluated by a team of experts, leading to a consolidated set of 14 new patterns that are reported in the paper. Reusing pattern-based design solutions instead of building new solutions from scratch enables novice and expert designers to build good UIs for Recommendation-empowered content intensive multimedia applications more effectively, and ultimately can improve the UX experience in this class of systems. From a broader perspective, our work can stimulate future research bridging Recommender Systems, Web Engineering and Interface Design by means of Design Patterns, and highlights new research directions also discussed in the paper.

The reception of multimedia applications often depends on the quality of processed and displayed visual content. This is the main reason for the development of automatic image quality assessment (IQA) techniques which try to mimic properties of human visual system and produce objective scores for evaluated images. Most of them require a training step in which subjective scores, obtained in tests with human subjects, are used for parameters tuning. In this paper, it is shown that pairwise score differences (PSD) can be successfully used for training a full-reference hybrid IQA measure based on the least absolute shrinkage and selection operator (lasso) regression. The results of extensive experimental evaluation on four largest IQA benchmarks show that the proposed IQA technique is statistically better than its version trained using raw scores, and both approaches are statistically better than state-of-the-art full-reference IQA measures. They are also better than other hybrid approaches. In the paper, the evaluation protocol is extended with tests using PSD.

This paper presents an approach to gait recognition based on a single consumer accelerometer, built in most present mobile devices. It does not propose a completely novel algorithm, but rather investigates better ways to exploit the Dynamic Time Warping (DTW), which is still one of the most used at present in literature. To this aim, the paper presents both a new segmentation algorithm to split the gait signal into cycles/steps, and investigates the best way to use the possibly segmented signal for recognition. Summarizing, the first contribution of the present work is the proposal of a new segmentation algorithm for the gait signal, which does not require any pre-processing, either interpolation or noise reduction, to enhance the original signal, and its comparison with two other state-of-the-art step segmentation algorithms. The second contribution is related to the extensive tests performed with the five different investigated matching methods. The tests are carried out exploiting all compared segmentation algorithms and three different datasets, collected using different sensors: the originally exploited BWR dataset, that includes walk templates from 30 volunteers, and two huge datasets used for this kind of testing, namely the ZJU-gaitacc and the OU-ISIR Inertial Sensor Database. Tests have been performed in both verification mode, either single-template or multiple-template, and identification mode, both closed and open set. The latter is rarely found in literature though representing the most frequently predictable applicative setting. It is worth underlining that the final goal is to allow using low-cost, built-in sensors that nowadays equip most smartphones. The best result in closed set identification, which is the identification mode usually reported in literature, is achieved using the most constrained method, i.e., limiting the walks in the gallery and in the probe to have a similar number of steps. It reaches ź93 % of Recognition Rate (RR) on ZJU-gaitacc dataset. The best result obtained with methods exploiting segmentation to overcome the mentioned limitation reaches ź83 % of Recognition Rate (RR) on the same dataset, using our proposed algorithm. The best results in verification is achieved using multiple templates per user, again without segmentation, with an Equal Error Rate (EER) of 0.09, while the best results with segmentation is achieved again with our algorithm and is and EER of 0.10. This is a very good result for a soft biometrics as gait if often considered. As expected, open set identification achieves lower performance.

Facial expressions are one of the most powerful, natural and immediate means for human being to communicate their emotions and intensions. Recognition of facial expression has many applications including human-computer interaction, cognitive science, human emotion analysis, personality development etc. In this paper, we propose a new method for the recognition of facial expressions from single image frame that uses combination of appearance and geometric features with support vector machines classification. In general, appearance features for the recognition of facial expressions are computed by dividing face region into regular grid (holistic representation). But, in this paper we extracted region specific appearance features by dividing the whole face region into domain specific local regions. Geometric features are also extracted from corresponding domain specific regions. In addition, important local regions are determined by using incremental search approach which results in the reduction of feature dimension and improvement in recognition accuracy. The results of facial expressions recognition using features from domain specific regions are also compared with the results obtained using holistic representation. The performance of the proposed facial expression recognition system has been validated on publicly available extended Cohn-Kanade (CK+) facial expression data sets., Facial expressions, Local representation, Appearance features, Geometric features, Support vector machines

At present, the most reliable method to obtain end-user perceived quality is through subjective tests. In this paper, the impact of automatic region-of-interest (ROI) coding on perceived quality of mobile video is investigated. The evidence, which is based on perceptual comparison analysis, shows that the coding strategy improves perceptual quality. This is particularly true in low bit rate situations. The ROI detection method used in this paper is based on two approaches: - (1) automatic ROI by analyzing the visual contents automatically, and; - (2) eye-tracking based ROI by aggregating eye-tracking data across many users, used to both evaluate the accuracy of automatic ROI detection and the subjective quality of automatic ROI encoded video. The perceptual comparison analysis is based on subjective assessments with 54 participants, across different content types, screen resolutions, and target bit rates while comparing the two ROI detection methods. The results from the user study demonstrate that ROI-based video encoding has higher perceived quality compared to normal video encoded at a similar bit rate, particularly in the lower bit rate range.

In this paper we propose a novel approach for pattern recognition problems with non-uniform classes of images. The main concept of this classification method is to describe classes of images with their fuzzy portraits. This approach is a good generalization of the algorithm. The fuzzy set is calculated as a preliminary result of the algorithm before the final decision or rejection that solves the problem of uncertainty at the boundaries of classes. We use the method to solve the problem of knife detection in still images. The main aim of this paper is to test fuzzy classification with feature vectors in a real environment. We used selected MPEG-7 descriptor schemes as feature vectors. The method was experimentally validated on a dataset of over 12,000 images. The article describes the results of six experiments which confirm the accuracy of our method. In addition we conducted a test with enhanced images, achieved with two state-of-the-art super-resolution algorithms.

The recommendation system are widely adopted in today's mainstream online sharing services, providing useful prediction of user's rating or user's preferences of sharing items (such as products, movies, books, and news articles). A key challenge of recommendation systems in sharing economy is to employ prediction algorithms to estimate the matching items with considering their interests and needs. The environment-context has been recognized as an important factor to consider in personalized recommender systems. Since dynamic information in environment-context describes the situation of items and users, the information affects the user's decision process essentially to apply in recommender systems. However, most model-based collaborative filtering approaches such as Matrix Factorization do not provide an easy way of integrating context information into the model. In this paper, we introduce a Multidimensional Trust model based on Tensor Factorization. The generalization of Matrix Factorization allows for a flexible and generic integration of contextual information. According to the different types of context, the Multidimensional Trust model considers the additional dimensions for the representation of the data as a tensor. This is achieved by going through the collecting user's behavior based on rating analysis and identification of users' historical activity and viewing patterns. The benefits behavior solutions, which use the handle intelligently to meet the users' needs, are the focus of this paper.

The diagnosis of breast tumors will be affected by many factors, which will directly lead to deviations in the diagnosis results. In order to improve the diagnosis of breast tumors and improve the cure rate, this study based on mammography dynamic nuclear magnetic resonance imaging technology, proposed an image processing and image segmentation technology suitable for breast tumor structure to achieve effective segmentation of breast tumor sequence image segmentation. In addition, this paper uses genetic algorithm to select all features and obtain the best classification subset. Finally, this paper designs experiments to analyze the effectiveness of the proposed algorithm. Studies have shown that the image data of this study contains relevant information that can predict the efficacy of neoadjuvant chemotherapy for breast cancer and has excellent effects on the diagnosis of breast tumors and can provide a theoretical reference for subsequent related research.