Search

This paper describes our winning entry in the ImageCLEF 2015 image sentence generation task. We improve Google’s CNN-LSTM model by introducing concept-based sentence reranking, a data-driven approach which exploits the large amounts of concept-level annotations on Flickr. Different from previous usage of concept detection that is tailored to specific image captioning models, the propose approach reranks predicted sentences in terms of their matches with detected concepts, essentially treating the underlying model as a black box. This property makes the approach applicable to a number of existing solutions. We also experiment with fine tuning on the deep language model, which improves the performance further. Scoring METEOR of 0.1875 on the ImageCLEF 2015 test set, our system outperforms the runner-up (METEOR of 0.1687) with a clear margin.

The mechanisms underlying the effects of electroconvulsive therapy (ECT) in major depressive disorder (MDD) are not fully understood. Resting-state functional magnetic resonance imaging (rs-fMRI) is a new tool to study the effects of brain stimulation interventions, particularly ECT. The authors aim to investigate the mechanisms of ECT in MDD by rs-fMRI. They used rs-fMRI to measure functional changes in the brain of first-episode, treatment-naive MDD patients (n = 23) immediately before and then following 8 ECT sessions (brief-pulse square-wave apparatus, bitemporal). They also computed voxel-wise amplitude of low-frequency fluctuation (ALFF) as a measure of regional brain activity and selected the left subgenual anterior cingulate cortex (sgACC) to evaluate functional connectivity between the sgACC and other brain regions. Increased regional brain activity measured by ALFF mainly in the left sgACC following ECT. Functional connectivity of the left sgACC increased in the ipsilateral parahippocampal gyrus, pregenual ACC, contralateral middle temporal pole, and orbitofrontal cortex. Importantly, reduction in depressive symptoms were negatively correlated with increased ALFF in the left sgACC and left hippocampus, and with distant functional connectivity between the left sgACC and contralateral middle temporal pole. That is, across subjects, as depression improved, regional brain activity in sgACC and its functional connectivity increased in the brain. Eight ECT sessions in MDD patients modulated activity in the sgACC and its networks. The antidepressant effects of ECT were negatively correlated with sgACC brain activity and connectivity. These findings suggest that sgACC-associated prefrontal-limbic structures are associated with the therapeutic effects of ECT in MDD.

This paper introduces the 3D numerical simulation of unsteady turbulent flow in the entire flow passage of a Francis turbine with computational fluid dynamics (CFD) technology. The boundary conditions have been implemented based on the 3D averaged Navier-Stokes equations. The governing equations are discreted on space by the finite volume method and on time step by the finite difference method. The 3D unsteady turbulent flow in an entire Francis turbine model is calculated successfully using the CFX-TASCflow software and RNG k-ɛ turbulence model. Transient flow fields are simulated in the spiral case, the distributor, the runner, and the draft tube for the optimum operating condition. Meanwhile, the velocity and pressure at any points in the flow fields can be obtained so as to provide the great value on the performance prediction. By means of the numerical simulation in a flow field, it is verified that every component in a Francis turbine model is designed reasonably. The basis for the further researches on hydraulic turbines is also built.

Dermatopontin (DPT) was recently found as a downstream target of vitamin D receptor, which is a key molecule in the 1,25-dihydroxy-vitamin D(3) anti-hepatoma proliferation pathway. MCTx-1 from Millepora, a homolog of DPT, is identified as a cytotoxin towards leukemia cells. The aim of this study was to analyze DPT expression in hepatocellular carcinoma (HCC) based on the analysis for DPT gene in normal tissues in order to estimate its function in the progression of HCC. DPT mRNA expression was analyzed in normal tissues and HCC cell lines by RT-PCR, and in HCC tissue by RT-PCR and real-time PCR. Its protein was examined in HCC tissues by Western blot and immunohistochemistry assays. Meanwhile, transforming growth factor-beta1 (TGF-beta1) that is closely associated with HCC and DPT was observed by immunohistochemistry in HCC tissues. The results showed that DPT mRNA was strongly expressed in human fetal and adult liver, kidney, and spleen, weakly in ovary and heart, and absent in other tissues and HCC cell lines examined. Its mRNA was significantly downregulated in HCC tissues, while its protein was weakly expressed in tumor compared with non-tumor. DPT is located mainly in the cytoplasm of several cell types in the liver; it has been identified also in the extracellular matrix of the skin. TGF-beta1 was observed in extensive tumor tissue of HCC. This fact suggests that DPT can play various roles in different tissues and might be a molecule related to carcinogenesis and the progression of HCC via possible interaction with TGF-beta1 and other potential mechanisms.

Social image analysis and retrieval is important for helping people organize and access the increasing amount of user tagged multimedia. Since user tagging is known to be uncontrolled, ambiguous, and overly personalized, a fundamental problem is how to interpret the relevance of a user-contributed tag with respect to the visual content the tag is describing. Intuitively, if different persons label visually similar images using the same tags, these tags are likely to reflect objective aspects of the visual content. Starting from this intuition, we propose in this paper a neighbor voting algorithm which accurately and efficiently learns tag relevance by accumulating votes from visual neighbors. Under a set of well-defined and realistic assumptions, we prove that our algorithm is a good tag relevance measurement for both image ranking and tag ranking. Three experiments on 3.5 million Flickr photos demonstrate the general applicability of our algorithm in both social image retrieval and image tag suggestion. Our tag relevance learning algorithm substantially improves upon baselines for all the experiments. The results suggest that the proposed algorithm is promising for real-world applications.

Although it has been studied for years by the computer vision and machine learning communities, image annotation is still far from practical. In this paper, we propose a novel attempt at model-free image annotation, which is a data-driven approach that annotates images by mining their search results. Some 2.4 million images with their surrounding text are collected from a few photo forums to support this approach. The entire process is formulated in a divide-and-conquer framework where a query keyword is provided along with the uncaptioned image to improve both the effectiveness and efficiency. This is helpful when the collected data set is not dense everywhere. In this sense, our approach contains three steps: 1) the search process to discover visually and semantically similar search results, 2) the mining process to identify salient terms from textual descriptions of the search results, and 3) the annotation rejection process to filter out noisy terms yielded by Step 2. To ensure real-time annotation, two key techniques are leveraged - one is to map the high-dimensional image visual features into hash codes, the other is to implement it as a distributed system, of which the search and mining processes are provided as Web services. As a typical result, the entire process finishes in less than 1 second. Since no training data set is required, our approach enables annotating with unlimited vocabulary and is highly scalable and robust to outliers. Experimental results on both real Web images and a benchmark image data set show the effectiveness and efficiency of the proposed algorithm. It is also worth noting that, although the entire approach is illustrated within the divide-and- conquer framework, a query keyword is not crucial to our current implementation. We provide experimental results to prove this.

This tutorial focuses on challenges and solutions for content-based image annotation and retrieval in the context of online image sharing and tagging. We present a unified review on three closely linked problems, i.e., tag assignment, tag refinement, and tag-based image retrieval. We introduce a taxonomy to structure the growing literature, understand the ingredients of the main works, clarify their connections and difference, and recognize their merits and limitations. Moreover, we present an open-source testbed, with training sets of varying sizes and three test datasets, to evaluate methods of varied learning complexity. A selected set of eleven representative works have been implemented and evaluated. During the tutorial we provide a practice session for hands on experience with the methods, software and datasets. For repeatable experiments all data and code are online at http://www.micc.unifi.it/tagsurvey

How to estimate cross-media relevance between a given query and an unlabeled image is a key question in the MSR-Bing Image Retrieval Challenge. We answer the question by proposing cross-media relevance fusion, a conceptually simple framework that exploits the power of individual methods for cross-media relevance estimation. Four base cross-media relevance functions are investigated, and later combined by weights optimized on the development set. With DCG25 of 0.5200 on the test dataset, the proposed image retrieval system secures the first place in the evaluation.

This abstract paper sketches our research towards Structured Semantic Embedding of multimedia data. Though a tag may have multiple senses with completely different visual imagery, current semantic embedding methods represent the tag by a single vector regardless of its senses. We challenge this convention, arguing the importance of adding semantic structures into semantic embedding. In particular, we develop Hierarchical Semantic Embedding, a simple model that exploits the WordNet hierarchy to make the semantic embedding structured to some extent. We demonstrate the viability of structured semantic embedding for tag disambiguation and zero-shot image tagging.

Given the difficulty of acquiring labeled examples for many fine-grained visual classes, there is an increasing interest in zero-shot image tagging, aiming to tag images with novel labels that have no training examples present. Using a semantic space trained by a neural language model, the current state-of-the-art embeds both images and labels into the space, wherein cross-media similarity is computed. However, for labels of relatively low occurrence, its similarity to images and other labels can be unreliable. This paper proposes Hierarchical Semantic Embedding (HierSE), a simple model that exploits the WordNet hierarchy to improve label embedding and consequently image embedding. Moreover, we identify two good tricks, namely training the neural language model using Flickr tags instead of web documents, and using partial match instead of full match for vectorizing a WordNet node. All this lets us outperform the state-of-the-art. On a test set of over 1,500 visual object classes and 1.3 million images, the proposed model beats the current best results (18.3% versus 9.4% in hit@1).

Background Modified electroconvulsive therapy (MECT) has been regarded as the most effective antidepressant therapy, despite its cognitive side effects. However, how MECT influences problem-solving capacity in major depressive disorder (MDD), as well as its underlying neurobiological mechanisms, remains unclear. The present study aimed to assess alterations in problem-solving capacity after MECT and to explore spontaneous brain activity using amplitudes of low-frequency fluctuations (ALFF)/fractional ALFF. Methods Thirteen first-episode, treatment-naive MDD patients treated by MECT were recruited. We collected resting-state functional magnetic resonance imaging, and we evaluated their Modified Card Sorting Test performance before and after single-session MECT. Another 11 MDD patients without MECT were also recruited and interviewed with Modified Card Sorting Test twice as a control group. Results After a single MECT, MDD patients showed significantly decreased ALFF in the right cerebellar posterior lobe. Compared to the control group, perseverative errors significantly decreased after MECT, controlling for practice effects. Some cognitive functional changes significantly correlated to changed ALFF in several brain regions, including Brodmann areas BA9, BA19, BA 21, and BA48, right thalamus, left cerebellum, and right postcentral gyrus. Conclusions The MECT could improve problem-solving capacity, even after controlling for practice effects, and it could induce changes in spontaneous brain activity. These changes in cognitive functioning might result from changes in the cerebral functions of some regions, including frontal cortex, a key region for problem-solving capacity.

This paper proposed a framework to assist highlighting and annotating music utilization situation automatically within videos, further to supervise and protect music copyrights. Nowadays, music copyrighters pay attention to their rights increasingly, thus music embedded in TV channel videos should be validated to avoid infringing. Our framework supports Music Copyrighter Society of China(MCSC) to do statistic works to protect the copyright owners. In our framework, through AV separation, feature extractor, classification and assemblage functions, music positioning could be confirmed effectively. Then applying music fingerprint retrieving, music could be annotated automatically with high accuracy. Moreover, our framework is a closed loop self-adaptation system as it can be re-trained regularly to expand annotation database and enhance classifier's efficiency. The system based on our framework has been implemented in MCSC and its effectiveness has been evaluated in a real-life scenario. The results, on experiments of the real-life TV stations and comparisons of former works, show that the music positioning and annotation completed automatically by our system have significant improvement about over 30 times enhancement on the working efficiency.

In this paper, an algorithm, named CCHN, is proposed to solve the partitional clustering problem. An outer chaotic mechanism with annealing strategy is introduced into the competitive Hopfield neural network to construct CCHN for expecting better opportunities of converging to the optimal solution. In addition to retain the competitive characteristics of the conventional competitive Hopfield neural network, CCHN displays a rich range of complex and flexible chaotic dynamics. The chaotic dynamics and the annealing strategy guarantee the powerful searching ability and the effective convergence of CCHN. Results simulated on clustering benchmark problems show that CCHN algorithm is more likely to find an optimal or near-optimal solution with a higher successful ratio than previous algorithms.

With the increasing use of audio sensors in user generated content collection, how to detect semantic concepts using audio streams has become an important research problem. In this paper, we present a semantic concept annotation system using soundtracks/ audio of the video. We investigate three different acoustic feature representations for audio semantic concept annotation and explore fusion of audio annotation with visual annotation systems. We test our system on the data collection from HUAWEI Accurate and Fast Mobile Video Annotation Grand Challenge 2014. The experimental results show that our audio-only concept annotation system can detect semantic concepts significantly better than random guess. It can also provide significant complementary information to the visual-based concept annotation system for performance boost. Further detailed analysis shows that for interpreting a semantic concept both visually and acoustically, it is better to train concept models for the visual system and audio system using visual-driven and audio-driven ground truth separately.

Objective. Electroconvulsive therapy (ECT) is considered one of the most effective and fast-acting treatment options for depressive episodes. Little is known, however, about ECT’s enabling brain (neuro)plasticity effects, particular for plasticity of white matter pathway.Materials and Methods. We collected longitudinal diffusion tensor imaging in the first-episode, drug-naïve major depressive disorder (MDD) patientsn=24before and after a predefined time window ECT treatment. We constructed large-scale anatomical networks derived from white matter fiber tractography and evaluated the topological reorganization using graph theoretical analysis. We also assessed the relationship between topological reorganization with improvements in depressive symptoms.Results. Our investigation revealed three main findings: (1) the small-worldness was persistent after ECT series; (2) anatomical connections changes were found in limbic structure, temporal and frontal lobes, in which the connection changes between amygdala and parahippocampus correlate with depressive symptom reduction; (3) significant nodal strength changes were found in right paralimbic network.Conclusions. ECT elicits neuroplastic processes associated with improvements in depressive symptoms that act to specific local ventral frontolimbic circuits, but not small-world property. Overall, ECT induced topological reorganization in large-scale brain structural network, opening up new avenues to better understand the mode of ECT action in MDD.

Emotion recognition from speech is a challenging research area with wide applications. In this paper we explore one of the key aspects of building an emotion recognition system: generating suitable feature representation. We extract features from four angles: (1) low-level acoustic features such as intensity, F0, jitter, shimmer and spectral contours etc. and statistical functions over these features, (2) a set of features derived from segmental cepstral-based features scored against emotion-dependent Gaussian mixture models, (3) a set of features derived from a set of low-level acoustic codewords and (4) GMM supervectors constructed by stacking the means or covariance or weights of the adapted mixture components on each utterance. We apply these features for emotion recognition independently and jointly and compare their performance within this task. We build a support vector machine (SVM) classifier based on these features on the IEMOCAP database. The four-class emotion recognition accuracy of 71.9% of our system outperforms the previously reported best results on this dataset.

In this paper, we extract the core idea of state perturbation from Hopfield-type neural networks and define state perturbation formulas to describe the general way of optimization methods. Departing from the core idea and the formulas, we propose a novel optimization method related to neural network structure, named structure perturbation optimization. Our method can produce a structure transforming process to retrain Hopfield-type neural networks to get better problem-solving ability. Experiments validate that our method effectively helps Hopfield-type neural networks to escape from local minima and get superior solutions.

Due to the subjective nature of social tagging, measuring the relevance of social tags with respect to the visual content is crucial for retrieving the increasing amounts of social-networked images. Witnessing the limit of a single measurement of tag relevance, we introduce in this paper tag relevance fusion as an extension to methods for tag relevance estimation. We present a systematic study, covering tag relevance fusion in early and late stages, and in supervised and unsupervised settings. Experiments on a large present-day benchmark set show that tag relevance fusion leads to better image retrieval. Moreover, unsupervised tag relevance fusion is found to be practically as effective as supervised tag relevance fusion, but without the need of any training efforts. This finding suggests the potential of tag relevance fusion for real-world deployment.

With the increasing use of audio sensors in user generated content UGC collection, semantic concept annotation using audio streams has become an important research problem. Huawei initiates a grand challenge in the International Conference on Multimedia & Expo ICME 2014: Huawei Accurate and Fast Mobile Video Annotation Challenge. In this paper, we present our semantic concept annotation system using audio stream only for the Huawei challenge. The system extracts audio stream from the video data and low-level acoustic features from the audio stream. Bag-of-feature representation is generated based on the low-level features and is used as input feature to train the support vector machine SVM concept classifier. The experimental results show that our audio-only concept annotation system can detect semantic concepts significantly better than random guess. It can also provide important complementary information to the visual-based concept annotation system for performance boost.

Cubic and orthorhombic KNbO3 have been synthesized under hydrothermal conditions respectively by tuning impurities and alkali concentration in initial reaction compositions, and characterized by X-ray diffraction, Laser Raman, Infrared and X-ray photoelectronic spectroscopies. Phase transitions of resulting niobates induced by pressure at high temperatures were investigated in comparison with hydrothermal pressure effect. Mechanisms of stabilization of cubic KNbO3 were discussed.

Hydrogarnets Sr3M2(OH)12 (M = Cr, Fe, and Al) were synthesized from mild hydrothermal systems at 240 °C with Sr(OH)2·8H2O, Cr(NO3)3·9H2O, Fe(NO3)3·9H2O, and Al(OH)3 as the starting materials. High solubility of the input species and basic concentration in the reaction systems considerably lower crystallization temperatures and enhance the crystallinity of the resulting hydrogarnets. The structural stability and thermal properties of Sr based hydrogarnets were investigated by X-ray diffraction, DTA-TG, infrared, and X-ray photoelectron spectroscopies. With increasing temperature, Sr based hydrogarnets underwent fast dehydration due to the loss of the nearest OH- species of Sr−OH dodecahedra. During the course of decomposition, valence variations of the hydrogarnet framework ions, e.g., Fe and Cr directly determined the decomposition process and phase compositions in final decomposition products. The decomposition product of Sr3Cr2(OH)12 in air was a mixture of SrCrO4 and SrO, but in N2, a mixture of a rhom...

Image tag relevance estimation aims to automatically determine what people label about images is factually present in the pictorial content. Different from previous works, which either use only positive examples of a given tag or use positive and random negative examples, we argue the importance of relevant positive and relevant negative examples for tag relevance estimation. We propose a system that selects positive and negative examples, deemed most relevant with respect to the given tag from crowd-annotated images. While applying models for many tags could be cumbersome, our system trains efficient ensembles of Support Vector Machines per tag, enabling fast classification. Experiments on two benchmark sets show that the proposed system compares favorably against five present day methods. Given extracted visual features, for each image our system can process up to 3,787 tags per second. The new system is both effective and efficient for tag relevance estimation.

This paper proposes an efficient hybrid approach based on the stochastic competitive Hopfield neural networkSCHNN and artificial bee colony ABC, which named SCH-ABC. The hybrid algorithm aims to cope with the frequency assignment problem FAP. The objective of FAP is to minimize the cochannel interference between satellite communication systems by rearranging the frequency assignments so that they can accommodate the increasing demands. In fact, as our SCH-ABC algorithm owns good adaptability, it can not only deal with the frequency assignment problem, but also cope with other problems including the clustering, classification, the maximum clique problem etc. With the help of hybridization, SCH-ABC makes up for the defects in the Hopfield neural network and ABC while fully utilizing the advantages of the two algorithms.

One highly multi-drug-resistant, mucus-producing and foul-smelling strain of Klebsiella ozaenae was isolated from a patient in the ICU of a Chinese tertiary hospital. MICs of several clinical antimicrobials against the strain were obtained using the Vitek-2 Compact System with AST-GN13 cards and resistance genes were evaluated by PCR and gene sequencing. The strain was resistant to most of the β-lactams and quinolones tested and carried several antibiotic resistance genes, including bla(KPC-2), bla(TEM-98), bla(CTX-M-3), bla(SHV-26) and qnrS. To our knowledge, this is the first report of β-lactam and quinolone resistance genes co-existing in a K. ozaenae strain in China.

Searching for the co-occurrence of two visual concepts in unlabeled images is an important step towards answering complex user queries. Traditional visual search methods use combinations of the confidence scores of individual concept detectors to tackle such queries. In this paper we introduce the notion of bi-concepts, a new concept-based retrieval method that is directly learned from social-tagged images. As the number of potential bi-concepts is gigantic, manually collecting training examples is infeasible. Instead, we propose a multimedia framework to collect de-noised positive as well as informative negative training examples from the social web, to learn bi-concept detectors from these examples, and to apply them in a search engine for retrieving bi-concepts in unlabeled images. We study the behavior of our bi-concept search engine using 1.2 M social-tagged images as a data source. Our experiments indicate that harvesting examples for bi-concepts differs from traditional single-concept methods, yet the examples can be collected with high accuracy using a multi-modal approach. We find that directly learning bi-concepts is better than oracle linear fusion of single-concept detectors, with a relative improvement of 100%. This study reveals the potential of learning high-order semantics from social images, for free, suggesting promising new lines of research.

Given that millions of images are uploaded to the social web in a single day, how to effectively retrieve such increasing amounts of unstructured data becomes crucial. While current social platforms find images on the base of user-contributed tags, the tags are known to be subjective and noisy, and consequently put the effectiveness of social image retrieval into question. Different from existing work which focuses on analyzing individual sources of information such as textual and visual separately, in this paper we propose to fuse the heterogeneous information. We investigate the state of the art weighting methods within a linear fusion framework. Image retrieval experiments on a present day benchmark with 46 visual concept queries show encouraging results. Compared to the best search results obtained by a single source of information, the Uniform method and the Coordinate Ascent method obtain relative performance gain of 3.9% and 5.7% in terms of mean average precision. This work provides some practical guidelines for fusing multiple sources of information for improving social image retrieval.

In the equilibrium immiscible Ru-Pd system, the total energies for five possible structures, i.e., A15, D0 9 , D0 19 , L1 2 , and L6 0 , with a stoichiometry of RuPd 3 were calculated as a function of lattice constants, based on the Vienna Ab Initio Simulation Package (VASP). The calculation results predicted that the D0 19 , L1 2 , and L6 0 structures were relatively stable and had similar energies, if the spin polarization effect of the Ru atoms was considered, and that the L1 2 structure was the most stable one, if the spin polarization effect was considered to be depressed by the crystal field. Interestingly, in ion mixing experiments, a metastable crystalline Ru-Pd phase of fcc structure was indeed obtained in the Ru-Pd multilayered films with an overall composition of Ru 20 Pd 80 and its lattice constant determined by diffraction analysis also confirmed the relevance of the ab initio calculations.

Transmission characteristics of photon in the subgrade tissue is the theoretical basis of using laser imaging to detect density of compacted subgrade. After explaining the basic methods of using Monte Carlo simulation theory to study the propagation of photon in the subgrade tissue, the paper selects optical properties of several subgrade tissues, simulates the propagation of photon in the subgrade tissue by using program written with C language and presents the photon density in different radius with the light source as the center. Research shows that applying Monte Carlo to study the propagation of photon in the subgrade tissue is of great help to find out the factors which will affect the detection accuracy of non-destructive testing in principle.

Although it has been studied for years by computer vision and machine learning communities, image annotation is still far from practical. In this chapter, the authors propose a novel attempt of modeless image annotation, which investigates how effective a data-driven approach can be, and suggest annotating an uncaptioned image by mining its search results. The authors collected 2.4 million images with their surrounding texts from a few photo forum Web sites as our database to support this data-driven approach. The entire process contains three steps: (1) the search process to discover visually and semantically similar search results; (2) the mining process to discover salient terms from textual descriptions of the search results; and (3) the annotation rejection process to filter noisy terms yielded by step 2. To ensure real time annotation, two key techniques are leveraged – one is to map the high dimensional image visual features into hash codes, the other is to implement it as a distributed system, of which the search and mining processes are provided as Web services. As a typical result, the entire process finishes in less than 1 second. Since no training dataset is required, our proposed approach enables annotating with unlimited vocabulary, and is highly scalable and robust to outliers. Experimental results on real Web images show the effectiveness and efficiency of the proposed algorithm.

To learn classifiers for many visual categories, obtaining labeled training examples in an efficient way is crucial. Since a classifier tends to misclassify negative examples which are visually similar to positive examples, inclusion of such informative negatives should be stressed in the learning process. However, they are unlikely to be hit by random sampling, the de facto standard in literature. In this paper, we go beyond random sampling by introducing a novel social negative bootstrapping approach. Given a visual category and a few positive examples, the proposed approach adaptively and iteratively harvests informative negative examples from a large amount of social-tagged images. To label negative examples without human interaction, we design an effective virtual labeling procedure based on simple tag reasoning. Virtual labeling, in combination with adaptive sampling, enables us to select the most misclassified negatives as the informative samples. Learning from the positive set and a series of informative negative sets results in visual classifiers with higher accuracy. Experiments on two present-day image benchmarks employing 650K virtually labeled negative examples show the viability of the proposed approach. On a popular visual categorization benchmark our precision at 20 increases by 34%, compared to baselines trained on randomly sampled negatives. The robustness of the proposed approach is verified by a cross-dataset experiment. The results clearly show the advantage of our approach: more accurate visual categorization without the need of manually labeling any negatives.

Annotating the increasing amounts of user-contributed imagesin a personalized manner is in great demand. However,this demand is largely ignored by the mainstream of automatedimage annotation research. In this paper we aimfor personalizing automated image annotation by jointly exploitingpersonalized tag statistics and content-based imageannotation. We propose a cross-entropy based learning algorithmwhich personalizes a generic annotation model bylearning from a user’s multimedia tagging history. Usingcross-entropy-minimization basedMonte Carlo sampling, theproposed algorithm optimizes the personalization process interms of a performance measurement which can be flexiblychosen. Automatic image annotation experiments with5,315 realistic users in the social web show that the proposedmethod compares favorably to a generic image annotationmethod and a method using personalized tag statistics only.For 4,442 users the performance improves, where for 1,088users the absolute performance gain is at least 0.05 in termsof average precision. The results show the value of the proposedmethod.

Hydrogarnets Sr3M2(OH)12 (M = Cr, Fe, and Al) were synthesized from mild hydrothermal systems at 240 °C with Sr(OH)2·8H2O, Cr(NO3)3·9H2O, Fe(NO3)3·9H2O, and Al(OH)3 as the starting materials. High solubility of the input species and basic concentration in the reaction systems considerably lower crystallization temperatures and enhance the crystallinity of the resulting hydrogarnets. The structural stability and thermal properties of Sr based hydrogarnets were investigated by X-ray diffraction, DTA-TG, infrared, and X-ray photoelectron spectroscopies. With increasing temperature, Sr based hydrogarnets underwent fast dehydration due to the loss of the nearest OH- species of Sr−OH dodecahedra. During the course of decomposition, valence variations of the hydrogarnet framework ions, e.g., Fe and Cr directly determined the decomposition process and phase compositions in final decomposition products. The decomposition product of Sr3Cr2(OH)12 in air was a mixture of SrCrO4 and SrO, but in N2, a mixture of a rhom...

Interpreting the relevance of a user-contributed tag with respect to the visual content of an image is an emerging problem in social image retrieval. In the literature this problem is tackled by analyzing the correlation between tags and images represented by specific visual features. Unfortunately, no single feature represents the visual content completely, e.g., global features are suitable for capturing the gist of scenes, while local features are better for depicting objects. To solve the problem of learning tag relevance given multiple features, we introduce in this paper two simple and effective methods: one is based on the classical Borda Count and the other is a method we name UniformTagger. Both methods combine the output of many tag relevance learners driven by diverse features in an unsupervised, rather than supervised, manner.Experiments on 3.5 million social-tagged images and two test sets verify our proposal. Using learned tag relevance as updated tag frequency for social image retrieval, both Borda Count and UniformTagger outperform retrieval without tag relevance learning and retrieval with single-feature tag relevance learning. Moreover, the two unsupervised methods are comparable to a state-of-the-art supervised alternative, but without the need of any training data.

Automatic visual categorization is critically dependent on labeled examples for supervised learning. As an alternative to traditional expert labeling, social-tagged multimedia is becoming a novel yet subjective and inaccurate source of learning examples. Different from existing work focusing on collecting positive examples, we study in this paper the potential of substituting social tagging for expert labeling for creating negative examples. We present an empirical study using 6.5 million Flickr photos as a source of social tagging. Our experiments on the PASCAL VOC challenge 2008 show that with a relative loss of only 4.3% in terms of mean average precision, expert-labeled negative examples can be completely replaced by social-tagged negative examples for consumer photo categorization.

In this paper, we provide a new formulation for video queries as structured combination of concept threads, contributing to the general query-by-concept paradigm. Occupying a low-dimensional region in the concept space, concept thread defines a ranked list of video documents ordered by their combined concept predictions. This localized representation incorporates the previous concept based formulation as a special case and extends the restricted AND concept combination logic to a two-level concept inference network. We apply this new formulation to interactive video retrieval and utilize abundant feedback information to mine the latent semantic concept threads for answering complex query semantics. Simulative experiments which are conducted on two years’ TRECVID data sets with two sets of concept lexicons demonstrate the advantage of the proposed formulation. The proposed query formulation offers some 60% improvements over the simple browsing search baseline in nearly real time. It has clear advantages over c-tf-idf and achieves better results over the state-of-the-art online ordinal reranking approach. Meanwhile, it not only alleviates user’s workload significantly but also is robust to user mislabeling errors.

Automatic image tagging is important yet challenging due to the semantic gap and the lack of learning examples to model a tag's visual diversity. Meanwhile, social user tagging is creating rich multimedia content on the Web. In this paper, we propose to combine the two tagging approaches in a search-based framework. For an unlabeled image, we first retrieve its visual neighbors from a large user-tagged image database. We then select relevant tags from the result images to annotate the unlabeled image. To tackle the unreliability and sparsity of user tagging, we introduce a joint-modality tag relevance estimation method which efficiently addresses both textual and visual clues. Experiments on 1.5 million Flickr photos and 10 000 Corel images verify the proposed method.

Social image retrieval is important for exploiting the increasing amounts of amateur-tagged multimedia such as Flickr images. Since amateur tagging is known to be uncontrolled, ambiguous, and personalized, a fundamental problem is how to reliably interpret the relevance of a tag with respect to the visual content it is describing. Intuitively, if different persons label similar images using the same tags, these tags are likely to reflect objective aspects of the visual content. Starting from this intuition, we propose a novel algorithm that scalably and reliably learns tag relevance by accumulating votes from visually similar neighbors. Further, treated as tag frequency, learned tag relevance is seamlessly embedded into current tag-based social image retrieval paradigms. Preliminary experiments on one million Flickr images demonstrate the potential of the proposed algorithm. Overall comparisons for both single-word queries and multiple-word queries show substantial improvement over the baseline by learning and using tag relevance. Specifically, compared with the baseline using the original tags, on average, retrieval using improved tags increases mean average precision by 24%, from 0.54 to 0.67. Moreover, simulated experiments indicate that performance can be improved further by scaling up the amount of images used in the proposed neighbor voting algorithm.

In this technical demonstration, we showcase the SBIA system - a search-based image annotation system. At the heart of the system lies a very large-scale image search engine which indexed three million Web images and supports both text and visual queries. Given an image (with initial annotations), SBIA first finds semantically/visually similar images via the search engine, and then mines representative keywords from the retrieved images. These keywords, after annotation rejection and relevance ranking, are finally used to annotate the query image.

A new video retrieval paradigm of query-by-concept emerges recently. However, it remains unclear how to exploit the detected concepts in retrieval given a multimedia query. In this paper, we point out that it is important to map the query to a few relevant concepts instead of search with all concepts. In addition, we show that solving this problem through both text and image inputs are effective for search, and it is possible to determine the number of related concepts by a language modeling approach. Experimental evidence is obtained on the automatic search task of TRECVID 2006 using a large lexicon of 311 learned semantic concept detectors.

In the paper, our video retrieval system is presented. The system acts as a decision support system to help users to find what they want with many analysis and visualization tools provided by the system. It consists of three basic retrieval models which searches shots in text, image and concept space respectively. The results from different modalities are fused to achieve better performance. The relevance shots are shown to users in different threads and expanded in different ways to help users try their best to make correct decision during the retrieval procedure.

Though both quantity and quality of semantic concept detection in video are continuously improving, it still remains unclear how to exploit these detected concepts as semantic indices in video search, given a specific query. In this paper, we tackle this problem and propose a video search framework which operates like searching text documents. Noteworthy for its adoption of the well-founded text search principles, this framework first selects a few related concepts for a given query, by employing a tf-idf like scheme, called c-tf-idf, to measure the informativeness of the concepts to this query. These selected concepts form a concept subspace. Then search can be conducted in this concept subspace, either by a Vector Model or a Language Model. Further, two algorithms, i.e., Linear Summation and Random Walk through Concept-Link, are explored to combine the concept search results and other baseline search results in a reranking scheme. This framework is both effective and efficient. Using a lexicon of 311 concepts from the LSCOM concept ontology, experiments conducted on the TRECVID 2006 search data set show that: when used solely, search within the concept subspace achieves the state-of-the-art concept search result; when used to rerank the baseline results, it can improve over the top 20 automatic search runs in TRECVID 2006 on average by approx. 20%, on the most significant one by approx. 50%, all within 180 milliseconds on a normal PC.

AimThe aim of this study is to assess the depression of pregnant women in the aftermath of an earthquake, and to identify the social support that they obtained, their coping styles and socio-demographic factors associated with depression.MethodsA total of 128 pregnant women from three hospitals in the epicenter area were recruited immediately after the Ya'an earthquake. Their depression was investigated using the Edinburgh Postnatal Depression Scale (EPDS) with a cutoff score of 14; the social support that they obtained was measured using the Social Support Questionnaire; and their coping styles were assessed using the Coping Styles Questionnaire.ResultsImmediately after the earthquake, the incidence rate of depression in pregnant women was 35.2%, higher than that of the general pregnant population (7%-14%). The EPDS scores were significantly correlated with gestation age at the time of the earthquake, objective support, subjective support, use of support, negative coping style, and positive coping style. The regression analysis indicated that risk factors of prenatal depression include the number of children, relatives wounded, subjective support, and coping styles. A further analysis of the interaction between social support and two types of coping styles with depression showed that there was interaction effect between subjective social support and positive coping styles in relation to EPDS scores. There was an inverse relationship between low EPDS scores and positive coping styles and high social support, and vice versa.ConclusionThe timing of the occurrence of the earthquake may not necessarily affect the progress of the illness and recovery from depression, and psychological intervention could be conducted in the immediate aftermath after the earthquake. The impact of coping styles on prenatal depression appeared to be linked with social support. Helping pregnant women to adopt positive coping styles with good social support after a recent major earthquake, which is a stressor, may reduce their chances of developing prenatal depression.

Image annotation has been an active research topic in recent years due to its potentially large impact on both image understanding and Web image search. In this paper, we target at solving the automatic image annotation problem in a novel search and mining framework. Given an uncaptioned image, first in the search stage, we perform content-based image retrieval (CBIR) facilitated by high-dimensional indexing to find a set of visually similar images from a large-scale image database. The database consists of images crawled from the World Wide Web with rich annotations, e.g. titles and surrounding text. Then in the mining stage, a search result clustering technique is utilized to find most representative keywords from the annotations of the retrieved image subset. These keywords, after salience ranking, are finally used to annotate the uncaptioned image. Based on search technologies, this framework does not impose an explicit training stage, but efficiently leverages large-scale and well-annotated images, and is potentially capable of dealing with unlimited vocabulary. Based on 2.4 million real Web images, comprehensive evaluation of image annotation on Corel and U. Washington image databases show the effectiveness and efficiency of the proposed approach.

In a world with increasing amounts of digital pictures, content-based visual search is an important and scientifically challenging problem in ICT research. This thesis tackles the problem by learning from social media. The fundamental question addressed in this thesis is: what is the value of socially tagged images for visual search?