Search

This paper addresses the feature selection problem for gene expression profiles. Feature selection based on fuzzy neighborhood rough sets is very important in gene expression profiles. The process of extracting key features in gene expression profiles may have two disadvantages: (1) it may generate many redundant features, thus reducing the classification accuracy, and (2) it may ignore some information. To address the above problems, this paper proposes a fuzzy neighborhood joint entropy model based on feature selection; this model adopts the nonnegative principle of fuzzy neighborhood joint entropy to evaluate the importance of a candidate feature gene. Based on the fuzzy neighborhood joint entropy model, this paper proposes a new feature gene selection algorithm: the fuzzy neighborhood joint entropy (FNJE) algorithm. In the model, first, fuzzy neighborhood particles and fuzzy decision-making are combined with the uncertainty measure of joint entropy to construct a fuzzy neighborhood joint entropy model. Second, the importance degree of a feature is introduced as the measurement standard of candidate feature genes to evaluate the importance of each feature. Third, the algorithm uses the importance of candidate features in selecting features to reduce the redundancy of the selected features and improve the classification accuracy. In this paper, based on the UCI and gene datasets, we conduct a series of experiments. The experimental results show that the proposed algorithm can select fewer feature genes and achieve higher classification accuracy. Compared with the other four algorithms, our proposed algorithm can improve the accuracy by 0.4 % - 10.42 % and 1.16 % - 15.18 % and can reach maximum accuracy values of 90.52 % and 87.02 % with the linear-SVM and KNN classifiers, respectively. [ABSTRACT FROM AUTHOR]

Abstract This paper addresses the feature selection problem for gene expression profiles. Feature selection based on fuzzy neighborhood rough sets is very important in gene expression profiles. The process of extracting key features in gene expression profiles may have two disadvantages: (1) it may generate many redundant features, thus reducing the classification accuracy, and (2) it may ignore some information. To address the above problems, this paper proposes a fuzzy neighborhood joint entropy model based on feature selection; this model adopts the nonnegative principle of fuzzy neighborhood joint entropy to evaluate the importance of a candidate feature gene. Based on the fuzzy neighborhood joint entropy model, this paper proposes a new feature gene selection algorithm: the fuzzy neighborhood joint entropy (FNJE) algorithm. In the model, first, fuzzy neighborhood particles and fuzzy decision-making are combined with the uncertainty measure of joint entropy to construct a fuzzy neighborhood joint entropy model. Second, the importance degree of a feature is introduced as the measurement standard of candidate feature genes to evaluate the importance of each feature. Third, the algorithm uses the importance of candidate features in selecting features to reduce the redundancy of the selected features and improve the classification accuracy. In this paper, based on the UCI and gene datasets, we conduct a series of experiments. The experimental results show that the proposed algorithm can select fewer feature genes and achieve higher classification accuracy. Compared with the other four algorithms, our proposed algorithm can improve the accuracy by 0.4 $$\%$$ % $$-$$ - 10.42 $$\%$$ % and 1.16 $$\%$$ % $$-$$ - 15.18 $$\%$$ % and can reach maximum accuracy values of 90.52 $$\%$$ % and 87.02 $$\%$$ % with the linear-SVM and KNN classifiers, respectively.

Information collected through sensor measurements has the potential to improve knowledge of complex-system behavior, leading to better decisions related to system management. In this situation, and particularly when using digital twins, the quality of sensor data determines the improvement that sensors have on decision-making. The choice of the monitoring system, including sensor types and their configuration, is typically made using engineering judgement alone. As the price of sensor devices is usually low, large sensor networks have been implemented. As sensors are often used to monitor at high frequencies over long periods, very large data sets are collected. However, model predictions of system behavior are often influenced by only a few parameters. Informative data sets are thus difficult to extract as they are often hidden amid redundant and other types of irrelevant data when updating key parameter values. This study presents a methodology for selecting informative measurements within large data sets for a given model-updating task. By selecting the smallest set that maximizes the information gain, data sets can be significantly refined, leading to increased data-interpretation efficiency. Results of an excavation case study show that the information gains with refined measurement sets that are much smaller than the entire data set are better than using the data set prior to refinement for the same probability of identification, while the computational time of model updating is significantly reduced. This methodology thus supports engineers for significant data filtering to improve model-updating performance. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Jilin University (Science Edition) / Jilin Daxue Xuebao (Lixue Ban) is the property of Zhongguo Xue shu qi Kan (Guang Pan Ban) Dian zi Za zhi She and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The preprocessing of a digital image affects the statistical properties of image pixels and steganographic security. Modification of digital media is quite a trivial task nowadays, and it is required intentionally for various reasons. Therefore, it is significant to discuss the usage of the preprocessed image in steganography and its impact on the security of embedded secret information. In this work, we attempt to estimate the impact of image operations, namely scaling, cropping, and contrast enhancement over LSB steganography. We first attack the original image dataset to produce a cover image dataset, then we apply steganography methods and measure their detection accuracy concerning the original image dataset. Steganalysis is conducted by utilizing the higher-order Markov features and newly introduced Joint Entropy-based features with SVM and ensemble classifiers. Experimental results show that the detection rate of embedding degrades by 20% to 40% due to preprocessing of cover images. Results also indicate that the Markov-based SPAM and Joint Entropy-based features work well for steganalysis. [ABSTRACT FROM AUTHOR]

In several nuclear cardiac imaging applications (SPECT and PET), images are formed by reconstructing tomographic data using an iterative reconstruction algorithm with corrections for physical factors involved in the imaging detection process and with corrections for cardiac and respiratory motion. The physical factors are modeled as coefficients in the matrix of a system of linear equations and include attenuation, scatter, and spatially varying geometric response. The solution to the tomographic problem involves solving the inverse of this system matrix. This requires the design of an iterative reconstruction algorithm with a statistical model that best fits the data acquisition. The most appropriate model is based on a Poisson distribution. Using Bayes Theorem, an iterative reconstruction algorithm is designed to determine the maximum a posteriori estimate of the reconstructed image with constraints that maximizes the Bayesian likelihood function for the Poisson statistical model. The a priori distribution is formulated as the joint entropy (JE) to measure the similarity between the gated cardiac PET image and the cardiac MRI cine image modeled as a FE mechanical model. The developed algorithm shows the potential of using a FE mechanical model of the heart derived from a cardiac MRI cine scan to constrain solutions of gated cardiac PET images.

Train wheel flats are formed when wheels slip on rails. Crucial for passenger comfort and the safe operation of train systems, early detection and quantification of wheel-flat severity without interrupting railway operations is a desirable and challenging goal. Our method involves identifying the wheel-flat size by using a model updating strategy based on dynamic measurements. Although measurement and modelling uncertainties influence the identification results, they are rarely taken into account in most wheel-flat detection methods. Another challenge is the interpretation of time series data from multiple sensors. In this article, the size of the wheel flat is identified using a model-falsification approach that explicitly includes uncertainties in both measurement and modelling. A two-step important point selection method is proposed to interpret high-dimensional time series in the context of inverse identification. Perceptually important points, which are consistent with the human visual identification process, are extracted and further selected using joint entropy as an information gain metric. The proposed model-based methodology is applied to a field train track test in Singapore. The results show that the wheel-flat size identified using the proposed methodology is within the range of true observations. In addition, it is also shown that the inclusion of measurement and modelling uncertainties is essential to accurately evaluate the wheel-flat size because identification without uncertainties may lead to an underestimation of the wheel-flat size. [ABSTRACT FROM AUTHOR]

Three-way incomplete multi-attribute decision-making model (TWIMADM) for multiple decision makers is an important topic in data science, which is an effective tool for decision making. However, calculation results of neighborhood blocks are still affected by neighborhood radius. In addition, most of the existing models without considering the information from the evaluations of multiple decision makers separately is difficult to obtain a comprehensive assessment. To address these problems, this paper builds a novel multi-source TWD model based on multi-granularity ball. First, the well-established granular ball is extended to multi-granularity, by which a weighted conditional probability is generated to divide the concerned region into three parts: positive, negative and boundary region. The calculation of the multi-granularity ball is independence of the neighborhood radius parameter. Subsequently, the joint entropy is developed by using the defined multi-granularity to obtain the optimal decision. Furthermore, the relative utility function is utilized to capture the preferences of decision makers, the classification and ranking of each object are determined by calculating the threshold value and expected utility value. Evaluation information from all decision makers are combined and fused. Finally, a novel TWD model is developed to address the challenges of incomplete multi-attribute decision-making problems involving multiple decision makers. The comparative case analyses and experimental evaluations have demonstrated the stability, validity, and feasibility of our model. • We first extend the well-established granular ball to multi-granularity. • We utilize multi-granularity to compute neighborhood blocks. • We design the corresponding algorithm for multi-granularity ball generation. • Consider the multiple decision makers scenario. • We establish joint entropy to fully consider the information of each decision maker. [ABSTRACT FROM AUTHOR]

In this paper, the design of low probability of intercept (LPI) radar waveforms considers not only the performance of passive interception systems (PISs), but also radar detection and resolution performance. Waveform design is an important considerations for the LPI ability of radar. Since information theory has a powerful performance-bound description ability from the perspective of information flow, LPI waveforms are designed in this paper within the constraints of the detection performance metrics of radar and PISs, both of which are measured by the Kullback–Leibler divergence, and the resolution performance metric, which is measured by joint entropy. The designed optimization model of LPI waveforms can be solved using the sequential quadratic programming (SQP) method. Simulation results verify that the designed LPI waveforms not only have satisfactory target-detecting and resolution performance, but also have a superior low interception performance against PISs. [ABSTRACT FROM AUTHOR]

The inflow and infiltration (I&I) is an issue for many urban sewer networks (USNs), which can significantly affect system functioning. Placing sensors within the USNs is a typical approach to detect large I&I event, but deploying a limited number of sensors while achieving maximum detection reliability is challenging. While some methods are available for sensor placement, they are generally heuristic search-based methods (HSBMs) and hence the resultant sensor placement strategies (SPSs) are variable over different algorithm runs or parameterizations. This paper develops a new deterministic two-stage clustering method for SPS optimization based on information entropy. Within the first stage, the Spectral Clustering method is applied to assign USN nodes to different clusters according to their joint entropy. In the second stage, the topology structure property is considered to enable further clustering for improving detection reliability. Average I&I detection reliability is used to select clusters and the optimal SPS is identified by maximizing joint entropy of all possible solutions where a single sensor is assigned to each selected cluster. The proposed method and two existing HSBMs are applied to a real USN and their performance is compared. The results obtained show that: (i) a strong correlation coefficient R (R > 0.95) is observed between joint entropy and SPS's detection reliability, which has not been revealed before, (ii) the proposed method consistently outperforms the other two approaches in efficiently offering SPSs with about 7-15 % higher detection reliability, and (iii) the proposed method provides the optimal SPS in a deterministic manner, which makes it attractive for engineering applications., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

The process of aligning one image, in the coordinate system of another is called registration. Image registration is vital in the medical domain and is used for diagnosis, therapy planning, and treatment of diseases. Due to its huge applicability and importance, medical image registration has emerged to be a separate domain of research. Beginning from the invasive landmark based registration, innumerable algorithms have been proposed to register two images of the human physiology. However, a breakthrough in the literature occurred when an information theory based measure, mutual information was used to register two images. Since then a large number of new algorithms have developed, which use mutual information for fully automatic registration of medical images. This paper is a survey of these algorithms. Beginning from its development, it discusses about some of the major works done on mutual information based image registration. Some comparative studies with other algorithms have also been discussed. The paper ends with a discussion of the developments in medical image registration algorithms post mutual information, which primarily include deep neural network (DNN) based algorithms. [ABSTRACT FROM AUTHOR]

Existing model-free ultra-high-dimensional feature screening methods mainly focus on the individual covariate. However, many variables have a community structure, such as grouped covariates in which all variates have high correlation and some associations in one group. There are some research studies about grouped feature screening for ultra-high-dimensional data for a linear model by promoting sure independence screening and high-dimensional ordinary least-squares projector to the group version. Hence, we propose a new ultra-high-dimensional grouped feature screening method for a classification model, which is model-free and suitable for continuous and categorical covariates. Compared with individual covariate feature screening, the proposed method yields a better screening performance and classification accuracy. The grouped feature screening and ranking consistency properties of the proposed method are established. We illustrate the finite sample performance of the proposed method by simulation and real data analysis. [ABSTRACT FROM AUTHOR]

Distributed Denial of Service (DDoS) attacks are increasingly harmful to the cyberspace nowadays. The attackers can now easily launch a bigger and more challenging DDoS attack both towards and with Internet-of-Things (IoT) devices, due to the fast popularization of them. Because of the characteristic of fast overwhelming, it is important to make fast as well as accurate response to DDoS attacks, and the real-time performance can be even more important to prevent and legitimate the attacks. Among the methods proposed by researchers, the entropy-based detection method provides a sensitive and reliable performance. However, the balance between computational complexity and recognition accuracy remains a challenge. In this paper, we propose a detection method that consists of 3 main parts in different aspects: a sliding time window to fasten the entropy calculation, a single-directional filter to realize early detection during the DDoS progress but not after the crash, and a quintile deviation check algorithm to optimize the detection result. These will eventually lead to a real-time and high-efficient performance to recognize IoT DDoS attacks as soon as possible.

The quality of sensor placement in water distribution systems (WDSs) has considerable impact on leak detection accuracy and identification efficiency. Sensor-placement optimization has long been a major issue in the research of WDS. Conventional hierarchical algorithms based on joint entropy have been widely applied to sensor-placement optimization in the engineering-safety monitoring field. Although such algorithms can effectively reduce information redundancy and are computationally very efficient, they fail to fully account for various practical failure scenarios. Sensor failures introduce bias into monitored data and disrupt the monitoring process, thereby compromising the monitoring performance of a given sensor network. To overcome this drawback, this study proposes a new optimal sensor-placement scheme for WDSs based on an improved hierarchical algorithm. The new optimization scheme works by sequentially selecting sensors using joint entropy as a sensor-placement objective function and fully considers various scenarios of a single-sensor failure to ensure minimal information entropy loss during sensor failure. Simulation is performed on an example network (i.e., Net 3 from EPANET), revealing that when a sensor fails, information entropy loss is 0.011 for conventional sensor-placement schemes, but it is only 0.007 for the improved scheme. Statistical analysis shows that the improved scheme has a higher robustness and adaptability for leak detection and identification, owing to the full consideration of various scenarios of a single-sensor failure. [ABSTRACT FROM AUTHOR]

In this paper, the design of low probability of intercept (LPI) radar waveforms considers not only the performance of passive interception systems (PISs), but also radar detection and resolution performance. Waveform design is an important considerations for the LPI ability of radar. Since information theory has a powerful performance-bound description ability from the perspective of information flow, LPI waveforms are designed in this paper within the constraints of the detection performance metrics of radar and PISs, both of which are measured by the Kullback–Leibler divergence, and the resolution performance metric, which is measured by joint entropy. The designed optimization model of LPI waveforms can be solved using the sequential quadratic programming (SQP) method. Simulation results verify that the designed LPI waveforms not only have satisfactory target-detecting and resolution performance, but also have a superior low interception performance against PISs.

Image segmentation is the initial task in image processing which is extensively utilized in object recognition and detection. In the field of image segmentation, multilevel thresholding is one of the leading methods. Though, the computational cost of this method scales exponentially as the number of the threshold value increases, which directs to exercise of optimization method to determine the optimal value of the thresholds. In this article, a newly modified algorithm called quasi opposite multiverse optimization (QOMVO) is proposed. The proposed QOMVO is based on quasi opposite based learning and multiverse optimization (MVO) algorithm. The quasi opposite based learning helps to improve the exploration phase of QOMVO. QOMVO is coupled with a new proposed entropy called Joint entropy (Renyi-Tsalii) to perform image segmentation by finding the optimal threshold value. The outcome of the proposed algorithm is compared with other evolutionary algorithm based on objective function value, feature similarity index, structural similarity index , quality index based on local variance, uniformity, normalized absolute error and computational time. A non-parametric test called the Wilcoxon test is done to justify the response of these parameters. Along with comparing with other algorithms, a comparison has also been made with other entropy, i.e. Renyi's and Tsallis. The experimental outcomes confirmed that the proposed algorithm provides more reliable results than other existing methods. [ABSTRACT FROM AUTHOR]

The management of existing civil infrastructure is challenging due to evolving functional requirements, aging and climate change. Civil infrastructure often has hidden reserve capacity because of conservative approaches used in design and during construction. Information collected through sensor measurements has the potential to improve knowledge of structural behavior, leading to better decisions related to asset management. In this situation, the design of the monitoring system is an important task since it directly affects the quality of the information that is collected. Design of optimal measurement systems depends on the choice of behavior-model parameters to identify using monitoring data and non-parametric uncertainty sources. A model that contains a representation of these parameters as variables is called a model class. Selection of the most appropriate model class is often difficult prior to acquisition of information regarding the structural behavior, and this leads to suboptimal sensor placement. This study presents strategies to efficiently design measurement systems when multiple model classes are plausible. This methodology supports the selection of a sensor configuration that provides significant information gain for each model class using a minimum number of sensors. A full-scale bridge, The Powder Mill Bridge (USA), and an illustrative beam example are used to compare methodologies. A modification of the hierarchical algorithm for sensor placement has led to design of configurations that have fewer sensors than previously proposed strategies without compromising information gain. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Purpose: Clinical image pairs provide the most realistic test data for image registration evaluation. However, the optimal registration is unknown. Using combinatorial rigid registration optimization (CORRO) we demonstrate a method to estimate the optimal alignment for rigid‐registration of clinical image pairs. Methods: Expert selected landmark pairs were selected for each CT/CBCT image pair for six cases representing head and neck, thoracic, and pelvic anatomic regions. Combination subsets of a k number of landmark pairs (k‐combination set) were generated without repeat to form a large set of k‐combination sets (k‐set) for k = 4,8,12. The rigid transformation between the image pairs was calculated for each k‐combination set. The mean and standard deviation of these transformations were used to derive final registration for each k‐set. Results: The standard deviation of registration output decreased as the k‐size increased for all cases. The joint entropy evaluated for each k‐set of each case was smaller than those from two commercially available registration programs indicating a stronger correlation between the image pair after CORRO was used. A joint histogram plot of all three algorithms showed high correlation between them. As further proof of the efficacy of CORRO the joint entropy of each member of 30 000 k‐combination sets in k = 4 were calculated for one of the thoracic cases. The minimum joint entropy was found to exist at the estimated mean of registration indicating CORRO converges to the optimal rigid‐registration results. Conclusions: We have developed a methodology called CORRO that allows us to estimate optimal alignment for rigid‐registration of clinical image pairs using a large set landmark point. The results for the rigid‐body registration have been shown to be comparable to results from commercially available algorithms for all six cases. CORRO can serve as an excellent tool that can be used to test and validate rigid registration algorithms. [ABSTRACT FROM AUTHOR]

The Internet of Things creates opportunities to develop data-driven design methodologies for smart cities. However, effects rather than causes are often measured in complex urban systems, requiring robust data-interpretation methodologies. Additionally, effective monitoring of large urban components, such as civil infrastructure, often involves multiple sensor devices and invasive sensor systems. In these situations, the design of measurement systems is an important task. Usually, this task is carried out by engineers using only qualitative rules of thumb and experience. Recently, researchers have developed quantitative sensor-placement methodologies to maximize the information gain of measurement systems. Nonetheless, these methodologies are only weakly validated using field measurements due to the small amount of data collected and the difficulties comparing the predicted information gain with observations. This paper proposes a validation strategy for sensor-placement methodologies. In this strategy, predictions of both individual sensor and sensor-configuration performances are compared with observations using statistical tests and hypothesis testing. The validation procedure is illustrated through three full-scale-bridge case studies. This strategy helps engineers select an appropriate methodology to design measurement systems in order to optimize data collection using sensors. [ABSTRACT FROM AUTHOR]

Cloud computing provides enormous storage capacity and huge computation power. Cloud enables users to deploy data-intensive applications without substructure investment. Intermediate data are generated under the cloud applications and stored again in cloud. An opponent may analyse multiple intermediate data sets to access the privacy sensitive information.The proposed technique to have good privacy and utility trade off, joint entropy with adaptive optimization process is used to maintain the privacy in cloud. Here optimal entropy value process using adaptive particle swarm optimization (APSO) process. After get the optimal entropy database difference model was processed, Entropy and database difference ratio is taken as the evaluation matrices and is tested using various datasets. The technique is also compared to the existing PSO optimization process on privacy preservation in cloud. An adaption feature of Particle swarm optimization show that APSO enhances privacy and preserving cost of intermediate data sets can be significantly saved. [ABSTRACT FROM AUTHOR]

The signature matrix of an n -component three-state network (system), which depends only on the network structure, is a useful tool for comparing the reliability and stochastic properties of networks. In this paper, we consider a three-state network with states up, partial performance , and down. We assume that the network remains in state up, for a random time T 1 and then moves to state partial performance until it fails at time T > T 1. The signature-based expressions for the conditional entropy of T given T 1, the joint entropy, Kullback-Leibler (K-L) information, and mutual information of the lifetimes T and T 1 are presented. It is shown that the K-L information, and mutual information between T 1 and T depend only on the network structure (i.e., depend only to the signature matrix of the network). Some signature-based stochastic comparisons are also made to compare the K-L of the state lifetimes in two different three-state networks. Upper and lower bounds for the K-L divergence and mutual information between T 1 and T are investigated. Finally the results are extended to n -component multi-state networks. Several examples are examined graphically and numerically. [ABSTRACT FROM AUTHOR]

Abstract Meditation, a self-regulation practice, attracts attention on its physiological and psychological effects, and its effects on nonlinear complexity of heartbeats are analyzed in our contribution. Heartbeats of Chinese Chi, Kundalini Yoga practitioners and non-meditation metronomic breathing subjects, all from physionet, contribute to our research. We introduce the meditative heart rates in time and frequency domains, and extract their nonlinear dynamics by local dynamic and global static symbolic time-series analysis. Shannon entropy for each individual symbolization and joint entropy for the two kinds of symbolic transformations are applied to detect heartbeats’ nonlinear complexity. Test results prove that symbolic entropy is an exquisite quantitative analysis to characterize and distinguish nonlinear complexity of the physiological signals. Heart rates, during meditation, show highest regularities and lowest nonlinear complexities, and meditators, even in pre-meditation status, have lower heartbeats nonlinear complexity than the non-meditation volunteers, which suggest that meditation enhances cardiac self-regulation and reduces the nonlinear dynamic complexity of heart rates. Highlights • Static and dynamic symbolic nonlinear complexities are extracted from meditative heartbeats. • Shannon entropy for individual symbolization and joint entropy for two symbolic methods are employed. • Meditative heartbeats have increased regularity and reduced nonlinear complexity. • Meditation training, we suppose, improve self-control to cardiac regulation system. [ABSTRACT FROM AUTHOR]

Virtually all modern imaging devices collect electromagnetic or acoustic waves and use the energy carried by these waves to determine pixel values to create what is basically an “energy” picture. However, waves also carry “information”, as quantified by some form of entropy, and this may also be used to produce an “information” image. Numerous published studies have demonstrated the advantages of entropy, or “information imaging”, over conventional methods. The most sensitive information measure appears to be the joint entropy of the collected wave and a reference signal. The sensitivity of repeated experimental observations of a slowly-changing quantity may be defined as the mean variation (i.e., observed change) divided by mean variance (i.e., noise). Wiener integration permits computation of the required mean values and variances as solutions to the heat equation, permitting estimation of their relative magnitudes. There always exists a reference, such that joint entropy has larger variation and smaller variance than the corresponding quantities for signal energy, matching observations of several studies. Moreover, a general prescription for finding an “optimal” reference for the joint entropy emerges, which also has been validated in several studies.

As one classic technique of computer vision, image retrieval could retrieve the target images from hundreds of thousands of images effectively. Furthermore, with the rapid development of deep learning, the quality of retrieval is increased obviously. However, under normal conditions, the high-quality retrieval is supported by a large number of learning instances. The large number of learning instances not only need much human source in the process of selection, but also need much computing source in the process of computation. More importantly, for some special categories, it’s difficult to obtain a large number of learning instances. Aiming at the problem above, we proposed one joint entropy based learning model which could reduce the number of learning instances through optimizing the distribution of learning instances. Firstly, the learning instances are pre-selected using improved watershed segmentation method. Then, joint entropy model is used for reducing the possibility of double, useless even mistaken instances existence. After that, a database using a large number of images is built up. Sufficient experiments based on the database show the model’s superiority that our model not only could reduce the number of learning instances but also could keep the accuracy of retrieval. [ABSTRACT FROM AUTHOR]

Histograms have extensively been used as a simple tool for nonparametric probability density function estimation. However, practically, the accuracy of some histogram-based derived quantities, such as the marginal entropy (ME), the joint entropy (JE), or the mutual information (MI) depends on the number of bins chosen for the histogram. In this paper, we investigate the binning problem of bi-histogram for the estimation of JE. By minimizing a theoretical mean square error (MSE) of JE estimation, we derive a new formula for the optimal number of bins of bi-histogram for continuous random variables. This novel JE estimation has been used in the MI estimation to avoid the error accumulation of joint MI between the class variable and feature subset in the feature selection. In a synthetic Gaussian feature selection problem, only the proposed method permits to retrieve the exact number of relevant features that explain the class variable when compared to a concurrent univariate estimator based on binning formula that has been proposed for ME estimation. In speech and speaker recognition applications, the proposed method permits to select a limited number of features which guaranties approximately the same or an even better recognition rate than using the total number of features. [ABSTRACT FROM AUTHOR]