Search

Pseudo-inverse learners (PILs) are a kind of feedforward neural network trained with the pseudoinverse learning algorithm, which can be traced back to 1995 originally. PIL is an approach for nongradient descent learning, and its main advantage is the lower computational cost and fast learning procedure, which is especially relevant in the edge computing research field. However, PIL is mostly applied to a deterministic learning problem, while in the real world, the greatest case that is of concern is the uncertainty learning problem. In this work, under the framework of the synergetic learning system (SLS), we introduce an approximated synergetic learning scheme, which can transform uncertainty learning into deterministic learning. We call this new learning framework the Bayesian PIL, and the advantages are also demonstrated in this work.

This study aims to analyze the k-value on K nearest neighbor classification. k-value is the distance used to find the closest data to label the class from the testing data. Each k-value can produce a different class label against the same testing data. The variants of k-value that we use are k=3, k=5 and k=7 to find the best k-value. There are 2 classes that are used in this research. Both classes are stairs descent and floor classes. The gray level co-occurrence matrix method is used to extract features. The data we use comes from videos obtained from the camera on the smart wheelchair taken by the frame. Refer to the results of our tests, the best k-value is obtained when using k=7 and angle 0° with accuracy is 92.5%. The stairs descent detection system will be implemented in a smart wheelchair

Momentum method has been used extensively in optimizers for deep learning. Recent studies show that distributed training through K-step averaging has many nice properties. We propose a momentum method for such model averaging approaches. At each individual learner level traditional stochastic gradient is applied. At the meta-level (global learner level), one momentum term is applied and we call it block momentum. We analyze the convergence and scaling properties of such momentum methods. Our experimental results show that block momentum not only accelerates training, but also achieves better results.

Descent & landing (D&L) on small planetary bodies are scientifically rewarding exploration missions but they are technically challenging due to the complex and poorly-known environment around those bodies. The standard guidance synthesis approach considers nominal conditions and applies optimal control theory to obtain guidance law gains, followed by intensive verification and validation. In this article, it is shown that the standard approach may yield gains that are not optimal once dispersions (and/or other optimality metrics) are taken into account and a tuning approach is then proposed based on a priori methodological system assessment. The proposed approach employs systematic high-fidelity simulations to generate trade-off maps. These maps can be generated by on ground operators based on the best estimated conditions and uploaded to the spacecraft as it approaches the target. The proposed systematic guidance tuning and resulting maps also provide a valuable understanding of the system dynamics towards the application of other industry-oriented tools such as structured Η∞ optimisation. It is shown that the proposed tuning enables propellant consumption reductions of around 40% compared to state-of-practice gain selections.

Gradient descent is a widely used optimization method. The adjustment of the learning rate is an important factor in improving its performance, and many researchers have investigated methods for automatically adjusting the learning rate. One such method, hypergradient descent, automatically adjusts the learning rate by using gradient descent. However, it introduces the “learning rate of the learning rate,” and an appropriate value for the learning rate of the learning rate must be chosen in order to effectively adjust the learning rate. We investigated the use of two datasets and two optimization methods for doing this and achieved an effective adjustment of the learning rate when the objective function was convex and $L$ -smooth.

The discrete logarithm problem (DLP) is a cornerstone in the public-key cryptography with various applications. Recently there is a new computational record for the DLP in a 795-bit prime field using the number field sieve for discrete logarithms (NFS-DL). The main technique used in the computation is choosing good parameters for each step of NFS-DL which reduces the time estimated by the asymptotic complexity significantly. In this article, we propose a new algorithm to find relations for high degree prime ideals and remove the previous restriction that only degree one prime ideals are allowed to appear in the descent step. This will increase the success probability of the descent step and reduce the actual running time. As the descent step could be done separately in some sense, improving the descent step is beneficial to find optimal parameters such as the smoothness bound which may result in better performance of the NFS-DL algorithm. We also give an experiment to demonstrate the effectiveness of our algorithm.

For the case of public transport, we consider the problem of demand estimation. Given an origin-destination matrix representing the public transport demand, the distribution of flow among different lines can be obtained assuming that it corresponds to a certain equilibrium characterized by an optimization problem. In particular we will focus on the assignment model proposed by Cepeda et al. (2006). However the knowledge of origin-destination matrix is expensive and sometimes unaffordable in practice. Traditionally, it is estimated using statistical or econometrical considerations. In this work, we explore the estimation through the numerical solution of a bilevel optimization problem. One disadvantage of this formulation is the difficulty of obtaining descent directions, hence, for the resolution of the optimization problem we use a derivative-free method. This method was applied for small networks getting good results.

It is well-known that for general convex optimization problems, block-coordinate descent can get stuck in poor local optima. Despite that, versions of this method known as convergent message passing are very successful to approximately solve the dual LP relaxation of the MAP inference problem in graphical models. In attempt to identify the reason why these methods often achieve good local minima, we argue that if in block-coordinate descent the set of minimizers over a variable block has multiple elements, one should choose an element from the relative interior of this set. We show that this rule is not worse than any other rule for choosing block-minimizers. Based on this observation, we develop a theoretical framework for block-coordinate descent applied to general convex problems. We illustrate this theory on convergent message-passing methods.

The physical support of the Internet infrastructure is fiber-optics network (FON). Since the global Internet traffic doubles each year, FONs are suffering the capacity crunch problem, and several proposals offer content-oriented solutions, combined with a smart spectral utilization.It is well known that the physical implementation of FONs is robust under single failures on its links, or 2-connected. Unfortunately, simultaneous failures occur, and the service is disrupted for some hours per year in average.In this paper we propose a smart network augmentation which solves the previous problems simultaneously. The resulting network is a locally most-reliable 3-connected augmentation. Our methodology to build 3-connected networks is supported by the theory of uniformly most-reliable networks, elementary properties of cubic networks and a careful selection of local searches in a Variable Neighborhood Descent (VND) metaheuristic.

This paper considers a refinement problem for the flight altitude, speed, and flight-path angle of an aircraft during aerodynamic deceleration in the atmosphere of Mars using inaccurate discrete g-load measurements. For solving this problem, several suboptimal discrete nonlinear filters of higher accuracy than the extended Kalman filter based on the Taylor linearization procedure with static (Gaussian) linearization of the nonlinearities are applied and compared with each other. The equations of these filters are derived, the Gaussian moments of their structural functions are calculated, and the resulting accuracy of these filters is analyzed in statistical terms.

This paper present algorithms for solving a single machine scheduling problem to minimize the sum of total completion times, total tardiness,maxim-um tardiness,and maximum earliness.The single machine total tardiness problem is already NP-hard, so the consider problem is strongly NP-hard, and several algorithms are used to solve it. Branch and bound algorithmwith dominance ruleand local search algorit- hms are proposed for the problem. For the Branch and bound algorithm results- show that using dominance rule improve the performance of the algorithm in both computation times and optimal values,but it need longer times.Thus we tackle the problemof large sizes with local search algorit- hms descent method, simulated annealing and tabusearch. The perfomance of these algorithms is evaluated on a large set of test problems and the results are compared.The computational results show that simulated annealing algorithm and Tabu search algorithm are better than Descent method with preference to simulated annealing algorithm,and show that the three algorithms find optimal or near optimal solutions inreasonable times.

Optic disc (OD) segmentation is an important step in analyzing the color fundus image. Most existing approaches are based on the shape prior and visual appearance of the OD boundary. However, the current ways of integrating the shape and appearance are simple. We argue that the performance of OD segmentation can be improved by better shape-appearance modeling. In this paper, we propose to learn a sequence of supervised descent directions between the coordinates of OD boundary and their surrounding visual appearances for OD segmentation. In addition, we introduce the histograms of gradient orientations to represent the OD appearance. Experimental results on six datasets clearly show that the proposed method improves the OD segmentation and outperforms the state-of-the-art.

One important problem in radiation therapy for cancer treatment is the selection of the set of beam angles radiation will be delivered from. A primary goal in this problem is to find a beam angle configuration (BAC) that leads to a clinically acceptable treatment plan. Further, this process must be done within clinically acceptable times. Since the problem of selecting beam angles in radiation therapy is known to be extremely hard to solve as well as time-consuming, both exact algorithms and population-based heuristics might not be suitable to solve this problem. In this paper, we compare two matheuristic methods based on local search algorithms, to approximately solve the beam angle optimisation problem (BAO). Although the steepest descent algorithm is able to find locally optimal BACs for the BAO problem, it takes too long before convergence, which is not acceptable in clinical practice. Thus, we propose to use a next descent algorithm that converges quickly to good quality solutions although no (local) optimality guarantee is given. We apply our two matheuristic methods on a prostate case which considers two organs at risk, namely, the rectum and the bladder. Results show that the matheuristic algorithm based on the next descent local search is able to quickly find solutions as good as the ones found by the steepest descent algorithm.

The goal of this paper is to examine the possible neurobiological basis of a defining property of the human language faculty: recursion. I suggest that recursion should be understood in light of Darwinian's descent with modification. Descent: that is, based on ingredients of neural circuitry found in ‘non-linguistic’ species; and modification: a reconfiguration that is specific to anatomically modern humans. I argue that the expansion of the parietal region associated with the globularization of the neurocranium in our species contributed to the transformation of the connection between Broca's and Wernicke's region via Geschwind's territory, and enabled the pairing of evolutionary ancient networks that together became capable of constructing and processing not just sequences, but sequences of sequences.

The authors considered the solution of boundary problems, arising from objects modeling with distributed parameters, of radial basis functions networks. The gradient algorithms analysis for learning radial basis functions networks when solving boundary value problems is carried out. The widely used the first order method-the gradient method descent does not provide high learning rate and the solution accuracy. The fastest of the second order method - the method of trust regions is very complicated. Developed implementations of the first order method-the accelerated Nesterov gradient and the second order method-Levenberg Marquardt for learning radial basic functions networks in solving boundary value problems. A comparison is made of the Levenberg Marquardt methods and trust regions for learning radial basis functions networks. The results are presented of solving model problem using the developed methods. The Levenberg-Marquardt method, with a simpler implementation, showed the results compared with the fastest method of the second order - the method of trust regions.

In this work, we study the application of the supervised descent method using neural networks (SDMNN) for 2D electrical impedance tomography. SDMNN contains the offline training and the online prediction stages. In the offline stage, neural networks are iteratively applied to learn a set of descent directions for minimizing the objective function; in the online stage, the trained neural networks can be directly used for the image reconstruction. This scheme combines the advantages of fast convergence speed of neural networks and good generalization ability of the supervised descent method (SDM). Numerical results verify the efficiency and accuracy of this method.

We propose an unsupervised method for saliency detection with multiple functions by a universal descent model. The basic descent model is concise, which is a minimization equation conducted by a relationship matrix and background prior. The relationship matrix is calculated by the contrast of features which contains much information between superpixels. So saliency maps with high performance can be obtained even if only one feature is included. Besides, various terms can be appended to the basic descent model to achieve multiple functions. For example, considering an neighbor matrix into the basic model can further smooth saliency maps. The refinement term can also refine existing results by introducing prior saliency maps, demonstrating the universal applications of our model. Extensive experiments on different datasets indicate that the basic model outperforms state-of-the-art methods. The universal descent models with smoothness and refinement terms are able to promote the performance of state-of-the-art methods by a large margin.

Supervised Descent Method (SDM) is an efficient and accurate approach for facial landmark locating and face alignment. In the training phase, it requires a large amount of training samples to learn the descent directions and get the corresponding regressors. Then in the test phase, it uses the corresponding regressors to estimate the descent directions and locate the facial landmarks. However, when the facial expression or direction changes too much, generally SDM cannot obtain good performance due to the large variation between the initial shape (the initial shape of SDM is the mean shape of the training samples) and the target shape. Therefore, we propose a coarse-to-fine SDM (CFSDM) method to improve the accuracy of the test results. This method predicts the approximate coordinates of the facial landmarks with a simple CNN (Convolutional Neural Network) network (here we introduce the channel-wise attention mechanism, which can predict the coordinates of the landmarks more accurately with a relatively simple structure) in advance, and then SDM will take the coordinates as its initial shape’s coordinates, which reduces the distance between the initial shape and the target shape, thereby solving the problem that SDM cannot achieve good results when the facial expression or direction changes greatly.

Variable neighborhood searches and evolutionary techniques have shown their effectiveness when dealing with many combinatorial optimisation problems. This study proposes to combine these two techniques for addressing the routing problem using electric and modular vehicles. This is a recent problem that aims to overcome recharging battery constraints while maintaining a certain performance regarding to the fleet cost and the traveled distance. An experimental study on benchmark instances is provided to show the relevance of the proposed algorithm.

The aim of this study is to develop a hybrid genetic algorithm (H-GA) that combines a genetic algorithm (GA) and a descent local search technique to pursue the optimization of a sandwich composite inter-stage skirt located in the upper bound of the launcher. Total number of plies forming the sandwich composite skin and fiber orientations (predefined ply angles) are considered as design variables. The H-GA is chosen as an optimization tool thanks to its ability to deal with discrete optimization problems, of which the design of composites stacking is an example. First, the proposed approach is presented and explained. Then, its performances are compared and discussed against those offered by the classical GA.The aim of this study is to develop a hybrid genetic algorithm (H-GA) that combines a genetic algorithm (GA) and a descent local search technique to pursue the optimization of a sandwich composite inter-stage skirt located in the upper bound of the launcher. Total number of plies forming the sandwich composite skin and fiber orientations (predefined ply angles) are considered as design variables. The H-GA is chosen as an optimization tool thanks to its ability to deal with discrete optimization problems, of which the design of composites stacking is an example. First, the proposed approach is presented and explained. Then, its performances are compared and discussed against those offered by the classical GA.

Recent research in machine learning pointed to the core problem of state-of-the-art models which impedes their widespread adoption in different domains. The models' inability to differentiate between noise and subtle, yet significant variation in data leads to their vulnerability to adversarial perturbations that cause wrong predictions with high confidence. The study is aimed at identifying whether the algorithms inspired by biological evolution may achieve better results in cases where brittle robustness properties are highly sensitive to the slight noise. To answer this question, we introduce the new robust gradient descent inspired by the stability and adaptability of biological systems to unknown and changing environments. The proposed optimization technique involves an open-ended adaptation process with regard to two hyperparameters inherited from the generalized Verhulst population growth equation. The hyperparameters increase robustness to adversarial noise by penalizing the degree to which hardly visible changes in gradients impact prediction. The empirical evidence on synthetic and experimental datasets confirmed the viability of the bio-inspired gradient descent and suggested promising directions for future research. The code used for computational experiments is provided in a repository at https://github.com/yukinoi/bio_gradient_descent.

In this work, we study the application of the neural network-based supervised descent method (NN-SDM) for 2D electrical impedance tomography.The NN-SDM contains two stages: offline training and online prediction. In the offline stage, neural networks are iteratively applied to learn a sequence of descent directions for minimizing the objective function, where the training data set is generated in advance according to prior information or historical data. In the online stage, the trained neural networks are directly used to predict the descent directions.Numerical and experimental results are reported to assess the efficiency and accuracy of the NN-SDM for both model-based and pixel-based inversions. In addition, the performance of the NN-SDM is compared with the linear SDM (LSDM), an end-to-end neural network (E2E-NN) and the Gauss-Newton (GN) method. The results demonstrate that the NN-SDM achieves faster convergence than the LSDM and GN method, and achieves a stronger generalization ability than the E2E-NN.The NN-SDM combines the strong non-linear fitting ability of the neural network and good generalization capability of the supervised descent method (SDM), which also provides good flexibility to incorporate prior information and accelerates the convergence of iteration.

Blind signal separation (BSS) has received much research attention due to its potential applicability in recent decades. A great deal of measurements acquired from practical application show some degree of periodicity. Information theoretic criterion is exploited to deduce an objective function for the separation of periodic signals. Minimizing the objective function with natural gradient descent rule, a novel learning algorithm is proposed correspondingly. Computer simulations on signals with temporal structure illustrate the efficiency of the proposed techniques.

A novel method for automatic facial landmark localization is presented. The method builds on the supervised descent framework, which was shown to successfully localize landmarks in the presence of large expression variations and mild occlusions, but struggles when localizing landmarks on faces with large pose variations. We propose an extension of the supervised descent framework that trains multiple descent maps and results in increased robustness to pose variations. The performance of the proposed method is demonstrated on the Bosphorus, the FRGC and the UND data sets for the problem of facial landmark localization from 3D data. Our experimental results show that the proposed method exhibits increased robustness to pose variations, while retaining high performance in the case of expression and occlusion variations.

We want to solve fuzzy linear programming problems with fuzzy coefficients in the objective function using a new variable neighborhood search algorithm. In our proposed algorithm, the local search is defined based on descent directions. We make use of our recently proposed modified Kerre's method for finding descent directions. The fuzzy optimization problem is solved directly, without changing it to a crisp program. We show the effectiveness of our proposed method in comparison with some available methods by using a non-parametric statistical sign test. The objective function values obtained by our proposed method turn to be more accurate the ones obtained by other methods.

A recently proposed clustering method, called the Nearest Descent (ND), can organize the whole dataset into a sparsely connected graph, called the In-tree. This ND-based In-tree structure proves able to reveal the clustering structure underlying the dataset, except one imperfect place, that is, there are some undesired edges in this In-tree which require to be removed. Here, we propose an effective way to automatically remove the undesired edges in In-tree via an effective combination of the In-tree structure with affinity propagation (AP). The key for the combination is to add edges between the reachable nodes in In-tree before using AP to remove the undesired edges. The experiments on both synthetic and real datasets demonstrate the effectiveness of the proposed method.

Realizing the pinpoint landing localization for the ChangE-3 system is a symbol of success for this mission, and it is also a basic requirement for the lunar Lander and Rover to carry out the following scientific exploration tasks. In this paper, jointly dealing with the descent image sequence and ChangE-2 DOM data, an approach based on the images processing techniques, such as SIFT feature matching, homography transportation and stereo vision measurement, is presented to realize the pinpoint localization of the ChangE-3 system. The localization result is proved to be reliable and accuracy and the result will be a critical component for the ChangE-3 mission planning and the completion of scientific exploration.

International audience; Sparsity-inducing penalties are useful tools in variational methods for machine learning. In this paper, we propose two block-coordinate descent strategies for learning a sparse multiclass support vector machine. The first one works by selecting a subset of features to be updated at each iteration, while the second one performs the selection among the training samples. These algorithms can be efficiently implemented thanks to the flexibility offered by recent randomized primal-dual proximal methods. Experiments carried out for the supervised classification of handwritten digits demonstrate the interest of considering the primal-dual approach in the context of block-coordinate descent, and the efficiency of the proposed algorithms is assessed through a comparison of execution times and classification errors. Index Terms— Sparsity, multiclass SVM, proximal algorithm, random update, block-coordinate descent.

Gradient descent in image distances can lead a navigating agent to the goal location, but in environments with an anisotropic distribution of landmarks, gradient home vectors deviate from the true home direction. These deviations can be reduced by applying Newton's method to matched-filter descent in image distances (MFDID). Based on several image databases we demonstrate that the home vectors produced by the Newton-based MFDID method are usually closer to the true home direction than those obtained from the original MFDID method. The greater accuracy of Newton-MFDID home vectors in the vicinity of the goal location would allow a navigating agent to approach the goal on straighter trajectories, improve the results of triangulation procedures, and enhance a robot's ability to detect its arrival at a goal.

Supervised Descent Method (SDM) is a highly efficient and accurate approach for facial landmark locating/face alignment. It learns a sequence of descent directions that minimize the difference between the estimated shape and the ground truth in HOG feature space during training, and utilize them in testing to predict shape increment iteratively. In this paper, we propose to modify SDM in three respects: (1) Multi-scale HOG features are applied orderly as a coarse-to-fine feature detector; (2) Global to local constraints of the facial features are considered orderly in regression cascade; (3) Rigid Regularization is applied to obtain more stable prediction results. Extensive experimental results demonstrate that each of the three modifications could improve the accuracy and robustness of the traditional SDM methods. Furthermore, enhanced by the three-fold improvements, the extended SDM compares favorably with other state-of-the-art methods on several challenging face data sets, including LFPW, HELEN and 300 Faces in-the-wild.

Many real-world problems entail multiple conflicting objectives, which makes multiobjective optimization an important subject. Much attention has been paid to Genetic Algorithm (GA) as a potent multiobjective optimization method, and the effectiveness of its hybridization with local search (LS) has recently been reported in the literature. However, there have been a relatively small number of studies on LS methods for multiobjective function optimization. Although each of the existing LS methods has some strong points, they have respective drawbacks such as high computational cost and inefficiency of improving objective functions. Hence, a more effective and efficient LS method is being sought, which can be used to enhance the performance of the hybridization. Pareto descent directions are defined in this paper as descent directions to which no other descent directions are superior in improving all objective functions. Moving solutions in such directions is expected to maximally improve all objective functions simultaneously. This paper proposes a new LS method, Pareto Descent Method (PDM), which finds Pareto descent directions and moves solutions in such directions. In the case part or all of them are infeasible, it finds feasible Pareto descent directions or descent directions as necessary and moves solutions in these directions. PDM finds these directions by solving linear programming problems. Thus, it is computationally inexpensive. Experiments have shown that PDM is superior to existing methods.

International audience; This paper studies the multitrip cumulative capacitated vehicle routing problem (mt-CCVRP), a variant of the classical capacitated vehicle routing problem (CVRP). In the mt-CCVRP the objective function becomes the minimization of the sum of arrival times at required nodes and each vehicle may perform more than one trip. Applications of this NP-Hard problem can be found in disaster logistics. This article presents a Multistart Evolutionary Local Search (MS-ELS) that alternates between giant tour and mt-CCVRP solutions, and uses an adapted split procedure and a variable neighborhood descent (VND). The results on two sets of instances show that this approach finds very good results in relatively short computing time compared with a multistart iterated local search which works directly on the mt-CCVRP solution space.

We present a cut-peak function method for finding a global minimizer of the bound constrained optimization problems. A simple cut-peak function and choice function are defined at a local minimizer. By minimizing the choice function, a global descent of the original objective function is assured. Since the pattern search method does not require the gradient of the choice function, smoothing technique is not employed. The new algorithm is simple to implement and numerical results indicate its efficiency.

Drawing upon data from the phonologies, morphologies, numeral systems, constituent orders, case systems, and lexicons of the world’s languages, Quantifying Language Dynamics introduces new, quantitative methodologies for understanding language contact, evolution, and patterns of phylogenetic descent.

This paper deals with the Unrelated Parallel Machine Scheduling Problem with Setup Times (UPMSPST). The objective is to minimize the maximum completion time of the schedule, the so-called makespan. This problem is commonly found in industrial processes like textile manufacturing and it belongs to NP-Hard class. It is proposed an algorithm named AIV based on Iterated Local Search (ILS) and Variable Neighborhood Descent (VND). This algorithm starts from an initial solution constructed on a greedy way by the Adaptive Shortest Processing Time (ASPT) rule. Then, this initial solution is refined by ILS, using as local search the Random VND procedure, exploring neighborhood structures based on swaps and multiple insertions. In this procedure, here called RVND, there is no fixed sequence of neighborhoods, because they are sorted on each application of the local search. The perturbations of ILS consist in reinserting jobs from one machine to another. AIV was tested using benchmark instances from literature and statistical analysis of the computational experiments showed that AIV outperformed the algorithms of the literature, setting new improved solutions.

The inference of the evolutionary history of a set of languages is a complex problem. Although some languages are known to be related through descent from common ancestral languages, for other languages determining whether such a relationship holds is itself a difficult problem. In this paper we report on new methods, developed by linguists Johanna Nichols (University of California, Berkeley), Donald Ringe and Ann Taylor (University of Pennsylvania, Philadelphia), and me, for answering some of the most difficult questions in this domain. These methods and the results of the analyses based on these methods were presented in November 1995 at the Symposium on the Frontiers of Science held by the National Academy of Sciences.

Virtual network mapping considers the problem of fitting multiple virtual networks into one physical network in a cost-optimal way. This problem arises in Future Internet research. One of the core ideas is to utilize different virtual networks to cater to different application classes, each with customized protocols that deliver the required Quality-of- Service. In this work we introduce a Greedy Randomized Adaptive Search Procedure (GRASP) and Variable Neighborhood Search (VNS) algorithm for solving the Virtual Network Mapping Problem. Both algorithms make use of a Variable Neighborhood Descent with ruin-and-recreate neighborhoods. We show that the VNS approach significantly outperforms the previously best known algorithms for this problem.

Multi-instance multi-label learning (MIML) is a framework for supervised classification where the objects to be classified are bags of instances associated with multiple labels. For example, an image can be represented as a bag of segments and associated with a list of objects it contains. Prior work on MIML has focused on predicting label sets for previously unseen bags. We instead consider the problem of predicting instance labels while learning from data labeled only at the bag level. We propose Rank-Loss Support Instance Machines, which optimize a regularized rank-loss objective and can be instantiated with different aggregation models connecting instance-level predictions with bag-level predictions. The aggregation models that we consider are equivalent to defining a "support instance" for each bag, which allows efficient optimization of the rank-loss objective using primal sub-gradient descent. Experiments on artificial and real-world datasets show that the proposed methods achieve higher accuracy than other loss functions used in prior work, e.g., Hamming loss, and recent work in ambiguous label classification.

The recently proposed Sparse Shape Composition (SSC) opens a new avenue for shape prior modeling. Instead of assuming any parametric model of shape statistics, SSC incorporates shape priors on-the-fly by approximating a shape instance (usually derived from appearance cues) by a sparse combination of shapes in a training repository. Theoretically, one can increase the modeling capability of SSC by including as many training shapes in the repository. However, this strategy confronts two limitations in practice. First, since SSC involves an iterative sparse optimization at run-time, the more shape instances contained in the repository, the less run-time efficiency SSC has. Therefore, a compact and informative shape dictionary is preferred to a large shape repository. Second, in medical imaging applications, training shapes seldom come in one batch. It is very time consuming and sometimes infeasible to re-construct the shape dictionary every time new training shapes appear. In this paper, we propose an online learning method to address these two limitations. Our method starts from constructing an initial shape dictionary using the K-SVD algorithm. When new training shapes come, instead of re-constructing the dictionary from the ground up, we update the existing one using a block-coordinates descent approach. Using the dynamically updated dictionary, sparse shape composition can be gracefully scaled up to model shape priors from a large number of training shapes without sacrificing run-time efficiency. Our method is validated on lung localization in X-Ray and cardiac segmentation in MRI time series. Compared to the original SSC, it shows comparable performance while being significantly more efficient.

The recently introduced family of natural evolution strategies (NES), a novel stochastic descent method employing the natural gradient, is providing a more principled alternative to the well-known covariance matrix adaptation evolution strategy (CMA-ES). Until now, NES could only be used for single-objective optimization. This paper extends the approach to the multi-objective case, by first deriving a (1+1) hillclimber version of NES which is then used as the core component of a multi-objective optimization algorithm. We empirically evaluate the approach on a battery of benchmark functions and find it to be competitive with the state-of-the-art.

Sized types are a modular and theoretically well-understood tool for checking termination of recursive and productivity of corecursive definitions. The essential idea is to track structural descent and guardedness in the type system to make termination checking robust and suitable for strong abstractions like higher-order functions and polymorphism. To study the application of sized types to proof assistants and programming languages based on dependent type theory, we have implemented a core language, MiniAgda, with explicit handling of sizes. New considerations were necessary to soundly integrate sized types with dependencies and pattern matching, which was made possible by concepts such as inaccessible patterns and parametric function spaces. This paper provides an introduction to MiniAgda by example and informal explanations of the underlying principles., Comment: In Proceedings PAR 2010, arXiv:1012.4555

Although many pattern classifiers based on artificial neural networks have been vigorous-ly studied, they are still inadequate from a viewpoint of classifyingdynamic (variable-and unspecified-duration) speech patterns. To cope with this problem, the generalized probabilistic descent method (GPD) has recently been proposed. GPD not only allows one to train a discriminative system to classify dynamic patterns, but also possesses a remarkable advantage, namely guaranteeing the learning optimality (in the sense of a probabilistic descent search). A practical implementation of this theory, however, remains to be evaluated. In this light, we particularly focus on evaluating GPD in designing a widely-used speech recognizer based on dynamic time warping distance-measurement. We also show that the design algorithm appraised in this paper can be considered a new version of learning vector quantization, which is incorporated with the dynamic programming. Experimental evaluation results in tasks of classifying syllables and phonemes clearly demonstrate GPD's superiority.

Learning from examples is the process of taking input-output examples of an unknown function ƒ and infering an implementation of ƒ. Learning from hints allows for general information about ƒ to be used instead of just input-output examples. We introduce a method for incorporating any invariance hint about ƒ in any descent method for learning from examples. We also show that learning in a neural network remains NP-complete with a certain, biologically plausible, hint about the network. We discuss the information value and the complexity value of hibts.

In the exploration of the other planets of our solar system, images taken during a lander's descent to the surface of a planet provide a critical link between orbital images and surface images. The descent images not only allow us to locate the landing site in a global coordinate frame, but also provide progressively higher-resolution maps for mission planning. The paper addresses the generation of depth maps from the descent images. Our approach has two steps: motion refinement and depth recovery. During motion refinement, we use an initial motion estimate in order to avoid the intrinsic ambiguity in descending motions. The objective of the motion refinement step is to adjust the motion parameters such that the epipolar constraints are valid between adjacent frames. The depth recovery step correlates adjacent frames to match pixels for triangulation. Due to the descending motion, the conventional rectification process is replaced by a set of anti-aliasing image warpings corresponding to a set of virtual parallel planes. We demonstrate experimental results on synthetic and real descent images.

In a prior paper, we described a haptic rendering algorithm for arbitrary polygonal models using a six degree-of-freedom haptic interface. In that paper, global search for local minimum distances provided repulsive forces between models. This paper augments that system with a local tracking algorithm that quickly updates and maintains globally computed minima. The global search continuously adds and deletes local minimum distance pairs being updated by the local search. This new system supports higher haptic interaction rates on more complex scenes than the previous approach.

The presence of speckle in ultrasound images makes it hard to segment them using active contours. Speckle causes the energy function of the active contours to have many local minima, and the gradient descent procedure used for evolving the contour gets trapped in these minima.

Generalized competitive learning algorithms are described. These algorithms comprise competition handicaps, cooperation and multiply descent cost property. Applications are made on single processing and combinatorial optimizations. Parallel computation of the algorithms presented is discussed. >

We propose a method for sound monitoring, which enables one to selectively detect unexpected irregular sounds and ignore the other sounds, i.e., regular sounds. The proposed method is based on the generalized probabilistic descent (GPD) method, which was originally developed as a general concept for the discriminative design of pattern recognizers, and is referred to as minimum detection error (MDE) training. The formulation and implementation of MDE training are described in detail, and its utility is demonstrated in a task of detecting irregular events; more specifically, sounds due to the mis-operation of a tool in a noisy environment.