Search

Manual annotation of video segmentation datasets requires an immense amount of human effort, thus, reduction of human annotation costs is an active topic of research. While many papers deal with the propagation of masks through frames of a video, only a few results attempt to optimize annotation task selection. In this paper we present a deep learning based solution to the latter problem and train it using Reinforcement Learning. Our approach utilizes a modified version of the Dueling Deep Q-Network sharing weight parameters across the temporal axis of the video. This technique enables the trained agent to select annotation tasks from the whole video. We evaluate our annotation task selection method by means of a hierarchical supervoxel segmentation based mask propagation algorithm.

Human pose estimation is a crucial step towards understanding and characterizing people’s behavior in images and videos. Current state of the art results on human pose estimation were achieved by large Deep Learning models that are restricted to cloud computing for real time applications. However, with the development of edge computing, Deep Learning is moving more from the cloud to the edge. In this work we present DeepRehab, a Deep Learning based 2D pose estimator optimized for Edge TPU processing. We first improve an existing Edge TPU compatible model named PoseNet by refining its predictions with filtering methods. Subsequently, as the performance of the filters is limited by the model’s inaccuracies, specifically on the lower body parts, we developed DeepRehab, trained on 23 keypoints from the COCO-WholeBody dataset. We achieve 0.65 AP with DeepRehab and quantize it, such that, losing only 3% of AP it runs at a speed of 15 FPS on the Coral USB Accelerator that suits real time evaluations.

Continuous identification of objects with identical appearance is crucial to analyze the behavior of laboratory animals. Most existing methods attempt to avoid this problem by excluding direct social interactions or facilitating it by implants or artificial markers. Unfortunately, these techniques may distort the results as they can affect the behavior of the observed animals. In this paper, we present a simple, deep learning-based approach that can overcome these problems by providing reliable segmentation and tracking of similar instances. Recognition of frames where the system could not reliably locate the objects and mark them suggests human supervision is central to the system since there should be no mistake in instance tracking. Manual annotation of these data improves tracking and decreases annotation needs quickly. The proposed method achieves higher segmentation accuracy and more stable tracking than previous methods despite requiring only a small set of manually annotated data.

Tracking and segmentation of moving objects in videos continues to be the central problem in the separation and prediction of concurrent episodes and situation understanding. Along with critical issues such as collision avoidance, tracking and segmentation have numerous applications in other disciplines, including medicine research. To infer the potential side effects of a given treatment, behaviour analysis of la-boratory animals should be performed, which can be achieved via tracking. This presents a difficult task due to the special circumstances, such as the highly similar shape and the unpredictable movement of the subject creatures, but a precise solution would accelerate research by eliminating the need of manual supervision. To this end, we propose Cluster R-CNN, a deep architecture that uses clustering to segment object instances in a given image and track them across subsequent frames. We show that pairwise clustering coupled with a recurrent unit successfully extends Mask R-CNN to a model capable of tracking and segmenting highly similar moving and occluded objects, providing proper results even in certain cases where related networks fail. In addition to theoretical background and reasoning, our work also features experiments on a unique rat tracking data set, with quantitative results to compare the aforementioned model with other architectures. The proposed Cluster R-CNN serves as a baseline for future work towards achieving an automatic monitoring tool for biomedical research.

Machine learning is making substantial progress in diverse applications. The success is mostly due to advances in deep learning. However, deep learning can make mistakes and its generalization abilities to new tasks are questionable. We ask when and how one can combine network outputs, when (i) details of the observations are evaluated by learned deep components and (ii) facts and rules are available. The Deep Consistence Seeking (DCS) machine seeks for consistent and deterministic event descriptions and improves the representation accordingly. The machine has an anomaly detection component that may trigger coherence seeking. Coherence seeking resolves conflicts between computational modules by preferring components with higher scores. We illustrate that context can help in correcting recognitions and in deriving training samples for self-training. We put these concepts into a general framework of cognition, by distinguishing creativity, rule extraction, verification, and symbol grounding. We demonstrate our approach in a driving scenario.

Previously, we have put forth the concept of Cartesian abstraction and argued that it can yield ‘cognitive maps’. We suggested a general mechanism and presented deep learning based numerical simulations: an observed factor (head direction) was non-linearly projected to form a discretized representation (head direction cells). That representation, in turn, enabled the development of a complementing factor (place cells) from high dimensional (visual) inputs. It has been shown that a related metric, in the form of oriented hexagonal grids, may also be derived. Elements of the algorithms were connected to the entorhinal-hippocampal complex (EHC loop). Here, we make one step further in the mapping to the neural substrate. We consider (i) the features of signals arriving at deep and superficial CA1 pyramidal cells, (ii) the interplay between lateral and medial entorhinal cortex efferents, and the nature of ‘instructive’ input timing-dependent plasticity, a feature of the loop. We suggest that the circuitry corresponds to a special form of Residual Networks that we call Sparsified and Twisted Residual Autoencoder (ST-RAE). We argue that ST-RAEs can learn Cartesian Factors and fit the structure and the working of the entorhinal-hippocampal complex to a reasonable extent, including certain oscillatory properties. We put forth the idea that the factor learning architecture of ST-RAEs has a double role in serving goal-oriented behavior, such as (a) the lowering the dimensionality of the task and (b) the mitigation of the problem of partial observation.

Ever since the discovery of columnar structures, their function remained enigmatic. As a potential explanation for this puzzling function, we introduce the ‘Columnar Machine’. We join two neural network types, Structured Sparse Coding (SSC) of generative nature exploiting sparse groups of neurons and Feed-Forward Networks (FFNs) into one architecture. Memories supporting recognition can be quickly loaded into SSC via supervision or can be learned by SSC in a self-organized manner. However, SSC evaluation is slow. We train FFNs for predicting the sparse groups and then the representation is computed by fast undercomplete methods. This two step procedure enables fast estimation of the overcomplete group sparse representations. The suggested architecture works fast and it is biologically plausible. Beyond the function of the minicolumnar structure it may shed light onto the role of fast feed-forward inhibitory thalamocortical channels and cortico-cortical feed-back connections. We demonstrate the method for natural image sequences where we exploit temporal structure and for a cognitive task where we explain the meaning of unknown words from their contexts.

It has been long debated how the so called cognitive map, the set of place cells, develops in rat hippocampus. The function of this organ is of high relevance, since the hippocampus is the key component of the medial temporal lobe memory system, responsible for forming episodic memory, declarative memory, the memory for facts and rules that serve cognition in humans. Here, a general mechanism is put forth: We introduce the novel concept of Cartesian factors. We show a non-linear projection of observations to a discretized representation of a Cartesian factor in the presence of a representation of a complementing one. The computational model is demonstrated for place cells that we produce from the egocentric observations and the head direction signals. Head direction signals make the observed factor and sparse allothetic signals make the complementing Cartesian one. We present numerical results, connect the model to the neural substrate, and elaborate on the differences between this model and other ones, including Slow Feature Analysis [17] .

In monocular 3D human pose estimation a common setup is to first detect 2D positions and then lift the detection into 3D coordinates. Many algorithms suffer from overfitting to camera positions in the training set. We propose a siamese architecture that learns a rotation equivariant hidden representation to reduce the need for data augmentation. Our method is evaluated on multiple databases with different base networks and shows a consistent improvement of error metrics. It achieves state-of-the-art cross-camera error rate among algorithms that use estimated 2D joint coordinates only., Comment: Accepted to Neurocomputing

We argue that recent technology developments hold great promises for health and wellbeing. In our view, recent advances of (1) smart tools and wearable sensors of diverse kinds, (2) data collection and data mining methods, (3) 3D visual recording and visual processing methods, (4) 3D models of the environment with robust physics engine, and last but not least, (5) new applications of human computing and crowdsourcing started the revolution. We are neither claiming nor excluding that human intelligence will be reached in some years from now, but make the above claim, which is both weaker and stronger. We believe that fast developments for health and wellbeing are the question of active collaboration between health and wellbeing experts and motivated engineers.

The existence of place cells (PCs), grid cells (GCs), border cells (BCs), and head direction cells (HCs) as well as the dependencies between them have been enigmatic. We make an effort to explain their nature by introducing the concept of Cartesian Factors. These factors have specific properties: (i) they assume and complement each other, like direction and position and (ii) they have localized discrete representations with predictive attractors enabling implicit metric-like computations. In our model, HCs make the distributed and local representation of direction. Predictive attractor dynamics on that network forms the Cartesian Factor “direction.” We embed these HCs and idiothetic visual information into a semi-supervised sparse autoencoding comparator structure that compresses its inputs and learns PCs, the distributed local and direction independent (allothetic) representation of the Cartesian Factor of global space. We use a supervised, information compressing predictive algorithm and form direction sensitive (oriented) GCs from the learned PCs by means of an attractor-like algorithm. Since the algorithm can continue the grid structure beyond the region of the PCs, i.e., beyond its learning domain, thus the GCs and the PCs together form our metric-like Cartesian Factors of space. We also stipulate that the same algorithm can produce BCs. Our algorithm applies (a) a bag representation that models the “what system” and (b) magnitude ordered place cell activities that model either the integrate-and-fire mechanism, or theta phase precession, or both. We relate the components of the algorithm to the entorhinal-hippocampal complex and to its working. The algorithm requires both spatial and lifetime sparsification that may gain support from the two-stage memory formation of this complex.

We introduce a learning architecture that can serve compression while it also satisfies the constraints of factored reinforcement learning. Our novel Cartesian factors enable one to decrease the number of variables being relevant for the ongoing task, an exponential gain in the size of the state space. We demonstrate the working, the limitations and the promises of the abstractions: we develop a representation of space in allothetic coordinates from egocentric observations and argue that the lower dimensional allothetic representation can be used for path planning. Our results on the learning of Cartesian factors indicate that (a) shallow autoencoders perform well in our numerical example and (b) if deeper networks are needed, e.g., for classification or regression, then sparsity should also be enforced at (some of) the intermediate layers.

Facial expression communicates emotion, intention, and physical state, and regulates interpersonal behavior. Automated face analysis (AFA) for the detection, synthesis, and understanding of facial expression is a vital focus of basic research with applications in behavioral science, mental and physical health and treatment, marketing, and human-robot interaction among other domains. In previous work, facial action unit (AU) detection becomes seriously degraded when head orientation exceeds \(15^{\circ }\) to \(20^{\circ }\). To achieve reliable AU detection over a wider range of head pose, we used 3D information to augment video data and a deep learning approach to feature selection and AU detection. Source video were from the BP4D database (n = 41) and the FERA test set of BP4D-extended (n = 20). Both consist of naturally occurring facial expression in response to a variety of emotion inductions. In augmented video, pose ranged between \(-18^{\circ }\) and \(90^{\circ }\) for yaw and between \(-54^{\circ }\) and \(54^{\circ }\) for pitch angles. Obtained results for action unit detection exceeded state-of-the-art, with as much as a 10 % increase in \(F_1\) measures.

[This corrects the article DOI: 10.1371/journal.pone.0195131.].

We treat the problem of searching for hidden multi-dimensional independent auto-regressive processes (auto-regressive independent process analysis, AR-IPA). Independent subspace analysis (ISA) can be used to solve the AR-IPA task. The so-called separation theorem simplifies the ISA task considerably: the theorem enables one to reduce the task to one-dimensional blind source separation task followed by the grouping of the coordinates. However, the grouping of the coordinates still involves two types of combinatorial problems: (a) the number of the independent subspaces and their dimensions, and then (b) the permutation of the estimated coordinates are to be determined. Here, we generalize the separation theorem. We also show a non-combinatorial procedure, which—under certain conditions—can treat these two combinatorial problems. Numerical simulations have been conducted. We investigate problems that fulfill sufficient conditions of the theory and also others that do not. The success of the numerical simulations indicates that further generalizations of the separation theorem may be feasible.

The problem of anomaly detection is a critical topic across application domains and is the subject of extensive research. Applications include finding frauds and intrusions, warning on robot safety, and many others. Standard approaches in this field exploit simple or complex system models, created by experts using detailed domain knowledge.

It is an intriguing task to develop efficient connectionist representations for learning long time series. Recurrent neural networks have great promises here. We model the learning task as a minimization problem of a nonlinear least-squares cost function, that takes into account both one-step and multi-step prediction errors. The special structure of the cost function is constructed to build a bridge to reinforcement learning. We exploit this connection and derive a convergent, policy iteration-based algorithm, and show that RNN training can be made to fit the reinforcement learning framework in a natural fashion. The relevance of this connection is discussed. We also present experimental results, which demonstrate the appealing properties of the unique parameter structure prescribed by reinforcement learning. Experiments cover both sequence learning and long-term prediction.

Identification of mixed independent subspaces is thought to suffer from combinatorial explosion of two kinds: the minimization of mutual information between the estimated subspaces and the search for the optimal number and dimensions of the subspaces. Here we show that independent autoregressive process analysis, under certain conditions, can avoid this problem using a two-phase estimation process. We illustrate the solution by computer demonstration.

In this paper, we present a probabilistic method that can improve the efficiency of document classification when applied to structured documents. The analysis of the structure of a document is the starting point of document classification. Our method is designed to augment other classification schemes and complement pre-filtering information extraction procedures to reduce uncertainties. To this end, a probabilistic distribution on the structure of XML documents is introduced. We show how to parameterise existing learning methods to describe the structure distribution efficiently. The learned distribution is then used to predict the classes of unseen documents. Novelty detection making use of the structure-based distribution function is also discussed. Demonstration on model documents and on Internet XML documents are presented.

The slogan that information is powerhas undergone a slight change. Today, information updatingis in the focus of interest. The largest source of information is the world-wide web. Fast search methods are in need for this enormous source. In this paper a hybrid architecture that combines soft support vector classification and reinforcement learning for value estimation is introduced for the evaluation of a link (a document) and its neighboring links (or documents), called the context of a document. The method is motivated by (i) large differences between such contexts on the web, (ii) the facilitation of goal oriented search using context classifiers, and (iii) attractive fast adaptation properties, that could counteract diversity of web environments. We demonstrate that value estimation-based fast adaptation offers considerable improvement over other known search methods. © 2002 Elsevier Science B.V. All rights reserved.

We extend the scope of Wikification to novel words by relaxing two premises of Wikification: (i) we wikify without using the surface form of the word (ii) to a mixture of Wikipedia senses instead of a single sense. We identify two types of “novel” words: words where the connection between their surface form and their meaning is broken (e.g., a misspelled word), and words where there is no meaning to connect to—the meaning itself is also novel. We propose a method capable of wikifying both types of novel words while also dealing with the inherently large-scale disambiguation problem. We show that the method can disambiguate between up to 1,000 Wikipedia senses, and it can explain words with novel meaning as a mixture of other, possibly related senses. This mixture representation compares favorably to the widely used bag of words representation.

In many behavioral domains, such as facial expression and gesture, sparse structure is prevalent. This sparsity would be well suited for event detection but for one problem. Features typically are confounded by alignment error in space and time. As a consequence, high-dimensional representations such as SIFT and Gabor features have been favored despite their much greater computational cost and potential loss of information. We propose a Kernel Structured Sparsity (KSS) method that can handle both the temporal alignment problem and the structured sparse reconstruction within a common framework, and it can rely on simple features. We characterize spatio-temporal events as time-series of motion patterns and by utilizing time-series kernels we apply standard structured-sparse coding techniques to tackle this important problem. We evaluated the KSS method using both gesture and facial expression datasets that include spontaneous behavior and differ in degree of difficulty and type of ground truth coding. KSS outperformed both sparse and non-sparse methods that utilize complex image features and their temporal extensions. In the case of early facial event classification KSS had 10% higher accuracy as measured by F1 score over kernel SVM methods1.

It has been suggested that sensory information processing makes use of a factorial code. It has been shown that the major components of the hippocampal-entorhinal loop can be derived by conjecturing that the task of this loop is forming and encoding independent components (ICs), one type of factorial codes. However, continuously changing environment poses additional requirements on the coding that can be (partially) satisfied by extending the analysis to the temporal domain and performing IC analysis on concatenated inputs of time slices. We use computer simulations to decide whether IC analysis on temporal sequences can produce place fields in labyrinths or not.

A common technique in neurocontrol is that of controlling a plant by static state feedback using the plant's inverse dynamics, which is approximated through a learning process. It is well known that in this control mode even small approximation errors or, which is the same, small perturbations of the plant may lead to instability. Here, a novel approach is proposed to overcome the problem of instability by using the inverse dynamics both for the static and for the error-compensating dynamic state feedback control. This scheme is termed SDS feedback control. It is shown that as long as the error of the inverse dynamics model is “signproper” the SDS feedback control is stable, i.e., the error of tracking may be kept small. The proof is based on a modification of Liapunov's second method. The problem of on-line learning of the inverse dynamics when using the controller simultaneously for both forward control and for dynamic feedback is dealt with, as are questions related to noise sensitivity and robust control of robotic manipulators. Simulations of a simplified sensorimotor loop serve to illustrate the approach.

It is shown that local, extended objects of a metrical topological space shape the receptive fields of competitive neurons to local filters. Self-organized topology learning is then solved with the help of Hebbian learning together with extended objects that provide unique information about neighborhood relations. A topographical map is deduced and is used to speed up further adaptation in a changing environment with the help of Kohonen-type learning that teaches the neighbors of winning neurons as well.

A brain model-based alternative to reinforcement learning is presented that integrates artificial neural networks and knowledge-based systems into one unit or agent for goal-oriented problem solving. The agent may possess inherited and learned artificial neural networks and knowledge-based subsystems. The agent has and develops ANN cues to the environment for dimensionality reduction (data compression) to ease the problem of combinatorial explosion. Here, a dynamical concept model is put forward that builds cue models of the phenomena in the world, designs dynamical action sets (concepts), and makes them compete in a spreading-activation neural stage to reach decision. The agent works under closed-loop control. Here we examine a simple robotlike object in a two-dimensional conditionally probabilistic space.

The exploration-exploitation dilemma has been an intriguing and unsolved problem within the framework of reinforcement learning. "Optimism in the face of uncertainty" and model building play central roles in advanced exploration methods. Here, we integrate several concepts and obtain a fast and simple algorithm. We show that the proposed algorithm finds a near-optimal policy in polynomial time, and give experimental evidence that it is robust and efficient compared to its ascendants.

In this paper a novel approach to neurocognitive modeling is proposed in which the central constraints are provided by the theory of reinforcement learning. In this formulation learning is (1) exploiting the statistical properties of the system’s environment, (2) constrained by biologically inspired Hebbian interactions and (3) based only on algorithms which are consistent and stable. In the resulting model some of the most enigmatic problems of artificial intelligence have to be addressed. In particular, considerations on combinatorial explosion lead to constraints on the concepts of state-action pairs: these concepts have the peculiar flavor of determinism in a partially observed and thus highly uncertain world. We will argue that these concepts of factored reinforcement learning result in an intriguing learning task that we call the symbol learning problem. For this task we sketch an information theoretic framework and point towards a possible resolution.

Independent subspace analysis (ISA) that deals with multi-dimensional independent sources, is a generalization of independent component analysis (ICA). However, all known ISA algorithms may become ineffective when the sources possess temporal structure. The innovation process instead of the original mixtures has been proposed to solve ICA problems with temporal dependencies. Here we show that this strategy can be applied to ISA as well. We demonstrate the idea on a mixture of 3D processes and also on a mixture of facial pictures used as two-dimensional deterministic sources. ISA on innovations was able to find the original subspaces, while plain ISA was not.

This paper introduces an unsupervised algorithm that collects senses contained in WordNet to explain words, whose meaning is unknown, but plenty of documents are available that contain the word in that unknown sense. Based on the widely accepted idea that the meaning of a word is characterized by its context, a neural network architecture was designed to reconstruct the meaning of the unknown word. The connections of the network were derived from word co-occurrences and word-sense statistics. The method was tested on 80 TOEFL synonym questions, from which 63 questions were answered correctly. This is comparable to other methods tested on the same questions, but using a larger corpus or richer lexical database. The approach was found robust against details of the architecture.

In this paper we compare our selection based learning algorithm with the reinforcement learning algorithm in Web crawlers. The task of the crawlers is to find new information on the Web. We performed simulations based on data collected from the Web. The collected portion of the Web is typical and exhibits scale-free small world (SFSW) structure. We have found that on this SFSW, the weblog update algorithm performs better than the reinforcement learning algorithm. It finds the new information faster than the reinforcement learning algorithm and has better new information/all submitted documents ratio.

A multi-component cooperating system have been designed for severely disabled children having various disabilities. Different input tools have been developed to exploit possible ‘outputs', e.g., head motion or leg motion. Specific software tools serve to convert such ‘outputs' in different computer aided tasks. Extendable software enables configurable networking. Robotic personal agent helps the communication.

We propose a method to determine unintelligible words based on the textual context of the word determined. As there can be many different possibilities for the word, a robust, large-scale method is needed. The large scale makes the problem sensitive to spurious similarities of contexts: when the contexts of two, different words are similar. To reduce this effect, we induce structured sparsity on the words by formulating the task as a group Lasso problem. We compare this formulation to a k-nearest neighbor and a support vector machine based approach, and find that group Lasso outperforms both by a large margin. We achieve up to 75% of accuracy when determining the word from among 1000 words both on the Brown corpus and on the British National Corpus. Unintelligible words are often the result of errors in Optical Character Recognition (OCR) algorithms. As the proposed method utilizes information independent from information used in OCR, we expect that a combined approach could be very successful, as OCR and the proposed method complement each other.