Search

Mobile and edge computing devices for always-on classification tasks require energy-efficient neural network architectures. In this paper we present several changes to neural architecture searches that improve the chance of success in practical situations. Our search simultaneously optimizes for network accuracy, energy efficiency and memory usage. We benchmark the performance of our search on real hardware, but since running thousands of tests with real hardware is difficult, we use a random forest model to roughly predict the energy usage of a candidate network. We present a search strategy that uses both Bayesian and regularized evolutionary search with particle swarms, and employs early stopping to reduce the computational burden. Our search, evaluated on a sound event classification dataset based upon AudioSet, results in an order of magnitude less energy per inference and a much smaller memory footprint than our baseline MobileNetV1/V2 implementations while slightly improving task accuracy. We also demonstrate how combining a 2D spectrogram with a convolution with many filters causes a computational bottleneck for audio classification and that alternative approaches reduce the computational burden but sacrifice task accuracy.

Population Based Training (PBT) is a recent approach that jointly optimizes neural network weights and hyperparameters which periodically copies weights of the best performers and mutates hyperparameters during training. Previous PBT implementations have been synchronized glass-box systems. We propose a general, black-box PBT framework that distributes many asynchronous "trials" (a small number of training steps with warm-starting) across a cluster, coordinated by the PBT controller. The black-box design does not make assumptions on model architectures, loss functions or training procedures. Our system supports dynamic hyperparameter schedules to optimize both differentiable and non-differentiable metrics. We apply our system to train a state-of-the-art WaveNet generative model for human voice synthesis. We show that our PBT system achieves better accuracy, less sensitivity and faster convergence compared to existing methods, given the same computational resource., 9 pages

A diversity of signals can be recorded with extracellular electrodes. It remains unclear whether different signal types convey similar or different information and whether they capture the same or different underlying neural phenomena. Some researchers focus on spiking activity, while others examine local field potentials, and still others posit that these are fundamentally the same signals. We examined the similarities and differences in the information contained in four signal types recorded simultaneously from multielectrode arrays implanted in primary motor cortex: well-isolated action potentials from putative single units, multiunit threshold crossings, and local field potentials (LFPs) at two distinct frequency bands. We quantified the tuning of these signal types to kinematic parameters of reaching movements. We found 1) threshold crossing activity is not a proxy for single-unit activity; 2) when examined on individual electrodes, threshold crossing activity more closely resembles LFP activity at frequencies between 100 and 300 Hz than it does single-unit activity; 3) when examined across multiple electrodes, threshold crossing activity and LFP integrate neural activity at different spatial scales; and 4) LFP power in the “beta band” (between 10 and 40 Hz) is a reliable indicator of movement onset but does not encode kinematic features on an instant-by-instant basis. These results show that the diverse signals recorded from extracellular electrodes provide somewhat distinct and complementary information. It may be that these signal types arise from biological phenomena that are partially distinct. These results also have practical implications for harnessing richer signals to improve brain-machine interface control.

We propose a Multivariate Gaussian Process Factor Model to estimate low dimensional spatio-temporal patterns of finger motion in repeated reach-to-grasp movements. Our model decomposes and reduces the dimensionality of variation of the multivariate functional data. We first account for time variability through multivariate functional registration, then decompose finger motion into a term that is shared among replications and a term that encodes the variation per replication. We discuss variants of our model, estimation algorithms, and we evaluate its performance in simulations and in data collected from a non-human primate executing a reach-to-grasp task. We show that by taking advantage of the repeated trial structure of the experiments, our model yields an intuitive way to interpret the time and replication variation in our kinematic dataset.

Postspike effects (PSEs) in averages of spike-triggered EMG snippets provide physiological evidence of connectivity between CMN cells and spinal motoneurons innervating skeletal muscles. They are typically detected by visual inspection of spike-triggered averages (SpTAs) or by multiple-fragment/single-snippet analyses [MFA (Poliakov AV, Schieber MH. J Neurosci Methods 79: 143–150, 1998) and SSA (Perel S, Schwartz AB, Ventura V. Neural Comput 26: 40–56, 2014)]; the latter are automatic tests that yield P values. However, MFA/SSA are only effective to detect PSEs that occur at about 6–16 ms posttrigger. Our first contribution is the scan test, an automatic test that has the same utility as SpTA, i.e., it can detect a wide range of PSEs at any latency, but it also yields a P value. Our second contribution is a thorough investigation of the statistical properties of PSE detection tests. We show that when the PSE is weak or the sample size is small, visual inspections of SpTAs have low power, because it is difficult to distinguish PSEs from background EMG variations. We also show that the scan test has better power and that its rate of spurious detections matches the chosen significance level α. This is especially important for investigators because, when a PSE is detected, this guarantees that the probability of a spurious PSE is less than α. Finally, we illustrate the operational characteristics of the PSE detection tests on 2,059 datasets from 5 experiments. The scan test is particularly useful to identify candidate PSEs, which can then be subject to further evaluation by SpTA inspection, and when PSEs are small and visual detection is ambiguous.

Brain-machine interfaces have mostly been used previously to move cursors on computer displays. Now experiments on macaque monkeys show that brain activity signals can control a multi-jointed prosthetic device in real-time. The monkeys used motor cortical activity to control a human-like prosthetic arm in a self-feeding task, with a greater sophistication of control than previously possible. This work could be important for the development of more practical neuro-prosthetic devices in the future. A system where monkeys use their motor cortical activity to control a robotic arm in a real-time self-feeding task, showing a significantly greater sophisitication of control than in previous studies, is demonstrated. This work could be important for the development of more practical neuro-prosthetic devices in the future. Arm movement is well represented in populations of neurons recorded from the motor cortex1,2,3,4,5,6,7. Cortical activity patterns have been used in the new field of brain–machine interfaces8,9,10,11 to show how cursors on computer displays can be moved in two- and three-dimensional space12,13,14,15,16,17,18,19,20,21,22. Although the ability to move a cursor can be useful in its own right, this technology could be applied to restore arm and hand function for amputees and paralysed persons. However, the use of cortical signals to control a multi-jointed prosthetic device for direct real-time interaction with the physical environment (‘embodiment’) has not been demonstrated. Here we describe a system that permits embodied prosthetic control; we show how monkeys (Macaca mulatta) use their motor cortical activity to control a mechanized arm replica in a self-feeding task. In addition to the three dimensions of movement, the subjects’ cortical signals also proportionally controlled a gripper on the end of the arm. Owing to the physical interaction between the monkey, the robotic arm and objects in the workspace, this new task presented a higher level of difficulty than previous virtual (cursor-control) experiments. Apart from an example of simple one-dimensional control23, previous experiments have lacked physical interaction even in cases where a robotic arm16,19,24 or hand20 was included in the control loop, because the subjects did not use it to interact with physical objects—an interaction that cannot be fully simulated. This demonstration of multi-degree-of-freedom embodied prosthetic control paves the way towards the development of dexterous prosthetic devices that could ultimately achieve arm and hand function at a near-natural level.

We propose a Multivariate Gaussian Process Factor Model to estimate low dimensional spatio-temporal patterns of finger motion in repeated reach-to-grasp movements. Our model decomposes and reduces the dimensionality of variation of the multivariate functional data. We first account for time variability through multivariate functional registration, then decompose finger motion into a term that is shared among replications and a term that encodes the variation per replication. We discuss variants of our model, estimation algorithms, and we evaluate its performance in simulations and in data collected from a non-human primate executing a reach-to-grasp task. We show that by taking advantage of the repeated trial structure of the experiments, our model yields an intuitive way to interpret the time and replication variation in our kinematic dataset.

Corticomotoneuronal cells (CMN), located predominantly in the primary motor cortex, project directly to alpha motoneuronal pools in the spinal cord. The effects of CMN spikes on motoneuronal excitability are traditionally characterized by visualizing postspike effects (PSEs) in spike-triggered averages (SpTA; Fetz, Cheney, & German, 1976 ; Fetz & Cheney, 1980 ; McKiernan, Marcario, Karrer, & Cheney, 1998 ) of electromyography (EMG) data. Poliakov and Schieber ( 1998 ) suggested a formal test, the multiple-fragment analysis (MFA), to automatically detect PSEs. However, MFA's performance was not statistically validated, and it is unclear under what conditions it is valid. This paper's contributions are a power study that validates the MFA; an alternative test, the single-snippet analysis (SSA), which has the same functionality as MFA but is easier to calculate and has better power in small samples; a simple bootstrap simulation to estimate SpTA baselines with simulation bands that help visualize potential PSEs; and a bootstrap adjustment to the MFA and SSA to correct for nonlinear SpTA baselines.

Primary motor-cortex multi-unit activity (MUA) and local-field potentials (LFPs) have both been suggested as potential control signals for brain-computer interfaces (BCIs) aimed at movement restoration. Some studies report that LFP-based decoding is comparable to spiking-based decoding, while others offer contradicting evidence. Differences in experimental paradigms, tuning models and decoding techniques make it hard to directly compare these results. Here, we use regression and mutual information analyses to study how MUA and LFP encode various kinematic parameters during reaching movements. We find that in addition to previously reported directional tuning, MUA also contains prominent speed tuning. LFP activity in low-frequency bands (15-40Hz, LFPL) is primarily speed tuned, and contains more speed information than both high-frequency LFP (100-300Hz, LFPH) and MUA. LFPH contains more directional information compared to LFPL, but less information when compared with MUA. Our results suggest that a velocity and speed encoding model is most appropriate for both MUA and LFPH, whereas a speed only encoding model is adequate for LFPL.

Objective. A traditional goal of neural recording with extracellular electrodes is to isolate action potential waveforms of an individual neuron. Recently, in brain–computer interfaces (BCIs), it has been recognized that threshold crossing events of the voltage waveform also convey rich information. To date, the threshold for detecting threshold crossings has been selected to preserve single-neuron isolation. However, the optimal threshold for single-neuron identification is not necessarily the optimal threshold for information extraction. Here we introduce a procedure to determine the best threshold for extracting information from extracellular recordings. We apply this procedure in two distinct contexts: the encoding of kinematic parameters from neural activity in primary motor cortex (M1), and visual stimulus parameters from neural activity in primary visual cortex (V1). Approach. We record extracellularly from multi-electrode arrays implanted in M1 or V1 in monkeys. Then, we systematically sweep the voltage detection threshold and quantify the information conveyed by the corresponding threshold crossings. Main Results. The optimal threshold depends on the desired information. In M1, velocity is optimally encoded at higher thresholds than speed; in both cases the optimal thresholds are lower than are typically used in BCI applications. In V1, information about the orientation of a visual stimulus is optimally encoded at higher thresholds than is visual contrast. A conceptual model explains these results as a consequence of cortical topography. Significance. How neural signals are processed impacts the information that can be extracted from them. Both the type and quality of information contained in threshold crossings depend on the threshold setting. There is more information available in these signals than is typically extracted. Adjusting the detection threshold to the parameter of interest in a BCI context should improve our ability to decode motor intent, and thus enhance BCI control. Further, by sweeping the detection threshold, one can gain insights into the topographic organization of the nearby neural tissue.