Search

We characterized the generalization capabilities of DNN-based encoding models when predicting neuronal responses from the visual cortex. We collected \textit{MacaqueITBench}, a large-scale dataset of neural population responses from the macaque inferior temporal (IT) cortex to over $300,000$ images, comprising $8,233$ unique natural images presented to seven monkeys over $109$ sessions. Using \textit{MacaqueITBench}, we investigated the impact of distribution shifts on models predicting neural activity by dividing the images into Out-Of-Distribution (OOD) train and test splits. The OOD splits included several different image-computable types including image contrast, hue, intensity, temperature, and saturation. Compared to the performance on in-distribution test images -- the conventional way these models have been evaluated -- models performed worse at predicting neuronal responses to out-of-distribution images, retaining as little as $20\%$ of the performance on in-distribution test images. The generalization performance under OOD shifts can be well accounted by a simple image similarity metric -- the cosine distance between image representations extracted from a pre-trained object recognition model is a strong predictor of neural predictivity under different distribution shifts. The dataset of images, neuronal firing rate recordings, and computational benchmarks are hosted publicly at: https://bit.ly/3zeutVd.

One of the most impactful findings in computational neuroscience over the past decade is that the object recognition accuracy of deep neural networks (DNNs) correlates with their ability to predict neural responses to natural images in the inferotemporal (IT) cortex. This discovery supported the long-held theory that object recognition is a core objective of the visual cortex, and suggested that more accurate DNNs would serve as better models of IT neuron responses to images. Since then, deep learning has undergone a revolution of scale: billion parameter-scale DNNs trained on billions of images are rivaling or outperforming humans at visual tasks including object recognition. Have today's DNNs become more accurate at predicting IT neuron responses to images as they have grown more accurate at object recognition? Surprisingly, across three independent experiments, we find this is not the case. DNNs have become progressively worse models of IT as their accuracy has increased on ImageNet. To understand why DNNs experience this trade-off and evaluate if they are still an appropriate paradigm for modeling the visual system, we turn to recordings of IT that capture spatially resolved maps of neuronal activity elicited by natural images. These neuronal activity maps reveal that DNNs trained on ImageNet learn to rely on different visual features than those encoded by IT and that this problem worsens as their accuracy increases. We successfully resolved this issue with the neural harmonizer, a plug-and-play training routine for DNNs that aligns their learned representations with humans. Our results suggest that harmonized DNNs break the trade-off between ImageNet accuracy and neural prediction accuracy that assails current DNNs and offer a path to more accurate models of biological vision.

Primates constantly explore their surroundings via saccadic eye movements that bring different parts of an image into high resolution. In addition to exploring new regions in the visual field, primates also make frequent return fixations, revisiting previously foveated locations. We systematically studied a total of 44,328 return fixations out of 217,440 fixations. Return fixations were ubiquitous across different behavioral tasks, in monkeys and humans, both when subjects viewed static images and when subjects performed natural behaviors. Return fixations locations were consistent across subjects, tended to occur within short temporal offsets, and typically followed a 180-degree turn in saccadic direction. To understand the origin of return fixations, we propose a proof-of-principle, biologically-inspired and image-computable neural network model. The model combines five key modules: an image feature extractor, bottom-up saliency cues, task-relevant visual features, finite inhibition-of-return, and saccade size constraints. Even though there are no free parameters that are fine-tuned for each specific task, species, or condition, the model produces fixation sequences resembling the universal properties of return fixations. These results provide initial steps towards a mechanistic understanding of the trade-off between rapid foveal recognition and the need to scrutinize previous fixation locations., Comment: 9 main figs and 24 supp figs, accepted in PLOS Computational Biology

Artificial neural networks (ANNs) are considered the current best models of biological vision. ANNs are the best predictors of neural activity in the ventral stream; moreover, recent work has demonstrated that ANN models fitted to neuronal activity can guide the synthesis of images that drive pre-specified response patterns in small neuronal populations. Despite the success in predicting and steering firing activity, these results have not been connected with perceptual or behavioral changes. Here we propose an array of methods for creating minimal, targeted image perturbations that lead to changes in both neuronal activity and perception as reflected in behavior. We generated 'deceptive images' of human faces, monkey faces, and noise patterns so that they are perceived as a different, pre-specified target category, and measured both monkey neuronal responses and human behavior to these images. We found several effective methods for changing primate visual categorization that required much smaller image change compared to untargeted noise. Our work shares the same goal with adversarial attack, namely the manipulation of images with minimal, targeted noise that leads ANN models to misclassify the images. Our results represent a valuable step in quantifying and characterizing the differences in perturbation robustness of biological and artificial vision.

It has long been thought that the pulsed X-ray properties of rotation-powered pulsars are stable on long time scales. However, long-term, systematic studies of individual sources have been lacking. Furthermore, dramatic X-ray variability has now been observed from two pulsars having inferred sub-critical dipole magnetic fields. Here we present an analysis of the long-term pulsed X-ray properties of the young, energetic pulsar PSR B1509-58 using data from the Rossi X-ray Timing Explorer. We measured the 2-50 keV pulsed flux for 14.7 yr of X-ray observations and found that it is consistent with being constant on all relevant time scales, and place a 3 sigma upper limit on day-to-week variability of <28%. In addition, we searched for magnetar-like X-ray bursts in all observations and found none, which we use to constrain the measurable burst rate to less than one per 750 ks of observations. We also searched for variability in the pulse profile and found that it is consistent with being stable on time scales of days to decades. This supports the hypothesis that X-ray properties of rotation-powered X-ray pulsars can be stable on decade-long time scales. In addition, we extend the existing timing solution by 7.1 yr to a total of 28.4 yr and report updated values of the braking index, n=2.832+/-0.003 and the second braking index, m=17.6+/-1.9., Comment: 8 pages, 5 figures. Accepted for publication in the Astrophysical Journal

On 2011 July 14, a new magnetar candidate, Swift J1822.3-1606, was identified via a rate trigger on the Swift/Burst Alert Telescope. Here we present an initial analysis of the X-ray properties of the source, using data from the Rossi X-ray Timing Explorer, Swift, and the Chandra X-ray Observatory, spanning 2011 July 16--September 22. We measure a precise spin period of P=8.43771963(5) s and a spin-down rate of 2.97(28)E-13, at MJD 55761.0, corresponding to an inferred surface dipole magnetic field strength of B=5.1E13 G, the second lowest thus far measured for a magnetar, though similar to 1E~2259+586 as well as to several high-magnetic field radio pulsars. We show that the pulsed X-ray flux decay in the 2--10 keV band is best fit by an exponential with a time constant of 16.4+/-0.3 days. After increasing from ~35% during the first week after the onset of the outburst, the pulsed fraction in the 2--10 keV band remained constant at ~45. We argue that these properties confirm this source to be a new member of the class of objects known as magnetars., Comment: 7 pages, 3 figures. Submitted to the Astrophysical Journal Letters

The ~800 yr-old pulsar J1846-0258 is a unique transition object between rotation-powered pulsars and magnetars: though behaving like a rotation-powered pulsar most of the time, in 2006 it exhibited a distinctly magnetar-like outburst accompanied by a large glitch. Here we present X-ray timing observations taken with the Rossi X-ray Timing Explorer over a 2.2-yr period after the X-ray outburst and glitch had recovered. We observe that the braking index of the pulsar, previously measured to be n=2.65+/-0.01, is now n=2.16+/-0.13, a decrease of 18+/-5%. We also note a persistent increase in the timing noise relative to the pre-outburst level. Despite the timing changes, a 2009 Chandra X-ray Observatory observation shows that the X-ray flux and spectrum of the pulsar and its wind nebula are consistent with the quiescent levels observed in 2000., Comment: 15 pages, 7 figures. ApJ, in press. Upper limit on blackbody component added, as well as additional discussion

We report a large spin-up glitch in PSR J1846-0258 which coincided with the onset of magnetar-like behavior on 2006 May 31. We show that the pulsar experienced an unusually large glitch recovery, with a recovery fraction of Q=8.7+/- 2.5, resulting in a net decrease of the pulse frequency. Such a glitch recovery has never before been observed in a rotation-powered pulsar, however, similar but smaller glitch over-recovery has been recently reported in the magnetar AXP 4U~0142+61 and may have occurred in the SGR 1900+14. We also report a large increase in the timing noise of the source. We discuss the implications of the unusual timing behavior in PSR J1846-0258 on its status as the first identified magnetically active rotation-powered pulsar., Comment: 14 pages, 5 figures, Accepted for publication in the Astrophysical Journal. Incorporates changes from an anonymous referee; additional analysis and discussion included

We present timing data spanning 6.4 yr for the young and energetic PSR J0205+6449, in the supernova remnant 3C 58. Data were obtained with the Rossi X-ray Timing Explorer, the Jodrell Bank Observatory and the Green Bank Telescope. We present phase-coherent timing analyses showing timing noise and two spin-up glitches with fractional frequency increases of ~3.4E-7 near MJD 52555, and ~3.8E-6 between MJDs 52777 and 53062. These glitches are unusually large if the pulsar was created in the historical supernova in 1181 as has been suggested. For the X-ray timing we developed a new unbinned maximum-likelihood method for determining pulse arrival times which performs significantly better than the traditional binned techniques. In addition, we present an X-ray pulse profile analysis of four years of RXTE data showing that the pulsar is detected up to ~40 keV. We also present the first measurement of the phase offset between the radio and X-ray pulse for this source, showing that the radio pulse leads the X-ray pulse by phi=0.10+/-0.01 in phase. We compile all known measurements of the phase offsets between radio and X-ray and radio and gamma-ray pulses for X-ray and gamma-ray pulsars. We show that there is no relationship between pulse period and phase offset, supported by our measurement of the phase offset for PSR J0205+6449., Comment: 19 pages, 12 figures. Published in the Astrophysical Journal. Includes additional data analysis and two new figures

The rotation-powered pulsar PSR J1846-0258 in the supernova remnant Kes 75 was recently shown to have exhibited magnetar-like X-ray bursts in mid-2006. Radio emission has not yet been observed from this source, but other magnetar-like sources have exhibited transient radio emission following X-ray bursts. We report on a deep 1.9 GHz radio observation of PSR J1846-0258 with the 100-m Green Bank Telescope in late 2007 designed to search for radio pulsations or bursts from this target. We have also analyzed three shorter serendipitous 1.4 GHz radio observations of the source taken with the 64-m Parkes telescope during the 2006 bursting period. We detected no radio emission from PSR J1846-0258 in either the Green Bank or Parkes datasets. We place an upper limit of 4.9 \mu Jy on coherent pulsed emission from PSR J1846-0258 based on the 2007 November 2 observation, and an upper limit of 27 \mu Jy around the time of the X-ray bursts. Serendipitously, we observed radio pulses from the nearby RRAT J1846-02, and place a 3\sigma confidence level upper limit on its period derivative of 1.7 * 10^{-13}, implying its surface dipole magnetic field is less than 2.6 * 10^{13} G., Comment: 15 pages, 2 figures, submitted to ApJ

Anomalous X-ray Pulsars (AXPs), thought to be magnetars, exhibit poorly understood deviations from a simple spin-down called "timing noise". AXP timing noise has strong low-frequency components which pose significant challenges for quantification. We describe a procedure for extracting two quantities of interest, the intensity and power spectral index of timing noise. We apply this procedure to timing data from three sources: a monitoring campaign of five AXPs, observations of five young pulsars, and the stable rotator PSR B1937+21., Comment: submitted to the proceedings of the "40 Years of Pulsars" conference

Pulsar braking indices offer insight into the physics that underlies pulsar spin-down. Only five braking indices have been measured via phase-coherent timing; all measured values are less than 3, the value expected from magnetic dipole radiation. Here we present new measurements for three of the five pulsar braking indices, obtained with phase-coherent timing for PSRs J1846-0258 (n=2.65+/-0.01), B1509-58 (n=2.839+/-0.001) and B0540-69 (n=2.140+/-0.009). We discuss the implications of these results and possible physical explanations for them., Comment: 7 pages, 5 figures. To be published in the proceedings of the conference "Isolated Neutron Stars: from the Interior to the Surface" (April 24-28, 2006, London, UK), eds. D. Page, R. Turolla, & S. Zane

We present the first phase-coherent measurement of a braking index for the young, energetic rotation-powered pulsar PSR J1846-0258. This 324 ms pulsar is located at the center of the supernova remnant Kes 75 and has a characteristic age of tau_c = 723 years, a spin-down energy of 8.3x10^{36}erg/s, and inferred magnetic field of 4.9x10^{13} G. Two independent phase-coherent timing solutions are derived which together span 5.5 yr of data obtained with the Rossi X-ray Timing Explorer. In addition, a partially phase-coherent timing analysis confirms the fully phase-coherent result. The measured value of the braking index, n=2.65+/-0.01, is significantly less than 3, the value expected from magnetic dipole radiation, implying another physical process must contribute to the pulsar's rotational evolution. Assuming the braking index has been constant since birth, we place an upper limit on the spin-down age of J1846-0258 of 884 yr, the smallest age estimate of any rotation-powered pulsar., Comment: Includes major changes to analysis. Updated postion used for the pulsar which significantly changed the result and conclusions of the paper. 21 pages, 6 Figures. ApJ submitted

We present a new phase-coherent timing analysis for the young, energetic pulsar PSR B0540-69 using 7.6 yr of data from the Rossi X-Ray Timing Explorer. We measure the braking index, n=2.140+/-0.009, and discuss our measurement in the context of other discordant values reported in the literature. We present an improved source position from the phase-coherent timing of the pulsar, to our knowledge, the first of its kind from X-ray pulsar timing. In addition, we detect evidence for a glitch which has been previously reported but later disputed. The glitch occured at MJD 51335+/-12 with a fractional change in frequency of (1.4+/-0.2)E{-9} and a fractional change in frequency derivative of (1.33+/-0.02)E{-4}. We calculate that the glitch activity parameter for PSR B0540-69 is two orders of magnitude smaller than that of the Crab pulsar which has otherwise very similar properties. This suggests that neutron stars of similar apparent ages, rotation properties and inferred dipolar B fields can have significantly different internal properties., Comment: 18 Preprint pages, 6 Figures. Accepted to the Astrophysical Journal

We present an updated timing solution for the young, energetic pulsar PSR B1509-58 based on 21.3 years of radio timing data and 7.6 years of X-ray timing data. No glitches have occurred in this time span, in contrast to other well-studied young pulsars, which show frequent glitches. We report a measurement of the third frequency derivative of (-1.28+/-0.21)x10^(-31) s^(-4). This value is 1.65 standard deviations from, i.e. consistent with, that predicted by the simple constant magnetic dipole model of pulsar spin-down. We measured the braking index to be n=2.839+/-0.003 and show that it varies by 1.5% over 21.3 yr due to contamination from timing noise. Results of a low-resolution power spectral analysis of the significant noise apparent in the data yield a spectral index of alpha=-4.6+/-1.0 for the red noise component., Comment: 21 pages, 4 figures, 2 tables. Accepted by ApJ

We present three new binary pulsars discovered during a search for pulsations in 56 unidentified mid-latitude EGRET gamma-ray error boxes with the Parkes multibeam receiver. Timing observations of these sources is on-going with both the Parkes and the Green Bank telescopes. We discuss the place of these new systems in the population of binary pulsars and suggest that they are all somewhat atypical systems., Comment: To appear in Binary Radio Pulsars, proceedings of an Aspen Winter Conference, eds. F. Rasio, I. Stairs, 4 pages, 2 figures

We present an updated phase-coherent timing solution for the young, energetic pulsar PSR B1509-58 for twenty years of data. Using a partially phase-coherent timing analysis, we show that the second frequency derivative is changing in time implying a third frequency derivative of (-9+/-1)x10^-32 s^(-4). This value is consistent with the simple power law model of pulsar rotation., Comment: 2 pages. To appear in "Young Neutron Stars and Their Environments"(IAU Symposium 218, ASP Conference Proceedings), eds F. Camilo and B.M. Gaensler

We present the results of radio searches for pulsars within unidentified EGRET source error boxes. Using the Parkes multibeam system, we have surveyed 56 sources at Galactic latitudes |b| > 5 deg which do not appear to be associated with blazars. This population has been suggested to be distributed like the local star forming region known as the Gould Belt, the Galactic Halo, and/or the millisecond pulsars. We have discovered several new pulsars in this survey, including three new binary systems., Comment: 4 pages, to appear in the Proceedings of Young Neutron Stars and their Environment, International Astronomical Union. Symposium no. 218, held 14-17 July, 2003 in Sydney, Australia