Search

Directly parameterizing and learning gradients of functions has widespread significance, with specific applications in inverse problems, generative modeling, and optimal transport. This paper introduces gradient networks (GradNets): novel neural network architectures that parameterize gradients of various function classes. GradNets exhibit specialized architectural constraints that ensure correspondence to gradient functions. We provide a comprehensive GradNet design framework that includes methods for transforming GradNets into monotone gradient networks (mGradNets), which are guaranteed to represent gradients of convex functions. Our results establish that our proposed GradNet (and mGradNet) universally approximate the gradients of (convex) functions. Furthermore, these networks can be customized to correspond to specific spaces of potential functions, including transformed sums of (convex) ridge functions. Our analysis leads to two distinct GradNet architectures, GradNet-C and GradNet-M, and we describe the corresponding monotone versions, mGradNet-C and mGradNet-M. Our empirical results demonstrate that these architectures provide efficient parameterizations and outperform existing methods by up to 15 dB in gradient field tasks and by up to 11 dB in Hamiltonian dynamics learning tasks.

Signal processing (SP) excels at analyzing, processing, and inferring information defined over regular (first continuous, later discrete) domains such as time or space. Indeed, the last 75 years have shown how SP has made an impact in areas such as communications, acoustics, sensing, image processing, and control, to name a few. With the digitalization of the modern world and the increasing pervasiveness of data-collection mechanisms, information of interest in current applications oftentimes arises in non-Euclidean, irregular domains. Graph SP (GSP) generalizes SP tasks to signals living on non-Euclidean domains whose structure can be captured by a weighted graph. Graphs are versatile, able to model irregular interactions, easy to interpret, and endowed with a corpus of mathematical results, rendering them natural candidates to serve as the basis for a theory of processing signals in more irregular domains.

This article reviews significant advances in networked signal and information processing (SIP), which have enabled in the last 25 years extending decision making and inference, optimization, control, and learning to the increasingly ubiquitous environments of distributed agents. As these interacting agents cooperate, new collective behaviors emerge from local decisions and actions. Moreover, and significantly, theory and applications show that networked agents, through cooperation and sharing, are able to match the performance of cloud or federated solutions while offering the potential for improved privacy, increased resilience, and conserved resources. A longer version of this manuscript, with examples and illustrative applications, is available at https://arxiv.org/abs/2210.13767.

Signal processing (SP) is a “hidden” technology that has transformed the digital world and changed our lives in so many ways. The field of digital SP (DSP) took off in the mid-1960s, aided by the integrated circuit and increasing availability of digital computers. Since then, the field of DSP has grown tremendously and fueled groundbreaking advances in technology across a wide range of fields with profound impact on society. The IEEE Signal Processing Society (SPS) is the world’s premier professional society for SP scientists and professionals. Through its high-quality publications, conferences, and technical and educational activities, the SPS has played a pivotal role in advancing the theory and applications of SP. It has been instrumental in promoting cross-disciplinary collaboration and knowledge sharing among researchers, practitioners, and students in the field. This article highlights the SP advances between 1998 and mid-2023 and the evolution of the SPS to empower the growth of SP.