Search

This paper pertains to an emerging machine learning paradigm: learning higher-order functions, i.e. functions whose inputs are functions themselves, $\textit{particularly when these inputs are Neural Networks (NNs)}$. With the growing interest in architectures that process NNs, a recurring design principle has permeated the field: adhering to the permutation symmetries arising from the connectionist structure of NNs. $\textit{However, are these the sole symmetries present in NN parameterizations}$? Zooming into most practical activation functions (e.g. sine, ReLU, tanh) answers this question negatively and gives rise to intriguing new symmetries, which we collectively refer to as $\textit{scaling symmetries}$, that is, non-zero scalar multiplications and divisions of weights and biases. In this work, we propose $\textit{Scale Equivariant Graph MetaNetworks - ScaleGMNs}$, a framework that adapts the Graph Metanetwork (message-passing) paradigm by incorporating scaling symmetries and thus rendering neuron and edge representations equivariant to valid scalings. We introduce novel building blocks, of independent technical interest, that allow for equivariance or invariance with respect to individual scalar multipliers or their product and use them in all components of ScaleGMN. Furthermore, we prove that, under certain expressivity conditions, ScaleGMN can simulate the forward and backward pass of any input feedforward neural network. Experimental results demonstrate that our method advances the state-of-the-art performance for several datasets and activation functions, highlighting the power of scaling symmetries as an inductive bias for NN processing. The source code is publicly available at https://github.com/jkalogero/scalegmn., Comment: Accepted at NeurIPS 2024. 34 pages

The potential of Edge Computing technologies is yet to be exploited for multi-domain, multi-party data-driven systems. One aspect that needs to be tackled for the realization of envisioned open edge Ecosystems, is the secure and trusted exchange of data services among diverse stakeholders. In this work, we present a novel approach for integrating mechanisms for trustworthy and sovereign data exchange, into Multi-access Edge Computing (MEC) environments. To this end, we introduce an architecture that extends the ETSI MEC Architectural Framework with artifacts from the International Data Spaces Reference Architecture Model, accompanied by processes that automatically enrich Edge Computing applications with data space capabilities in an as-a-service paradigm. To validate our approach, we implement an open-source prototype solution and we conduct experiments that showcase its functionality and scalability. To our knowledge, this is one of the first concrete architectural specifications for enabling data space features in MEC systems.