Search

Growing privacy concerns regarding personal data disclosure are contrasting with the constant need of such information for data-driven applications. To address this issue, the combination of federated learning and differential privacy is now well-established in the domain of machine learning. These techniques allow to train deep neural networks without collecting the data and while preventing information leakage. However, there are many scenarios where simpler and more robust machine learning models are preferable. In this paper, we present a federated and differentially-private version of the Naive Bayes algorithm for classification. Our results show that, without data collection, the same performance of a centralized solution can be achieved on any dataset with only a slight increase in the privacy budget. Furthermore, if certain conditions are met, our federated solution can outperform a centralized approach., QC 20220926Part of proceedings: ISBN 978-989-758-590-6

Heterogeneous Information Network (HIN) embedding has been a prevalent approach to learn representations off semantically-rich heterogeneous networks. Most HIN embedding methods exploit meta-paths to retain high-order structures, yet, their performance is conditioned on the quality of the (generated/manually-defined) meta-paths and their suitability for the specific label set. Whereas other methods adjust random walks to harness or skip certain heterogeneous structures (e.g. node type(s)), in doing so, the adjusted random walker may casually omit other node/edge types. Our key insight is with no domain knowledge, the random walker should hold no assumptions about heterogeneous structure (i.e. edge types). Thus, aiming for a flexible and general method, we utilize network schema as a unique blueprint of HIN, and propose SchemaWalk, a random walk to uniformly sample all edge types within the network schema. Moreover, we identify the starvation phenomenon which induces random walkers on HINs to under- or over-sample certain edge types. Accordingly, we design SchemaWalkHO to skip local deficient connectivity to preserve uniform sampling distribution. Finally, we carry out node classification experiments on four real-world HINs, and provide in-depth qualitative analysis. The results highlight the robustness of our method regardless to the graph structure in contrast with the state-of-the-art baselines., QC 20230523

The growing computational demands of model training tasks and the increased privacy awareness of consumers call for the development of new techniques in the area of machine learning. Fully decentralized approaches have been proposed, but are still in early research stages. This study analyses gossip learning, one of these state-of-the-art decentralized machine learning protocols, which promises high scalability and privacy preservation, with the goal of assessing its applicability to real-world scenarios. Previous research on gossip learning presents strong and often unrealistic assumptions on the distribution of the data, the communication speeds of the devices and the connectivity among them. Our results show that lifting these requirements can, in certain scenarios, lead to slow convergence of the protocol or even unfair bias in the produced models. This paper identifies the conditions in which gossip learning can and cannot be applied, and introduces extensions that mitigate some of its limitations., QC 20201027

Recent years have seen a sharp increase in the ubiquity and power of connected devices, such as smartphones, smart appliances and smart sensors. These de- vices produce large amounts of data that can be extremely precious for training larger, more advanced machine learning models. Unfortunately, it is some- times not possible to collect and process these datasets on a central system, due either to their size or to the growing privacy requirements of digital data handling.To overcome this limit, researchers developed protocols to train global models in a decentralised fashion, exploiting the computational power of these edge devices. These protocols do not require any of the data on the device to be shared, relying instead on communicating partially-trained models.Unfortunately, real-world systems are notoriously hard to control, and may present a wide range of challenges that are easily overlooked in academic stud- ies and simulations. This research analyses the gossip learning protocol, one of the main results in the area of decentralised machine learning, to assess its applicability to real-world scenarios.Specifically, this work identifies the main assumptions built into the pro- tocol, and performs carefully-crafted simulations in order to test its behaviour when these assumptions are lifted. The results show that the protocol can al- ready be applied to certain environments, but that it fails when exposed to certain conditions that appear in some real-world scenarios. In particular, the models trained by the protocol may be biased towards the data stored in nodes with faster communication speeds or a higher number of neighbours. Further- more, certain communication topologies can have a strong negative impact on the convergence speed of the models.While this study also suggests effective mitigations for some of these is- sues, it appears that the gossip learning protocol requires further research ef- forts, in order to ensure a wider industrial applicability., Under de senaste åren har vi sett en kraftig ökning av närvaron och kraften hos anslutna enheter, såsom smartphones, smarta hushållsmaskiner, och smarta sensorer. Dessa enheter producerar stora mängder data som kan vara extremt värdefulla för att träna stora och avancerade maskininlärningsmodeller. Dessvärre är det ibland inte möjligt att samla in och bearbeta dessa dataset på ett centralt system, detta på grund av deras storlek eller de växande sekretesskraven för digital datahantering.För att lösa problemet har forskare utvecklar protokoller för att träna globala modeller på ett decentraliserat sätt och utnyttja beräkningsförmågan hos dessa enheter. Dessa protokoll kräver inte datan på enheter delas utan förlitar sig istället på att kommunicera delvis tränade modeller.Dessvärre så är verkliga system svåra att kontrollera och kan presentera ett brett spektrum av utmaningar som lätt överskådas i akademiska studier och simuleringar. Denna forskning analyserar gossip inlärning protokollet vilket är av de viktigaste resultaten inom decentraliserad maskininlärning, för att bedöma dess tillämplighet på verkliga scenarier.Detta arbete identifierar de huvudsakliga antagandena om protokollet och utför noggrant utformade simuleringar för att testa protokollets beteende när dessa antaganden tas bort. Resultaten visar att protokollet redan kan tillämpas i vissa miljöer, men att det misslyckas när det utsätts för vissa förhållanden som i verklighetsbaserade scenarier. Mer specifikt så kan modellerna som utbildas av protokollet vara partiska och fördelaktiga mot data lagrade i noder med snabbare kommunikationshastigheter eller ett högre antal grannar. Vidare kan vissa kommunikationstopologier få en stark negativ inverkan på modellernas konvergenshastighet.Även om denna studie kom fram till en förmildrande effekt för vissa av dessa problem så verkar det som om gossip inlärning protokollet kräver ytterligare forskningsinsatser för att säkerställa en bredare industriell tillämpligh

Large scale contextual representation models have significantly advanced NLP in recent years, understanding the semantics of text to a degree never seen before. However, they need to process large amounts of data to achieve high-quality results. Joining and accessing all these data from multiple sources can be extremely challenging due to privacy and regulatory reasons. Federated Learning can solve these limitations by training models in a distributed fashion, taking advantage of the hardware of the devices that generate the data. We show the viability of training NLP models, specifically Word2Vec, with the Federated Learning protocol. In particular, we focus on a scenario in which a small number of organizations each hold a relatively large corpus. The results show that neither the quality of the results nor the convergence time in Federated Word2Vec deteriorates as compared to centralised Word2Vec., QC 20210512

Large scale contextual representation models have significantly advanced NLP in recent years, understanding the semantics of text to a degree never seen before. However, they need to process large amounts of data to achieve high-quality results. Joining and accessing all these data from multiple sources can be extremely challenging due to privacy and regulatory reasons. Federated Learning can solve these limitations by training models in a distributed fashion, taking advantage of the hardware of the devices that generate the data. We show the viability of training NLP models, specifically Word2Vec, with the Federated Learning protocol. In particular, we focus on a scenario in which a small number of organizations each hold a relatively large corpus. The results show that neither the quality of the results nor the convergence time in Federated Word2Vec deteriorates as compared to centralised Word2Vec., QC 20210512

This is an extended version of our work in [16]. In this paper,we introduce two novel algorithms to collaboratively train Naive Bayesmodels across multiple private data sources: Federated Naive Bayes andGossip Naive Bayes. Instead of directly providing access to their data,the data owners compute local updates that are then aggregated to builda global model. In order to also prevent indirect privacy leaks from theupdates or from the final model, our algorithms protect the exchangedinformation with differential privacy. We experimentally evaluate ourproposed approaches, examining different scenarios and focusing on potentialreal-world issues, such as different data owner offering differentamounts of data or requesting different levels of privacy. Our results showthat both Federated and Gossip Naive Bayes achieve similar accuracy toa “vanilla” Naive Bayes while maintaining reasonable privacy guarantees,while being extremely robust to heterogeneous data owners., QC 20230405

Early-stage detection of botnets during their spreadingphase, before any attack, is fundamental to IoT security.Recently introduced lightweight memory networks represent thestate of the art in this domain. However, they require a centralsystem to capture and analyze all traffic in the network, whichmay not always be feasible in real-world scenarios.In this paper, we introduce a decentralized and collaborativealternative, in which the IoT devices themselves are responsiblefor this task without any central observer or coordinator. Ourresults show that the performance of this novel approach iscompetitive with similar centralized solutions, despite the lackof a global view of the network at any participating device.We also provide an extensive analysis of the security limitationsof our fully-decentralized detection system. We identify thepotential exploits that an attacker may attempt to perform, assesstheir impact on the IoT network as well as propose and evaluateeffective countermeasures., QC 20230405

In existing literature, GNN training has been performed mostly in centralized, and sometimes federated, settings. In this work, we consider a fully-decentralized data-private scenario, where each node has limited knowledge of the surrounding graph. We propose the first architecture that enables GNN training in this fully-decentralized setting, by carefully combining several techniques, including decoupled learning, self-supervision and Gossip Learning. We implement two simulation tools to experimentally evaluate our solution. The results show that the proposed technique can be effectively used in scenarios where centralized or federated approaches are unfeasible or undesirable., QC 20230322

This is an extended version of our work in [16]. In this paper,we introduce two novel algorithms to collaboratively train Naive Bayesmodels across multiple private data sources: Federated Naive Bayes andGossip Naive Bayes. Instead of directly providing access to their data,the data owners compute local updates that are then aggregated to builda global model. In order to also prevent indirect privacy leaks from theupdates or from the final model, our algorithms protect the exchangedinformation with differential privacy. We experimentally evaluate ourproposed approaches, examining different scenarios and focusing on potentialreal-world issues, such as different data owner offering differentamounts of data or requesting different levels of privacy. Our results showthat both Federated and Gossip Naive Bayes achieve similar accuracy toa “vanilla” Naive Bayes while maintaining reasonable privacy guarantees,while being extremely robust to heterogeneous data owners., QC 20230405

Early-stage detection of botnets during their spreadingphase, before any attack, is fundamental to IoT security.Recently introduced lightweight memory networks represent thestate of the art in this domain. However, they require a centralsystem to capture and analyze all traffic in the network, whichmay not always be feasible in real-world scenarios.In this paper, we introduce a decentralized and collaborativealternative, in which the IoT devices themselves are responsiblefor this task without any central observer or coordinator. Ourresults show that the performance of this novel approach iscompetitive with similar centralized solutions, despite the lackof a global view of the network at any participating device.We also provide an extensive analysis of the security limitationsof our fully-decentralized detection system. We identify thepotential exploits that an attacker may attempt to perform, assesstheir impact on the IoT network as well as propose and evaluateeffective countermeasures., QC 20230405

In existing literature, GNN training has been performed mostly in centralized, and sometimes federated, settings. In this work, we consider a fully-decentralized data-private scenario, where each node has limited knowledge of the surrounding graph. We propose the first architecture that enables GNN training in this fully-decentralized setting, by carefully combining several techniques, including decoupled learning, self-supervision and Gossip Learning. We implement two simulation tools to experimentally evaluate our solution. The results show that the proposed technique can be effectively used in scenarios where centralized or federated approaches are unfeasible or undesirable., QC 20230322

In existing literature, GNN training has been performed mostly in centralized, and sometimes federated, settings. In this work, we consider a fully-decentralized data-private scenario, where each node has limited knowledge of the surrounding graph. We propose the first architecture that enables GNN training in this fully-decentralized setting, by carefully combining several techniques, including decoupled learning, self-supervision and Gossip Learning. We implement two simulation tools to experimentally evaluate our solution. The results show that the proposed technique can be effectively used in scenarios where centralized or federated approaches are unfeasible or undesirable., QC 20230322

Early-stage detection of botnets during their spreadingphase, before any attack, is fundamental to IoT security.Recently introduced lightweight memory networks represent thestate of the art in this domain. However, they require a centralsystem to capture and analyze all traffic in the network, whichmay not always be feasible in real-world scenarios.In this paper, we introduce a decentralized and collaborativealternative, in which the IoT devices themselves are responsiblefor this task without any central observer or coordinator. Ourresults show that the performance of this novel approach iscompetitive with similar centralized solutions, despite the lackof a global view of the network at any participating device.We also provide an extensive analysis of the security limitationsof our fully-decentralized detection system. We identify thepotential exploits that an attacker may attempt to perform, assesstheir impact on the IoT network as well as propose and evaluateeffective countermeasures., QC 20230405

This is an extended version of our work in [16]. In this paper,we introduce two novel algorithms to collaboratively train Naive Bayesmodels across multiple private data sources: Federated Naive Bayes andGossip Naive Bayes. Instead of directly providing access to their data,the data owners compute local updates that are then aggregated to builda global model. In order to also prevent indirect privacy leaks from theupdates or from the final model, our algorithms protect the exchangedinformation with differential privacy. We experimentally evaluate ourproposed approaches, examining different scenarios and focusing on potentialreal-world issues, such as different data owner offering differentamounts of data or requesting different levels of privacy. Our results showthat both Federated and Gossip Naive Bayes achieve similar accuracy toa “vanilla” Naive Bayes while maintaining reasonable privacy guarantees,while being extremely robust to heterogeneous data owners., QC 20230405

Early-stage detection of botnets during their spreadingphase, before any attack, is fundamental to IoT security.Recently introduced lightweight memory networks represent thestate of the art in this domain. However, they require a centralsystem to capture and analyze all traffic in the network, whichmay not always be feasible in real-world scenarios.In this paper, we introduce a decentralized and collaborativealternative, in which the IoT devices themselves are responsiblefor this task without any central observer or coordinator. Ourresults show that the performance of this novel approach iscompetitive with similar centralized solutions, despite the lackof a global view of the network at any participating device.We also provide an extensive analysis of the security limitationsof our fully-decentralized detection system. We identify thepotential exploits that an attacker may attempt to perform, assesstheir impact on the IoT network as well as propose and evaluateeffective countermeasures., QC 20230405

This is an extended version of our work in [16]. In this paper,we introduce two novel algorithms to collaboratively train Naive Bayesmodels across multiple private data sources: Federated Naive Bayes andGossip Naive Bayes. Instead of directly providing access to their data,the data owners compute local updates that are then aggregated to builda global model. In order to also prevent indirect privacy leaks from theupdates or from the final model, our algorithms protect the exchangedinformation with differential privacy. We experimentally evaluate ourproposed approaches, examining different scenarios and focusing on potentialreal-world issues, such as different data owner offering differentamounts of data or requesting different levels of privacy. Our results showthat both Federated and Gossip Naive Bayes achieve similar accuracy toa “vanilla” Naive Bayes while maintaining reasonable privacy guarantees,while being extremely robust to heterogeneous data owners., QC 20230405

In existing literature, GNN training has been performed mostly in centralized, and sometimes federated, settings. In this work, we consider a fully-decentralized data-private scenario, where each node has limited knowledge of the surrounding graph. We propose the first architecture that enables GNN training in this fully-decentralized setting, by carefully combining several techniques, including decoupled learning, self-supervision and Gossip Learning. We implement two simulation tools to experimentally evaluate our solution. The results show that the proposed technique can be effectively used in scenarios where centralized or federated approaches are unfeasible or undesirable., QC 20230322

Large scale contextual representation models have significantly advanced NLP in recent years, understanding the semantics of text to a degree never seen before. However, they need to process large amounts of data to achieve high-quality results. Joining and accessing all these data from multiple sources can be extremely challenging due to privacy and regulatory reasons. Federated Learning can solve these limitations by training models in a distributed fashion, taking advantage of the hardware of the devices that generate the data. We show the viability of training NLP models, specifically Word2Vec, with the Federated Learning protocol. In particular, we focus on a scenario in which a small number of organizations each hold a relatively large corpus. The results show that neither the quality of the results nor the convergence time in Federated Word2Vec deteriorates as compared to centralised Word2Vec., QC 20210512

Large scale contextual representation models have significantly advanced NLP in recent years, understanding the semantics of text to a degree never seen before. However, they need to process large amounts of data to achieve high-quality results. Joining and accessing all these data from multiple sources can be extremely challenging due to privacy and regulatory reasons. Federated Learning can solve these limitations by training models in a distributed fashion, taking advantage of the hardware of the devices that generate the data. We show the viability of training NLP models, specifically Word2Vec, with the Federated Learning protocol. In particular, we focus on a scenario in which a small number of organizations each hold a relatively large corpus. The results show that neither the quality of the results nor the convergence time in Federated Word2Vec deteriorates as compared to centralised Word2Vec., QC 20210512

Large scale contextual representation models have significantly advanced NLP in recent years, understanding the semantics of text to a degree never seen before. However, they need to process large amounts of data to achieve high-quality results. Joining and accessing all these data from multiple sources can be extremely challenging due to privacy and regulatory reasons. Federated Learning can solve these limitations by training models in a distributed fashion, taking advantage of the hardware of the devices that generate the data. We show the viability of training NLP models, specifically Word2Vec, with the Federated Learning protocol. In particular, we focus on a scenario in which a small number of organizations each hold a relatively large corpus. The results show that neither the quality of the results nor the convergence time in Federated Word2Vec deteriorates as compared to centralised Word2Vec., QC 20210512