Search

Recently embedding-based retrieval or dense retrieval have shown state of the art results, compared with traditional sparse or bag-of-words based approaches. This paper introduces a model-agnostic doc-level embedding framework through large language model (LLM) augmentation. In addition, it also improves some important components in the retrieval model training process, such as negative sampling, loss function, etc. By implementing this LLM-augmented retrieval framework, we have been able to significantly improve the effectiveness of widely-used retriever models such as Bi-encoders (Contriever, DRAGON) and late-interaction models (ColBERTv2), thereby achieving state-of-the-art results on LoTTE datasets and BEIR datasets.

Due to the drawbacks of Federated Learning (FL) such as vulnerability of a single central server, centralized federated learning is shifting to decentralized federated learning, a paradigm which takes the advantages of blockchain. A key enabler for adoption of blockchain-based federated learning is how to select suitable participants to train models collaboratively. Selecting participants by storing and querying the metadata of data owners on blockchain could ensure the reliability of selected data owners, which is helpful to obtain high-quality models in FL. However, querying multi-dimensional metadata on blockchain needs to traverse every transaction in each block, making the query time-consuming. An efficient query method for multi-dimensional metadata in the blockchain for selecting participants in FL is absent and challenging. In this paper, we propose a novel data structure to improve the query efficiency within each block named MerkleRB-Tree. In detail, we leverage Minimal Bounding Rectangle(MBR) and bloom-filters for the query process of multi-dimensional continuous-valued attributes and discrete-valued attributes respectively. Furthermore, we migrate the idea of the skip list along with an MBR and a bloom filter at the head of each block to enhance the query efficiency for inter-blocks. The performance analysis and extensive evaluation results on the benchmark dataset demonstrate the superiority of our method in blockchain-based FL.

Auction-based Federated Learning (AFL) enables open collaboration among self-interested data consumers and data owners. Existing AFL approaches are commonly under the assumption of sellers' market in that the service clients as sellers are treated as scarce resources so that the aggregation servers as buyers need to compete the bids. Yet, as the technology progresses, an increasing number of qualified clients are now capable of performing federated learning tasks, leading to shift from sellers' market to a buyers' market. In this paper, we shift the angle by adapting the procurement auction framework, aiming to explain the pricing behavior under buyers' market. Our modeling starts with basic setting under complete information, then move further to the scenario where sellers' information are not fully observable. In order to select clients with high reliability and data quality, and to prevent from external attacks, we utilize a blockchain-based reputation mechanism. The experimental results validate the effectiveness of our approach.

In recent years, there has been a significant increase in attention towards designing incentive mechanisms for federated learning (FL). Tremendous existing studies attempt to design the solutions using various approaches (e.g., game theory, reinforcement learning) under different settings. Yet the design of incentive mechanism could be significantly biased in that clients' performance in many applications is stochastic and hard to estimate. Properly handling this stochasticity motivates this research, as it is not well addressed in pioneering literature. In this paper, we focus on cross-device FL and propose a multi-level FL architecture under the real scenarios. Considering the two properties of clients' situations: uncertainty, correlation, we propose FL Incentive Mechanism based on Portfolio theory (FL-IMP). As far as we are aware, this is the pioneering application of portfolio theory to incentive mechanism design aimed at resolving FL resource allocation problem. In order to more accurately reflect practical FL scenarios, we introduce the Federated Learning Agent-Based Model (FL-ABM) as a means of simulating autonomous clients. FL-ABM enables us to gain a deeper understanding of the factors that influence the system's outcomes. Experimental evaluations of our approach have extensively validated its effectiveness and superior performance in comparison to the benchmark methods.

COVID-19 significantly disrupted how educational contents are delivered in academic institutions, rapidly accelerating the adoption of online and blended learning. This thesis explores the creation and refinement of optimized student learning communities as a mean to support online and blended learning in the pandemic and post-pandemic setting. Students enrolled in university courses can be modeled as an enrollment network akin to a circuit netlist. Learning communities are created by clustering students into groups, optimized for maximum internal connection to support student learning, and minimum external connection to reduce disease transmission. Three VLSI-based clustering algorithms: Hyperedge Coarsening, Modified Hyperedge Coarsening, and Best Choice, are modified to cluster student enrollment networks. The learning communities created by the clustering algorithms are further refined by the Simulated Annealing algorithm using the same optimization criteria. The Learning Community Creation and Refinement Framework combines all three stages of network modeling, learning community creation, and learning community refinement. The proposed framework is tested on both the 3rd year Electrical Engineering Fall 2020 enrollment dataset and a very large Fall 2020 and Winter 2021 enrollment dataset. Best Choice performed the best among the clustering algorithms, capable of creating learning communities for the optimization criteria for a given maximum cluster size. Simulated Annealing can refine the clustering results by significantly increase cluster quality. The framework is capable of creating and refining learning communities for both the small and the large enrollment networks, but it is better suited for creating tailored learning communities at a program level. Future work, including creating student learning communities based on other optimization criteria, should be explored., Comment: Unpublished MSc Thesis

The combination of grating-based frequency-selective optical feedback mechanisms, such as distributed feedback (DFB) or distributed Bragg reflector (DBR) structures, with quantum dot (QD) gain materials is a main approach towards ultra-high-performance semiconductor lasers for many key novel applications, either as stand-alone sources or as on-chip sources in photonic integrated circuits. However, the fabrication of conventional buried Bragg grating structures on GaAs, GaAs/Si, GaSb and other material platforms have been met with major material regrowth difficulties. We report a novel and universal approach of introducing laterally coupled dielectric Bragg gratings to semiconductor lasers that allows highly controllable, reliable and strong coupling between the grating and the optical mode. We implement such a grating structure in a low-loss amorphous silicon material alongside GaAs lasers with InAs/GaAs QD gain layers. The resulting DFB laser arrays emit at pre-designed 0.8 THz LWDM frequency intervals in the 1300 nm band with record performance parameters, including side mode suppression ratios as high as 52.7 dB, continuous-wave output power of 27.7 mW (room-temperature) and 10 mW (at 70{\deg}C), and ultra-low relative intensity noise (RIN) of < -165 dB/Hz (2.5-25 GHz). The devices are also capable of operating isolator-free under very high external reflection levels of up to -12.3 dB whilst maintaining the high spectral and ultra-low RIN qualities. These results validate the novel laterally coupled dielectric grating as a technologically superior and potentially cost-effective approach for fabricating DFB and DBR lasers free of their semiconductor material constraints, thus universally applicable across different material platforms and wavelength bands., Comment: 18 Pages,5 Figures

We introduce a simple and efficient lossless image compression algorithm. We store a low resolution version of an image as raw pixels, followed by several iterations of lossless super-resolution. For lossless super-resolution, we predict the probability of a high-resolution image, conditioned on the low-resolution input, and use entropy coding to compress this super-resolution operator. Super-Resolution based Compression (SReC) is able to achieve state-of-the-art compression rates with practical runtimes on large datasets. Code is available online at https://github.com/caoscott/SReC., Comment: Tech report