Search

Machine learning models often suffer from catastrophic forgetting of previously learned knowledge when learning new classes. Various methods have been proposed to mitigate this issue. However, rehearsal-based learning, which retains samples from previous classes, typically achieves good performance but tends to memorize specific instances, struggling with Out-of-Distribution (OOD) generalization. This often leads to high forgetting rates and poor generalization. Surprisingly, the OOD generalization capabilities of these methods have been largely unexplored. In this paper, we highlight this issue and propose a simple yet effective strategy inspired by contrastive learning and data-centric principles to address it. We introduce Adaptive Contrastive Replay (ACR), a method that employs dual optimization to simultaneously train both the encoder and the classifier. ACR adaptively populates the replay buffer with misclassified samples while ensuring a balanced representation of classes and tasks. By refining the decision boundary in this way, ACR achieves a balance between stability and plasticity. Our method significantly outperforms previous approaches in terms of OOD generalization, achieving an improvement of 13.41\% on Split CIFAR-100, 9.91\% on Split Mini-ImageNet, and 5.98\% on Split Tiny-ImageNet.

Machine learning models that are overfitted/overtrained are more vulnerable to knowledge leakage, which poses a risk to privacy. Suppose we download or receive a model from a third-party collaborator without knowing its training accuracy. How can we determine if it has been overfitted or overtrained on its training data? It's possible that the model was intentionally over-trained to make it vulnerable during testing. While an overfitted or overtrained model may perform well on testing data and even some generalization tests, we can't be sure it's not over-fitted. Conducting a comprehensive generalization test is also expensive. The goal of this paper is to address these issues and ensure the privacy and generalization of our method using only testing data. To achieve this, we analyze the null space in the last layer of neural networks, which enables us to quantify overfitting without access to training data or knowledge of the accuracy of those data. We evaluated our approach on various architectures and datasets and observed a distinct pattern in the angle of null space when models are overfitted. Furthermore, we show that models with poor generalization exhibit specific characteristics in this space. Our work represents the first attempt to quantify overfitting without access to training data or knowing any knowledge about the training samples.

Over the past decade, polar codes have received significant traction and have been selected as the coding method for the control channel in fifth-generation (5G) wireless communication systems. However, conventional polar codes are reliant solely on binary (2x2) kernels, which restricts their block length to being only powers of 2. In response, multi-kernel (MK) polar codes have been proposed as a viable solution to attain greater code length flexibility. This paper proposes an unrolled architecture for encoding both systematic and non-systematic MK polar codes, capable of high-throughput encoding of codes constructed with binary, ternary (3x3), or binary-ternary mixed kernels. The proposed scheme exhibits an unprecedented level of flexibility by supporting 83 different codes and offering various architectures that provide trade-offs between throughput and resource consumption. The FPGA implementation results demonstrate that a partially-pipelined polar encoder of size N=4096 operating at a frequency of 270 MHz gives a throughput of 1080 Gbps. Additionally, a new compiler implemented in Python is given to automatically generate HDL modules for the desired encoders. By inserting the desired parameters, a designer can simply obtain all the necessary VHDL files for FPGA implementation.

Semantic communication represents a promising roadmap toward achieving end-to-end communication with reduced communication overhead and an enhanced user experience. The integration of semantic concepts with wireless communications presents novel challenges. This paper proposes a flexible simulation software that automatically transmits semantic segmentation map images over a communication channel. An additive white Gaussian noise (AWGN) channel using binary phase-shift keying (BPSK) modulation is considered as the channel setup. The well-known polar codes are chosen as the channel coding scheme. The popular COCO-Stuff dataset is used as an example to generate semantic map images corresponding to different signal-to-noise ratios (SNRs). To evaluate the proposed software, we have generated four small datasets, each containing a thousand semantic map samples, accompanied by comprehensive information corresponding to each image, including the polar code specifications, detailed image attributes, bit error rate (BER), and frame error rate (FER). The capacity to generate an unlimited number of semantic maps utilizing desired channel coding parameters and preferred SNR, in conjunction with the flexibility of using alternative datasets, renders our simulation software highly adaptable and transferable to a broad range of use cases.

Semantic communication is considered the future of mobile communication, which aims to transmit data beyond Shannon's theorem of communications by transmitting the semantic meaning of the data rather than the bit-by-bit reconstruction of the data at the receiver's end. The semantic communication paradigm aims to bridge the gap of limited bandwidth problems in modern high-volume multimedia application content transmission. Integrating AI technologies with the 6G communications networks paved the way to develop semantic communication-based end-to-end communication systems. In this study, we have implemented a semantic communication-based end-to-end image transmission system, and we discuss potential design considerations in developing semantic communication systems in conjunction with physical channel characteristics. A Pre-trained GAN network is used at the receiver as the transmission task to reconstruct the realistic image based on the Semantic segmented image at the receiver input. The semantic segmentation task at the transmitter (encoder) and the GAN network at the receiver (decoder) is trained on a common knowledge base, the COCO-Stuff dataset. The research shows that the resource gain in the form of bandwidth saving is immense when transmitting the semantic segmentation map through the physical channel instead of the ground truth image in contrast to conventional communication systems. Furthermore, the research studies the effect of physical channel distortions and quantization noise on semantic communication-based multimedia content transmission., Comment: Accepted for IEEE Access

Polar codes have received growing attention in the past decade and have been selected as the coding scheme for the control channel in the fifth generation (5G) wireless communication systems. However, the conventional polar codes have only been constructed by binary (2x2) kernel which poses block length limitation to powers of 2. To attain more flexible block lengths, multi-kernel polar codes are proposed. In this paper, a combinational architecture for multi-kernel polar codes with high throughput is proposed based on successive cancellation decoding algorithm. The proposed scheme can decode pure-binary, pure-ternary (3x3), and binary-ternary mixed polar codes. The decoder's architecture is rate-flexible meaning that a new code rate can be assigned to the decoder at every clock cycle. The proposed architecture is validated by FPGA implementation and the results reveal that a code of size N=81 gains the coded throughput of 1664.5 Mbps. A novel Python-based polar compiler is also proposed to automatically generate the HDL modules for target decoders. A designer can input the target block length and kernel ordering of a polar code, and get the required VHDL files automatically. Based on our simulations, the majority of the required HDL files can be generated in less than 0.4 seconds.

Polar codes have been selected as the channel coding scheme for control channel in the fifth generation (5G) communication system thanks to their capacity achieving characteristics. However, the traditional polar codes support only codes constructed by binary (2x2) kernel which limits the code lengths to powers of 2. Multi-kernel polar codes are proposed to achieve flexible block length. In this paper, the first combinational decoder for multi-kernel polar codes based on successive cancellation algorithm is proposed. The proposed decoder can decode pure-binary and binary-ternary (3x3) mixed polar codes. The architecture is rate-flexible with the capability of online rate assignment and supports any kernel sequences. The FPGA implementation results reveal that for a code of length N = 48, the coded throughput of 812.1 Mbps can be achieved.