Search

Recent advancements in speech-language models have yielded significant improvements in speech tokenization and synthesis. However, effectively mapping the complex, multidimensional attributes of speech into discrete tokens remains challenging. This process demands acoustic, semantic, and contextual information for precise speech representations. Existing speech representations generally fall into two categories: acoustic tokens from audio codecs and semantic tokens from speech self-supervised learning models. Although recent efforts have unified acoustic and semantic tokens for improved performance, they overlook the crucial role of contextual representation in comprehensive speech modeling. Our empirical investigations reveal that the absence of contextual representations results in elevated Word Error Rate (WER) and Word Information Lost (WIL) scores in speech transcriptions. To address these limitations, we propose two novel distillation approaches: (1) a language model (LM)-guided distillation method that incorporates contextual information, and (2) a combined LM and self-supervised speech model (SM)-guided distillation technique that effectively distills multimodal representations (acoustic, semantic, and contextual) into a comprehensive speech tokenizer, termed DM-Codec. The DM-Codec architecture adopts a streamlined encoder-decoder framework with a Residual Vector Quantizer (RVQ) and incorporates the LM and SM during the training process. Experiments show DM-Codec significantly outperforms state-of-the-art speech tokenization models, reducing WER by up to 13.46%, WIL by 9.82%, and improving speech quality by 5.84% and intelligibility by 1.85% on the LibriSpeech benchmark dataset. The code, samples, and model checkpoints are available at https://github.com/mubtasimahasan/DM-Codec.

Healthcare technology has advanced with the development of an IoT-based real-time monitoring system for post-operative heart disease patients. This system integrates advanced sensor technologies with cloud-connected platforms to continuously track vital signs, enabling more effective post-operative care. The system employs the HW-827 Heart Rate Sensor and DS18B20 Temperature Sensor, integrated with an ESP32 for wireless connectivity, to collect real-time data. Data was collected from 20 participants in each of the following categories: healthy individuals, pre-operative heart disease patients, and post-operative heart disease patients. The results revealed distinct physiological variations across the groups. Healthy individuals maintained stable temperatures (mean 36.3{\deg}C), while post-operative patients exhibited slightly elevated temperatures (mean 36.6{\deg}C). Heart rate analysis indicated higher rates in pre-operative (mean 87.8 bpm) and post-operative patients (mean 82.0 bpm) compared to healthy individuals (mean 72.4 bpm). ECG analysis showed elevated readings in pre-operative patients (mean 0.16 mV) compared to healthy individuals (mean 0.10 mV) and post-operative patients (mean 0.12 mV). Statistical analyses, including ANOVA tests, confirmed significant differences among the groups in terms of temperature (F(2, 12) = 4.57, p < 0.05), heart rate (F(2, 12) = 5.91, p < 0.01), and ECG readings (F(2, 12) = 3.24, p < 0.05). This study demonstrates the potential of IoT technologies in enhancing post-operative care through continuous monitoring and data-driven insights. It emphasizes the role of real-time patient data in improving personalized healthcare management., Comment: 17 Pages, 7 figures, 7 Tables, Journal

Land Use Land Cover (LULC) analysis on satellite images using deep learning-based methods is significantly helpful in understanding the geography, socio-economic conditions, poverty levels, and urban sprawl in developing countries. Recent works involve segmentation with LULC classes such as farmland, built-up areas, forests, meadows, water bodies, etc. Training deep learning methods on satellite images requires large sets of images annotated with LULC classes. However, annotated data for developing countries are scarce due to a lack of funding, absence of dedicated residential/industrial/economic zones, a large population, and diverse building materials. BD-SAT provides a high-resolution dataset that includes pixel-by-pixel LULC annotations for Dhaka metropolitan city and surrounding rural/urban areas. Using a strict and standardized procedure, the ground truth is created using Bing satellite imagery with a ground spatial distance of 2.22 meters per pixel. A three-stage, well-defined annotation process has been followed with support from GIS experts to ensure the reliability of the annotations. We performed several experiments to establish benchmark results. The results show that the annotated BD-SAT is sufficient to train large deep learning models with adequate accuracy for five major LULC classes: forest, farmland, built-up areas, water bodies, and meadows., Comment: 26 pages, 15 figures and 12 tables

The nearby elliptical galaxy M87 contains one of the only two supermassive black holes whose emission surrounding the event horizon has been imaged by the Event Horizon Telescope (EHT). In 2018, more than two dozen multi-wavelength (MWL) facilities (from radio to gamma-ray energies) took part in the second M87 EHT campaign. The goal of this extensive MWL campaign was to better understand the physics of the accreting black hole M87*, the relationship between the inflow and inner jets, and the high-energy particle acceleration. Understanding the complex astrophysics is also a necessary first step towards performing further tests of general relativity. The MWL campaign took place in April 2018, overlapping with the EHT M87* observations. We present a new, contemporaneous spectral energy distribution (SED) ranging from radio to very high energy (VHE) gamma-rays, as well as details of the individual observations and light curves. We also conduct phenomenological modelling to investigate the basic source properties. We present the first VHE gamma-ray flare from M87 detected since 2010. The flux above 350 GeV has more than doubled within a period of about 36 hours. We find that the X-ray flux is enhanced by about a factor of two compared to 2017, while the radio and millimetre core fluxes are consistent between 2017 and 2018. We detect evidence for a monotonically increasing jet position angle that corresponds to variations in the bright spot of the EHT image. Our results show the value of continued MWL monitoring together with precision imaging for addressing the origins of high-energy particle acceleration. While we cannot currently pinpoint the precise location where such acceleration takes place, the new VHE gamma-ray flare already presents a challenge to simple one-zone leptonic emission model approaches, and emphasises the need for combined image and spectral modelling., Comment: 46 pages, 23 figures, accepted by Astronomy & Astrophysics on August. 29, 2024

In this paper, we use a very prominent technique, Hirota Bilinear Method (HBM) to survey the lump structures of the Kadomtsev-Petviashvili (KP) equation in the frame of a collisionless magnetized plasma system composed of dust grains, ions, and nonextensive electrons. Nonlinearity has worldwide applications, and soliton theory is a powerful appliance to illustrate its qualitative behaviors. So, lump solitons are very significant and also interesting. We have observed that lump structures differ due to the correlated parameters of the plasma system. It has also been found that the nonextensive parameter crucially changes the lump features., Comment: 13 pages, 25 figures

Out of the estimated few trillion galaxies, only around a million have been detected through radio frequencies, and only a tiny fraction, approximately a thousand, have been manually classified. We have addressed this disparity between labeled and unlabeled images of radio galaxies by employing a semi-supervised learning approach to classify them into the known Fanaroff-Riley Type I (FRI) and Type II (FRII) categories. A Group Equivariant Convolutional Neural Network (G-CNN) was used as an encoder of the state-of-the-art self-supervised methods SimCLR (A Simple Framework for Contrastive Learning of Visual Representations) and BYOL (Bootstrap Your Own Latent). The G-CNN preserves the equivariance for the Euclidean Group E(2), enabling it to effectively learn the representation of globally oriented feature maps. After representation learning, we trained a fully-connected classifier and fine-tuned the trained encoder with labeled data. Our findings demonstrate that our semi-supervised approach outperforms existing state-of-the-art methods across several metrics, including cluster quality, convergence rate, accuracy, precision, recall, and the F1-score. Moreover, statistical significance testing via a t-test revealed that our method surpasses the performance of a fully supervised G-CNN. This study emphasizes the importance of semi-supervised learning in radio galaxy classification, where labeled data are still scarce, but the prospects for discovery are immense., Comment: 9 pages, 6 figures, accepted in INNS Deep Learning Innovations and Applications (INNS DLIA 2023) workshop, IJCNN 2023, to be published in Procedia Computer Science

While describing Spatio-temporal events in natural language, video captioning models mostly rely on the encoder's latent visual representation. Recent progress on the encoder-decoder model attends encoder features mainly in linear interaction with the decoder. However, growing model complexity for visual data encourages more explicit feature interaction for fine-grained information, which is currently absent in the video captioning domain. Moreover, feature aggregations methods have been used to unveil richer visual representation, either by the concatenation or using a linear layer. Though feature sets for a video semantically overlap to some extent, these approaches result in objective mismatch and feature redundancy. In addition, diversity in captions is a fundamental component of expressing one event from several meaningful perspectives, currently missing in the temporal, i.e., video captioning domain. To this end, we propose Variational Stacked Local Attention Network (VSLAN), which exploits low-rank bilinear pooling for self-attentive feature interaction and stacking multiple video feature streams in a discount fashion. Each feature stack's learned attributes contribute to our proposed diversity encoding module, followed by the decoding query stage to facilitate end-to-end diverse and natural captions without any explicit supervision on attributes. We evaluate VSLAN on MSVD and MSR-VTT datasets in terms of syntax and diversity. The CIDEr score of VSLAN outperforms current off-the-shelf methods by $7.8\%$ on MSVD and $4.5\%$ on MSR-VTT, respectively. On the same datasets, VSLAN achieves competitive results in caption diversity metrics., Comment: To be published in Winter Conference on Applications of Computer Vision 2022