Search

Background: While extensive studies have elucidated the relationships between exposure to air pollution and chronic diseases, such as cardiovascular disorders and diabetes, the intricate effects on specific kidney diseases, notably primary glomerulonephritis (GN)—an immune-mediated kidney ailment—are less well understood. Considering the escalating incidence of GN and conspicuous lack of investigative focus on its association with air quality, investigation is dedicated to examining the long-term effects of air pollutants on renal function in individuals diagnosed with primary GN. Methods: This retrospective cohort analysis was conducted on 1394 primary GN patients who were diagnosed at Seoul National University Bundang Hospital and Seoul National University Hospital. Utilizing time-varying Cox regression and linear mixed models (LMM), we examined the effect of yearly average air pollution levels on renal function deterioration (RFD) and change in estimated glomerular filtration rate (eGFR). In this context, RFD is defined as sustained eGFR of less than 60 mL/min per 1.73 m2. Results: During a mean observation period of 5.1 years, 350 participants developed RFD. Significantly, elevated interquartile range (IQR) levels of air pollutants—including PM10 (particles ≤10 micrometers, HR 1.389, 95 % CI 1.2–1.606), PM2.5 (particles ≤2.5 micrometers, HR 1.353, 95 % CI 1.162–1.575), CO (carbon monoxide, HR 1.264, 95 % CI 1.102–1.451), and NO2 (nitrogen dioxide, HR 1.179, 95 % CI 1.021–1.361)—were significantly associated with an increased risk of RFD, after factoring in demographic and health variables. Moreover, exposure to PM10 and PM2.5 was associated with decreased eGFR. Conclusions: This study demonstrates a substantial link between air pollution exposure and renal function impairment in primary GN, accentuating the significance of environmental determinants in the pathology of immune-mediated kidney diseases.

We address an important and practical problem facing deep-learning-based image classification: class incremental learning with a large domain shift. Most previous efforts on class incremental learning focus on one aspect of the problem, i.e., learning to classify additional new classes (with a little shift). However, in the real world, when new classes are added, the domain changes simultaneously (with a large domain shift). To obtain a model that is robust to these situations, we need to consider incrementally learning not only new labels but also domain-shifted labels. We target a continual and simultaneous shift of class and domain distribution and propose a new incremental learning method named Momentum Contrastive learning enhancing Orthogonality of Negative pairs (MoCo-ON). We employ a momentum encoder framework augmented with rehearsal memory to mitigate the risk of forgetting while leveraging contrastive learning to extract versatile features capable of adapting to the progressively shifting domain. Specifically, when training with a knowledge distillation loss, we introduce a novel supervised contrastive loss designed to closely embed positive pairs of the same class, even in the presence of a substantial domain gap. Additionally, we leverage feature embedding from momentum encoders for exemplar selection, aiming to mitigate the risk of forgetting previously acquired information from earlier tasks. We conduct comprehensive experiments involving inter-domain shifted class incremental learning scenarios using widely adopted datasets commonly employed for studying domain generalization in image classification. Our proposed method consistently outperforms other methods by a significant margin.

In modern radars, the target detection probability is increased by lowering the detection threshold via signal processing to detect a point target with a small radar cross-section value. However, a lower threshold increases the number of false targets. In the conventional tracking method, which uses a general tracking filter, the measurement data between scans should be compared. Therefore, for a large amount of acquired measurement data, the computational complexity can be reduced by accumulating the acquired measurement data over time, recognizing the target movement as a pattern, and training a convolutional neural network (CNN) model. Here, we propose a method to create a desired target scenario by transfer learning and estimate the target position using the activation map of a binary detector CNN model. The model can detect a target using the actual acquired radar data, and the processing time remains constant, regardless of the number of false alarms.

Deep learning models in image synthesis have proven their applicability in various image translation areas. However, although the synthesized image may reflect the user’s intention, some of its properties may be different from those of real images. In this study, we introduce an undesirable property that we discovered in the multi-domain label-based image translation techniques: Once the image is translated to one domain, the translated image cannot be adequately translated again to another domain. We refer to this problem as the failure of recursive translation, and analyze this phenomenon from the viewpoint of attribute disentanglement and establish a hypothesis: Unlabeled or unknown attributes that are correlated with the direction of translation hinder the network from learning the correct direction of translation. Based on our hypothesis, we also devise a solution that endows the generator with the power of recursive translation, which is achieved by injecting additive perturbations during model training. Our method is simple and easy to implement on various translation models without requiring much hyperparameter adjustment. Beyond enabling recursive translation, it is worth noting that solving the recursive translation problem improves the disentanglement of single translations, which eventually strengthens its practicability.

To reduce memory usage, edge devices such as TVs use Super-resolution(SR) with dedicated hardware networks. Dedicated hardware has the disadvantage of being difficult to change and difficult to improve performance. We propose a dual-structured SR system that can extend this dedicated hardware, suggesting a way to improve the restoration performance for compression with minimal change in hardware. Edge devices such as general TVs deal with moving videos rather than still images. In this case, the video includes video compression, and a preprocessing step of restoring deterioration is useful. We propose as the main idea that these steps can be handled while removing only pixel shuffle layers from existing networks. Our proposed SR method consists of two stages: 1) restoring from the video compression without changing the size of LR images and 2) increasing the resolution. The first stage can be learned by reducing the difference between video-LR (Low-Resolution video images with codec degradation) and downscaled-HR (video images made without codec degradation by simply reducing the size of High-Resolution video). The second stage, resolution enhancement, performs the same task as the traditional SISR task, except that it focuses on restoring the output of the first stage rather than a downscaled-HR. Our new dataset for this processing, HD2UHD, consists of (video-LR, downscaled-HR, and HR) tuples. We also propose a new scheme of input distillation that utilizes video-LR and downscaled-HR at the same time.

The design factors of anchor boxes, such as shape, placement, and target assignment policy, greatly influence the performance and latency of the 3D object detectors. Unlike image-based 2D anchors, 3D anchors must be placed in a 3D space and determined differently for each class of different sizes. This imposes a significant burden on the design complexity. To tackle this issue, various studies have been conducted on how to set the anchor form. However, for practical reasons, anchor-based methods select the anchor design by compromising between performance and latency. Consequently, only objects that are similar in shape and size to an anchor can obtain high accuracy. In this paper, we propose a Mixture-Density-based 3D Object Detection (MD3D) in point clouds to predict the distribution of 3D bounding boxes using a Gaussian Mixture Model (GMM). With an anchor-free detection head, MD3D requires few hand-crafted design factors and eliminates the inefficiency of separating the regression channel for each class, and thus offering both latency and memory benefits. MD3D is designed to utilize various types of feature encoding; therfore, it can be applied flexibly by replacing only the detection head of the existing detectors. Experimental results on the KITTI and Waymo open datasets show that the proposed method outperforms its counterparts that are based on the conventional anchor-based detection head in its overall performance, latency, and memory. The code is publicly available at https://github.com/sky77764/MD3D

We propose the position-based scaled gradient (PSG) that scales the gradient depending on the position of a weight vector to make it more compression-friendly. First, we theoretically show that applying PSG to the standard gradient descent (GD), which is called PSGD, is equivalent to the GD in the warped weight space, a space made by warping the original weight space via an appropriately designed invertible function. Second, we empirically show that PSG acting as a regularizer to the weight vectors is favorable for model compression domains such as quantization, pruning, and knowledge distillation. PSG reduces the gap between the weight distributions of a full-precision model and its compressed counterpart. This enables the versatile deployment of a model either as an uncompressed mode or as a compressed mode depending on the availability of resources. The experimental results on CIFAR-10/100 and ImageNet datasets show the effectiveness of the proposed PSG in model compression including an iterative pruning method and the knowledge distillation.

With the advancement of computer hardware and communication technologies, deep learning technology has made significant progress, enabling the development of systems that can accurately estimate human emotions. Factors such as facial expressions, gender, age, and the environment influence human emotions, making it crucial to understand and capture these intricate factors. Our system aims to recommend personalized images by accurately estimating human emotions, age, and gender in real time. The primary objective of our system is to enhance user experiences by recommending images that align with their current emotional state and characteristics. To achieve this, our system collects environmental information, including weather conditions and user-specific environment data through APIs and smartphone sensors. Additionally, we employ deep learning algorithms for real-time classification of eight types of facial expressions, age, and gender. By combining this facial information with the environmental data, we categorize the user’s current situation into positive, neutral, and negative stages. Based on this categorization, our system recommends natural landscape images that are colorized using Generative Adversarial Networks (GANs). These recommendations are personalized to match the user’s current emotional state and preferences, providing a more engaging and tailored experience. Through rigorous testing and user evaluations, we assessed the effectiveness and user-friendliness of our system. Users expressed satisfaction with the system’s ability to generate appropriate images based on the surrounding environment, emotional state, and demographic factors such as age and gender. The visual output of our system significantly impacted users’ emotional responses, resulting in a positive mood change for most users. Moreover, the system’s scalability was positively received, with users acknowledging its potential benefits when installed outdoors and expressing a willingness to continue using it. Compared to other recommender systems, our integration of age, gender, and weather information provides personalized recommendations, contextual relevance, increased engagement, and a deeper understanding of user preferences, thereby enhancing the overall user experience. The system’s ability to comprehend and capture intricate factors that influence human emotions holds promise in various domains, including human–computer interaction, psychology, and social sciences.

We are motivated by the observation that for problems where inputs and outputs are in the same form such as in image enhancement, deep neural networks can be reinforced by retraining the network using a new target set to the output for the original target. As an example, we introduce a new learning strategy for super-resolution by recurrently training the same simple network. Unlike the existing self-trained SR, which involves a single stage of learning with multiple runs at test time, our method trains the same SR network multiple times with increasingly better targets requiring only a single inference at test time. At each stage of the proposed learning scheme, a new target for training is obtained by applying the most recently trained SR network to the original image and downscaling the resultant SR image to normalize the size. Even if downscaling is involved, we argue that the downscaled SR image acts as a better target compared to the old target. We could mathematically demonstrate that this process is similar to unsharp masking when it is linearly approximated and that this process makes the image sharper. However, unlike unsharp masking, the proposed recurrent learning tends to converge to a specific target. By retraining the existing network aiming at a more enhanced target, the proposed method can achieve a similar effect of applying SR multiple times without increasing implementation cost and inference time. To objectively verify the supremacy of our approach by experiments, we propose to use VIQET MOS, which does not require a reference image as a measure of image quality. As far as we know, our work of using an objective quality measure in image enhancement is the first one whose validity was verified by showing similar results to the actual user's subjective evaluation. The proposed recurrent learning scheme makes existing SR algorithms more useful by clearly improving the effect of SR. Codes are available at https://github.com/rtsr82/rtsr.git.

Semi-supervised video object segmentation (semi-VOS) is required for many visual applications. This task is tracking class-agnostic objects from a given segmentation mask. Various approaches have been developed and achieved high accuracy in this field, but these previous models are hard to be utilized in real-world applications due to slow inference time and tremendous complexity. To significantly speed up inference while reducing performance gaps from those previous models, we introduce a fast segmentation model based on a template matching method and auxiliary loss with a transition map. Our template matching method consists of short-term and long-term matching. The short-term matching enhances target object localization by focusing on neighboring frames, while long-term matching improves fine details and handles object shape-changing by considering long-range frames. However, since both matching processes generate each template based on the previously estimated masks, this incurs error propagation for tracking objects in the next frames. To mitigate this problem, we add auxiliary loss with a newly proposed transition map for encouraging correction power to create accurate masks of the target object. Our model obtains $81.1\%~J\&F$ score at the speed of 78.3 FPS on the DAVIS16 benchmark and achieves $1.4\times $ faster speed and 11.3% higher accuracy than SiamMask, one of the fast semi-VOS models.

We tackle the issue of data imbalance between different poses in the human pose estimation problem. We explore unusual poses that are rare which occupy a small portion in a pose dataset. In order to identify a rare pose without additional learning, a simple $K$ -means clustering algorithm is applied to a given dataset. Experimental results on MPII and COCO datasets show that outliers which are far from the nearest cluster center can be defined as rare poses and the accuracy decreases as the distance between the data point and the cluster center increases. In order to improve the performance on the rare poses, we proposed three methods for the problem of data scarcity, which are addition of rare pose duplicates, addition of synthetic rare pose data and weighted loss based on the distance from the cluster. In the proposed methods, the highest increasing score is 13.5 mAP at the rare pose data.

Most face datasets target adults who can make their own decisions. In the case of children, consent from parents or guardians is necessary to collect biometric information, thus making it very difficult. As a result, the amount of data on children is quite small and inevitably private. In this work, we built a database by collecting face data of 74 children aged 2–7 years in daycare facilities. In addition, we conducted an experiment to determine the best location to perform face recognition on children by installing cameras in various locations. This study presents the points and methods to be considered to build a children’s face dataset and also studies the optimal camera installation setups for the face recognition of children.

Deep learning technology has rapidly evolved in recent years. Bone age assessment (BAA) is a typical object detection and classification problem that would benefit from deep learning. Convolutional neural networks (CNNs) and their variants are hence increasingly used for automating BAA, and they have shown promising results. In this paper, we propose a complete end-to-end BAA system to automate the entire process of the Tanner-Whitehouse 3 method, starting from localization of the epiphysis-metaphysis growth regions within 13 different bones and ending with estimation of the corresponding BA. Specific modifications to the CNNs and other stages are proposed to improve results. In addition, an annotated database of 3300 X-ray images is built to train and evaluate the system. The experimental results show that the average top-1 and top-2 prediction accuracies for skeletal bone maturity levels for 13 regions of interest are 79.6% and 97.2%, respectively. The mean absolute error and root mean squared error in age prediction are 0.46 years and 0.62 years, respectively, and accuracy within one year of the ground truth of 97.6% is achieved. The proposed system is shown to outperform a commercially available Greulich-Pyle-based system, demonstrating the potential for practical clinical use.

With the upsurge in the use of Unmanned Aerial Vehicles (UAVs) in various fields, detecting and identifying them in real-time are becoming important topics. However, the identification of UAVs is difficult due to their characteristics such as low altitude, slow speed, and small radar cross-section (LSS). With the existing deterministic approach, the algorithm becomes complex and requires a large number of computations, making it unsuitable for real-time systems. Hence, effective alternatives enabling real-time identification of these new threats are needed. Deep learning-based classification models learn features from data by themselves and have shown outstanding performance in computer vision tasks. In this paper, we propose a deep learning-based classification model that learns the micro-Doppler signatures (MDS) of targets represented on radar spectrogram images. To enable this, first, we recorded five LSS targets (three types of UAVs and two different types of human activities) with a frequency modulated continuous wave (FMCW) radar in various scenarios. Then, we converted signals into spectrograms in the form of images by Short time Fourier transform (STFT). After the data refinement and augmentation, we made our own radar spectrogram dataset. Secondly, we analyzed characteristics of the radar spectrogram dataset with the ResNet-18 model and designed the ResNet-SP model with less computation, higher accuracy and stability based on the ResNet-18 model. The results show that the proposed ResNet-SP has a training time of 242 s and an accuracy of 83.39%, which is superior to the ResNet-18 that takes 640 s for training with an accuracy of 79.88%.

We propose a stacking method for ensemble learning to distinguish micro-Doppler signals generated by human walking from background noises using radar sensors. We collected micro-Doppler signals caused by four types of background noise (line of sight (LoS), fan, snow and rain) and additionally considered micro-Doppler signals caused by human walking combined with these four types of background noise. We firstly verified the effectiveness of a fully connected deep neural network (DNN) to classify 8 types of signals. The average accuracy was 88.79% for the test set. Then, we propose a stacking method to combine two base classifiers of different structures. The average accuracy of the stacking method on the test set was 91.43%. Lastly, we designed a modified stacking method to reuse feature information stored at the previous stage and the average test accuracy increased to 95.62%. This result shows that the proposed stacking methods can be an effective approach to improve classifier’s accuracy in recognizing human walking using micro-Doppler signals with background noise.