Search

As a representative large-scale data management technology, Apache Hadoop is an open-source framework for processing a variety of data such as SNS, medical, weather, and IoT data. Hadoop largely consists of HDFS, MapReduce, and YARN. Among them, we focus on improving the HDFS metadata management scheme responsible for storing and managing big data. We note that the current HDFS incurs many problems in system utilization due to its file-based metadata management. To solve these problems, we propose a novel metadata management scheme based on RDBMS for improving the functional aspects of HDFS. Through analysis of the latest HDFS, we first present five problems caused by its metadata management and derive three requirements of robustness, availability, and scalability for resolving these problems. We then design an overall architecture of the advanced HDFS, A-HDFS, which satisfies these requirements. In particular, we define functional modules according to HDFS operations and also present the detailed design strategy for adding or modifying the individual components in the corresponding modules. Finally, through implementation of the proposed A-HDFS, we validate its correctness by experimental evaluation and also show that A-HDFS satisfies all the requirements. The proposed A-HDFS significantly enhances the HDFS metadata management scheme and, as a result, ensures that the entire system improves its stability, availability, and scalability. Thus, we can exploit the improved distributed file system based on A-HDFS for various fields and, in addition, we can expect more applications to be actively developed. [ABSTRACT FROM AUTHOR]

Abstract: In this paper we address the scaling-invariant problem in boundary image matching. Supporting the invariant property against the horizontal or vertical image scaling is very important in boundary image matching to get more intuitive and more accurate matching results. We note that supporting the scaling-invariance is a challenging problem since the number of possible scaling factors is infinite. In this paper we solve this scaling-invariant problem in the time-series domain instead of the image domain. We first define the scaling distance between boundary images and present an interpolation-based method to compute that distance in the time-series domain. We then propose the notion of scaling-invariant distance between boundary images, which is the minimum distance among all possible scaling distances of two images. We use this scaling-invariant distance as the similarity measure in scaling-invariant boundary image matching. Computing the scaling-invariant distance, however, is very difficult or almost impossible, and we instead present how to compute its upper and lower bounds on the given scaling range. Using these lower and upper bounds we also propose divide-and-conquer algorithms to determine the scaling-invariant similarity between boundary images. Finally, we propose sequential and index-based matching methods, respectively, that perform the scaling-invariant boundary image matching correctly. Experimental results show that our scaling-invariant matching gets more intuitive results than the previous (scaling-variant) matching. In addition, compared with the simple sequential scan, our index-based matching method improves performance by one or two orders of magnitude. [ABSTRACT FROM AUTHOR]

For the best results in quantitative polymerase chain reaction (qPCR) experiments, it is essential to design high-quality primers considering a multitude of constraints and the purpose of experiments. The constraints include many filtering constraints, homology test on a huge number of off-target sequences, the same constraints for batch design of primers, exon spanning, and avoiding single nucleotide polymorphism (SNP) sites. The target sequences are either in database or given as FASTA sequences, and the experiment is for amplifying either each target sequence with each corresponding primer pairs designed under the same constraints or all target sequences with a single pair of primers. Many websites have been proposed, but none of them including our previous MRPrimerW fulfilled all the above features. Here, we describe the MRPrimerW2, the update version of MRPrimerW, which fulfils all the features by maintaining the advantages of MRPrimerW in terms of the kinds and sizes of databases for valid primers and the number of search modes. To achieve it, we exploited GPU computation and a disk-based key-value store using PCIe SSD. The complete set of 3 509 244 680 valid primers of MRPrimerW2 covers 99% of nine important organisms in an exhaustive manner. Free access: http://MRPrimerW2.com., (© The Author(s) 2019. Published by Oxford University Press on behalf of Nucleic Acids Research.)