Search

Cameras that capture color and depth information have become an essential imaging modality for applications in robotics, autonomous driving, virtual, and augmented reality. Existing RGB-D cameras rely on multiple sensors or active illumination with specialized sensors. In this work, we propose a method for monocular single-shot RGB-D imaging. Instead of learning depth from single-image depth cues, we revisit double-refraction imaging using a birefractive medium, measuring depth as the displacement of differently refracted images superimposed in a single capture. However, existing double-refraction methods are orders of magnitudes too slow to be used in real-time applications, e.g., in robotics, and provide only inaccurate depth due to correspondence ambiguity in double reflection. We resolve this ambiguity optically by leveraging the orthogonality of the two linearly polarized rays in double refraction -- introducing uneven double refraction by adding a linear polarizer to the birefractive medium. Doing so makes it possible to develop a real-time method for reconstructing sparse depth and color simultaneously in real-time. We validate the proposed method, both synthetically and experimentally, and demonstrate 3D object detection and photographic applications.

We present a novel method that can enhance the spatial resolution of stereo images using a parallax prior. While traditional stereo imaging has focused on estimating depth from stereo images, our method utilizes stereo images to enhance spatial resolution instead of estimating disparity. The critical challenge for enhancing spatial resolution from stereo images: how to register corresponding pixels with subpixel accuracy. Since disparity in traditional stereo imaging is calculated per pixel, it is directly inappropriate for enhancing spatial resolution. We, therefore, learn a parallax prior from stereo image datasets by jointly training two-stage networks. The first network learns how to enhance the spatial resolution of stereo images in luminance, and the second network learns how to reconstruct a high-resolution color image from high-resolution luminance and chrominance of the input image. Our two-stage joint network enhances the spatial resolution of stereo images significantly more than single-image super-resolution methods. The proposed method is directly applicable to any stereo depth imaging methods, enabling us to enhance the spatial resolution of stereo images.

This paper proposes a novel scheme of integrating episodic memory into semantic memory based task planner. Task planners have taken an important role in AI research along with semantic memory to better perform tasks for robots. Episodic memory memorizes and retrieves temporal sequence of situated behaviors by which temporal relationship between behaviors can be defined. None of any research, however, has implemented it into their work for task planning. By introducing episodic memory into task planner, the temporal causal relationship between situated behaviors, which are stored in semantic memory, is taken into consideration. The integrated architecture proves its effectiveness by notably reducing the number of nodes traversed in finding solutions. Robots can reduce time complexity in solving given problems by retrieving previous memories. Deep Adaptive Resonance Theory (Deep-ART) neural model and cogency-based hierarchical behavior planner are used for the episodic memory and the task planner, respectively. Cogency-based hierarchical behavior planner proves its capability of solving given problems in experiment with humanoid robot Mybot, and Deep-ART is augmented to the planner and tested in simulations. Therefore, the contribution of this approach lies on developing a framework which takes advantage of implementing episodic memory and planner in one place.

This paper proposes a novel hierarchical behavior planner with a multi-layered confabulation based behavior selection structure for robots to perform tasks. The proposed planner integrates a STRIPS based behavior selection approach and cogent confabulation approach. The STRIPS based behavior selection approach is a goal tree search that induces goal-oriented sequences of behaviors, while the cogent confabulation approach is based on conditional probabilities between input symbols and target behaviors, aims to model human thinking mechanism. Our planner is applied with a set of behaviors defined in a multi-layered structure to show that it can plan a hierarchical sequences of behaviors to perform given tasks. The effectiveness and applicability of the proposed scheme is demonstrated through the experiments with the robot Mybot, developed in the Robot Intelligence Technology Lab. at KAIST.

This paper proposes a novel algorithm, named dual multiobjective quantum-inspired evolutionary (DMQEA) algorithm augmented fuzzy cognitive map (FCM). DMQEA was developed to help users select preferable solutions out of the non-dominated ones and has been proven to be an effective way compared to other multi-objective optimization methods, such as MQEA, MQEA-PS, etc. DMQEA, in this paper, has been coupled with decision supporting tool, fuzzy cognitive maps (FCMs) to support selecting best models which can reflect users' preferences. Even though the attempts with single optimization such as genetic algorithms (GAs) or particle swarm optimization (PSO) have been frequently carried out, there have been only few attempt to incorporate FCM with multicriteria decision making algorithm, especially to reflect user's preference. This study aims to integrate DMQEA with FCM to build models according to user's preference. In robotics field, the interaction with human operators is an important issue and DMQEA-FCM can aid robots in their decision making process in the context of the interaction.

We present a multiview image completion method that provides geometric consistency among different views by propagating space structures. Since a user specifies the region to be completed in one of multiview photographs casually taken in a scene, the proposed method enables us to complete the set of photographs with geometric consistency by creating or removing structures on the specified region. The proposed method incorporates photographs to estimate dense depth maps. We initially complete color as well as depth from a view, and then facilitate two stages of structure propagation and structure-guided completion. Structure propagation optimizes space topology in the scene across photographs, while structure-guide completion enhances, and completes local image structure of both depth and color in multiple photographs with structural coherence by searching nearest neighbor fields in relevant views. We demonstrate the effectiveness of the proposed method in completing multiview images.

This paper reports MEMS parallel plate rotation (PPR) with electrothermal actuator for real-time stereoscopic imaging. MEMS PPR allows stereoscopic imaging by generating camera shift effect with displacement in ray of the light. The device was designed opto-mechanically to operate in dynamic mode for stability, large rotation angle. The electrothermal actuation rotates over 25° in dynamic mode under low voltage, generating over 80μm baseline that indicates distance between two cameras and decides depth resolution. Finally stereoscopic images was captured by MEMS PPR and high-speed camera and reconstructed. This method can be widely used in miniaturized stereoscopic imaging system based on single camera.

Recently, interactive whiteboard (IWB) is often considered to be used in education. Most of IWB systems, however, cannot accurately describe complicated 3D structures because they only handle 2D projected images. In this paper, we present an IWB system which allows drawing in 3D space. This system is divided into two parts: a gesture recognition part and a 3D rendering part. A pen-like tool is used to determine the drawing state and an RGB-D camera to obtain its 3D coordinate data. By using the system, we could easily describe complicated 3D structures, which is difficult in the previous IWB systems.

This paper describes an efficient method for piezoelectric energy harvesting (PEH) system from impact forces using an array of piezoelectric modules. Specifically, the rectifier circuit and array configuration are considered. It is shown that rectifying the signal from each piezoelectric module separately generates more energy and charges a capacitor faster than using a single rectifier for all modules connected in series. In addition, the array can be designed such that the modules are impacted simultaneously or sequentially with a phase difference. We show that the latter allows faster energy harvesting. These results could be used in the design of efficient impact-based PEH system.

The relationship between deformation degree and curvature to the output voltage of piezoelectric system was analyzed. The calculated data and experimental data were compared. The experiment was held by using tip mass of 10 g, 20 g, and 30 g for the various thickness of piezoelectric material as 0.15 mm and 0.2 mm for various type design of supplemental plate. As results, at the vibration acceleration of 4 m/s2, the lowest average Ix value was resulted the highest voltage of 26.12 V. However, when the vibration acceleration of 40 m/s2, the lowest standard deviation of curvature ratio, ρ, and resulted the highest voltage of 58.98 V. This study proves that the deformation degree and curvature ration can be controlled.

We designed a piezoelectric energy harvesting system that can shift the resonant frequency to match the fixed external energy frequency. A permanent magnet (10 × 10 × 5 mm3) was attached to the free end of a cantilever beam, and a permanent magnet was attached to each of the four faces of a rotor at 90° angles. The effect of the size of the permanent magnets (40 × 20 × 10 mm3, 30 × 20 × 10 mm3, and 20 × 20 × 10 mm3) and the effect of the pole array distribution (NNNN, SSSS, NSNS, and NNSS) were experimentally investigated. The optimum conditions were determined by testing various distances between the magnets at different rpm's. The experiments showed that the maximum output power was generated for the minimum distance and largest magnet. The most effective approach to adjust the resonance frequency was to change the pole arrays of the magnets attached to the rotor. Furthermore, the optimum conditions were determined at each distance by changing the pole array and rpm. Software simulations support the experimental results.

Two models for energy harvesting system imparting rotational energy to piezoelectric materials are presented in order to compare the effects of applying identical amounts of energy to a cantilever beam by strain-changing and shock application. For a piezoelectric system given a high impulse with low displacement, higher power outputs were generated at lower resistive loads. Conversely, for a system with high displacement at a low impulse, power output was higher at high resistive loads. At matched impedance, the secondary shock system generated higher power output than the hitting system did at low resistive load. Optimized response of secondary shock system was obtained at a frequency of 60 Hz with a low resistive load of 1 k Ω. The generated output power was measured 124 mW, which corresponds to power density of 140 mW/cm3 for entire cantilever beam and power density of 342 mW/cm3 for only piezoelectric material volume. For a system without a secondary impulse at low resistive loads (1 k Ω), the optimizing frequency was between 20 and 30 Hz, with an output power of 22 mW, which corresponds to a 25 mW/cm3 power density for entire cantilever beam and power density of 60 mW/ cm3 for only piezoelectric material volume.

This study investigated the effects of strain and crack formation on the performance of piezoelectric materials as well as methods to repair damaged electrodes and reinforce the piezoelectric material. Specifically, this study investigated whether Ag paste (thickness: 0.1 mm) could restore electrical contact in damaged electrodes, and whether a coating of UV curable resin could provide reinforcement against crack formation. Experiments were conducted with a piezoelectric material on a steel cantilever substrate. The substrate was subjected to impact at various distances from the free end of the piezoelectric material to vary the applied strain. It was found that the output voltage increased with the strain (distance of impact from free end) until crack formation, which led to a large decrease in output. However, Ag paste could successfully restore electrical contact. Furthermore, before crack formation, coating with UV curable resin increased the maximum strain, and therefore, maximum electrical output, as well as cycle life.

Effective perimeter protection mechanisms for industrial sites and critical infrastructure must contend with a large variety of potential threats as well as with the fact that normal site activity can be both complex and diverse. This paper documents the development of a system level approach capable of functioning under such challenging conditions. A multi-view tracking system is used to provide real-time site wide trajectories of all observed individuals. A Radar-based system is also used for tracking if and when camera coverage of various regions is not available. Track information is then analyzed with respect to articulated motion analysis, complex event analysis and normalcy analysis. In addition, object recognition is used to classify left behind objects using high resolution PTZ imagery. A real-time integrated version of this comprehensive approach to perimeter protection was deployed using a single standard off-the-shelf desktop computer.

Summary form only given. The rapid evolution of tools and software systems to design experiments, automatically monitor, collect and warehouse large amounts of data, from applications such as life sciences and industrial processes has resulted in a new paradigm shift. This change of paradigm is so fast that some of the practices for optimization and management of these processes that were valid only 5–10 years ago may no longer be fully acceptable or sufficient for today's business optimization and management. This has a direct influence on the best practices for knowledge discovery and management of the discovered knowledge in real-world data mining applications. Establishing and managing a real-world data mining project in any domain, in particular in today's life science industry, is not a trivial task. A few approaches have been proposed in the literature. However, initiation and successful management of such efforts may depend on where a given case study fits in the overall classification of data mining approaches. Today's knowledge discovery from data can be classified in several ways: (i) data mining on engineered systems (e.g. complex equipment) or systems designed by nature (e.g. life sciences), (ii) explanatory or predictive data mining, (iii) data mining from static data (e.g. data warehouse) or dynamic data (e.g. data streams), (iv) user operated or automated data mining. There could still be other ways to classify data mining applications. This talk provides an overview of the above listed knowledge discovery applications. We provide examples where we demonstrate how small or large amounts of data, when understood from a real-world data mining point of view and the required data is properly integrated, can result in novel knowledge discovery case studies. We explain motivations and challenges of establishing real-world dat

WL-CSP (wafer level - chip scale package) has many advantages such as low cost, easy fabrication and ultimate miniature size, even though solder joint reliability (SJR) of conventional WL-CSP is critical weak point of the technology. Therefore, many advanced structure of WL-CSP has been developed to improve SJR such as using Cu post covered with encapsulation material. One of advanced WL-CSP is using encapsulated e double solder bump structure that the first bump is covered with epoxy molding compound (EMC) to protect. Fig.1 shows both conventional and encapsulated double bump type WL-CSP structure. However, the warpage of this advanced WL-CSP is much higher than conventional one due to CTE mismatch between silicon wafer and EMC material, which generates failure such as wafer crack or manufacturing difficulty such as process handling. In this paper, WL-CSP which has 120 mum EMC thickness on 6 inch wafer has been developed for advanced high I/O density applications. The warpage of WL-CSP after EMC curing process is considered. 3D thermo-mechanical FEM simulation is carried out the warpage distribution after curing process. The results also present the main factor in materials to affect the WL-CSP warpage. The lower Youngpsilas modulus and EMC CTE for encapsulation achieves less warpage. Furthermore, we studied some structures to reduce the warpage of WLCSP such as adding other material on backside of wafer and using patterned EMC. Both structures can make the WL-CSP structure balanced.

A Wide range and adjustable temperature coefficient (TC) Proportional To Absolute Temperature (PTAT) reference using negative TC CMOS FET Resistor is proposed which is built with CMOS Technology without passive resistor. It shows low sensitivity with respect to process, supply voltage. The temperature coefficient of proposed linear PTAT voltage reference is range of 1.48 to 3.48 mV/degC.

There are several important representative features in music. Among them beat is highly distinguishable one. Based on the pattern of beats' appearance in the music, we propose a music identification method that can be used for background music retrieval from video. In this paper, a music signal is primarily segmented into a number of small frames in the time domain. The features representing energy contents of each frame is determined by computing beats. Also, the time stamps' difference of two neighboring beats in frames is used as another feature. Extracted features from both the training and query music are classified using similarity matching algorithm. The retrieval accuracy of 95% of our method is a pretty good result. It can provide services like music or video indexing, summarization etc.

This paper presents a simple and robust face detection algorithm that can be utilized to video summary. Because the characteristic of the face can be change from unexpected condition like as shadows or lighting, we firstly process the illumination-compensation for maintaining face components. And then, we analyze color-based face region in YCbCr space to obtain the skin color. Also, we try the morphological image processing called closing algorithm to improve the detection performance and reducing the number of false detected face regions. The steps of the closing algorithm are dilation and erosion that find the candidates in face region and removes noise from the non-face region. For localization of the face region, we make a proper face location based on golden ratio. We evaluate our algorithm in the various genres. Experimental results demonstrate the effectiveness of our face detection algorithm that leads 96.7 % precision ratio on average. The proposed method is applicable to video summary because of these high performances with low complexity.

Scene boundary detection is an essential research field in content-based video summary, retrieval, and browsing. In this paper, we propose an efficient and robust scene extraction algorithm by using both video and audio information. The proposed algorithm consists of three stages: shot boundary detection, scene boundary detection, and scene boundary correction. Low level features based on visual information are used in shot and scene boundary detection method. To correct extracted scene boundary in the last stage, music detection is performed by using audio information on the assumption that if a shot contains background music, most of all shots in a scene which includes corresponding shot have same background music in general

We propose a new occluded object recognition method using extended local features and hashing. First we present some methods for extracting the extended local features such as corners, arcs, parallel-lines, and corner-arcs from the preprocessed images. Then we construct the knowledge-base using hashing, which can reduce the searching time significantly. In order to match the hypothesized objects, we find a geometric transform using clustering, which brings a model point to the corresponding image point. Our methods were tested on a hypercube-topology multiprocessor computer, the Intel iPSC/2. >

Feature extraction is the most important phase in object recognition because accuracy of the system relies on how well the features are extracted. In this paper a new parallel extended local feature extraction method is proposed which can be implemented on a distributed memory machine. In order to reduce the complexity in the extended local feature extraction, an efficient algorithm is developed which is capable of exploiting a high degree of parallelism. Our parallel algorithm is implemented and tested on an Intel iPSC/2 hypercube computer. Some resulting figures and execution times according to various number of nodes and object features are presented. >

We present a novel method for stabilizing the Sagnac optical fibre current sensors (SOFCS) constructed with all-standard single mode fibers (SMFs). It is characterized by simply adjusting the azimuth angle of the half-waveplate to compensate the output of the sensor drifts with temperature.