Search

Deep learning improves the realistic expression of virtual simulations specifically to solve multi-criteria decision-making problems, which are generally rely on high-performance artificial intelligence. This study was inspired by the motivation theory and natural life observations. Recently, motivation-based control has been actively studied for realistic expression, but it presents various problems. For instance, it is hard to define the relation among multiple motivations and to select goals based on multiple motivations. Behaviors should generally be practiced to take into account motivations and goals. This paper proposes a deep Q-network (DQN)-based multi-criteria decision-making framework for virtual agents in real time to automatically select goals based on motivations in virtual simulation environments and to plan relevant behaviors to achieve those goals. All motivations are classified according to the five-level Maslow’s hierarchy of needs, and the virtual agents train a double DQN by big social data, select optimal goals depending on motivations, and perform behaviors relying on a predefined hierarchical task networks (HTNs). Compared to the state-of-the-art method, the proposed framework is efficient and reduced the average loss from 0.1239 to 0.0491 and increased accuracy from 63.24 to 80.15%. For behavioral performance using predefined HTNs, the number of methods has increased from 35 in the Q network to 1511 in the proposed framework, and the computation time of 10,000 behavior plans reduced from 0.118 to 0.1079 s.

Recently, Internet of Drones (IoD) are issued to utilize the diverse kinds of drones for leisure, education and so on. Researchers study to prevent the situations that drones are disabled by cyber-attackers by embedding malwares into the drones and Ground Control Stations (GCS). Therefore, it is required to protect the malwares considering the diverse kinds of features of the drones and GCSs. Signature-based detection approaches are traditionally utilized. However, given that those approaches only scan files partially, some of malwares are not detected. This paper proposes a novel method for finding the malwares in GCSs that utilizes a fastText model to create lower-dimension vectors than those the vectors by one-hot encoding and a bidirectional LSTM model to analyze the correlation with sequential opcodes. In addition, API function names are utilized to increase the classification accuracy of the sequential opcodes. In the experiments, the Microsoft malware classification challenge dataset was utilized and the malwares in the dataset were classified by family types. The proposed method showed the performance improvement of 1.87% comparing with the performance by a one-hot encoding-based approach. When the proposed method was compared with a similar decision tree-based malware detection approach, the performance of the proposed method was improved by 0.76%.

3D (3-Dimensional) object recognition is a hot research topic that benefits environment perception, disease diagnosis, and the mobile robot industry. Point clouds collected by range sensors are a popular data structure to represent a 3D object model. This paper proposed a 3D object recognition method named Dynamic Graph Convolutional Broad Network (DGCB-Net) to realize feature extraction and 3D object recognition from the point cloud. DGCB-Net adopts edge convolutional layers constructed by weight-shared multiple-layer perceptrons (MLPs) to extract local features from the point cloud graph structure automatically. Features obtained from all edge convolutional layers are concatenated together to form a feature aggregation. Unlike stacking many layers in-depth, our DGCB-Net employs a broad architecture to extend point cloud feature aggregation flatly. The broad architecture is structured utilizing a flat combining architecture with multiple feature layers and enhancement layers. Both feature layers and enhancement layers concatenate together to further enrich the features’ information of the point cloud. All features work on the object recognition results thus that our DGCB-Net show better recognition performance than other 3D object recognition algorithms on ModelNet10/40 and our scanning point cloud dataset.

A Deep-Q-Network (DQN) controls a virtual agent as the level of a player using only screenshots as inputs. Replay memory selects a limited number of experience replays according to an arbitrary batch size and updates them using the associated Q-function. Hence, relatively fewer experience replays of different states are utilized when the number of states is fixed and the state of the randomly selected transitions becomes identical or similar. The DQN may not be applicable in some environments where it is necessary to perform the learning process using more experience replays than is required by the limited batch size. In addition, because it is unknown whether each action can be executed, a problem of an increasing amount of repetitive learning occurs as more non-executable actions are selected. In this study, an enhanced DQN framework is proposed to resolve the batch size problem and reduce the learning time of a DQN in an environment with numerous non-executable actions. In the proposed framework, non-executable actions are filtered to reduce the number of selectable actions to identify the optimal action for the current state. The proposed method was validated in Gomoku, a strategy board game, in which the application of a traditional DQN would be difficult.

With unlabeled music data widely available, it is necessary to build an unsupervised latent music representation extractor to improve the performance of classification models. This paper proposes an unsupervised latent music representation learning method based on a deep 3D convolutional denoising autoencoder (3D-DCDAE) for music genre classification, which aims to learn common representations from a large amount of unlabeled data to improve the performance of music genre classification. Specifically, unlabeled MIDI files are applied to 3D-DCDAE to extract latent representations by denoising and reconstructing input data. Next, a decoder is utilized to assist the 3D-DCDAE in training. After 3D-DCDAE training, the decoder is replaced by a multilayer perceptron (MLP) classifier for music genre classification. Through the unsupervised latent representations learning method, unlabeled data can be applied to classification tasks so that the problem of limiting classification performance due to insufficient labeled data can be solved. In addition, the unsupervised 3D-DCDAE can consider the musicological structure to expand the understanding of the music field and improve performance in music genre classification. In the experiments, which utilized the Lakh MIDI dataset, a large amount of unlabeled data was utilized to train the 3D-DCDAE, obtaining a denoising and reconstruction accuracy of approximately 98%. A small amount of labeled data was utilized for training a classification model consisting of the trained 3D-DCDAE and the MLP classifier, which achieved a classification accuracy of approximately 88%. The experimental results show that the model achieves state-of-the-art performance and significantly outperforms other methods for music genre classification with only a small amount of labeled data.

Recently, with the development of computer technology, deep learning has expanded to the field of art, which requires creativity, which is a unique ability of humans, and an understanding of the human emotions expressed in art to process them as data. The field of art is integrating with various industrial fields, among which artificial intelligence (AI) is being used in stage art, to create visual images. As it is difficult for a computer to process emotions expressed in songs as data, existing stage background images for song performances are human designed. Recently, research has been conducted to enable AI to design stage background images on behalf of humans. However, there is no research on reflecting emotions contained in song lyrics to stage background images. This paper proposes a style transformation method to reflect emotions in stage background images. First, multiple verses and choruses are derived from song lyrics, one at a time, and emotion words included in each verse and chorus are extracted. Second, the probability distribution of the emotion words is calculated for each verse and chorus, and the image with the most similar probability distribution from an image dataset with emotion word tags in advance is selected for each verse and chorus. Finally, for each verse and chorus, the stage background images with the transferred style are outputted. Through an experiment, the similarity between the stage background and the image transferred to the style of the image with similar emotion words probability distribution was 38%, and the similarity between the stage background image and the image transferred to the style of the image with completely different emotion word probability distribution was 8%. The proposed method reduced the total variation loss of change from 1.0777 to 0.1597. The total variation loss is the sum of content loss and style loss based on weights. This shows that the style transferred image is close to edge information about the content of the input image, and the style is close to the target style image.

Signature-based malware detection approaches are inadequate for detecting the increasingly intelligent and large number of malware programs emerging today. Therefore, alternative approaches are required. The effects of malware can be estimated by analyzing the opcodes in its executable files. It can then be classified into families using a long short-term memory (LSTM) network. Vectorizing opcodes and application programming interface (API) function names using one-hot encoding results in high-dimensional vectors because each case is represented using one dimension. Therefore, this paper proposes a word2vec-based LSTM method to analyze opcodes and API function names using fewer dimensions. The results of opcode and API function name classification using the proposed method and one-hot encoding were compared using the Microsoft Malware Classification Challenge dataset. The proposed method showed approximately 0.5% higher performance than the one-hot encoding-based approach.

This paper proposes an enhanced Bayesian probability-based method to generate high-quality scenarios using a small number of collected sensory data. Diverse kinds of new scenarios can be generated ...

Affective social multimedia computing provides us the opportunity to improve our daily lives. Various things, such as devices in ubiquitous computing environments and autonomous vehicles in real environments considering human beings, can be controlled by analyzing and learning affective social big data. Deep learning is a core learning algorithm for autonomous control; however, it requires huge amounts of learning data, and the process of collecting various types of learning data is expensive. The collection limit of affective social videos for deep learning is resolved by analyzing affective social videos, such as YouTube and Closed Circuit Television (CCTV) videos collected in advance, and generating new affective social videos more as learning data without human beings autonomously controlling other cameras. The control signals of the cameras are generated by Convolutional Neural Network (CNN)-based end-to-end controls. However, images captured consecutively need to be analyzed to improve the quality of the generated control signals. This paper proposes a system that generates affective social videos for deep learning by Convolutional Recurrent Neural Network (CRNN)-based end-to-end controls. The extracted images in affective social videos are utilized for calculating the control signals based on the CRNN. Additional affective social videos are then generated by the extracted consecutive images and camera control signals. The effectiveness of the proposed method was verified in the experiments by comparing the results obtained using the proposed method with those obtained using the traditional CNN. The results showed that the accuracy of the control signals obtained using the proposed method was 56.30% higher than that of the control signals obtained using the traditional CNN.

Three-dimensional virtual environments can be configured as test environments of autonomous things, and remote sensing by 3D point clouds collected by light detection and range (LiDAR) can be used to detect virtual human objects by segmenting collected 3D point clouds in a virtual environment. The use of a traditional encoder-decoder model, such as DeepLabV3, improves the quality of the low-density 3D point clouds of human objects, where the quality is determined by the measurement gap of the LiDAR lasers. However, whenever a human object with a surrounding environment in a 3D point cloud is used by the traditional encoder-decoder model, it is difficult to increase the density fitting of the human object. This paper proposes a DeepLabV3-Refiner model, which is a model that refines the fit of human objects using human objects whose density has been increased through DeepLabV3. An RGB image that has a segmented human object is defined as a dense segmented image. DeepLabV3 is used to make predictions of dense segmented images and 3D point clouds for human objects in 3D point clouds. In the Refiner model, the results of DeepLabV3 are refined to fit human objects, and a dense segmented image fit to human objects is predicted. The dense 3D point cloud is calculated using the dense segmented image provided by the DeepLabV3-Refiner model. The 3D point clouds that were analyzed by the DeepLabV3-Refiner model had a 4-fold increase in density, which was verified experimentally. The proposed method had a 0.6% increase in density accuracy compared to that of DeepLabV3, and a 2.8-fold increase in the density corresponding to the human object. The proposed method was able to provide a 3D point cloud that increased the density to fit the human object. The proposed method can be used to provide an accurate 3D virtual environment by using the improved 3D point clouds.

Deep learning has made significant progress in the field of automatic music generation. At present, the research on music generation via deep learning can be divided into two categories: predictive models and generative models. However, both categories have the same problems that need to be resolved. First, the length of the music must be determined artificially prior to generation. Second, although the convolutional neural network (CNN) is unexpectedly superior to the recurrent neural network (RNN), CNN still has several disadvantages. This paper proposes a conditional generative adversarial network approach using an inception model (INCO-GAN), which enables the generation of complete variable-length music automatically. By adding a time distribution layer that considers sequential data, CNN considers the time relationship in a manner similar to RNN. In addition, the inception model obtains richer features, which improves the quality of the generated music. In experiments conducted, the music generated by the proposed method and that by human composers were compared. High cosine similarity of up to 0.987 was achieved between the frequency vectors, indicating that the music generated by the proposed method is very similar to that created by a human composer.

Gomoku is a two-player board game that originated in ancient China. There are various cases of developing Gomoku using artificial intelligence, such as a genetic algorithm and a tree search algorithm. Alpha-Gomoku, Gomoku AI built with Alpha-Go’s algorithm, defines all possible situations in the Gomoku board using Monte-Carlo tree search (MCTS), and minimizes the probability of learning other correct answers in the duplicated Gomoku board situation. However, in the tree search algorithm, the accuracy drops, because the classification criteria are manually set. In this paper, we propose an improved reinforcement learning-based high-level decision approach using convolutional neural networks (CNN). The proposed algorithm expresses each state as One-Hot Encoding based vectors and determines the state of the Gomoku board by combining the similar state of One-Hot Encoding based vectors. Thus, in a case where a stone that is determined by CNN has already been placed or cannot be placed, we suggest a method for selecting an alternative. We verify the proposed method of Gomoku AI in GuPyEngine, a Python-based 3D simulation platform.

The latest research on smart city technologies mainly focuses on utilizing cities’ resources to improve the quality of the lives of citizens. Diverse kinds of control signals from massive systems and devices such as adaptive traffic light systems in smart cities can be collected and utilized. Unfortunately, it is difficult to collect a massive dataset of control signals as doing so in the real-world requires significant effort and time. This paper proposes a deep generative model which integrates a long short-term memory model with generative adversarial network (LSTM-GAN) to generate agent control signals based on the words extracted from newspaper articles to solve the problem of collecting massive signals. The discriminatory network in the LSTM-GAN takes continuous word embedding vectors as inputs generated by a pre-trained Word2Vec model. The agent control signals of sequential actions are simultaneously predicted by the LSTM-GAN in real time. Specifically, to collect the training data of smart city simulations, the LSTM-GAN is trained based on the Corpus of Contemporary American English (COCA) newspaper dataset, which contains 5,317,731 sentences, for a total of 93,626,203 word tokens, from written texts. To verify the proposed method, agent control signals were generated and validated. In the training of the LSTM-GAN, the accuracy of the discriminator converged to 50%. In addition, the losses of the discriminator and the generator converged from 4527.04 and 4527.94 to 2.97 and 1.87, respectively.

Malware detection and classification methods are being actively developed to protect personal information from hackers. Global images of malware (in a program that includes personal information) can be utilized to detect or classify it. This method is efficient, given that small changes in the program can be detected while maintaining the overall structure of the program. However, if any obfuscation approach that encrypts malware code is implemented, it becomes difficult to extract features such as opcodes and application programming interface functions. Given that malware detection and classification are performed differently depending on whether malware is obfuscated or not, methods that can simultaneously detect and classify general and obfuscated malware are required. This paper proposes a method that uses a generative adversarial network (GAN) and global image-based local image to classify unobfuscated and obfuscated malware. Global and local images of unobfuscated malware are generated using pixel and local feature visualizers. The GAN is utilized to visualize local features and generate local images of obfuscated malware by learning global and local images of unobfuscated malware. The local image of unobfuscated malware is merged with the global image generated via the pixel visualizer. To merge the global and local images of unobfuscated and obfuscated malware, the pixels extracted from global and local images are stored in a two-dimensional array, and then merged images are generated. Finally, unobfuscated and obfuscated malware are classified using a convolutional neural network (CNN). The results of experiments conducted on the Microsoft Malware Classification Challenge (BIG 2015) dataset indicate that the proposed method has a malware classification accuracy of 99.65%, which is 2.18% higher than that of the malware classification approach based on only global images and local features.

Point clouds have been widely used in three-dimensional (3D) object classification tasks, i.e., people recognition in unmanned ground vehicles. However, the irregular data format of point clouds and the large number of parameters in deep learning networks affect the performance of object classification. This paper develops a 3D object classification system using a broad learning system (BLS) with a feature extractor called VB-Net. First, raw point clouds are voxelized into voxels. Through this step, irregular point clouds are converted into regular voxels which are easily processed by the feature extractor. Then, a pre-trained VoxNet is employed as a feature extractor to extract features from voxels. Finally, those features are used for object classification by the applied BLS. The proposed system is tested on the ModelNet40 dataset and ModelNet10 dataset. The average recognition accuracy was 83.99% and 90.08%, respectively. Compared to deep learning networks, the time consumption of the proposed system is significantly decreased.

Three-dimensional (3D) point clouds are important for many applications, including object tracking and 3D scene reconstruction. Point clouds are usually obtained from laser scanners, but their high cost impedes the widespread adoption of this technology. We propose a method to generate the 3D point cloud corresponding to a single red–green–blue (RGB) image. The method retrieves high-quality 3D data from two-dimensional (2D) images captured by conventional cameras, which are generally less expensive. The proposed method comprises two stages. First, a generative adversarial network generates a depth image estimation from a single RGB image. Then, the 3D point cloud is calculated from the depth image. The estimation relies on the parameters of the depth camera employed to generate the training data. The experimental results verify that the proposed method provides high-quality 3D point clouds from single 2D images. Moreover, the method does not require a PC with outstanding computational resources, further reducing implementation costs, as only a moderate-capacity graphics processing unit can efficiently handle the calculations.

Fast and accurate obstacle detection is essential for accurate perception of mobile vehicles&rsquo, environment. Because point clouds sensed by light detection and ranging (LiDAR) sensors are sparse and unstructured, traditional obstacle clustering on raw point clouds are inaccurate and time consuming. Thus, to achieve fast obstacle clustering in an unknown terrain, this paper proposes an elevation-reference connected component labeling (ER-CCL) algorithm using graphic processing unit (GPU) programing. LiDAR points are first projected onto a rasterized x&ndash, z plane so that sparse points are mapped into a series of regularly arranged small cells. Based on the height distribution of the LiDAR point, the ground cells are filtered out and a flag map is generated. Next, the ER-CCL algorithm is implemented on the label map generated from the flag map to mark individual clusters with unique labels. Finally, obstacle labeling results are inverse transformed from the x&ndash, z plane to 3D points to provide clustering results. For real-time 3D point cloud clustering, ER-CCL is accelerated by running it in parallel with the aid of GPU programming technology.

The data computing process is utilized in various areas such as autonomous driving. Autonomous vehicles are intended to detect and track nearby moving objects avoiding collisions and to navigate in complex situations, such as heavy traffic and dense pedestrian areas. Therefore, object tracking is the core technology in the environment perception systems of autonomous vehicles and requires the monitoring of surrounding objects and the prediction of the moving states of objects in real time. In this paper, a multiple object tracking method based on light detection and ranging (LiDAR) data is proposed by using a Kalman filter and data computing process. We suppose that the movements of the tracking objects are captured consecutively as frames; thus, model-based detection and tracking of dynamic objects are possible. A Kalman filter is applied for predicting posterior state of tracking object based on anterior state of the tracking object. State denotes the positions, shapes, and sizes of objects. By computing the likelihood probability between predicted tracking objects and clusters which registered from tracking objects, the data association process of the tracking objects can be generated. Experimental results showed enhanced object tracking performance in a dynamic environment. The average matching probability of the tracking object was greater than 92.9%.

Imbalanced datasets can be found in a number of fields; they are commonly regarded as big data because of their sheer volume and high attribute dimensions. As the name suggests, imbalanced big datasets come with an extremely imbalanced ratio between the amount of major class and minority class samples. Traditional methods: have been attempted but still cannot fully, effectively, and reliably solve the imbalanced class classification problem, especially when the distribution of the classes is exceedingly imbalanced. In this paper, we propose a collection of algorithms to solve the problem of imbalanced datasets in binary data classification. Most traditional methods: rebalance the imbalanced dataset merely by matching the data quantities of the two classes. Our proposed algorithms, which take the form of a suite of variants, focus on guaranteeing the credibility of the classification model and reaching the greatest possible accuracy by dynamically rebalancing the training dataset with multi-objective swarm intelligence optimisation. The new algorithms are extended from those we proposed earlier, which had a single objective – first find a set of solutions that satisfy the Kappa criterion, then search for the solution in the set that offers the highest accuracy. Two main modifications are made in the new algorithms. Multi-objective optimisation is aimed at finding a solution that satisfies several criteria at the same time, such as accuracy and identifying a list of credibility indicators. The other enhancement is the incremental operation of the multi-objective optimisation. Incremental optimisation is imperative for processing data feeds that may arrive in a streaming manner. Instead of waiting for the full data archive to be available before optimisation, incremental optimisation rebalances the data feed segment by segment on the fly. The experimental results from the suite of proposed algorithms show that they can effectively attain better and more stable performances from the classification model and are accompanied by much greater credibility than the other five traditional methods when imbalanced datasets are used as training datasets for inducing a classifier.

This paper proposes an unmanned aerial vehicle (UAV) flight control method where a graph-based path is generated after the collected UAV flight data by a pilot are analyzed. UAV flights are planned by using hierarchical A* search algorithms based on graph-based generated flight paths to take images at multiple surveillance points. Generating a graph-based path makes it possible for UAVs to fly autonomously along paths shorter than that of the pilot collecting UAV flight data given that the shorter paths can be derived by connecting partially flied paths. A* search algorithms can be applied hierarchically to a graph-based path that contains circulation paths. The proposed method was experimentally verified through an analysis of the collected UAV flight data to generate graph-based and planned paths. The pilot flew the UAV six times and obtained 8115 UAV flight data points. The generated graph-based path included 17 monitoring points for taking surveillance images and 90 intermediate flight points. The length of the flight paths collected by six time flights was 1364.32 m, and the length of the flight paths by the proposed method was 764.27 m. Given that 8115 flight points were collected and 109 flight points were selected by the proposed method, the complexity of the generated graph-based path consisted of flight points was reduced to 1.34% by hierarchical A* search algorithms.

This study contributes to knowledge by advancing the transportation management plan (TMP) development efforts for expressway work zones. Using text mining techniques to a large-scale transportation data set that contains descriptively narrated texts, this research analyzes the association between words related to the type of work being performed and the type of lane closure in expressway work zone areas. It found that recurrent everyday tasks and bridge repair works tend to cause shoulder lane closure, while works—such as tunnel repair, night work, pavement, median barrier, road surface repair, and line marking—are more associated with main lane closure. Moreover, the findings further clarify the characteristic patterns shared between the number of closed lanes, and the respective lane position in two- and three-lane expressways. These offer significant insights into the decision-making process for the development of work zone TMPs, which can further be integrated into the various components of TMP to make the plan more effective and, at the same time, ensure an efficient throughput flow throughout the work zone, reduced congestion, and improved safety.

Smart city is an emerging research field nowadays, with emphasis of using big data to enhance citizens' quality of life. One of the prevalent smart city projects is to use big traffic data collected from road users over time, for road planning, traffic light scheduling, traffic jam relief, and public security. In particular, being able to know a road user's current location and predict his/her next move is important in today's intelligent transportation systems. Trajectory prediction has become a prudential research study direction, by which many algorithms have been published before. In this paper, we present a simple probabilistic model which predicts road users' next locations based on the "concept of segments" abstracted from historical trails which the users have taken and accumulated over time in some data archive. Given a trajectory and a current location, the road user's next move in terms of road direction can be predicted at the junction. It is found that each road user would have his/her unique travel pattern hidden in the aggregate big traffic data. These patterns could be modeled from connected segments for simplicity. With the longer the trail and more frequently this trail was travelled, the more accurate that the next turn can be predicted. Simulation experiment was conducted based on summing up the segments from empirical trajectory data that was used in trajectory data mining by Microsoft. The results of our alternative model in contrast to the state of the arts demonstrated good efficacy.

Social network services (SNSs) have become one of the core Internet-based application services in recent years. Through SNSs, diverse kinds of private data are shared with users' friends and SNS plug-in applications. However, these data can be exposed via abnormal private data access. For example, the addition of fake friends to a user's account is one approach to gain access to a private user's data. Private user data can be protected from being accessed by using an automated method to assess information. This paper proposes a method that evaluates private data accesses for social network security. By defining normal private data access patterns in advance, abnormal private data access patterns can be exposed. Normal private data access patterns are generated by analyzing all of the consecutive private data accesses of users based on Bayesian probability. We have proven the effectiveness of our approach by conducting experiments where the private data access signals of Twitter accounts were collected and analyzed.

Scientists have explored the human body for hundreds of years, and yet more relationships between the behaviors and health are still to be discovered. With the development of data mining, artificial intelligence technology, and human posture detection, it is much more possible to figure out how behaviors and movements influence people’s health and life and how to adjust the relationship between work and rest, which is needed urgently for modern people against this high-speed lifestyle. Using smart technology and daily behaviors to supervise or predict people’s health is a key part of a smart city. In a smart city, these applications involve large groups and high-frequency use, so the system must have low energy consumption, a portable system, and a low cost for long-term detection. To meet these requirements, this paper proposes a posture recognition method based on multisensor and using LoRa technology to build a long-term posture detection system. LoRa WAN technology has the advantages of low cost and long transmission distances. Combining the LoRa transmitting module and sensors, this paper designs wearable clothing to make people comfortable in any given posture. Aiming at LoRa’s low transmitting frequency and small size of data transmission, this paper proposes a multiprocessing method, including data denoising, data enlarging based on sliding windows, feature extraction, and feature selection using Random Forest, to make 4 values retain the most information about 125 data from 9 axes of sensors. The result shows an accuracy of 99.38% of extracted features and 95.06% of selected features with the training of 3239 groups of datasets. To verify the performance of the proposed algorithm, three testers created 500 groups of datasets and the results showed good performance. Hence, due to the energy sustainability of LoRa and the accuracy of recognition, this proposed posture recognition using multisensor and LoRa can work well when facing long-term detection and LoRa fits smart city well when facing long-distance transmission.

Micro unmanned aircraft systems (micro UAS)-related technical research is important because micro UAS has the advantage of being able to perform missions remotely. When an omnidirectional camera is mounted, it captures all surrounding areas of the micro UAS. Normal field of view (NFoV) refers to a view presented as an image to a user in a 360-degree video. The 360-degree video is controlled using an end-to-end controls method to automatically provide the user with NFoVs without the user controlling the 360-degree video. When using the end-to-end controls method that controls 360-degree video, if there are various signals that control the 360-degree video, the training of the deep learning model requires a considerable amount of training data. Therefore, there is a need for a method of autonomously determining the signals to reduce the number of signals for controlling the 360-degree video. This paper proposes a method to autonomously determine the output to be used for end-to-end control-based deep learning model to control 360-degree video for micro UAS controllers. The output of the deep learning model to control 360-degree video is automatically determined using the K-means algorithm. Using a trained deep learning model, the user is presented with NFoVs in a 360-degree video. The proposed method was experimentally verified by providing NFoVs wherein the signals that control the 360-degree video were set by the proposed method and by user definition. The results of training the convolution neural network (CNN) model using the signals to provide NFoVs were compared, and the proposed method provided NFoVs similar to NFoVs of existing user with 24.4% more similarity compared to a user-defined approach.

Nowadays, deep learning methods based on a virtual environment are widely applied to research and technology development for autonomous vehicle&rsquo, s smart sensors and devices. Learning various driving environments in advance is important to handle unexpected situations that can exist in the real world and to continue driving without accident. For training smart sensors and devices of an autonomous vehicle well, a virtual simulator should create scenarios of various possible real-world situations. To create reality-based scenarios, data on the real environment must be collected from a real driving vehicle or a scenario analysis process conducted by experts. However, these two approaches increase the period and the cost of scenario generation as more scenarios are created. This paper proposes a scenario generation method based on deep learning to create scenarios automatically for training autonomous vehicle smart sensors and devices. To generate various scenarios, the proposed method extracts multiple events from a video which is taken on a real road by using deep learning and generates the multiple event in a virtual simulator. First, Faster-region based convolution neural network (Faster-RCNN) extracts bounding boxes of each object in a driving video. Second, the high-level event bounding boxes are calculated. Third, long-term recurrent convolution networks (LRCN) classify each type of extracted event. Finally, all multiple event classification results are combined into one scenario. The generated scenarios can be used in an autonomous driving simulator to teach multiple events that occur during real-world driving. To verify the performance of the proposed scenario generation method, experiments using real driving video data and a virtual simulator were conducted. The results for deep learning model show an accuracy of 95.6%, furthermore, multiple high-level events were extracted, and various scenarios were generated in a virtual simulator for smart sensors and devices of an autonomous vehicle.

Internet of Things simulations play significant roles in the diverse kinds of activities in our daily lives and have been extensively researched. Creating and controlling virtual agents in three-dimensional Internet of Things simulations is a key technology for achieving realism in three-dimensional simulations. Given that traditional virtual agent-based approaches have limitations for realism, it is necessary to improve the realism of three-dimensional Internet of Things simulations. This article proposes a Q-Network-based motivation framework that applies a Q-Network to select motivations from desires and hierarchical task network planning to execute actions based on goals of the selected motivations. The desires are to be identified and calculated based on states. Selected motivations will be chosen to determine the goals that agents must achieve. In the experiments, the proposed framework achieved an average accuracy of up to 85.5% when the Q-Network-based motivation model was trained. To verify the Q-Network-based motivation framework, a traditional Q-learning is also applied in the three-dimensional virtual environment. Comparing the results of the two frameworks, the Q-Network-based motivation framework shows better results than those of traditional Q-learning, as the accuracy of the Q-Network-based motivation is higher by 15.58%. The proposed framework can be applied to the diverse kinds of Internet of Things systems such as a training autonomous vehicle. Moreover, the proposed framework can generate big data on animal behaviors for other training systems.

With the widespread use of unmanned aerial vehicles (UAVs), there is an increasing demand for the development of their control technology. The key interaction technology between humans and UAVs needs to focus on the human body language, which comprises rich interactive information, as it is the most natural, intuitive, and easy to master approach of interpersonal communication for humans. Therefore, the research on human motion estimation for UAV control is of considerable practical significance. Recently, deep learning has made breakthroughs in speech, image recognition and, other fields, and has crushed the performance of traditional methods in many fields. However, in the field of human motion estimation, deep learning has been progressing slowly. To overcome the limitations of the traditional methods and explore the application of deep learning methods in the field of motion estimation, this study proposes a method to estimate human arm motion using deep learning networks. We proposed a bidirectional two-layer LSTM fusion network to estimate the forearms' motion according to the hand position measured by HTC Vive. The performance was verified using a real data set. The average Euclidean distance similarity can reach up to 56%. In comparison with the traditional methods, the proposed method demonstrated wider applicability and better performance.

Leap Motion has become widely used due to its ability to recognize intuitive hand gestures or accurate finger positions. Attaching a Leap Motion to a virtual reality head-mounted display (VR HMD) is highly interoperable with virtual objects in virtual reality. However, it is difficult for a virtual reality application to identify the accurate position where the Leap Motion is attached to the HMD. This causes errors in the positions of the actual user’s hands and the virtual hands, which makes the interaction in virtual reality difficult. In this paper, a method that calibrates an output area in VR HMD and a sensing area in Leap Motion is proposed. The difference in the origin coordinate between VR HMD and Leap Motion is derived using the proposed method. The position of the Leap Motion attached to the HMD was determined through an experiment using the proposed calibration technique, and the error was approximately 0.757 cm. Accordingly, it enables more intuitive interactions in virtual reality applications.

Location-based services on GPS-enabled smartphones are undergoing strong growth. Capitalizing on the popularity of this geo-location social media, a mobile app called Foursquare is developed to recommend its users places where they may be interested in, to travel from their current proximities. Such location data, in the form of check-ins by Foursquare, have huge business potentials including marketing, advertising and consumers' behaviors analysis. Many researchers from both academia and industries are seriously looking into this location-based big data which comes in high velocity (with millions of users and frequent geo-tagging), and wide variety (with potentially many meta-data and associations), accumulating into a huge volume. One of the fundamental analytics in such big data is to guess which check-in locations a user would move to, as a prerequisite for sequential mining and other lifestyle pattern analysis. This paper reports a novel, but simple big data analytic by sampling a portion of location data for predicting the next check-in locations. This proposed analytic does not need every individual user's history path and ID to match the history path of the current user in the database in order to infer a prediction. We show by a simulation experiment based on a Foursquare dataset that a minimum of two pairs of coordinates are required to provide a prediction. Several variables such as segment lengths, number of check-ins, and time factors are investigated in the experiment in relation to the prediction accuracy.

Recently, the surveillance system research has focused because Unmanned Aerial Vehicle(UAV) has the ability to monitor wide area. When the wide area are monitored, controlling UAVs repeatedly by pilots invokes the cost problem to operate UAVs. If monitoring path can be defined in advance, the cost problem can be solved by controlling UAVs autonomously based on the monitoring path. The traditional approach generates multiple motor primitives based on flied GPS locations. However, the monitoring points by UAVs are not considered by the generated motor primitives, the surveillance by UAVs is not performed properly. This paper proposes a motor primitive structure for surveillance UAVs to be flied autonomously. Motor primitives are generated automatically by setting surveillance points to denote surveillance targets accurately.Keywords:Unmanned Aerial Vehicle, Ground Control System, Motor Primitive, Global Positioning System, Demonstration-Based Learning 감시 위치 기반의 UAV 모터프리미티브의 구조 설계

It is possible to design an English vocabulary learning system using a text to speech (TTS) engine and a module which creates push alarms for new words or sentences at a specific time every day. The function of selecting new words and sentences and showing them to the user is a core technology in vocabulary learning systems, and it has a great effect on the systems’ learning effectiveness. This paper proposes a method for creating a system which selects words and sentences. The system collects English words and sentences via social networks, refines them so that they are in a form which is appropriate for user learning, and allows the user to learn words which are related to their current location and weather.

To provide a realistic environment for remote sensing applications, point clouds are used to realize a three-dimensional (3D) digital world for the user. Motion recognition of objects, e.g., humans, is required to provide realistic experiences in the 3D digital world. To recognize a user's motions, 3D landmarks are provided by analyzing a 3D point cloud collected through a light detection and ranging (LiDAR) system or a red green blue (RGB) image collected visually. However, manual supervision is required to extract 3D landmarks as to whether they originate from the RGB image or the 3D point cloud. Thus, there is a need for a method for extracting 3D landmarks without manual supervision. Herein, an RGB image and a 3D point cloud are used to extract 3D landmarks. The 3D point cloud is utilized as the relative distance between a LiDAR and a user. Because it cannot contain all information the user's entire body due to disparities, it cannot generate a dense depth image that provides the boundary of user's body. Therefore, up-sampling is performed to increase the density of the depth image generated based on the 3D point cloud, the density depends on the 3D point cloud. This paper proposes a system for extracting 3D landmarks using 3D point clouds and RGB images without manual supervision. A depth image provides the boundary of a user's motion and is generated by using 3D point cloud and RGB image collected by a LiDAR and an RGB camera, respectively. To extract 3D landmarks automatically, an encoder&ndash, decoder model is trained with the generated depth images, and the RGB images and 3D landmarks are extracted from these images with the trained encoder model. The method of extracting 3D landmarks using RGB depth (RGBD) images was verified experimentally, and 3D landmarks were extracted to evaluate the user's motions with RGBD images. In this manner, landmarks could be extracted according to the user's motions, rather than by extracting them using the RGB images. The depth images generated by the proposed method were 1.832 times denser than the up-sampling-based depth images generated with bilateral filtering.

The importance of estimating human movement has increased in the field of human motion capture. HTC VIVE is a popular device that provides a convenient way of capturing human motions using several sensors. Recently, the motion of only users&rsquo, hands has been captured, thereby greatly reducing the range of motion captured. This paper proposes a framework to estimate single-arm orientations using soft sensors mainly by combining a Bi-long short-term memory (Bi-LSTM) and two-layer LSTM. Positions of the two hands are measured using an HTC VIVE set, and the orientations of a single arm, including its corresponding upper arm and forearm, are estimated using the proposed framework based on the estimated positions of the two hands. Given that the proposed framework is meant for a single arm, if orientations of two arms are required to be estimated, the estimations are performed twice. To obtain the ground truth of the orientations of single-arm movements, two Myo gesture-control sensory armbands are employed on the single arm: one for the upper arm and the other for the forearm. The proposed framework analyzed the contextual features of consecutive sensory arm movements, which provides an efficient way to improve the accuracy of arm movement estimation. In comparison with the ground truth, the proposed method estimated the arm movements using a dynamic time warping distance, which was the average of 73.90% less than that of a conventional Bayesian framework. The distinct feature of our proposed framework is that the number of sensors attached to end-users is reduced. Additionally, with the use of our framework, the arm orientations can be estimated with any soft sensor, and good accuracy of the estimations can be ensured. Another contribution is the suggestion of the combination of the Bi-LSTM and two-layer LSTM.

The importance of cybersecurity has recently been increasing. A malware coder writes malware into normal executable files. A computer is more likely to be infected by malware when users have easy access to various executables. Malware is considered as the starting point for cyber-attacks, thus, the timely detection, classification and blocking of malware are important. Malware visualization is a method for detecting or classifying malware. A global image is visualized through binaries extracted from malware. The overall structure and behavior of malware are considered when global images are utilized. However, the visualization of obfuscated malware is tough, owing to the difficulties encountered when extracting local features. This paper proposes a merged image-based malware classification framework that includes local feature visualization, global image-based local feature visualization, and global and local image merging methods. This study introduces a fastText-based local feature visualization method: First, local features such as opcodes and API function names are extracted from the malware, second, important local features in each malware family are selected via the term frequency inverse document frequency algorithm, third, the fastText model embeds the selected local features, finally, the embedded local features are visualized through a normalization process. Malware classification based on the proposed method using the Microsoft Malware Classification Challenge dataset was experimentally verified. The accuracy of the proposed method was approximately 99.65%, which is 2.18% higher than that of another contemporary global image-based approach.

Recent research on deep learning has been applied to a diversity of fields. In particular, numerous studies have been conducted on self-driving vehicles using end-to-end approaches based on images captured by a single camera. End-to-end controls learn the output vectors of output devices directly from the input vectors of available input devices. In other words, an end-to-end approach learns not by analyzing the meaning of input vectors, but by extracting optimal output vectors based on input vectors. Generally, when end-to-end control is applied to self-driving vehicles, the steering wheel and pedals are controlled autonomously by learning from the images captured by a camera. However, high-resolution images captured from a car cannot be directly used as inputs to Convolutional Neural Networks (CNNs) owing to memory limitations; the image size needs to be efficiently reduced. Therefore, it is necessary to extract features from captured images automatically and to generate input images by merging the parts of the images that contain the extracted features. This paper proposes a learning method for end-to-end control that generates input images for CNNs by extracting road parts from input images, identifying the edges of the extracted road parts, and merging the parts of the images that contain the detected edges. In addition, a CNN model for end-to-end control is introduced. Experiments involving the Open Racing Car Simulator (TORCS), a sustainable computing environment for cars, confirmed the effectiveness of the proposed method for self-driving by comparing the accumulated difference in the angle of the steering wheel in the images generated by it with those of resized images containing the entire captured area and cropped images containing only a part of the captured area. The results showed that the proposed method reduced the accumulated difference by 0.839% and 0.850% compared to those yielded by the resized images and cropped images, respectively.

This paper proposes a user interface control framework of sexual violence prevention education Apps. To conduct the control of the properties of UI elements such as colors and sizes, the frequency, the utilization, the relationship of UI elements are considered. In the experiment, the application, “Sexual violence! How should I do?”, was utilized applying the proposed framework. The sizes of the buttons of the application was resized based on the data of the utilization of four subjects.

Demand for interactive wearable devices is rapidly increasing with the development of smart devices. To accurately utilize wearable devices for remote robot controls, limited data should be analyzed and utilized efficiently. For example, the motions by a wearable device, called Myo device, can be estimated by measuring its orientation, and calculating a Bayesian probability based on these orientation data. Given that Myo device can measure various types of data, the accuracy of its motion estimation can be increased by utilizing these additional types of data. This paper proposes a motion estimation method based on weighted Bayesian probability and concurrently measured data, orientations and electromyograms (EMG). The most probable motion among estimated is treated as a final estimated motion. Thus, recognition accuracy can be improved when compared to the traditional methods that employ only a single type of data. In our experiments, seven subjects perform five predefined motions. When orientation is measured by the traditional methods, the sum of the motion estimation errors is 37.3%; likewise, when only EMG data are used, the error in motion estimation by the proposed method was also 37.3%. The proposed combined method has an error of 25%. Therefore, the proposed method reduces motion estimation errors by 12%.

This study aimed to develop and evaluate the effects of a sexual abuse prevention mobile application, SAP_MobAPP, for primary school children. Forty-five subjects were trained for 40 minutes once a week. The experimental group received education that utilized the SAP_MobAPP. Control group A received Web based sexual abuse prevention education, while control group B received textbook based sexual abuse prevention education. Effectiveness was verified through a survey on child sexual abuse recognition and avoidance skills administered before and after training. The SAP_MobAPP program improved recognition (awareness) and the child's skills to avoid child sexual abuse situations, and the effects were long-lasting. However, differences between groups were not statistically significant. This study developed a sexual abuse prevention application and verified its effectiveness. Awareness and skills to avoid child sexual abuse after app education increased immediately after training and four weeks later. The SAP_MobAPP could be used for sexual abuse prevention education in schools.

The strong relationship between music and health has helped prove that soft and peaceful classical music can significantly reduce people&rsquo, s stress, however, it is difficult to identify and collect examples of such music to build a library. Therefore, a system is required that can automatically generate similar classical music selections from a small amount of input music. Melody is the main element that reflects the rhythms and emotions of musical works, therefore, most automatic music generation research is based on melody. Given that melody varies frequently within musical bars, the latter are used as the basic units of composition. As such, there is a requirement for melody extraction techniques and bar-based encoding methods for automatic generation of bar-based music using melodies. This paper proposes a method that handles melody track extraction and bar encoding. First, the melody track is extracted using a pitch-based term frequency&ndash, inverse document frequency (TFIDF) algorithm and a feature-based filter. Subsequently, four specific features of the notes within a bar are encoded into a fixed-size matrix during bar encoding. We conduct experiments to determine the accuracy of track extraction based on verification data obtained with the TFIDF algorithm and the filter, an accuracy of 94.7% was calculated based on whether the extracted track was a melody track. The estimated value demonstrates that the proposed method can accurately extract melody tracks. This paper discusses methods for automatically extracting melody tracks from MIDI files and encoding based on bars. The possibility of generating music through deep learning neural networks is facilitated by the methods we examine within this work. To help the neural networks generate higher quality music, which is good for human health, the data preprocessing methods contained herein should be improved in future works.

In traditional music composition, the composer has a special knowledge of music and combines emotion and creative experience to create music. As computer technology has evolved, various music-related technologies have been developed. To create new music, a considerable amount of time is required. Therefore, a system is required that can automatically compose music from input music. This study proposes a novel melody composition method that enhanced the original generative adversarial network (GAN) model based on individual bars. Two discriminators were used to form the enhanced GAN model: one was a long short-term memory (LSTM) model that was used to ensure correlation between the bars, and the other was a convolutional neural network (CNN) model that was used to ensure rationality of the bar structure. Experiments were conducted using bar encoding and the enhanced GAN model to compose a new melody and evaluate the quality of the composition melody. In the evaluation method, the TFIDF algorithm was also used to calculate the structural differences between four types of musical instrument digital interface (MIDI) file (i.e., randomly composed melody, melody composed by the original GAN, melody composed by the proposed method, and the real melody). Using the TFIDF algorithm, the structures of the melody composed were compared by the proposed method with the real melody and the structure of the traditional melody was compared with the structure of the real melody. The experimental results showed that the melody composed by the proposed method had more similarity with real melody structure with a difference of only 8% than that of the traditional melody structure.

With intelligent big data, a variety of gesture-based recognition systems have been developed to enable intuitive interaction by utilizing machine learning algorithms. Realizing a high gesture recognition accuracy is crucial, and current systems learn extensive gestures in advance to augment their recognition accuracies. However, the process of accurately recognizing gestures relies on identifying and editing numerous gestures collected from the actual end users of the system. This final end-user learning component remains troublesome for most existing gesture recognition systems. This paper proposes a method that facilitates end-user gesture learning and recognition by improving the editing process applied on intelligent big data, which is collected through end-user gestures. The proposed method realizes the recognition of more complex and precise gestures by merging gestures collected from multiple sensors and processing them as a single gesture. To evaluate the proposed method, it was used in a shadow puppet performance that could interact with on-screen animations. An average gesture recognition rate of 90% was achieved in the experimental evaluation, demonstrating the efficacy and intuitiveness of the proposed method for editing visualized learning gestures.

Unmanned aerial vehicles may collide with obstacles, such as trees or other unmanned aerial vehicles, while flying. A waypoint-based flight path is an approach to avoid such obstacles. To specify waypoints for the safe flight of unmanned aerial vehicles, it is necessary to define a flight path in advance by analyzing the flight records of unmanned aerial vehicles and thereby designate the waypoints automatically. However, there is a problem in that pilots tend to make errors in controlling unmanned aerial vehicles and collecting flight records. This article proposes a method to generate candidate waypoints for a flight path by removing such unintended flight records. In this method, images representing the positions in the collected flight records are generated. The candidate waypoints are generated as positions corresponding to the overlapping pixels of the images generated via image accumulation based on the flight records and the ones generated by accumulating the images reconstructed using an Autoencoder. The unmanned aerial vehicles can be set the waypoints for an autonomous flight using the candidate waypoints. An experiment was conducted in a university to generate candidate waypoints for road monitoring. The results obtained using the proposed method and K-means algorithm were compared. The candidate waypoints generated using the proposed method were reduced by 84.21% compared to those generated using the K-means algorithm.

Unmanned Aerial Vehicles (UAVs) have been utilized in various applications in many fields in recent years. The paths the pilots flew can be measured and collected to be utilized to create routes for autonomous flight. However, there is a problem in that GPS errors result in the path being irregularly represented. The measured path can be optimized by using the Douglas-Peucker algorithm. Our research led to the proposal of a method to optimize this path by applying the Douglas-Peucker algorithm, which has been shown to be suitable for a two-dimensional path, in three-dimensional space. Optimization of the 3D path by the proposed method was possible by deleting unnecessary points from the three-dimensional space. Thus, the flight paths that were measured and collected can be utilized to define the autonomous flight path.

Unmanned aerial vehicles (UAVs) with auto-pilot capabilities are often used for surveillance and patrol. Pilots set the flight points on a map in order to navigate to the imaging point where surveillance or patrolling is required. However, there is the limit denoting the information such as absolute altitudes and angles. Therefore, it is required to set the information accurately. This paper hereby proposes a method to construct environmental symmetric big data using an unmanned aerial vehicle (UAV) during flight by designating the imaging and non-imaging points for surveillance and patrols. The K-Means-based algorithm proposed in this paper is then employed to divide the imaging points, which is set by the pilot, into K clusters, and K imaging points are determined using these clusters. Flight data are then used to set the points to which the UAV will fly. In our experiment, flight records were gathered through an UAV in order to monitor a stadium and the imaging and non-imaging points were set using the proposed method and compared with the points determined by a traditional K-Means algorithm. Through the proposed method, the cluster centroids and cumulative distance of its members were reduced by 87.57% more than with the traditional K-Means algorithm. With the traditional K-Means algorithm, imaging points were not created in the five points desired by the pilot, and two incorrect points were obtained. However, with the proposed method, two incorrect imaging points were obtained. Due to these two incorrect imaging points, the two points desired by the pilot were not generated.

Background An imbalanced dataset is defined as a training dataset that has imbalanced proportions of data in both interesting and uninteresting classes. Often in biomedical applications, samples from the stimulating class are rare in a population, such as medical anomalies, positive clinical tests, and particular diseases. Although the target samples in the primitive dataset are small in number, the induction of a classification model over such training data leads to poor prediction performance due to insufficient training from the minority class. Results In this paper, we use a novel class-balancing method named adaptive swarm cluster-based dynamic multi-objective synthetic minority oversampling technique (ASCB_DmSMOTE) to solve this imbalanced dataset problem, which is common in biomedical applications. The proposed method combines under-sampling and over-sampling into a swarm optimisation algorithm. It adaptively selects suitable parameters for the rebalancing algorithm to find the best solution. Compared with the other versions of the SMOTE algorithm, significant improvements, which include higher accuracy and credibility, are observed with ASCB_DmSMOTE. Conclusions Our proposed method tactfully combines two rebalancing techniques together. It reasonably re-allocates the majority class in the details and dynamically optimises the two parameters of SMOTE to synthesise a reasonable scale of minority class for each clustered sub-imbalanced dataset. The proposed methods ultimately overcome other conventional methods and attains higher credibility with even greater accuracy of the classification model.