Search

Based on binocular stereo vision measurement and wireless sensing theory, this paper studies the measurement and analysis methods of athletes' movement displacement parameters. The main purpose is to explore the relationship between sports biomechanical parameters. The analysis reveals the laws of athletes 'movements, and provides theoretical guidance on athletes' attitude recognition and physical training. Based on the principle of binocular stereo vision, a non-contact displacement measurement system was developed. The displacement measurement system uses template matching to extract the image coordinates of the measurement points and restore the spatial coordinates of the measurement points through European 3D reconstruction, which improves the practicality of the measurement system. In the cantilever beam vibration test, the traditional displacement sensor and binocular image displacement measurement system are used to simultaneously measure the cantilever beam displacement. The experimental results show that the integrated displacement measurement system in this paper has higher accuracy and better stability.

Despite the impressive performance of some recent state-of-the-art detectors, small target detection, scale variation, and label ambiguities remain challenges. To tackle these issues, we present a coordinate-based anchor-free (CBAF) module for object detection. It can be used as a branch of a single-shot detector (e.g., RetinaNet or SSD) or predict the output probabilities and coordinates directly. The main idea of the CBAF module is to predict the category and the adjustments to the box of the object by part feature and its contextual part features, which are based on feature maps divided by spatial coordinates. This is inspired by the fact that human beings can infer an entire object by observing the part of the surrounding environment. The CBAF module will encode and decode boxes in the anchor-free manner per feature map with different resolutions during training and testing. During training, we first use the proposed spatial coordinate partition layer to divide feature maps into several parts of size n × n and then propose a contextual building layer to fuse the part and its contextual parts together. We will demonstrate the CBAF module through a concrete implementation. The CBAF module improves AP scores of object detection with nearly no additional computation when working in conjunction with the anchor-based RetinaNet. Furthermore, experimental results on the MS-COCO dataset show that the mAP of the CBAF module has increased by 1.1%, compared with RetinaNet. When the CBAF module works in conjunction with the anchor-based RetinaNet, the mAP increased by 2.2%.

The authors present the results of numerical experiments aimed to determine the mean square error, which can be used to obtain the spatial coordinates of the object’s under study points through processing images obtained from UAVs using only on-Board global satellite and inertial navigation equipment on drones. At those experiments, aerial photography from the UAVs was modeled for mapping and solving the engineering task of creating a three-dimensional building (structure) model. A pair of images was simulated for the research. The results using the characteristics of the GNSS inertial solutions Trimble АРХ-15 UAV and Trimble АРХ-20 UAV are presented. The experiments show that at solving engineering tasks under conditions corresponding to an arbitrary case of aerial photography, in order to determinie the spatial coordinates with the mean square error not more than 15 mm, the same error of determining the linear elements of external orientation should be 2–5 mm, and the angular elements 10–20ʺ. This significantly exceeds the characteristics corresponding to the post-processing mode for the specified GNSS-inertial solutions. The authors show the effect of angular elements’ external orientation errors to resulting the mean square error of each coordinate and a possible approach to reducing it.

Multicamera systems have important applications in industrial online measurement, attracting wide interest due to their encouraging performance. However, how to develop a convenient and high-accuracy multicamera calibration model is a big challenge. Traditional calibration methods based on 2-D objects require the preparation of high-precision and large-scale calibration objects and have poor adaptability to the spatial distribution of cameras. To address these issues, a convenient and accurate multicamera calibration method is proposed based on an imperfect spherical object. Specifically, a special calibration object, i.e., an imperfect sphere with many coded targets is designed for calibrating the multicamera system, where the optical axes of the cameras are different in orientations and converge into the measurement field. Then, a Euclidean reconstruction method is employed to calculate the camera poses and the spatial coordinates of feature points in the local coordinate system after the adaptive grouping of cameras is completed. Moreover, a graph theory-based optimal path transformation algorithm is developed to obtain the camera poses and spatial point coordinates in the global coordinate system (GCS), and a spherical projection optimization algorithm is designed to spheroidize the spatial coordinates of feature points. In the end, the camera poses and the spatial coordinates of feature points in the GCS are optimized by a bundle adjustment algorithm. We build a multicamera system and conduct extensive experiments to demonstrate the superiority of the proposed method.

For practical use, the relevance of indoor surveillance from multiple cameras to track the movement of people and reidentify them in video sequences is constantly increasing. This is a complex task due to the effect of uneven illumination, background inhomogeneity, overlap, uncertainty of the trajectories of people, and the similarity of their visual features. The article presents an approach to track people by video sequences and reidentify them in multicamera video surveillance systems that are used indoors. At the first step, people are detected using a YOLO v4 convolution neural network (CNN) and described by a rectangular area. Further, the search for the face area and the calculation of its features are carried out, which in the developed method are used when accompanying a person in a video sequence and during his intercamera reidentification. This approach improves the accuracy of tracking with a complex movement trajectory and multiple intersections of people with similar characteristics. The search for faces is carried out on the detected areas based on the multitasking MTCNN, and the MobileFaceNetwork model is used to form the vector of the features of the face. Human features are generated using a modified CNN based on ResNet34 and an HSV color tone channel histogram. The correspondence between people on different frames is established based on the analysis of the spatial coordinates of faces and people, as well as their CNN features, using the Hungarian algorithm. To ensure the accuracy of intercamera tracking, reidentification is performed based on the facial features. Five test video sequences of different numbers of people captured indoors with a fixed video camera were used to test and compare different approaches. The obtained experimental results confirmed the strength of the characteristics of the proposed approach.

The field data collection and in-house data processing of the aerial photography system of the uavouring aircraft are studied, combined with their working examples in this paper. The aerial survey of the Bowen School of Management of Guilin University of Technology is carried out by using the Southern Sky Patrol AS1200 aerial survey system, and the obtained data is processed and studied in the internal and external industry. In order to successfully complete this research, this article (1) first of all to the external industry collected data inspection, that is, the quality of the aircraft film inspection, the main content of the inspection is the detection of the image of the tone, resolution, level, whether the film can clearly identify the main objects, such as houses, roads, etc. can be clearly identified; - Whether the film is ghosting, whether there is misalignment and fuzziness, etc. , if the film has blurred or other quality problems, it is necessary to carry out retest or retest, this step is mainly to ensure the quality of research data, for the industry processing data to do a basic guarantee. (2) For the processing of in-house data, this paper uses a correction model for image data distortion pre-processing, the method of arithmetic and the edge detection operator method of extracting the image feature, and the image matching method Greenfeld-Schenk method based on the feature, Bamard-Thompson method and jump method and other methods to match the image, at the same time, this paper also takes the aerial triangulation method of the processing of internal data, complete the measurement control point encryption work, so as to calculate the spatial coordinates of the encryption control point, reduce the field laying work like control points, and thus reduce the working time of the field. (3) For improving the accuracy of the topographic elevation point in the aviation area, this paper uses polymorphic curve to be legal, using the field elevation measured value of the checkpoint and the three-dimensional measurement value does not match the difference, using the least square-by-multiply method to fit, so that the elevation checkpoint curve fitting value change and the actual difference is the smallest, with the measurement area elevation accuracy.

In this paper, a three-dimensional (3D) indoor visible light localization method based on machine learning and deep learning is presented, which is able to obtain accurate 3D spatial coordinates of the user, including the location on the plane and the height in a room. The machine learning approaches adopted for localization include two typical algorithms, i.e., support vector machine and random forest. For the deep learning based approach, deep neural networks composed of full connected layers are employed for training in different indoor visible light localization scenarios. In the formulated learning-based visible light localization framework, the received signal strength of light-emitting diodes are taken as the input of the learning algorithm, and the measured position coordinates are inferred as the output. Apart from obtaining the two-dimensional location on the plane accurately, we also take the height into account and accurate 3D coordinates with height are obtained. The experimental results show that centimeter-scale accuracy of 3D indoor localization can be achieved using the proposed learning-based visible light localization method. Moreover, the performance of the visible light localization methods with respect to the number and the spatial pattern of LEDs, and the number of neural network layers, are also investigated.

Aiming at the problems of low recognition accuracy and low real-time in the existing human motion recognition in the complex background, a registration model recognition algorithm of human posture limit learning machine(ELM) is proposed, which uses Kinect measurement to calculate the joint association angle. Through Kinect, the spatial coordinates of human joints are obtained. Then, three-point method is used to calculate the correlation angle of human joints, and the elm registration model corresponding to the correlation angle of the target attitude joints is defined. Finally, the movement recognition test of the elm registration model of human body posture is carried out with the joint correlation angle calculated based on Kinect real-time measurement. The experimental results show that this method has high real-time and accuracy, strong robustness and generalization ability, and can identify more pose by defining more joint angles of target pose to establish elm registration model of human pose.

Image recognition in smart systems and internet of things applications is rapidly developing. Significant advances in mobile computing technology and machine learning are expanding horizons to use image recognition in smart systems and IoT applications. This paper use the smart system based image matching system utilizing genetic algorithm to design an integrated service to improve the management of image recognition. An efficient genetic algorithm used to match an unknown pattern which represents a partial image with original image. This technique uses a parameter, namely, chromosome. It transformed the spatial coordinates position into a sequence of bits of small length depending on the dimension of the original image, whereby the matching will be obtained in a short time. The system was tested on a sample images showed high matching rate (97%).

Geo-spatial information science 22(2), 73-88 (2019). doi:10.1080/10095020.2019.1613778 special issue: "Special Issue: Ubiquitous Positioning, Indoor Navigation and Location-Based Services (UPINLBS)", Published by Wuhan Univ. Journals Press, Wuhan

The problem of detection and estimation of spatial coordinates of moving objects by a passive scanning system of vision is solved. Algebraic and algorithmic approaches to determining the spatial coordinates and parameters of object motion are proposed. These approaches differ by taking into account the object velocities in searching for conjugate vectors of directions to the objects. The algorithms developed on the basis of these approaches show that the approach with allowance for motion parameters offers significant advantages of higher accuracy of coordinate estimation and higher probability of detecting all objects.

Skull registration is one of the important steps in craniofacial reconstruction, and its registration accuracy and efficiency have an important impact on the reconstruction results. To solve the problem of low accuracy and efficiency of existing skull registration methods, a hierarchical skull point cloud registration method is proposed in this paper. The whole registration process is divided into a rough registration stage and a fine registration stage. Firstly, feature points are extracted from the pre-processed skull point cloud model, and a local coordinate reference system is established according to the feature points and their neighbor points. The improved spin image is used to construct the local feature descriptor. The feature matching is carried out according to the nearest neighbor algorithm, and the k-means algorithm is used to eliminate the mismatching points to achieve skull rough registration. Then, based on rough registration, we use an improved ICP algorithm to achieve fine registration of the skull. In this process, we use random sampling to reduce the search scale of points and add geometric feature constraints to further eliminate mismatched points. Finally, the whole registration algorithm is applied to the skull point cloud data to verify. The experimental results show that, compared with other methods, the registration effect and efficiency of the proposed method are superior to those of other methods. In order to verify the universality of the method, we also use a common data set for verification. Experiments show that the method is also very effective.

The paper presents a solution for the problem of the point spatial coordinates restoration in three-dimensional basic system of coordinates by its stereo images obtained by cameras with independent locations and orientations in space; the cameras settings can be different. In analytical photogrammetry such a problem is called direct photogrammetric intersection for the general photographing case. The solution presented in the paper is based on the approach different from that one adopted in analytical photogrammetry. Methodologically, this approach is based on a vector-matrix apparatus applied from the formulation of the problem up to the final solution. Structurally, the result presented in the paper provides the equivalence of the cameras, which equally produce an effect on the final result. Similarity transformations are not used in the solution, the result is obtained just in the base coordinate system. The number of stereo cameras can be easily increased without changing the decision algorithm. These facts distinguish the solution presented in the paper from the direct photogrammetric intersection, in which one of the cameras (usually the left one) is the main camera and the center of the model coordinate system with a special direction of the axes is placed in its projection center. Similarity transforms should be used repeatedly in calculations to move from one coordinate system to another before the result in the base coordinate system is obtained. The method presented in the paper has greater average accuracy in the presence of errors in determining the coordinates of the corresponding points on the images than the direct photogrammetric intersection. This advantage in the accuracy of determining the spatial coordinates of points in the performed experiments occurred to be more than 20 %.

Marine bioprospecting is the procedure of identifying the characteristics of marine organisms to develop them into commercial products. This paper proposes a 3D reconstruction algorithm to facilitate 3D visualization of underwater scene for the marine bioprospecting using remotely operated underwater vehicle (ROV). It allows to provide an operator with intuitive user interface and accordingly contribute to enhance the operability of the ROV in the bioprospecting operation. The reconstruction algorithm transforms 2D pixel coordinates of the sonar image into the corresponding 3D spatial coordinates of the scene surface by restoring the surface elevation missing in the sonar image. First, the algorithm segments the objects and shadows in the image by classifying the pixels based on the intensity value of the seafloor. Second, it computes the surface elevation on the object pixels based on their intensity values. Third, the elevation correction factor, which is derived by the ratio between height of the object and the length of the shadow, is multiplied to the surface elevation value. Finally, the 3D coordinates of the scene surfaces are reconstructed using the coordinate transformation from the image plane to the seafloor with the restored surface elevation values. The experimental results show the algorithm successfully reconstructs the surface of the reference object within an error range less than 10 % of the object dimension, and practical applicability to the marine bioprospecting operation.

The reconstruction of electrophysiological sources within the brain is sensitive to the constructed head model, which depends on the positioning accuracy of anatomical landmarks known as fiducials. In this work, we propose an algorithm for the automatic detection of fiducial landmarks of EEG electrodes on the 3D human head model. Our proposal combines a dimensional reduction approach with a perspective projection from 3D to 2D object space; the eye and ear automatic detection in a 2D face image by two cascades of classifiers and geometric transformations to obtain 3D spatial coordinates of the landmarks and to generate the head coordinate system, This is accomplished by considering the characteristics of the scanner information. Capturing the 3D model of the head is done with Occipital Inc. ST01 structure sensor and the implementation of our algorithm was carried out on MATLAB R2018b using the Computer Vision Toolbox and the FieldTrip Toolbox. The experimental results were aimed at recursively exploring the efficacy of the facial feature detectors as a function of the projection angle; they show that robust results are obtained in terms of false acceptance rate. Our proposal is an initial step of an approach for the automatic digitization of electrode locations. The experimental results demonstrate that the proposed method detects anatomical facial landmarks automatically, accurately, and rapidly.

In order to solve the dragon fruit recognition and positioning problem in the process of mechanized picking, A new method based on binocular vision was proposed in this paper. The analysis of dragon fruit, fruit branches, and fruit flakes in the HSV color space has been finished. Multi-channel fusion method for fruit segmentation has been used, then through the filter processing, mathematical morphology method and iterative traversal of area, a binary image of the area where the target fruit is located was obtained. Then, contour fitting was performed using the least circumscribed rectangle method and the least square method, and the two-dimensional coordinates of the appropriate ellipse center point were calculated. Finally, the coordinates of the fruit in three-dimensional space were obtained through SURF (Speeded Up Robust Features) stereo matching and calculation of spatial coordinates. The experimental results show that the image segmentation of dragon fruit can be effectively implemented in the HSV color space, with an average segmentation error of 3.46%. 3D position obtained through SURF matching and calculation. The average positioning error of the three-dimensional position is 6 mm. In summary, this method can provide a reference for the automatic picking of dragon fruit.

Актуальность исследования обусловлена назревшей необходимостью разработки модели оценки точности определения пространственного положения объектов, а также отслеживания положения и движения их границ системой технического зрения, которая в составе автоматизированных комплексов может применяться при проведении различных геодезических работ, для маркшейдерского обеспечения подземного строительства, 3D-реконструкции ландшафта поверхности земли, картографировании местности, при выполнении работ в горной промышленности, а также обслуживании крупногабаритных и протяжённых объектов. Особенно актуально применение систем технического зрения, позволяющих осуществлять пространственные измерения, в тех случаях, когда традиционные лазерные средства не обеспечивают высокую эффективность, например, из-за высокой степени рассеивания или наоборот поглощения лазерного излучения поверхностью наблюдаемого объекта или его малыми угловыми размерами. Цель: оценить геометрию измерительного пространства и определить закономерности распределения погрешности измерений в её границах. Постановка задачи: разработка имитационной модели, обеспечивающей с учётом технических данных камер, их взаимных положения и ориентации, а также влияния на измерительный процесс случайных составляющих, возникающих как в процессе сборки системы, так и при её эксплуатации, реконструкция формы и размеров измерительного пространства, а также визуализация распределения погрешностей измерения координат объектов. Методы: метод математического моделирования. Новизна разработанной модели характеризуется тем, что в ней впервые была реализована возможность оценивания распределения ошибок определения координат системой технического зрения по всей глубине её измерительного пространства. При моделировании нет ограничений по взаимному положению камер и обязательной их однотипности. В модели впервые учитываются случайные погрешности, связанные как с определением взаимной ориентации положения камер, так и с неточностями при их изготовлении, а также возможностями применённой методики калибровки системы. Результат. Приведены результаты анализа известных из уровня науки и техники моделей оценки погрешности определения трёхмерных координат объектов системой технического зрения, показаны их недостатки. Раскрыта сущность и основные этапы преобразований при разработке модели оценки погрешностей определения координат объектов и их распределения по измерительному пространству системой технического зрения. Выводы. Модель на основе множества внешних и внутренних параметров камер, а также с учётом ожидаемых случайных погрешностей при сборке и настройке измерительной системы позволяет оценивать погрешность определения координат объектов и геометрию измерительного пространна. Представленная модель позволяет оценивать точность определения пространственных координат объектов, положения и движения их границ при применении систем технического зрения в составе геодезических автоматизированных измерительных комплексов. Кроме этого, применение данной модели даёт возможность оптимизировать процесс формулирования технических требований к видео- или фотокамерам и их взаимному положению в процессе проведения измерений. The relevance of the study is caused by the urgent need to develop a model for assessing the accuracy of determining the spatial position of objects, as well as tracking the position and movement of their boundaries by a computer vision system, which, as part of automated systems, can be used when carrying out various geodetic works, for surveying support of underground construction, 3D reconstruction of the surface landscape land, terrain mapping, when performing work in the mining industry, as well as servicing large and extended objects. It is especially important to use computer vision systems that allow spatial measurements in cases where traditional laser means do not provide high efficiency, for example, due to a high degree of scattering or, conversely, absorption of laser radiation by the surface of the observed object or small angular dimensions. Purpose: assessment of the measurement space geometry and determine the distribution patterns of the measurement accuracy within its boundaries. Problem statement: development of a simulation model that provides reconstruction of the form and dimensions of the measurement space and visualization of the distribution of measurement accuracy of coordinates of objects, taking into account the technical data of the cameras, their relative positions and orientations, as well as the influence on the measuring process of random components arising both during the assembly of the system and during its operation. Methods: mathematical model method. The novelty of the developed model is characterized by the fact that it was the first one to realize the possibility of estimating the distribution of errors in determining coordinates by the technical vision system over the entire depth of its measuring space. When modeling, there are no restrictions on the relative position of the cameras and their obligatory uniformity. For the first time, the model takes into account random errors associated with both the determination of the relative orientation of the position of the cameras, and with inaccuracies in their manufacture, as well as the capabilities of the applied system calibration method. Results. The paper introduces the results of the analysis of models known from the level of science and technology for assessing the accuracy in determining the three-dimensional coordinates of objects by the technical vision system, their disadvantages are shown. The essence and main stages of transformations in the development of the model for assessing the accuracy in determining the coordinates of objects and their distribution over the measuring space by the computer vision system are disclosed. Conclusions. The model based on the set of external and internal parameters of the cameras, as well as taking into account the expected random errors in the assembly and adjustment of the measuring system, makes it possible to estimate the accuracy in determining the coordinates of objects and the geometry of the measuring space. The presented model makes it possible to evaluate the accuracy of determining the spatial coordinates of objects, the position and movement of their boundaries when using computer vision systems as part of geodetic automated measuring systems. In addition, the use of this model makes it possible to optimize the process of formulating technical requirements for video or cameras and their relative position in the course of measurements.

The angular elements of external orientation characterize the position of a shooting camera relative to the coordinate system in which the spatial coordinates of the points of the object under study are determined from the processing of its images. In many cases of aerial photography, e.g. shooting from an unmanned aerial vehicle, as well as in the case of space survey, the values of the orientation angles could be very significant. This paper presents the results of numerical experiments for five different systems of external orientation angles (Euler angles). The studies were performed using the condition of complanarity, which is the basis of space forward intersection. For a space forward intersection, a model of errors in determining spatial coordinates for five systems of shooting camera orientation angles has been developed. In the numerical experiments, the general case of aerial photography from an unmanned aerial vehicle and of space survey of the Earth were simulated. By comparing the root-mean-square errors (RMSE) in determining the spatial coordinates obtained using the studied systems of external orientation angles, the features of the use of these systems of orientation angles were revealed. The results of the research will allow to determine the spatial coordinates of the points of the studied objects with a greater degree of reliability by photogrammetry methods.

Humans, and several non-human species, possess the ability to make approximate but reliable estimates of the number of objects around them. Alike other perceptual features, numerosity perception is susceptible to adaptation: exposure to a high number of items causes underestimation of the numerosity of a subsequent set of items, and vice versa. Several studies have investigated adaptation in the auditory and visual modality, whereby stimuli are preferentially encoded in an external coordinate system. As tactile stimuli are primarily coded in an internal (body-centered) reference frame, here we ask whether tactile numerosity adaptation operates based on internal or external spatial coordinates as it occurs in vision or audition. Twenty participants performed an adaptation task with their right hand located either in the right (uncrossed) or left (crossed) hemispace, in order for the two hands to occupy either two completely different positions, or the same position in space, respectively. Tactile adaptor and test stimuli were passively delivered either to the same (adapted) or different (non-adapted) hands. Our results show a clear signature of tactile numerosity adaptation aftereffects with a pattern of over- and under-estimation according to the adaptation rate (low and high, respectively). In the uncrossed position, we observed stronger adaptation effects when adaptor and test stimuli were delivered to the "adapted" hand. However, when both hands were aligned in the same spatial position (crossed condition), the magnitude of adaptation was similar irrespective of which hand received adaptor and test stimuli. These results demonstrate that numerosity information is automatically coded in external coordinates even in the tactile modality, suggesting that such a spatial reference frame is an intrinsic property of numerosity processing irrespective of the sensory modality.

In recent years, human motion detection and analysis has been a research hotspot in the field of machine vision. It has been widely used in sports fitness, sports rehabilitation, sports analysis and virtual reality. Most of the existing human motion detection systems are based on optical equipment using retroreflective markers, which have many defects such as high deployment cost and complex operation. To this end, this article develops a household fitness motion evaluation system based on ToF camera with a mobile phone. The system firstly uses the ToF camera to collect the 3D human body point cloud information. Then Huawei augmented reality engine is employed to track the human skeleton. As all the spatial coordinates of joints are acquired, virtual human model can be reconstructed remotely on the phone. The mobile phone, working as the edge node between the person and the cloud server, transmits the captured human motion model parameters to the cloud platform. Then a hierarchical co-occurrence neural network recognizes and scores the detected user actions online. The experimental results show that the system proposed in this paper is portable and inexpensive. Real-time capabilities and measurement accuracy can also satisfy the requirements of evaluation for physical movements in sports rehabilitation.

The problem of detection and trajectory tracking of a group of small objects by a passive vision system is solved. The system consists of several space-oriented optical receivers that monitor the viewing area is considered. We propose an approach based on the distribution of optical receivers over stereo pairs, taking into account the orthogonality of the lines of sight on objects, and the redistribution of stereo pairs in the case of receiver failures to solve this problem. An appropriate algorithm has been developed to improve the reliability of the system, the probability of detecting all objects, and the accuracy of determining spatial coordinates. The results of the experimental examination of the developed algorithm are presented.

Inexperienced drivers always suffer from limited spatial information coming from side and review mirrors to complete parking tasks. The major obstacle is that they cannot easily estimate relative position of the parking space with respect to their vehicles. Therefore, this paper aims to develop a parking assistance display system that can continuously provide the top view of both the vehicle and the parking space for drivers. The system applies two wide-angle cameras mounted at the rear of the vehicle. In order to search for two farther corners of the parking space with efficiency, the FAST corner detection technique is employed. Three dimensional spatial coordinates of those corners can therefore be determined by the stereo vision framework. As a result, the position of the parking space relative to the vehicle can be estimated. To verify the effectiveness of the proposed parking assistance display, actual parking experiments with a golf cart were conducted. Experimental results demonstrate the parking tasks can be successfully accomplished with the help from the presented assistance display.

With the extensive development and utilization of urban underground space, coal mines, and other indoor areas, the indoor positioning technology of these areas has become a hot research topic. This paper proposes a robust localization method for indoor mobile platforms. Firstly, a series of coding graphics were designed for localizing the platform, and the spatial coordinates of these coding graphics were calculated by using a new method proposed in this paper. Secondly, two spatial resection models were constructed based on unit weight and Tukey weight to localize the platform in indoor environments. Lastly, the experimental results show that both models can calculate the position of the platform with good accuracy. The space resection model based on Tukey weight correctly identified the residuals of the observations for calculating the weights to obtain robust positioning results and has a high positioning accuracy. The navigation and positioning method proposed in this study has a high localization accuracy and can be potentially used in localizing practical indoor space mobile platforms.

Magnetic Resonance Imaging (MRI) has long been considered to be among the gold standards of today’s diagnostic imaging. The most significant drawback of MRI is long acquisition times, prohibiting its use in standard practice for some applications. Compressed sensing (CS) proposes to subsample the k-space (the Fourier domain dual to the physical space of spatial coordinates) leading to significantly accelerated acquisition. However, the benefit of compressed sensing has not been fully exploited; most of the sampling densities obtained through CS do not produce a trajectory that obeys the stringent constraints of the MRI machine imposed in practice. Inspired by recent success of deep learning-based approaches for image reconstruction and ideas from computational imaging on learning-based design of imaging systems, we introduce 3D FLAT, a novel protocol for data-driven design of 3D non-Cartesian accelerated trajectories in MRI. Our proposal leverages the entire 3D k-space to simultaneously learn a physically feasible acquisition trajectory with a reconstruction method. Experimental results, performed as a proof-of-concept, suggest that 3D FLAT achieves higher image quality for a given readout time compared to standard trajectories such as radial, stack-of-stars, or 2D learned trajectories (trajectories that evolve only in the 2D plane while fully sampling along the third dimension). Furthermore, we demonstrate evidence supporting the significant benefit of performing MRI acquisitions using non-Cartesian 3D trajectories over 2D non-Cartesian trajectories acquired slice-wise.

With the development of human-computer interaction technology, the interactive experience between human and machine becomes more and more simple and natural. The ideal state of human-computer interaction can be operated by natural devices such as voice, gestures and expressions, which have been tried and applied in the Kinect field. Kinect, as a new generation of motion sensing device, can achieve the input function by capturing the user’s motion and speech recognition in real time, so as to control the movement of the robot and effectively improve its intelligence. The research of this paper mainly includes skeleton tracking and robot control. Kinect1 generation was used to obtain the human body depth and skeleton images. VS software was used for programming. OpenNI and Kinect SDK libraries were called to identify the human arm joints and obtain the spatial coordinates. The robot controller USES the STMF series 32-bit processor, accepts serial port data, controls the robot arm steering gear, and completes the human body movement imitation. The motion trajectory is smooth and stable, and the motion effect is good.

An image may be defined as a two-dimensional function f(x,y) where x and y represent the spatial coordinates and the function (f) represents the amplitude at any given pair of coordinates (x,y). The amplitude (f) is also often referred to as the gray level or the intensity of that point in the image. A digital image is thus composed of a finite number of these x, y elements where all of them have an exact location and value. These picture elements are often referred to as pixels. 3D images may subsequently be defined by a three-dimensional function (x,y,z) and the individual elements are often referred to as voxels. The image quality is an important parameter in the field of radiology and the term spatial resolution refers to the capability of differentiating two neighboring objects in the image. Another important resolution property is the temporal resolution, which refers to the quality of the imaging with respect to time.

Three-dimensional (3D) scanning, 3D modelling and 3D printing are disrupting healthcare with the potential to create personalised, automated and customised treatments that are superior to current approaches. One application of these technologies is the production of ear and facial prosthetics to restore aesthetics in the case of congenital defects, trauma and tumour resection. Several studies have recently investigated the potential of this approach to increase patient comfort and outcomes while decreasing labour time for prosthetists. One challenge with translating these new technologies to clinical use is the expensive cost of equipment and software and the additional training and skill required to use them. Recent studies have shown the potential for a scanning technique called photogrammetry to be used with smartphones in place of expensive industrial 3D scanners to acquire 3D craniofacial models of both adults1 and infants.2 Photogrammetry works by detecting common features within a series of photographs taken from different locations to compute the 3D spatial coordinates of each camera. Following this, patient's 3D surface anatomy can be calculated. Although previous studies demonstrate the ease and accessibility of scanning with smartphones, they have not quantitatively assessed the accuracy. In this short communication, we investigate the potential of low-cost devices for 3D scanning of the external ear, and provide a quantitative comparison against a metrology rated 3D scanner. It is hoped the photogrammetry approach to 3D scanning can be used to help children born with microtia, a congenital condition which affects the formation of the external ear.

In order to solve the problem of the low efficiency and precision of manual eye inspection of vehicle dashboard, a method of dashboard image location based on machine vision is proposed. The relationship between spatial coordinates and pixel coordinates is established through nine-point calibration. In this coordinate system, PatQuick algorithm is used to roughly locate the dashboard. After that the image is marginalized by the Sobel Operator. Then the Blob algorithm is used to precisely locate the scale lines. At last, liquid crystal display character is located by combining PatMax algorithm and Blob algorithm. This method can locate the graph information accurately, reduce the deviation of dashboard display, improve automobile quality inspection rate and have important significance for the practical application of the image positioning of the automobile dashboard.

In this paper, we present an object detection and pose estimation algorithm that runs on low-cost and limited-processing hardware. The algorithm relies on two main steps: in the detection phase, it employs a Convolutional Neural Network (CNN) – namely MobileNet SSD – to recognize and track objects of interest in the scene; while in the pose estimation phase, the algorithm uses stereo correspondences to 3D reconstruct the spatial coordinates of multiple ORB features in the bounding box of the recognized object. The final position of the object in space is computed using a weighted average of the same spatial coordinates of the stereo-corresponded key points, where the weights are exponentially proportional to the level of ORB stereo matching. This algorithm was deployed on embedded systems and it achieved comparable results with respect to state-of-art, GPU-dependent, deep-learning algorithms for object pose estimation. We tested our approach in a calibrated environment where the object displacement (in either X or Y directions) can be easily measured. We used objects from the Yale-CMU-Berkeley (YCB) dataset to compare our estimation results with a deep learning based method, but our approach is not limited to this object selection. In the end, we achieved an estimated error of less than 10 mm. These results demonstrate that our approach can be used in affordable embedded systems for assistive technology in tasks such as for control of robotic arms for pick-and-place.

In this paper, we propose a method for the structured-light field (SLF) 3D measurement, involving ray calibration and phase mapping. The ray calibration is carried out to determine each light ray with metric spatio-angular parameters. Base on the ray parametric equation, the phase mapping in the SLF is developed so that spatial coordinates could be directly mapped from phase-encoding information. Then, a calibration strategy is designed to determine the mapping coefficients for each light ray, achieving high-efficiency SLF 3D reconstruction.

Problem statement: Optical-electronic systems can be used to control the elements of large objects such as radio telescope mirrors, floating docks, bridges etc. Often, large objects have smart electromechanical systems (SEMS) to adjust the position of their elements in space. The processing unit uses reference marks to calculate the spatial position of the object’s elements and gives commands to the adjustment mechanisms to set the elements at a given position. An external force may be exerted on the measuring system and the system may rotate about an arbitrary axis. Therefore, the spatial coordinates will be calculated with an error. Purpose of research: Development of an algorithm to compensate for the random rotation of the measuring system. Results: Algorithm for finding the direction of the rotation axis and the rotation angle of the measuring system is proposed. The results of calculating the spatial coordinates on the theoretical and experimental models were compared. The developed algorithm in combination with the optical scheme allows to create measuring systems with selective invariance. Practical significance: The developed algorithm allows to increase the accuracy of measuring the spatial coordinates of the object. The proposed optical-electronic system can be used together with such systems as SEMS and automatic control systems.

This paper describes the concept and methods of space-time processing of radio images of small objects in a multiposition radiometer system, which make it possible to increase the performance of this system. The spatial processing is carried out in each overlook cycle of scanning radiometers in order to determine the spatial coordinates of objects and mutual orientation of coordinate systems. The time processing is carried out in a sequence of overlook cycles of radiometers for determining the trajectory of the objects that change mutual positions.

Magnetic resonance imaging (MRI) is widely used in study of maxillofacial structures. While MRI is the modality of choice for soft tissues, it fails to capture hard tissues such as bone and teeth. Virtual dental models, acquired by optical 3D scanners, are becoming more accessible for dental practice and are starting to replace the conventional dental impressions. The goal of this research is to fuse the high-resolution 3D dental models with MRI to enhance the value of imaging for applications where detailed analysis of maxillofacial structures are needed such as patient examination, surgical planning, and modeling. A subject-specific dental attachment was digitally designed and 3D printed based on the subject’s face width and dental anatomy. The attachment contained 19 semi-ellipsoidal concavities in predetermined positions where oil-based ellipsoidal fiducial markers were later placed. The MRI was acquired while the subject bit on the dental attachment. The spatial position of the center of mass of each fiducial in the resultant MR Image was calculated by averaging its voxels’ spatial coordinates. The rigid transformation to fuse dental models to MRI was calculated based on the least squares mapping of corresponding fiducials and solved via singular-value decomposition. The target registration error (TRE) of the proposed fusion process, calculated in a leave-one-fiducial-out fashion, was estimated at 0.49 mm. The results suggest that 6–9 fiducials suffice to achieve a TRE of equal to half the MRI voxel size. Ellipsoidal oil-based fiducials produce distinguishable intensities in MRI and can be used as registration fiducials. The achieved accuracy of the proposed approach is sufficient to leverage the merged 3D dental models with the MRI data for a finer analysis of the maxillofacial structures where complete geometry models are needed.

A novel method for supervisory control of multilink manipulators mounted on underwater vehicles is considered. This method is designed to significantly increase the level of automation of manipulative operations, by the building of motion trajectories for a manipulator working tool along the surfaces of work objects on the basis of target indications given by the operator. This is achieved as follows: The operator targets the camera (with changeable spatial orientation of optical axis) mounted on the vehicle at the work object, and uses it to set one or more working point on the selected object. The geometric shape of the object in the work area is determined using clouds of points obtained from the technical vision system. Depending on the manipulative task set, the spatial motion trajectories and the orientation of the manipulator working tool are automatically set using the spatial coordinates of these points lying on the work object surfaces. The designed method was implemented in the C++ programming language. A graphical interface has also been created that provides rapid testing of the accuracy of overlaying the planned trajectories on the mathematically described surface of a work object. Supervisory control of an underwater manipulator was successfully simulated in the V-REP environment.

This article mainly designs an efficient fruit picking robot that combines rigidity and flexibility. The design of the robot adopts a mechanical arm with 5 degrees of freedom for the mecanum wheel. The mechanical arm is equipped with a gripper made using artificial muscle technology. The ZED2 binocular vision system is used to obtain spatial coordinates to realize the positioning and positioning of the mechanical arm. Video image acquisition. In order to realize the automatic recognition and picking of strawberries, this paper creates a strawberry training set, and uses the yolov3 algorithm to train the data and obtains a strawberry recognition model, which has a recall rate of larger strawberries and immature strawberries and the precision rate reached 90%, and the detection precision and precision rate of small target strawberries reached more than 80%. It can Meet the requirements of picking in real life.

In this paper, the method of spatial information extraction of tree branch was studied. The region matching method was used to get the disparity map of stereo image, extracted feature points combining with branch skeleton image by multi-segment approximation method, and calculated the spatial coordinates and the radius of branch feature points by using binocular stereo vision. Real-time model reconstruction for fruit tree has been researched on. Test proposed that each branch module was constructed by 12-prism in the coordinate origin, and then rotated twice and translated once to get correct posture, finally combined with other modules for the fruit tree model. Test has optimized extraction algorithm and matching algorithm of the branch region, improved matching rate, reduced matching errors, avoided matching confusion, accurately extracted branch spatial information and improved the success rate of robot path planning for obstacle avoidance.

Grapes are likely to have collisions and be damaged by manipulations when harvesting grape clusters. To conduct an undamaged robotic harvesting, this paper focuses mainly on locating the spatial coordinates of the cutting points on a peduncle of grape clusters for the end-effector and determining the bounding volume of the grape clusters for the motion planner of the manipulator. A method for acquiring spatial information from grape clusters is presented based on binocular stereo vision. This method includes four steps: (1) calibrating the binocular cameras and rectifying the images, (2) detecting the cutting points on the peduncle and the centres of the grape berries, (3) extracting three-dimensional spatial coordinates of the points detected in step 2, and (4) calculating the bounding volume of the grape clusters. A total of 300 images were captured in the vineyard and were tested to validate the method for the cutting point detection, and the success rate was approximately 87%. The accuracy of the localisation of the cutting points was determined under outdoor conditions, and the accuracy in the Z and X directions was 12 mm and 9 mm, respectively. The acquired bounding volume of the grape cluster was compared with manual measurements, and errors in the height and maximum diameter were less than 17 mm and 19 mm, respectively. The elapsed time of the whole algorithm was less than 0.7 s. The demonstrated performance of this developed method indicated that it could be used on harvesting robots.

Humans can only interact with part of the surrounding environment due to biological restrictions. Therefore, we learn to reason the spatial relationships across a series of observations to piece together the surrounding environment. Inspired by such behavior and the fact that machines also have computational constraints, we propose \underline{CO}nditional \underline{CO}ordinate GAN (COCO-GAN) of which the generator generates images by parts based on their spatial coordinates as the condition. On the other hand, the discriminator learns to justify realism across multiple assembled patches by global coherence, local appearance, and edge-crossing continuity. Despite the full images are never generated during training, we show that COCO-GAN can produce \textbf{state-of-the-art-quality} full images during inference. We further demonstrate a variety of novel applications enabled by teaching the network to be aware of coordinates. First, we perform extrapolation to the learned coordinate manifold and generate off-the-boundary patches. Combining with the originally generated full image, COCO-GAN can produce images that are larger than training samples, which we called "beyond-boundary generation". We then showcase panorama generation within a cylindrical coordinate system that inherently preserves horizontally cyclic topology. On the computation side, COCO-GAN has a built-in divide-and-conquer paradigm that reduces memory requisition during training and inference, provides high-parallelism, and can generate parts of images on-demand., Accepted to ICCV'19 (oral). All images are compressed due to size limit, please access the full-resolution version via Google Drive: http://bit.ly/COCO-GAN-full

For the measurement of three-dimensional (3D) shapes, active optical measurement systems based on pattern projection are widely used. These sensors work without contact and are non-destructive. Between one camera and the projector or between two cameras, the 3D reconstruction is performed by detection and triangulation of corresponding image points. Recently, we developed a 3D stereo sensor working in the visible range of light (VIS). It consists of two highspeed cameras and a GOBO projection-based high-speed pattern projector. Our system allows us to successfully reconstruct 3D point clouds of fast processes such as athletes in motion or even crash tests. Simultaneously measuring the surface temperature would be of great benefit as fast processes usually exhibit local temperature changes. In order to include thermal data into the evaluation, we have extended our existing high-speed 3D sensor by including an additional high-speed long-wave infrared (LWIR) camera. The thermal camera detects radiation in the spectral range between 7.5 and 12 μm. We map the measured temperatures as texture onto the reconstructed 3D points. In this contribution, we present the design of this novel 5D (three spatial coordinates, temperature, and time) sensor. The simultaneous calibration process of the VIS cameras and the LWIR camera in a common coordinate system is described. First promising measurements of an inflating airbag, a basketball player, and the crushing of a metal tube performed at a frame rate of 1 kHz are shown. Keywords: high-speed visible (VIS) and infrared (IR) cameras, 3D shape measurement, multimodal sensor, GOBO projection, aperiodic sinusoidal patterns, temperature mapping

When drivers interact with the car, their hands will leave the steering wheel, causing great safety risks. In order to solve this problem, this paper proposed a natural interaction method based on driver's head-eye moving behaviors by capturing driver's image and detecting multi-feature in driver's face. Firstly, 2D images of the driver's head and 3D point cloud data are collected through the infrared camera installed in front of the driver in the car. By using the SDM minimization nonlinear least square function to detect the human face in 2D infrared images, the features of human face are obtained. And the eye area images are obtained by using the feature point coordinates. The C-V extension model based on feature search was used to extract the pupil and the feature points of the Purkinje image in the eye image. Meanwhile, the spatial coordinates of the points in the three-dimensional point cloud were transformed by using the index of facial feature points to obtain the driver's head posture. Finally, the decision tree model is used to analyze the obtained eye feature vectors and head posture data, by this way, mapping these head-eye behaviors to the gazing object, and driver can use it to interaction with car. The experimental results show that the proposed method in this paper can accurately and quickly identify the driver's interaction intentions on the vehicle.

If a gabor pattern drifts in one direction while its internal texture drifts in the orthogonal direction, observers see a remarkable shift in its perceived direction when it is viewed in the periphery. The reported direction of the double-drift stimulus (also known as the infinite regress and curveball illusions) is some combination of the actual external motion of the gabor envelope and the internal motion of its texture (Tse & Hsieh, 2006). Here we find that if the observers track a fixation point that moves in tandem with the gabor, the illusion is undiminished. The pursuit of the moving fixation spot keeps the gabor roughly fixed at one location on the retina, cancelling its external motion, leaving only the internal motion. The gabor is seen to move in the world at roughly its actual speed as the motion of the eye is discounted at some point to recover velocities in world coordinates (e.g. Wallach, 1959). Our finding indicates that the combination of internal and external motion that produces the double drift illusion must happen after the eye movement signals have been factored into stimulus motions. We also test the double drift effect at various path lengths, durations, and speeds, with both mid-grey and black backgrounds, all with a static fixation. These results confirm that a simple vector combination of the two speeds alone accounts for virtually all the direction shifts on the grey background. On the black background, the illusion is eliminated. These results place constraints on where perceived spatial coordinates arise in the visual processing hierarchy to locations at or beyond where compensation for pursuit eye movements arise, specifically V3A, V6, MSTd, and VIP (e.g., Nau et al, 2018).

This paper elucidates a procedure for real time detection and tracking of a moving model car in the sequence of image frames acquired by a static camera. The vehicle tracking is performed by exploiting the modelling behavior of Kalman filter. The raw image acquisition, background subtraction, detection of vehicle in image frames is performed using image processing techniques. The vehicle tracking is performed by employing the modelling efficiency of a traditional Kalman filter. The raw image acquisition, background subtraction, to detect vehicle in image frames is performed using image processing techniques. The vehicle parameter estimation is further performed using the Kalman filter. This system takes raw image of two dimensional (2D) drivable surface with car as input, and outputs the spatial coordinates of estimated car position. The system was implemented on a ‘VEEROBOT’ robotic car in the laboratory environment. Real time images were taken using a ‘Logitech’ webcam, and the relevant processing is carried out using ‘Raspberry pi 3’. The analysis of experimental results between the measured and estimated position of the car is presented in this paper.

At present, in some real-time occasion like as stage performances, mixed reality effects are often generated by a mechanical means, or only between physical objects and virtual objects. This will lead to many uncontrollable shortcomings. in order to overcome the above disadvantage, the authors developed a mixed reality system for physical interaction. The experiment unfolds between the experiencer and the controllable Four-axis unmanned aerial vehicle with some performance props. The first step of the study is to build the stereoscopic scene through the camera calibration, the target coordinates in image are obtained by infrared sensors deployed on the experimenter and the aircraft respectively by Camshift algorithm and it can be transformed into the spatial coordinates by projection transformation. And the second step is based on different preset working modes and the coordinate relationship expressions between them that have been configured, with the action of the experimenter, Ant Colony Algorithm is used to move the Four-axis UAV to a certain position, thereby realizing this kind of precisely controllable interaction between entities. The final experiment in this paper has proven that our work provided a great mixed reality effect for occasions such as stage performances.

We propose a deep learning tool to localize fish objects in benthic underwater videos on a frame by frame basis. The deep network predicts fish object spatial coordinates and simultaneously segments the corresponding pixels of each fish object. The network follows a state of the art inception resnet v2 architecture that automatically generates informative features for object localization and mask segmentation tasks. Predicted masks are passed to dense Conditional Random Field (CRF) post-processing for contour and shape refinement. Unlike prior methods that rely on motion information to segment fish objects, our proposed method only requires RGB video frames to predict both box coordinates and object pixel masks. Independence from motion information makes our proposed model more robust to camera movements or jitters, and makes it more applicable to process underwater videos taken from unmanned water vehicles. We test the model in actual benthic underwater video frames taken from ten different sites. The proposed tool can segment fish objects despite wide camera movements, blurred underwater resolutions, and is robust to a wide variety of environments and fish species shapes.

Depth perception requires the use of an internal model of the eye-head geometry to infer distance from binocular retinal images and extraretinal 3D eye-head information, particularly ocular vergence. Similarly, for motion in depth perception, gaze angle is required to correctly interpret the spatial direction of motion from retinal images; however, it is unknown whether the brain can make adequate use of extraretinal version and vergence information to correctly transform binocular retinal motion into 3D spatial coordinates. Here we tested this hypothesis by asking participants to reconstruct the spatial trajectory of an isolated disparity stimulus moving in depth either peri-foveally or peripherally while participants' gaze was oriented at different vergence and version angles. We found large systematic errors in the perceived motion trajectory that reflected an intermediate reference frame between a purely retinal interpretation of binocular retinal motion (not accounting for veridical vergence and version) and the spatially correct motion. We quantify these errors with a 3D reference frame model accounting for target, eye, and head position upon motion percept encoding. This model could capture the behavior well, revealing that participants tended to underestimate their version by up to 17%, overestimate their vergence by up to 22%, and underestimate the overall change in retinal disparity by up to 64%, and that the use of extraretinal information depended on retinal eccentricity. Since such large perceptual errors are not observed in everyday viewing, we suggest that both monocular retinal cues and binocular extraretinal signals are required for accurate real-world motion in depth perception.

Nowadays, the technology of using computer vision to calibrate objects is widely used, which has a huge market demand in many fields. This paper provides a calibration method of computer vision system based on dual attention neural network. This paper uses the camera to simulate human eyes to obtain three-dimensional images. After obtaining the three-dimensional images, the images are input into the Residual Network (ResNet) model, and the weight of ResNet is repeatedly updated so as to accurately identify the images. On this basis, introduces dual attention mechanism that an algorithm is used in natural language to the visual image processing, using multistage feature extraction method to extract the three-dimensional image for each characteristic of regional. After extracting the feature area, the accuracy of the feature area is constantly updated to the minimum. Besides, the feature areas are brought into the calibration algorithm of Zhang Zhengyou system to obtain the spatial coordinates of the objects in the attention area. This method can realize the space position calibration of specific objects under various complex backgrounds and calculate the distance from the calibrated objects, which can not only calibrate the system but also identify it, and greatly improve the reliability and accuracy of the calibration process.

Moving objects gathering pattern represents a group events or incidents that involve congregation of moving objects, enabling the analysis of traffic system. However, effectively and efficiently discovering the specific gathering pattern turns to be a remaining challenging issue since the large number of moving objects will generate high volume of trajectory data. In order to address this issue, the authors propose a moving object gathering pattern retrieving method that aims to support the retrieving of gathering patterns based on spatio-temporal graph. In this method, firstly the authors use an improved R-tree based density clustering algorithm (RT-DBScan) to index the moving objects and collect clusters. Then, they maintain a spatio-temporal graph rather than storing the spatial coordinates to obtain the spatio-temporal changes in real time. Finally, a gathering retrieving algorithm is developed by searching the maximal complete graphs which meet the spatio-temporal constraints. To the best of their knowledge, effectiveness and efficiency of the proposed methods are outperformed other methods on both real and large trajectory data.

In this paper, a methodology for the calculation of rough exterior orientation (EO) parameters of multiple large-scale overlapping oblique aerial images, in the case that GPS/INS information is not available (e.g., for old datasets), is presented. It consists of five main steps; (a) the determination of the overlapping image pairs and the single image in which four ground control points have to be measured; (b) the computation of the transformation parameters from every image to the coordinate reference system; (c) the rough estimation of the camera interior orientation parameters; (d) the estimation of the true horizon line and the nadir point of each image; (e) the calculation of the rough EO parameters of each image. A developed software suite implementing the proposed methodology is tested using a set of UAV multi-perspective oblique aerial images. Several tests are performed for the assessment of the errors and show that the estimated EO parameters can be used either as initial approximations for a bundle adjustment procedure or as rough georeferencing information for several applications, like 3D modelling, even by non-photogrammetrists, because of the minimal user intervention needed. Finally, comparisons with a commercial software are made, in terms of automation and correctness of the computed EO parameters.