Search

This work presents Pictory, a mobile app that empowers users to transform photos into artistic renditions by using a combination of neural style transfer with user-controlled state-of-the-art nonlinear image filtering. The combined approach features merits of both artistic rendering paradigms: deep convolutional neural networks can be used to transfer style characteristics at a global scale, while image filtering is able to simulate phenomena of artistic media at a local scale. Thereby, the proposed app implements an interactive two-stage process: first, style presets based on pre-trained feed-forward neural networks are applied using GPU-accelerated compute shaders to obtain initial results. Second, the intermediate output is stylized via oil paint, watercolor, or toon filtering to inject characteristics of traditional painting media such as pigment dispersion (watercolor) as well as soft color blendings (oil paint), and to filter artifacts such as fine-scale noise. Finally, on-screen painting facilitates pixel-precise creative control over the filtering stage, e. g., to vary the brush and color transfer, while joint bilateral upsampling enables outputs at full image resolution suited for printing on real canvas.

Oblique is an iPhone app that allows users to apply real-time filtration and adjustments as they shoot photographs through a simple interface. Created by an artist who regularly combines photography and code, Oblique was born out of the desire to eliminate post-production and create dramatic images in real-time out in the world without having to adjust later. Oblique is an artist's studio within an app that alters the way users see and experience the world. Oblique includes dramatic filters and subtle adjustments that allow users to make slight brightness, contrast and brightness adjustments. A wide array of kaleidoscopic filters can be controlled and manipulated directly by touching a live view of the camera feed.

To use the advanced content creation functions of a game engine and develop contents in which displaying real-time integral photography images is important, we implemented multi-viewpoint rendering and IP image synthesis functions by adding a shader and C# scripts to the game engine.

In this work, we have developed an approach to include any cross-hatching technique into any rendering system with global illumination effects (see Figure 1). Our new approach provide a robust computation to obtain hand-drawn effects for a wide variety of diffuse and specular materials. Our contributions can be summarized as follows: (1) A Barycentric shader that can provide generalized cross-hatching with multi-textures; and (2) A texture synthesis method that can automatically produce crosshatching textures from any given image.

Ray-traced contours are inherently challenged by highly detailed geometry, as commonly found in organic shapes. Existing contour methods cannot reflect such complexity in an artistically pleasing way (Figure 1 A), and animations are prone to flicker.After a brief explanation of contour generation and its inherent challenges, this talk presents a novel approach to rendering aesthetic and flicker-free contours on highly detailed geometry (Figure 1 B). The new method employs sub-pixel-level sub-sampling to achieve a high level of detail quality, and supports contours in transparency, reflection and refraction.The implementation uses mental ray (Unified Sampling mode) but could be realized in other ray tracing renderers as well.

In this manuscript, we describe a process that can be used to create still and/or animated portrait paintings to be shown in Expressive Art Exhibit. Our process consists of two stages: (1) Creation of control textures for a Barycentric shader by using color information gathered from photographs to provide realistic looking skin rendering; (2) Filtering and compositing the layers of images that are obtained by control textures, which correspond to effects such as diffuse, specular and ambient. To demonstrate proof-of-concept, we have created a few rigid body animations of painterly portraits under different lighting conditions.

This paper presents MoonRay, a high performance production rendering architecture using Monte Carlo path tracing developed at DreamWorks Animation. MoonRay is the first production path tracer, to our knowledge, designed to fully leverage Single Instruction/Multiple Data (SIMD) vector units throughout. To achieve high SIMD efficiency, we employ Embree for tracing rays and vectorize the remaining compute intensive components of the renderer: the integrator, the shading system and shaders, and the texturing engine. Queuing is used to help keep all vector lanes full and improve data coherency. We use the ISPC programming language [Intel 2011; Pharr and Mark 2012] to achieve improved performance across SSE, AVX/AVX2 and AVX512 instruction sets. Our system includes two functionally equivalent uni-directional CPU path tracing implementations: a C++ scalar depth-first version and an ISPC vectorized breadth-first wavefront version. Using side by side performance comparisons on complex production scenes and assets we show our vectorized architecture, running on AVX2, delivers between a 1.3× to 2.3× speed-up in overall render time, and up to 3×, 6×, and 4×, speed-ups within the integration, shading, and texturing components, respectively.

Treball Final de Grau en Disseny i Desenvolupament de Videojocs The search for graphic quality in Unity is a complicated task to get the goal of achieving a hyperrealistic visual quality. For this, I have design an interactive demonstration called La Torre, where we can control a character in third person, demonstrating the potential of Unity to work with hyperrealistic graphics both in cutscenes sequences and during the gameplay. It will be possible to cross the main stage and the player will have a certain freedom to explore and observe the breadth of detail of the scene, with an optimized configuration in Unity to support this graphic level during the whole demo. This document intends to indicate the process to achieve this graphic quality showing the proper use and, in order, each one of the different resources presented by the engine such as shaders, lighting or FX and Post-effects; those allow to achieve these graphics maintaining a good performance throughout the demo. And the creative process through which has been designed, modeled, sculpted, animated and textured every element that can be seen during the demo, such as the protagonist character Baltor, or the architecture and furniture that covers the scene.

Although spherical displays become more and more pervasive, the design of interactive content for these displays still remains difficult as it requires most developers to get familiar with specific tools for managing the output and input. In this paper, we present a framework for developing applications for multitouch spherical displays that makes it possible to create interactive content by programming standard GUI applications, as for example interactive web pages. The principal idea is to adapt the window output and interaction input of classical GUIs outside the application. To this end, our framework consists of two standalone applications where the first one captures the window output and changes the projection via GPU shaders, and the second one adapts the input with a Node.js server and sends interaction and mouse events. In this way, the same application runs on a standard desktop, and on the spherical display. Advantages of our approach include fast prototyping, and the fact that masses of developers can create applications for spherical displays just as if it were, for example, classical web applications. We believe that our framework will contribute to making spherical displays even more pervasive in the future.

Gaining a better understanding of the human brain continues to be one of the greatest challenges for science, largely because of the overwhelming complexity of the brain and the difficulty of analyzing the features and behavior of dense neural networks. Regarding analysis, 3D visualization has proven to be a useful tool for the evaluation of complex systems. However, the large number of neurons in non-trivial circuits, together with their intricate geometry, makes the visualization of a neuronal scenario an extremely challenging computational problem. Previous work in this area dealt with the generation of 3D polygonal meshes that approximated the cells’ overall anatomy but did not attempt to deal with the extremely high storage and computational cost required to manage a complex scene. This paper presents NeuroTessMesh, a tool specifically designed to cope with many of the problems associated with the visualization of neural circuits that are comprised of large numbers of cells. In addition, this method facilitates the recovery and visualization of the 3D geometry of cells included in databases, such as NeuroMorpho, and provides the tools needed to approximate missing information such as the soma’s morphology. This method takes as its only input the available compact, yet incomplete, morphological tracings of the cells as acquired by neuroscientists. It uses a multiresolution approach that combines an initial, coarse mesh generation with subsequent on-the-fly adaptive mesh refinement stages using tessellation shaders. For the coarse mesh generation, a novel approach, based on the Finite Element Method, allows approximation of the 3D shape of the soma from its incomplete description. Subsequently, the adaptive refinement process performed in the graphic card generates meshes that provide good visual quality geometries at a reasonable computational cost, both in terms of memory and rendering time. All the described techniques have been integrated into NeuroTessMesh, available to the scientific community, to generate, visualize, and save the adaptive resolution meshes.

This research addresses a new approach for rendering colored light shafts using light texture of virtual environments in real time. Typically, most of the techniques have focused on tackling the artifact and aliasing of light shafts that arising because the sampling, however disregard of color effect to achieve the realism. This approach adopts ray marching using virtual planes of texture to compute light scattering along the view rays. The pixel shader is exploited the interaction between a light and the participating media by sampling of texture to display effects on screen space. The experiments results refer to achieve better appearance of colored light shafts phenomenon in high frame rates. This approach produces compelling light shafts and suitable can be used in interactive applications.

Presilicon simulation is one of the key toolsets for computer architects to evaluate and optimize their future designs. As Graphics Processing Units (GPUs) have become the platform of choice in many computing communities due to their impressive processing capabilities, computer architecture researchers need a simulation framework that allows them to quantitatively consider design tradeoffs. In this paper, we present the Multi2Sim Kepler simulator framework, a new detailed GPU microarchitecture performance simulator that supports NVIDIA's Kepler shader assembly (SASS) code execution. The toolset provides a disassembler, a functional simulator and a detailed cycle-based simulator. We provide insight into the architecture of the NVIDIA Kepler GPU, describing the details of the streaming multiprocessor, front-end and instruction pipelines. We compare the performance of this new simulator against an NVIDIA K20X, a high-end Kepler device. We also evaluate the performance of NVIDIA's CUDA benchmark suite on our GPU performance simulator.

Light stimulation with precise and complex spatial and temporal modulation is demanded by a series of research fields like visual neuroscience, optogenetics, ophthalmology, and visual psychophysics. We developed a user-friendly and flexible stimulus generating framework (GEARS GPU-based Eye And Retina Stimulation Software), which offers access to GPU computing power, and allows interactive modification of stimulus parameters during experiments. Furthermore, it has built-in support for driving external equipment, as well as for synchronization tasks, via USB ports. The use of GEARS does not require elaborate programming skills. The necessary scripting is visually aided by an intuitive interface, while the details of the underlying software and hardware components remain hidden. Internally, the software is a C++/Python hybrid using OpenGL graphics. Computations are performed on the GPU, and are defined in the GLSL shading language. However, all GPU settings, including the GPU shader programs, are automatically generated by GEARS. This is configured through a method encountered in game programming, which allows high flexibility: stimuli are straightforwardly composed using a broad library of basic components. Stimulus rendering is implemented solely in C++, therefore intermediary libraries for interfacing could be omitted. This enables the program to perform computationally demanding tasks like en-masse random number generation or real-time image processing by local and global operations.

We introduce a new data structure and algorithms that employ it to compute real-time global illumination from static environments. Light field probes encode a scene's full light field and internal visibility. They extend current radiance and irradiance probe structures with per-texel visibility information similar to a G-buffer and variance shadow map. We apply ideas from screen-space and voxel cone tracing techniques to this data structure to efficiently sample radiance on world space rays, with correct visibility information, directly within pixel and compute shaders. From these primitives, we then design two GPU algorithms to efficiently gather real-time, viewer-dependent global illumination onto both static and dynamic objects. These algorithms make different tradeoffs between performance and accuracy. Supplemental GLSL source code is included.

Two-dimensional heightfields are the most common data structure used for storing and rendering of terrain in offline rendering and especially real-time computer graphics. By its very nature, a heightfield cannot store terrain structures with multiple vertical layers such as overhanging cliffs, caves, or arches. This restriction does not apply to volumetric data structures. However, the workflow of manual modelling and editing of volumetric terrain usually is tedious and very time-consuming. Therefore, we propose to use three-dimensional curve-based primitives to efficiently model prominent, large-scale terrain features. We present a technique for volumetric generation of a complete terrain surface from the sparse input data by means of diffusion-based algorithms. By combining an efficient, feature-based toolset with a volumetric terrain representation, the modelling workflow is accelerated and simplified while retaining the full artistic freedom of volumetric terrains. All stages of our method are GPU-accelerated using compute shaders to ensure interactive editing of terrain.

In this paper, we offer an efficient real-time video abstraction technique that enables interaction with the real-world environment using a smart device for combining augmented reality (AR) and virtual content. The conventional video abstraction technique is inappropriate for the real-time processing of AR content. Therefore, real-time video abstraction through the GPU approach is necessary to minimize the CPU processing of AR content. The methods other than the abstraction technique using the Open GL-ES shader have constraints regarding hardware and the dependency of AR content data. Consequently, this paper proposes a real-time video abstraction technique using the Open GL-ES shader. The proposed real-time video abstraction technique is a method for minimizing the disharmony between virtual and real-world content occurring on AR performing in the limited computing resources of a smart device. To do this, we performed research on video abstraction and applied three filtering algorithms (Sobel operator, Striped pattern-based toon, Non-boundary-based color quantization). In addition, we considered the applicability of AR content through real-time video abstraction experiments using adjusted and monitoring parameters.

Now that you have learned how to “skin” your 3D primitives using the JavaFX Phong shader algorithms and their various color and effects mapping channels and you have created colorful, highly optimized game board square texture maps, it’s time to add some more custom methods for building the game board and setting up the Phong shader objects with the texture maps. We’ll need to create a createGameBoardNodes() method to organize the 3D primitive assets that comprise our 3D game, as the createBoardGameNodes should (and does) contain higher-level Node subobject instantiation and configuration, such as the Scene, Root, UI StackPane, 3D gameBoard Group, Camera, and Lighting, as well as four game board Quadrant Group objects named Q1 through Q4 (quadrants 1 through 4). We will also create the other 19 game board square objects, named Q1S1 through Q1S5, Q2S1 through Q2S5, Q3S1 through Q3S5, and Q4S1 through Q4S5 to keep object names short. Naming the objects the abbreviated version of Quadrant1Square1 (Q1S1) will make the Java code that uses these abbreviated terms much more readable.

Pyramid methods are useful for certain image processing techniques due to their linear time complexity. Implementing them using compute shaders provides a basis for rendering image effects with reduced impact on performance compared to conventional methods. Although pyramid methods are used in the game industry, they are not easily accessible to all developers because many game engines do not include built-in support. We present a framework for a popular game engine that allows users to take advantage of pyramid methods for developing image effects. In order to evaluate the performance and to demonstrate the framework, a few image effects were implemented. These effects were compared to built-in effects of the same game engine. The results showed that the built-in image effects performed slightly better. The performance of our framework could potentially be improved through optimisation, mainly on the GPU.

During this chapter, you will be learning about the JavaFX 3D shader class hierarchy, which is contained in the javafx.scene.paint package. In Java 9 and Android, the Paint class applies pixel colors and attributes to the Canvas and in this case the surface of 3D primitives.

A deferred multi resolution rendering for direct illumination from area lights.A sub-fragment visibility test detects discontinuity within a fragment.A fragment is subdivided by analyzing its gradient of visibility discontinuities.Fragments are upsampled using Radial Basis Functions to interpolate scattered data.Our method generates fewer fragments as well as renders better and faster. Display Omitted Interactive rendering of direct illumination from area lights in virtual worlds has always proven to be challenging. In this paper, we propose a deferred multi-resolution approach for rendering direct illumination from area lights. Our approach subdivides the screenspace into multi-resolution 2D-fragments in which higher resolution fragments are generated and placed in regions with geometric, depth and visibility-to-light discontinuities. Compared to former techniques that use inter-fragment binary visibility test, our intra-fragment technique is able to detect shadow more efficiently while using fewer fragments. We also make use of gradient information across our binary visibility tests to further allocate higher resolution fragments to regions with larger visibility discontinuities. Our technique utilizes the stream-compaction feature of the transform feedback shader (TFS) in the graphics shading pipeline to filter out fragments in multiple streams for soft shadow refinement. The bindless texture extension in graphics pipeline allows us to easily process all these generated fragments in an unsorted manner. A single pass screenspace irradiance upsampling scheme which uses radial basis functions (RBF) with an adaptive variance scaling factor is proposed for interpolating the generated fragments. This reduces artifacts caused by large fragments and it also requires fewer fragments to produce reasonable results. Our technique does not require precomputations and is able to render diffuse materials at interactive rates.

In this chapter a revision of the traditional tracing methods over petroglyphs through the employment of three dimensional models is proposed. The different techniques suggested here are: Radiance Scaling and Algebraic Point Set Surfaces (APSS). Radiance Scaling is a shader that adjusts reflected light intensities in a way dependent on both surface curvature and material characteristics. While APSS allows the colorization of the vertices of a mesh or point set using the curvature of the underlying surface. These methods applied to 3D models allow a better visualisation, comprehension, and objectification of the open-air rock art carvings, improving the researches over a more reliable database, but also for issues related to management and conservation.

Finding the best suitable environment for 3D navigation that utilizes at least six degrees-of-freedom is still difficult. Furthermore, creating a system to procedurally generate a large virtual environment provides an opportunity for researchers to understand this problem further. Therefore, we present a novel technique to render a parametric celestial skybox with the ability to light environments similar to natural color corrected images from telescopes. We first pre-compute a spherical ray map that corresponds to the cubemap coordinates, then generate stars and dust through a combination of different noise generation shaders.

Now that you have created the multitiered gameBoard Group Node (subclass) hierarchy and tested it to see whether it rotates as if it were one 3D model, it is time to add a Sphere 3D primitive so we can create a 3D user interface element for the user to use to create random “spins” during gameplay. We will also set up the Phong shader objects for these and again use GIMP 2.8.22 to create (from scratch) the rest of your diffuse texture maps for the game board quadrants, as well as a 3D “spinner” UI texture map for a Sphere primitive. This spinner will be used on each player’s turn to randomly spin the gameboard to select the topic category. You will always need to differentiate your pro Java 9 game from others out there, so we’ll be unique and spin the gameboard itself to pick the quadrant (topic category), which cannot be done with real-life game boards but which can be done with virtual i3D game boards. We’ll add Java 9 code to the top of your class as well as in the createMaterials(), addNodesToSceneGraph(), and loadImageAssets() method bodies. We’ll create custom PNG24 diffuse textures to add to your project source (/src) folder. We’ll rearrange your createGameBoardNodes() method to reorganize the quadrant 3D primitives that we’ll work on during this chapter to complete the interior parts of the game board design. We’ll place the quadrants together at the top of this method.

Combining procedural generation and user control is a fundamental challenge for the interactive design of natural scenery. This is particularly true for modelling complex waterfall scenes where, in addition to taking charge of geometric details, an ideal tool should also provide a user with the freedom to shape the running streams and falls, while automatically maintaining physical plausibility in terms of flow network, embedding into the terrain, and visual aspects of the waterfalls. We present the first solution for the interactive procedural design of coherent waterfall scenes. Our system combines vectorial editing, where the user assembles elements to create a waterfall network over an existing terrain, with a procedural model that parametrizes these elements from hydraulic exchanges; enforces consistency between the terrain and the flow; and generates detailed geometry, animated textures and shaders for the waterfalls and their surroundings. The tool is interactive, yielding visual feedback after each edit.

Real-time rendering with true motion and defocus blur remains an elusive goal for application developers. In recent years, substantial progress has been made in the areas of rasterization, shading, and reconstruction for stochastic rendering. However, we have yet to see an efficient method for decoupled sampling that can be implemented on current or near-future graphics processors. In this paper, we propose one such algorithm that leverages the capability of modern GPUs to perform unordered memory accesses from within shaders. Our algorithm builds per-pixel primitive lists in canonical shading space. All shading then takes place in a single, non-multisampled forward rendering pass using conservative rasterization. This pass exploits the rasterization and shading hardware to perform shading very efficiently, and only samples that are visible in the final image are shaded. Last, the shading samples are gathered and filtered to create the final image. The input to our algorithm can be generated using a variety of methods, of which we show examples of interactive stochastic and interleaved rasterization, as well as ray tracing.

In this paper, we present a new automatic shader simplification method using surface signal approximation. We regard the entire multi-stage rendering pipeline as a process that generates signals on surfaces, and we formulate the simplification of the fragment shader as a global simplification problem across multi-shader stages. Three new shader simplification rules are proposed to solve the problem. First, the code transformation rule transforms fragment shader code to other shader stages in order to redistribute computations on pixels up to the level of geometry primitives. Second, the surface-wise approximation rule uses high-order polynomial basis functions on surfaces to approximate pixel-wise computations in the fragment shader. These approximations are pre-cached and simplify computations at runtime. Third, the surface subdivision rule tessellates surfaces into smaller patches. It combines with the previous two rules to approximate pixel-wise signals at different levels of tessellations with different computation times and visual errors. To evaluate simplified shaders using these simplification rules, we introduce a new cost model that includes the visual quality, rendering time and memory consumption. With these simplification rules and the cost model, we present an integrated shader simplification algorithm that is capable of automatically generating variants of simplified shaders and selecting a sequence of preferable shaders. Results show that the sequence of selected simplified shaders balance performance, accuracy and memory consumption well.

The self-organizing map (SOM) model is an effective technique applicable in a wide range of areas, such as pattern recognition and image processing. In the SOM model, the most time-consuming procedure is the training process. It consists of two time-consuming parts. The first part is the calculation of the Euclidean distances between training vectors and codevectors. The second part is the update of the codevectors with the pre-defined neighborhood structure. This paper proposes a graphics processing unit (GPU) algorithm that accelerates these two parts using the graphics rendering ability of GPUs. The distance calculation is implemented in the form of matrix multiplication with compute shader, while the update process is treated as a point-rendering process with vertex shader and fragment shader. Experimental results show that our algorithm runs much faster than previous CUDA implementations, especially for the large neighborhood case. Also, our method can handle the case with large codebook size and high dimensional data.

The main challenge in achieving good image morphs is to create a map that aligns corresponding image elements. Our aim is to help automate this often tedious task. We compute the map by optimizing the compatibility of corresponding warped image neighborhoods using an adaptation of structural similarity. The optimization is regularized by a thin-plate spline and may be guided by a few user-drawn points. We parameterize the map over a halfway domain and show that this representation offers many benefits. The map is able to treat the image pair symmetrically, model simple occlusions continuously, span partially overlapping images, and define extrapolated correspondences. Moreover, it enables direct evaluation of the morph in a pixel shader without mesh rasterization. We improve the morphs by optimizing quadratic motion paths and by seamlessly extending content beyond the image boundaries. We parallelize the algorithm on a GPU to achieve a responsive interface and demonstrate challenging morphs obtained with little effort.

Received 2 July 2014; received in revised form 27 July 2014; accepted 29 July 2014ABSTRACTThis paper presents a real-time rendering algorithm of large-scale geometric data using GLSL(OpenGL shading language). It details the VAO (vertex array object) and VBO(vertex bufferobject) to be used for up-loading the large-scale point clouds and polygon meshes to a graphicvideo memory, and describes the shader program composed by a vertex shader and a fragmentshader, which manipulates those large-scale data to be rendered by GPU. In addition, weexplain the global rendering procedure that creates and runs the shader program with the VAOand VBO. Finally, a rendering performance will be measured with application examples, fromwhich it will be demonstrated that the proposed algorithm enables a real-time rendering of largeamount of geometric data, almost impossible to carry out by previous techniques.Key Words: Fragment shader, GLSL, Large-scale point cloud, Real-time rendering, Shaderprogram, Vertex shader, Video memory

We propose a powerful hardware architecture for pixel shading, which enables flexible control of shading rates and automatic shading reuse between triangles in tessellated primitives. The main goal is efficient pixel shading for moderately to finely tessellated geometry, which is not handled well by current GPUs. Our method effectively decouples the cost of pixel shading from the geometric complexity. It thereby enables a wider use of tessellation and fine geometry, even at very limited power budgets. The core idea is to shade over small local grids in parametric patch space, and reuse shading for nearby samples. We also support the decomposition of shaders into multiple parts, which are shaded at different frequencies. Shading rates can be locally and adaptively controlled, in order to direct the computations to visually important areas and to provide performance scaling with a graceful degradation of quality. Another important benefit of shading in patch space is that it allows efficient rendering of distribution effects, which further closes the gap between real-time and offline rendering.

Shading can smooth the appearance of polygons by reducing the effects of sharp edges, which makes drawn graphics more realistic. Consider factors related to the object surface brightness, analyze and compare several shading algorithms, focus on presenting spherical linear interpolation of the Phong shading, and use OpenGL to realize the procedures and effect diagrams of Flat shading and Gouraud shading in VC + + environment.

Redundancy is at the heart of graphical applications. In fact, generating an animation typically involves the succession of extremely similar images. In terms of rendering these images, this behavior translates into the creation of many fragment programs with the exact same input data. We have measured this fragment redundancy for a set of commercial Android applications, and found that more than 40% of the fragments used in a frame have been already computed in a prior frame. In this paper we try to exploit this redundancy, using fragment memoization. Unfortunately, this is not an easy task as most of the redundancy exists across frames, rendering most HW based schemes unfeasible. We thus first take a step back and try to analyze the temporal locality of the redundant fragments, their complexity, and the number of inputs typically seen in fragment programs. The result of our analysis is a task level memoization scheme, that easily outperforms the current state-of-the-art in low power GPUs More specifically, our experimental results show that our scheme is able to remove 59.7% of the redundant fragment computations on average. This materializes to a significant speedup of 17.6% on average, while also improving the overall energy efficiency by 8.9% on average.

In order to achieve maximization of parallelism, effective distribution of rendering tasks, balance between performance and flexibility in graphics processing pipeline, this article presents design, performance analysis and optimization for multi-core interactive graphics processing unit (MIGPU). This processor integrates twelve processing cores with specific instruction set architecture and many sophisticated application-specific accelerators into a 3D graphics engine. It is implemented on XC6VLX550T field programmable gate array (FPGA). MIGPU supports OpenGL2.0 with programmable front-end processor, vertex shader, plane clipper, geometry transformer, three-D clippers and pixel shaders. For boosting the performance of MIGPU, the relationship model is established between primitive types, vertices, pixels, and the effect of culling, clipping, and memory access, and shows a way to improve the speed up of the graphics pipeline. It is capable of assigning graphics rendering tasks to different processors for efficiency and flexibility. The pixel filling rate can reach to 40 Mpixel/s at its peak performance.

Exact order-independent transparency (OIT) techniques capture all fragments during rasterization. The fragments are then sorted per-pixel by depth and composited in order using alpha transparency. The sorting stage is a bottleneck for high depth complexity scenes, taking 70---95 % of the total time for those investigated. In this paper, we show that typical shader-based sorting speed is impacted by local memory latency and occupancy. We present and discuss the use of both registers and an external merge sort in register-based block sort to better use the memory hierarchy of the GPU for improved OIT rendering performance. This approach builds upon backwards memory allocation, achieving an OIT rendering speed up to 1.7 $$\times $$ × that of the best previous method and 6.3 $$\times $$ × that of the common straight forward OIT implementation. In some cases, the sorting stage is reduced to no longer be the dominant OIT component.

When rendering effects such as motion blur and defocus blur, shading can become very expensive if done in a naive way, i.e. shading each visibility sample. To improve performance, previous work often decouple shading from visibility sampling using shader caching algorithms. We present a novel technique for reusing shading in a stochastic rasterizer. Shading is computed hierarchically and sparsely in an object-space texture, and by selecting an appropriate mipmap level for each triangle, we ensure that the shading rate is sufficiently high so that no noticeable blurring is introduced in the rendered image. Furthermore, with a two-pass algorithm, we separate shading from reuse and thus avoid GPU thread synchronization. Our method runs at real-time frame rates and is up to 3 × faster than previous methods. This is an important step forward for stochastic rasterization in real time.

We propose a versatile pipeline to render B-Rep models interactively, precisely and without rendering-related artifacts such as cracks. Our rendering method is based on dynamic surface evaluation using both tesselation and ray-casting, and direct GPU surface trimming. An initial rendering of the scene is performed using dynamic tesselation. The algorithm we propose reliably detects then fills up cracks in the rendered image. Crack detection works in image space, using depth information, while crack-filling is either achieved in image space using a simple classification process, or performed in object space through selective ray-casting. The crack filling method can be dynamically changed at runtime. Our image space crack filling approach has a limited runtime cost and enables high quality, real-time navigation. Our higher quality, object space approach results in a rendering of similar quality than full-scene ray-casting, but is 2 to 6 times faster, can be used during navigation and provides accurate, reliable rendering. Integration of our work with existing tesselation-based rendering engines is straightforward.

In the context of real-time, GPU-based rendering of animated skinned meshes, we propose a new algorithm to compute surface normals with minimal overhead both in terms of the memory footprint and the required per-vertex operations. By accounting for the variation of the skinning weights over the surface, we achieve a higher visual quality compared to the standard approximation ubiquitously used in video-game engines and other real-time applications. Our method supports Linear Blend Skinning and Dual Quaternion Skinning. We demonstrate the advantages of our technique on a variety of datasets and provide a complete open-source implementation, including GLSL shaders.

There are two major ways to implement depth estimation, multiple image depth estimation and single image depth estimation, respectively. The former has a high hardware cost because it uses multiple cameras but it has a simple software algorithm. Conversely, the latter has a low hardware cost but the software algorithm is complex. One of the recent trends in this field is to make a system compact, or even portable, and to simplify the optical elements to be attached to the conventional camera. In this paper, we present an implementation of depth estimation with a single image using a graphics processing unit (GPU) in a desktop PC, and achieve real-time application via our evolutional algorithm and parallel processing technique, employing a compute shader. The methods greatly accelerate the compute-intensive implementation of depth estimation with a single view image from 0.003 frames per second (fps) (implemented in MATLAB) to 53 fps, which is almost twice the real-time standard of 30 fps. In the previous literature, to the best of our knowledge, no paper discusses the optimization of depth estimation using a single image, and the frame rate of our final result is better than that of previous studies using multiple images, whose frame rate is about 20fps.

Rendering time depends on number of faces in polygonal mesh. Displaying large number of objects or objects with a lot of faces can result in performance degradation. One of the ways to overcome this problem is to use several levels of detail for polygonal mesh. Levels of detail with low quality are used when object is far away from camera and difference in number of faces is not apparent to viewer. Coarse meshes can be generated manually which is difficult and time consuming. For many applications (computer graphics, computer vision, finite element analysis) it's preferable to use automated methods for polygonal surface simplification to increase performance. Traditional methods for surface simplification are designed for CPU. GPU methods provide improvements in performance and have comparable visual quality. This survey covers algorithms on GPU. They are based on either shader programming or general purpose computing frameworks like OpenCL and CUDA. Shader methods are very restrictive. They are designed to render images in real-time. General purpose computing frameworks provide flexible APIs but require attention to hardware implementation details to avoid performance bottlenecks. Main features of algorithms are provided for comparison as the result. The most used operation for geometry simplification is edge collapse. It's difficult to avoid expensive data exchange between GPU and CPU. It's important to take into account computational model of GPU to increase number of stream processors working in parallel.

In response to the remarkable increase in 3D applications in consumer electronics devices in recent years, graphics processing units (GPUs) have become widely available on mobile devices. These GPUs typically use hardware multithreaded shaders to improve their throughputs for real-time rendering, but they depend on duplicate register files to maintain the context of each hardware thread, increasing power consumption. However, the register usage of shading programs is often relatively low, which causes many registers to remain unused, thus wasting power. Long latency memory operations can also consume unnecessary power to activate registers. This study proposes a low-power register file with multiple power modes to reduce the power consumption of the register file. This study also presents an adaptive thread scheduling mechanism to achieve a tradeoff between the power consumption of the register file and frames per second (FPS). Results show that the average performance degradation from the proposed low-power register file is only 0.62%. The proposed adaptive thread scheduling has average under prediction ratio of 3.32%. The leakage reduction of the proposed low-power register file is 74.80%. This reduction can be improved to 81.49%, 82.22%, and 84.28% with adaptive thread scheduling at frame rates of 30, 25, and 20, respectively.

The importance of visual elements in video games is increasing, and players demand high quality graphics. To reach more of the target market, game developers need to maintain good performance while still delivering high quality graphical effects. Game developers want to use the most efficient ways to render game graphics, specifically particle systems, one of the most resource intensive elements in a game. Geometry shader is a new technology that was proposed to offer the right capability in rendering particle systems. The aim of this paper is to find out whether geometry shader is more efficient to render particle systems by analyzing the performance differences and resource consumption between rendering particle systems using a CPU and geometry shader. These research experiments support the power of a GPU-based geometry shader approach to render particle systems compared to the CPU rendering of particle systems. Geometry shader is shown to be effective and provides sufficient benefits for optimizing the geometry generations. However, the benefits of the geometry shader approach diminish when the particle size increases and the screen resolution gets higher.

During the using of 3D meshes, the different level of detail (LOD) meshes are needed for a model. This paper presents an algorithm of mesh reconstruction based on view maps. It bases the rendering pipeline, performs the coordinate transform and records the original coordinates of the mesh during the vertex shader. It interpolates the vertex's 3D coordinates for every pixel in the rasterization step. During the pixel shader, it extracts the visible pixels through the depth testing, and then outputs the corresponding 3D coordinates as a view map. According to pixel's relative position in the map, it connects the 3D vertices together to make a triangle mesh for the current view. The result LOD mesh can be obtained by merging these triangle meshes for different views. The experiments show that the algorithm is effective, speedy and convenient for adjusting the LOD.

To the EditorsYoung researchers and young clinicians in general and young psychopharmacologists, young psychiatric researchers, and young psychiatrists, in particular, should read Greenblatt and Shader's1 recent editorial titled, “The Impact Non-factor” before selecting a journal for publication

Virtual reality with computer technologies as the core and in combination with related sciences can generate a digital environment highly approximate to the real environment in aspects of visual sense, auditory sense and touch sense, etc., and characterized by high immersion and interactivity. It has been widely used in the medical field, but rarely used in disease prevention, especially in science popularization and prevention of eye diseases. This paper focuses on the study on the application of VR in eye disease prevention and presents a mobile phone App. It utilizes virtual reality as the presentation form, augmented reality as the connection tool, Unity3D as the development engine, and Oculus as the device carrier for further display and experience. It uses shader to add special effects for the camera lens in its simulation of eye disease's symptoms, which allows common people to experience the world seen by the patient of eye disease through the mobile phone camera, understand their inconveniences in daily life, and thus offer them more respects through understanding. At the same time, it can allow people to know the hazards of these diseases, and pay more attention to eye health, thus reaching the result of eye disease prevention.

This work presents a new representation used as a rendering primitive of surfaces. Our representation is defined by an arbitrary cubic cell complex: a projection-based parameterization domain for surfaces where geometry and appearance information are stored as tile textures. This representation is used by our ray casting rendering algorithm called projection mapping, which can be used for rendering geometry and appearance details of surfaces from arbitrary viewpoints. The projection mapping algorithm uses a fragment shader based on linear and binary searches of the relief mapping algorithm. Instead of traditionally rendering the surface, only front faces of our rendering primitive our arbitrary cubic cell complex are drawn, and geometry and appearance details of the surface are rendered back by using projection mapping. Alternatively, another method is proposed for mapping appearance information on complex surfaces using our arbitrary cubic cell complexes. In this case, instead of reconstructing the geometry as in projection mapping, the original mesh of a surface is directly passed to the rendering algorithm. This algorithm is applied in the texture mapping of cultural heritage sculptures.

Graphics processing units (GPUs) are now being widely adopted in system-on-a-chip designs, and they are often used in embedded systems for manipulating computer graphics or even for general-purpose computation. Energy management is of concern to both hardware and software designers. In this article, we present an energy-aware code-motion framework for a compiler to generate concentrated accesses to input and output (I/O) buffers inside a GPU. Our solution attempts to gather the I/O buffer accesses into clusters, thereby extending the time period during which the I/O buffers are clock or power gated. We performed experiments in which the energy consumption was simulated by incorporating our compiler-analysis and code-motion framework into an in-house compiler tool. The experimental results demonstrated that our mechanisms were effective in reducing the energy consumption of the shader processor by an average of 13.1% and decreasing the energy-delay product by 2.2%.

Existing GPU antialiasing techniques, such as MSAA or MLAA, focus on reducing aliasing artifacts along silhouette boundaries or edges in image space. However, they neglect aliasing from shading in case of high-frequency geometric detail. This may lead to a shading aliasing artifact that resembles Bailey's Bead Phenomenon-the degradation of continuous specular highlights to a string of pearls. These types of artifacts are particularly striking for high-quality surfaces. So far, the only way of removing aliasing from shading is by globally supersampling the entire image with a large number of samples. However, globally supersampling the image is slow and significantly increases bandwidth consumption. We propose three adaptive approaches that locally supersample triangles only where necessary on the GPU. Thereby, we efficiently remove artifacts from shading while aliasing along silhouettes is reduced by efficient hardware MSAA.

We propose a highly memory-efficient representation for modeling and rendering urban buildings composed predominantly of rectangular block structures, which can be used to completely or partially represent most modern buildings. With the proposed representation, the data size required for modeling most buildings is more than two orders of magnitude less than using the conventional mesh representation. In addition, it substantially reduces the dependency on conventional texture maps, which are not space-efficient for defining visual details of building facades. The proposed representation can be stored and transmitted as images and can be rendered directly without any mesh reconstruction. A ray-casting based shader has been developed to render buildings thus represented on the GPU with a high frame rate to support interactive fly-by as well as street-level walk-through. Comparisons with standard geometric representations and recent urban modeling techniques indicate the proposed representation performs well when viewed from a short and long distance.