Search

This Embedded Graphics libraries have evolved from OpenGL to next gen Vulkan libraries. OpenCL and CUDA have been used for many years for GPU compute on a range of SoC devices with powerful embedded GPU chipsets. OpenCL and CUDA were never able to be certified, due to the contributions from many developers. Next generation autonomous vehicles, unmanned aircraft, transportation and even military vehicles will require both graphics, as well as GPU compute to implement DSP or FPGA logic is software, as well as Deep Machine Learning. Vulkan libraries are available now and are defined by the Kronos Group, as well as this, a next generation of Vulkan safety critical library for safe Shader and GPU Compute is now available. This white paper will look at Vulkan architectures, embedded system use and the implications for using GPU Compute, where certification may be a critical requirement for ISO 26262, EN50182, IEC 61508, DO178C certifiable embedded systems.

Non-photorealistic rendering (NPR) of 3D models in a specific style has been a hot topic of NPR research, and people are working hard in the direction of a more similar rendering style and higher real-time performance, Chinese ink painting style is one of them. In general, previous studies of 3D model Chinese ink painting style rendering mainly in two methods: Set up a mathematical model of brushstroke to simulate the pressure and diffuse of brush, which is usually used in interactive painting. Another method is rendering the model in two parts (contour line the interior area) to automatically convert the 3D model into 2D Chinese ink painting style images. Combine with the application scenarios, this paper used the second method. We had improved the traditional algorithm. In the contour line rendering part, freehand style contour line extraction and rendering algorithm are used, and user-defined contour line thickness and omission control are added to give the artist more creative space. We use the improved dye shader and noise map in the interior rendering. Our results show a better aesthetics effect and real-time character.

Creating realistic physically-based systems such as weather, water, or light for digital environments (DEs) is an important topic in computer graphics. This paper describes a method utilizing compute shaders to implement dynamic manipulation of meshes to realistically simulate how loosely packed snow, or other transformable surfaces, behave when it is stepped on or otherwise manipulated by objects or agents in a VE. Additional features of the method include accumulation over time, wind direction, and other phenomena that serve to facilitating the simulation. Conditions of the environment were tested using physically-based reinforcement learning (RL) agents walking and running around the VE, leaving behind trails in the snow, to assess computational efficiency and robustness. Results show that the system is robust and computationally efficient running at +30 frames per second (FPS) with up to 100 agents under various conditions. Visual inspection of imagery which was captured during the tests also indicates possibilities in creating larger areas in which the snow has been altered in realistic human-like patterns.

The visual inspection of a hexahedral mesh with respect to element quality is difficult due to clutter and occlusions that are produced when rendering all element faces or their edges simultaneously. Current approaches overcome this problem by using focus on specific elements that are then rendered opaque, and carving away all elements occluding their view. In this work, we make use of advanced GPU shader functionality to generate a focus+context rendering that highlights the elements in a selected region and simultaneously conveys the global mesh structure in the surrounding. To achieve this, we propose a gradual transition from edge-based focus rendering to volumetric context rendering, by combining fragment shader-based edge and face rendering with per-pixel fragment lists. A fragment shader smoothly transitions between wireframe and face-based rendering, including focus-dependent rendering style and depth-dependent edge thickness and halos, and per-pixel fragment lists are used to blend fragments in correct visibility order. To maintain the global mesh structure in the context regions, we propose a new method to construct a sheet-based level-of-detail hierarchy and smoothly blend it with volumetric information. The user guides the exploration process by moving a lens-like hotspot. Since all operations are performed on the GPU, interactive frame rates are achieved even for large meshes., under review at IEEE Transactions on Visualization and Computer Graphics

This paper proposes a block solution method for the Extreme Learning Machine. It combines the speed of a direct non-iterative solver with minimal memory requirements. The method is suitable for edge computing scenarios running on a mobile device with GPU acceleration. The implementation tested on the GPU of iPad Pro outperforms a laptop CPU, and trains a 19,000-neuron model using under one gigabyte of memory. It confirms the feasibility of Big Data analysis on modern mobile devices.

Multimodality medical data visualization can convey a significant amount of complementary image information from various sources into a single and meaningful display. Even though there have been publications on integrating multiple medical images into a unique 3D representation, there are no researches on rendering registered multiple medical data in web browsers and synchronizing the rendering results over Internet, especially in cardiac applications. Web-based multimodality medical data display is hampered by the large size of the dataset that makes the rendering and information streaming too slow to be applied in clinical diagnosis and treatment. In this paper, we introduce new algorithms and a software platform to address this issue. The presented research results take advantages of bidirectional network connection and hardware accelerated graphics shader processing to real-time fuse and display inner organ structures in web browsers and stream the rendering results over Internet, which will provide medical users with intuitive feedback in collaborative cardiac diagnosis and image-guided remote therapy.

We explore the notion of dataflow programming in realtime graphics artistic practice. We complement the previous editions of the course at SIGGRAPH Asia (2015, 2016, 2018) and SIGGRAPH (2017, 2019) to include OpenISS and PureData/GEM. First, we explore a rapid prototyping of interactive graphical applications for stage and beyond using Jitter/Max and Processing with OpenGL, shaders, and featuring connectivity with RGBD cameras. Such rapid prototyping environment is ideal for entertainment computing, as well as for artists and live performances using real-time interactive graphics on stage. We share the expertise we developed in connecting the real-time graphics with on-stage performance with the Illimitable Space System (ISS) v2 and its OpenISS core framework for creative near-realtime broadcasting, and the use of AI and HCI techniques in art.

Developing a game that satisfies high expectations on visual quality while running at 30 fps on a 6 year old console platform, PlayStation 4, is quite a challenging task. The majority of the world of The Last of Us Part II contains alpha geometry which has poor rendering pipeline performance and full-screen expensive shaders with heavy resource usage which required us to use plenty of hacks and tricks to push the hardware to its limits and make sure we don’t have to compromise needlessly on the visual fidelity of the game. This talk will mainly focus on case studies from the actual game highlighting the most expensive parts of our game’s frame and how we were able to come up with solutions to these performance hurdles.

The soul characters in Disney/Pixar’s Soul have a stylized appearance that sets them into a unique world, which introduced many new challenges. Everyone from the art department and character modelers and shaders to the technical directors and developers in the effects, lighting, and software groups collaborated to bring this new visual style to screen. The soul world is abstract and ethereal; this needed to be balanced with visual clarity and design appeal. As our character rigging and animation tools use rigged surfaces, a key challenge was presenting a new representation derived from this data that meets our visual goals. To achieve softness of volumetric form and dynamically changing linework, we built a system to procedurally generate this data in Houdini. Significant numerical computation was required to create this data at the fidelity required. We developed an automated system for managing this computation in a configurable way, while keeping data for downstream renders in sync with changes to character performances.

In order to produce shots with hundreds of multi-volume crowd characters for Soul, we could not rely on the same I/O heavy pose-cache pipeline used for the hero characters [Coleman et al. 2020]. Our rendering and systems teams rated the total necessary storage for the ”soul world” at over 100 TBs for an average of two hero characters per shot.1 For expansive crowds of these characters to hit the same volumetric look while avoiding this I/O limitation, two new render-time technologies were developed. The first leveraged an existing volume rasterizer to pose volumes at render-time, informed by a lattice deformation. The second allowed for rasterization of surface primvars to be delivered to the volume shaders.

Using a commercial terrain engine [Englert 2012] as an example, we discuss both advantages and disadvantages of the continuous level-of-detail approach [Lindstrom and Pascucci 2001] to terrain-rendering while comparing our implementation with other hybrid [Dick et al. 2010] and implicit [Jad Khoury and Riccio 2018] approaches. We show how mesh shader and inline ray-tracing features of DirectX 12 Ultimate can be used to remedy those disadvantages.

This poster describes the implementation of a performant 2D drawing application in the browser that renders Signed Distance Functions (SDF) compiled from user input. SDFs are well suited for CAD applications because they reveal elegant boolean operations and efficiently allow for superior anti-aliasing. Because of their reliance on the GPU, SDFs have not traditionally lent themselves to graphical user input. By compiling shaders on the fly from user input we are able to seamlessly interact with rendered SDFs in a CAD interface.

In recent years several hobbyists and research institutions have turned to ARM based cluster computers to meet their high performance computing needs. These systems are typically low cost, power efficient machines made of 4 or more nodes. However, current applications running on these machines are limited to the computational capabilities of the on-board CPUs. An attempt to fully utilise the available hardware is to use the on-board GPUs for general purpose computation. The challenge with this approach is the difficulty of programming for embedded ARM GPUs, which support a very limited programming interface such as OpenGL ES 2.0. In this paper, we present a system which allows a high level program to utilise multiple embedded ARM based GPUs in parallel to perform computation, without the application developer concerning themselves with the details of OpenGL and writing shaders. We describe the vector and matrix functions which we implement and provide results detailing their speedup. For matrix multiplication, our system shows a speedup of nearly 7X on the 8 nodes of the PHINEAS cluster computer. We further describe how these functions may be composited with some CPU functionality to implement inferencing for a Convolutional Neural Network trained in Keras on a different machine. Our benchmarks demonstrate that the system’s performance scales well across several nodes for a variety of tasks. Our work brings the benefits of GPGPU computing to an as yet untouched class of cluster computers.

There is an ever increasing need for higher GPU performance to render sophisticated User Interface, latest high end 3D games and general purpose compute (GPGPU) applications. GPU SW programming models such as OpenGL have evolved over decades to cater to the unique mixed pipeline 3D GPU architectures. Due to the sticky nature of GPU SW programming model, leveraging other HW blocks to enhance graphics performance has been a most challenging task for SW architects. System designers have usually responded to the GFLOPS demand by increasing the GPU HW specifications. This paper proposes enhancing GPU performance by leveraging DSP transparently in background without impacting GPU software programming model. The proposed solution consists of multiple novel techniques namely ability to offload vertex shader to DSP, 3 stage pipelined execution and ability to re-use GPU internal pipeline. The proposed solution is prototyped in Jacinoto6 Platform from Texas Instruments. The default GPU spec performance is increased by up-to 41% by leveraging dual core C66x DSP in Jacinto6 Platform using proposed solution for different use-cases. The proposed solution is fully transparent to application software stack. In addition, the solution is directly applicable to any GPU + DSP architecture making it attractive approach for cost optimized solutions.

The design of video games including the planning and integration of various 3D assets that build up the game model and map will be discussed. There are various types of structures: terrain, foliage, lights, UI and other elements that are needed to create the engaging game world. In this chapter, the development of an interactive user interface (UI) in Unity including menus, extra features and tools will be explained. To enhance the quality in the game, real-time shadow and sun simulation to control the day and night cycle and shaders for real looking rivers will be presented. The authors will discuss the importance of narration and game logic enhanced by artificial intelligence (AI) in level design. Artificially generated terrain structures are a big part of the modeling of virtual environments for the video games industry and in military training simulations. Without them, we could not enjoy an open space world in many games like The Witcher, The Elder Scrolls, Fallout etc. In this chapter, we will focus on faster ways to generate terrain structures using height map algorithms and modify them according to our needs. The Unreal Engine Blueprint (UEB) system to generate procedurally parameterized trees and interactive gameplay features like force reaction, chopping and dynamic wind will be discussed. Utilizing the UEB system to implement the recursive surface align algorithm that can be used to create surface aligned vines, ropes, fences and other assets will be presented. The chapter contains four parts which were built independently and they may be used as a roadmap to develop interactive games following different approaches.

This paper reports on an intervention in introductory Computer Graphics courses, where practical coursework is transitioned from standard OpenGL Graphics APIs to the Unity Game Development Engine. Game Development Engines (GDEs) provide powerful programming tools and computing components. Software development in GDEs is facilitated by full-fledged IDEs. Complex scene modeling and image rendering are managed through a rich catalogue of rendering assets. Graphics APIs are evolving towards a lower level of hardware abstraction, reduced overhead, and multithreading capabilities. Such changes in APIs benefit software developers who build graphics engines; how much they benefit Computer Graphics education is yet to be determined. New graphics APIs represent a shift in graphics programming practices, nearly as big as programmable GPUs and shader-based APIs over a decade ago. As then, Computer Graphics instructors have to decide: either use outdated models, or again change how Computer Graphics is taught. This paper describes an educational approach that acknowledges the increasing divide between the ease of use and development speed of GDEs, and the complexity and vastness of new graphics APIs. We show the use of a GDE as software development framework, to support programming assignments covering core Computer Graphics concepts. Relying on a GDE, such graphics assignments don't have to shy away from shader-based graphics programming, while non-graphics low-level tasks are managed by the engine itself. With our approach, we hope to contribute to the discussion about evolving best practices in teaching Computer Graphics.

This paper proposes a terrain tessellation technique using the new GPU mesh pipeline, and compares it to the classically used tessellation pipeline. We explain how we can use it to greatly increase the tessellation capability within the draw call while keeping similar performances. We then show how it can be used to limit the CPU load as well as the data transfer to the GPU for rendering wide terrains.

International audience; Image-Based Rendering (IBR) has made impressive progress towards highly realistic, interactive 3D navigation for many scenes, including cityscapes. However, cars are ubiquitous in such scenes; multi-view stereo reconstruction provides proxy geometry for IBR, but has difficulty with shiny car bodies, and leaves holes in place of reflective, semi-transparent windows on cars. We present a new approach allowing free-viewpoint IBR of cars based on an approximate analytic reflection flow computation on curved windows. Our method has three components: a refinement step of reconstructed car geometry guided by semantic labels, that provides an initial approximation for missing window surfaces and a smooth completed car hull; an efficient reflection flow computation using an ellipsoid approximation of the curved car windows that runs in real-time in a shader and a reflection/background layer synthesis solution. These components allow plausible rendering of reflective, semi-transparent windows in free viewpoint navigation. We show results on several scenes casually captured with a single consumer-level camera, demonstrating plausible car renderings with significant improvement in visual quality over previous methods.

Time-of-Flight camera systems are an essential component in 3D scene analysis and reconstruction for many modern computer vision applications. The development and validation of such systems requires testing in a large variety of scenes and situations. Accurate room impulse response simulation greatly speeds up development and validation, as well as reducing its cost, but large computational overhead has so far limited its applicability.While the overall algorithmic requirements of this simulation differ significantly from 3D rendering, the recently introduced hardware raytracing support in GPUs nonetheless provides an interesting new implementation option. In this paper, we present a new room impulse simulation method, implemented with Vulkan compute shaders and leveraging NVIDIA VKRay hardware raytracing. We also extend this method to asynchronous streaming in order to overcome the limitations of on-board GPU memory when simulating very large scenes.Our implementation is, to the best of our knowledge, the first ever application of Vulkan hardware raytracing in a non-rendering simulation setting. Compared to a state-of-the-art multicore CPU implementation running on 12 CPU cores, we achieve an overall speedup of factor 20.9 when streaming is not required, and 17.6 with streaming, on a single consumer GPU.

International audience; We seek to cover a parametric domain with a set of evenly spaced bands which number and widthvaries according to a density field. We propose an implicit procedural algorithm, that generates theband pattern from a pixel shader and adapts to changes to the control fields in real time. Each band isuniquely identified by an integer. This allows a wide range of texturing effects, including specifying adifferent appearance in each individual bands. Our technique also affords for progressive gradationsof scales, avoiding the abrupt doubling of the number of lines of typical subdivision approaches. Thisleads to a general approach for drawing bands, drawing splitting and merging curves, and drawingevenly spaced streamlines. Using these base ingredients, we demonstrate a wide variety of texturingeffects.

Exploration of the challenges faced by individuals who are non sighted or visually impaired has primarily been limited to physical methods such as blindfolds and controlled environments. Using virtual reality, we provide a simulation of blind navigation using a shader effect with scattered light. The scattered light represents the auditory and haptic feedback that the player receives while using a walking cane to navigate the virtual environment. We discuss methods of development, and goals for this simulation.

This poster presents our experience in using a Game Development Engine (GDE, i.e. Unity) for teaching introductory Computer Graphics (CG) courses. About a decade ago, standard textbooks used in CG courses began updating to adopt shader-based APIs and programmable GPUs. More recently, Graphics APIs evolved towards a lower level of hardware abstraction, reduced overhead, and multithreading capabilities. Once again, CG instructors have to either use outdated models, or change the way they teach. Acknowledging the divide between the development speed of GDEs, and the complexity and vastness of new graphics APIs, we show how the use of a GDE can help students understand core CG concepts, and still include shader-based programming, while low-level tasks are managed by the engine itself.

We present a GPU algorithm to render path-based 3D surface detail in real-time. Our method models these features using a vector representation that is efficiently stored in two textures. First texture is used to specify the position of the features, while the second texture contains their paths, profiles and material information. A fragment shader is then proposed to evaluate this data on the GPU by performing an accurate and fast rendering of the details, including visibility computations and antialiasing. Some of our main contributions include a CSG approach to efficiently deal with intersections and similar cases, and an efficient antialiasing method for the GPU. This technique allows application of path-based features such as grooves and similar details just like traditional textures, thus can be used onto general surfaces.

As new technologies for CPUs and GPUs are released, games showcase improved graphics, physics simulations, and responsiveness. For limited form-factors such as virtual reality head-mounted displays though, it is possible to explore alternatives components to harness additional performance such as the GPU. This paper introduces a shader-based architecture for developing games using shared resources between the CPU and the GPU.

This paper presents a novel approach to using modern graphics card capabilities for predicting the target echo strength of an object. The key feature of this method is to inject code in the fragment shader stage of the graphics pipeline, in effect transforming the optics reflection problem of computer graphics to an acoustic reflection problem. This is exemplified with the simple situation of a rigid body and the monostatic detection case. Numerical accuracy is compared with Kirchhoff theory for simple bodies. Possibility of extending this approach to handle the multistatic case and transparent/absorbing/diffusing surfaces on the body is discussed.

©2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Modern computer graphics courses require students to complete assignments involving computer programming. The evaluation of student programs, either by the student (self-assessment) or by the instructors (grading) can take a considerable amount of time and does not scale well with large groups. Interactive judges giving a pass/fail verdict do constitute a scalable solution, but they only provide feedback on output correctness. In this article, we present a tool to provide extensive feedback on student submissions. The feedback is based both on checking the output against test sets, as well as on syntactic and semantic analysis of the code. These analyses are performed through a set of code features and instructor-defined rubrics. The tool is built with Python and supports shader programs written in GLSL. Our experiments demonstrate that the tool provides extensive feedback that can be useful to support self-assessment, facilitate grading, and identify frequent programming mistakes. This work has been partially funded by the Spanish Ministry of Economy and Competitiveness and FEDER Grant TIN2017-88515-C2-1-R.

With the development of science and technology, the medical level is becoming more and more advanced. Virtual medical technology is increasingly used in doctor training and surgical simulation. The core of rendering in virtual medical is that the rendering effect of the virtual target used for simulation should match the lighting effect of the object of the real environment. The traditional virtual medical training uses the organ or organ obtained by using the empirical model or the video image. The two rendering methods do not conform to the physical laws and the effect is greatly deviated from the real situation. The Brookfield model of Cook–Torrance is improved by combining the diffuse reflection model of Disney’s Bidirectional Reflectance Distribution Function(BRDF) algorithm, and the parameters of the algorithm model are adjusted. Continuously adjust the performance parameters on Unity 3D’s custom shader for rendering tests, resulting in a more realistic rendering result.

Ocean surface 3D visualization is an important aspect in marine information research and is of great significance to the construction of digital ocean. Due to the limitation of rendering efficiency and loading speed, the existing 3D visualization of ocean surface cannot meet the construction of ocean surface environment in a large area. Therefore, how to realize the global 3D visualization of the ocean surface is one of the key issues to be solved. For this, the paper proposes a 3D visualization method based on discrete global grids. Firstly, a multi-scale grid model for GPU based on the existing global discrete grid model is designed. Secondly, a dynamic wave simulation method based on GPU shader technology is proposed in order to realize the dynamic wave simulation. Finally, the effectiveness of the method is verified by comparative experiments. The experimental results show that the method of the paper achieves the effect of pixel-level ocean-land boundary and supports the dynamic simulation and real-time updating of the wave driven by the wind field. Compared with the existing 3D visualization method of ocean surface, the loading speed is faster and the rendering efficiency is higher. Therefore, the method is more suitable for the application needs of environment construction of the global ocean surface.

This article examines the development of the audiovisual duo Obi-Wan Codenobi and The Wookie (authors Shawn Lawson and Ryan Ross Smith respectively). The authors trace a now four-year trajectory of technological and artistic development, while highlighting the impact that a more recent physical displacement has had on the creative and collaborative aspects of the project. We seek to reflect upon the creative and technological journey through our collaboration, including Lawson’s development of The Force, an OpenGL shader-based live-coding environment for generative visuals, while illuminating our experiences with, and takeaways from, live coding in practice and performance, EDM in general and algorave culture specifically.

The growing parallelism in most of today's applications has led to an increased demand for parallel computing in processors. General Purpose Graphics Processing Units (GPGPUs) have been used extensively to support highly parallel applications in recent years. Such GPGPUs generate huge volumes of network traffic between memory controllers (MCs) and shader cores. As a result, the network-on-chip (NoC) fabric can become a performance bottleneck, especially for memory intensive applications running on GPGPUs. Traditional mesh-based NoC topologies are not suitable for GPGPUs as they possess high network latency that leads to congestion at MCs and an increase in application execution time. In this article, we propose a novel memory-aware NoC that has two (request and reply) planes tailored to exploit the traffic characteristics in GPGPUs. The request layer consists of low power, and low latency routers that are optimized for the many-to-few traffic pattern. In the reply layer, flits are sent on fast overlay circuits to reach their destinations in just three cycles (at 1 GHz). In addition, as traditional memory controllers are not aware of the application memory intensity that leads to higher waiting time for applications on the shader cores, we propose an enhanced memory controller that prioritizes burst packets to improve application performance on GPGPUs. Experimental results indicate that our framework yields an improvement of ${\mathrm{4}}-{\mathrm{10}}\times$ in NoC latency, up to 63 percent in execution time, and up to 4× in total energy consumption compared to the state-of-the-art.

The one-dimensional time axis on which the current meteorological observation platform depends is transformed into a two-dimensional coordinate system, i.e. one dimension, multi-source observation instruments, data and products are added. This dimension shows the observation occurring at a certain time node. The multi-source observations at the same time are vertically arranged to be distinguished by icons or colors. In this study, real-time display and rendering of scenes are realized through real-time efficient local multi-level of detail loading and adaptive multi-level caching of large-scale 3D spatial data and the dynamic interaction of multi-source observation information is realized by combining the CPU/GPU graphics hardware accelerated rendering technology of OpenGL+Shader. The platform is upgraded from one dimension of time to two dimensions of time and observation, which greatly expands the selection and change space of independent variables, thus the capacity of dependent variables is also greatly increased. Theoretically, it can accommodate all kinds of observation data and products. Through seamless connection on one page, it can freely schedule, switch and superimpose all kinds of observation information, enabling business researchers to efficiently and intensively obtain comprehensive meteorological observation data products, and carry out analysis and application.

I talked about the Blender user interface in Chapter 1. I talked about the Layout workspace, Modeling workspace, and Shader workspace in Chapters 2, 3, and 4. You even had a sneak peek of the UV editing workspace and many Blender features through sample projects. I also discussed the fundamentals of light and color and the various editors.

Virtual reality (VR) has been suggested for various purposes in the field of architecture, engineering, and construction (AEC). This research explores new roles for VR toward the super-smart society in the near future. In particular, we propose to develop post-processing rendering, segmentation-rendering and shadow-casting rendering algorithms for novel VR expressions to enable more versatile approaches than the normal photorealistic red, green, and blue (RGB) expressions. We succeeded in applying a wide variety of VR renderings in urban-design projects after implementation. The developed system can create images in real time to train deep-learning algorithms, can also be applied to landscape analysis and contribute to advanced user experience.

GPU has been applied to more and more visions which demand the hardware with higher performance, faster speed and lower computational cost. We propose a novel early z architecture named ZU1 who locates prior to the pixel shader stage for z range test. The module treats 8x8 pixel patch called tile as an input entry to calculate the tile’s z range, meanwhile acquiring cache zrange from z cache to which the tile zrange compares against according to the compare mode, so as to fastly evaluate z test for accepting or rejecting the tile. We benchmark our design on 3DMark set demonstrate substantial gains with 40% reject ratio and thousands of workloads per frame saving.

In this paper, we question the premise that graphics hardware uses a post-transform cache to avoid redundant vertex shader invocations. A large body of existing work on optimizing indexed triangle sets for rendering speed is based upon this widely-accepted assumption. We conclusively show that this assumption does not hold up on modern graphics hardware. We design and conduct experiments that demonstrate the behavior of current hardware of all major vendors to be inconsistent with the presence of a common post-transform cache. Our results strongly suggest that modern hardware rather relies on a batch-based approach, most likely for reasons of scalability. A more thorough investigation based on these initial experiments allows us to partially uncover the actual strategies implemented on graphics processors today. We reevaluate existing mesh optimization algorithms in light of these new findings and present a new mesh optimization algorithm designed from the ground up to target architectures that rely on batch-based vertex reuse. In an extensive evaluation, we measure and compare the real-world performance of various optimization algorithms on modern hardware. Our results show that some established algorithms still perform well. However, if the batching strategy of the target architecture is known, our approach can significantly outperform these previous state-of-the-art methods.

We propose a new by-example noise algorithm that takes as input a small example of a stochastic texture and synthesizes an infinite output with the same appearance. It works on any kind of random-phase inputs as well as on many non-random-phase inputs that are stochastic and non-periodic, typically natural textures such as moss, granite, sand, bark, etc. Our algorithm achieves high-quality results comparable to state-of-the-art procedural-noise techniques but is more than 20 times faster. Our approach is conceptually simple: we partition the output texture space on a triangle grid and associate each vertex with a random patch from the input such that the evaluation inside a triangle is done by blending 3 patches. The key to this approach is the blending operation that usually produces visual artifacts such as ghosting, softened discontinuities and reduced contrast, or introduces new colors not present in the input. We analyze these problems by showing how linear blending impacts the histogram and show that a blending operator that preserves the histogram prevents these problems. The main requirement for a rendering application is to implement such an operator in a fragment shader without further post-processing, i.e. we need a histogram-preserving blending operator that operates only at the pixel level. Our insight for the design of this operator is that, with Gaussian inputs, histogram-preserving blending boils down to mean and variance preservation, which is simple to obtain analytically. We extend this idea to non-Gaussian inputs by "Gaussianizing" them with a histogram transformation and "de-Gaussianizing" them with the inverse transformation after the blending operation. We show how to precompute and store these histogram transformations such that our algorithm can be implemented in a fragment shader.

Designers of real-time rendering engines must balance the conflicting goals of maintaining clear, extensible shading systems and achieving high rendering performance. In response, engine architects have established effective design patterns for authoring shading systems, and developed engine-specific code synthesis tools, ranging from preprocessor hacking to domain-specific shading languages, to productively implement these patterns. The problem is that proprietary tools add significant complexity to modern engines, lack advanced language features, and create additional challenges for learning and adoption. We argue that the advantages of engine-specific code generation tools can be achieved using the underlying GPU shading language directly, provided the shading language is extended with a small number of best-practice principles from modern, well-established programming languages. We identify that adding generics with interface constraints, associated types, and interface/structure extensions to existing C-like GPU shading languages enables real-time Tenderer developers to build shading systems that are extensible, maintainable, and execute efficiently on modern GPUs without the need for additional domain-specific tools. We embody these ideas in an extension of HLSL called Slang, and provide a reference design for a large, extensible shader library implemented using Slang's features. We rearchitect an open source Tenderer to use this library and Slang's compiler services, and demonstrate the resulting shading system is substantially simpler, easier to extend with new features, and achieves higher rendering performance than the original HLSL-based implementation.

We explore an integrated approach to sound generation that supports a wide variety of physics-based simulation models and computer-animated phenomena. Targeting high-quality offline sound synthesis, we seek to resolve animation-driven sound radiation with near-field scattering and diffraction effects. The core of our approach is a sharp-interface finite-difference time-domain (FDTD) wavesolver, with a series of supporting algorithms to handle rapidly deforming and vibrating embedded interfaces arising in physics-based animation sound. Once the solver rasterizes these interfaces, it must evaluate acceleration boundary conditions (BCs) that involve model-and phenomena-specific computations. We introduce acoustic shaders as a mechanism to abstract away these complexities, and describe a variety of implementations for computer animation: near-rigid objects with ringing and acceleration noise, deformable (finite element) models such as thin shells, bubble-based water, and virtual characters. Since time-domain wave synthesis is expensive, we only simulate pressure waves in a small region about each sound source, then estimate a far-field pressure signal. To further improve scalability beyond multi-threading, we propose a fully time-parallel sound synthesis method that is demonstrated on commodity cloud computing resources. In addition to presenting results for multiple animation phenomena (water, rigid, shells, kinematic deformers, etc.) we also propose 3D automatic dialogue replacement (3DADR) for virtual characters so that pre-recorded dialogue can include character movement, and near-field shadowing and scattering sound effects.

The Manuka rendering architecture has been designed in the spirit of the classic reyes rendering architecture: to enable the creation of visually rich computer generated imagery for visual effects in movie production. Following in the footsteps of reyes over the past 30 years, this means supporting extremely complex geometry, texturing, and shading. In the current generation of renderers, it is essential to support very accurate global illumination as a means to naturally tie together different assets in a picture. This is commonly achieved with Monte Carlo path tracing, using a paradigm often called shade on hit , in which the renderer alternates tracing rays with running shaders on the various ray hits. The shaders take the role of generating the inputs of the local material structure, which is then used by path-sampling logic to evaluate contributions and to inform what further rays to cast through the scene. We propose a shade before hit paradigm instead and minimise I/O strain on the system, leveraging locality of reference by running pattern generation shaders before we execute light transport simulation by path sampling. We describe a full architecture built around this approach, featuring spectral light transport and a flexible implementation of multiple importance sampling ( mis ), resulting in a system able to support a comparable amount of extensibility to what made the reyes rendering architecture successful over many decades.

This paper presents an image-space stroke rendering algorithm that provides temporally coherent placement of lines at particles that are moving with object surfaces. We generate particles in image space and move them according to an image-space velocity field. Consistent image-space density is achieved by a deterministic rejection-based algorithm that uses low-discrepancy series to filter out overpopulated areas and to fill in underpopulated regions. Our line stabilization method can solve the temporal continuity problems of image-space techniques. The multi-pass algorithm is implemented entirely on the GPU using geometry shaders and vertex transform feedback. Our method provides high-quality results and is implemented as an interactive post processing step. We also provide a wide toolset for artists to control the final rendering style and extended the method to process real-life RGBZ footage.

This paper introduces a general method to approximate the convolution of an arbitrary program with a Gaussian kernel. This process has the effect of smoothing out a program. Our compiler framework models intermediate values in the program as random variables, by using mean and variance statistics. Our approach breaks the input program into parts and relates the statistics of the different parts, under the smoothing process. We give several approximations that can be used for the different parts of the program. These include the approximation of Dorn et al., a novel adaptive Gaussian approximation, Monte Carlo sampling, and compactly supported kernels. Our adaptive Gaussian approximation is accurate up to the second order in the standard deviation of the smoothing kernel, and mathematically smooth. We show how to construct a compiler that applies chosen approximations to given parts of the input program. Because each expression can have multiple approximation choices, we use a genetic search to automatically select the best approximations. We apply this framework to the problem of automatically bandlimiting procedural shader programs. We evaluate our method on a variety of complex shaders, including shaders with parallax mapping, animation, and spatially varying statistics. The resulting smoothed shader programs outperform previous approaches both numerically, and aesthetically, due to the smoothing properties of our approximations.

Graphics processing units and video cards have seen a surge of usage in domains other than graphics computers, due to advances in hardware and software technologies, however, little uptake has been in the domain of systems engineering and real time control. This research article will demonstrate the use of video cards in multimodel adaptive robust control, using openGL and compute shaders. A software simulation will show the behavior of the adaptive robust multimodel control scheme as the target process is exposed to both parametric and structural disturbances and will show the viability of using graphics processing units in real time systems control.

A Perfectly Accurate, Synthetic dataset featuring a virtual railway EnVironment for Multi-View Stereopsis (RailEnV-PASMVS) is presented, consisting of 40 scenes and 79,800 renderings together with ground truth depth maps, extrinsic and intrinsic camera parameters, pseudo-geolocation metadata and binary segmentation masks of all the track components. Every scene is rendered from a set of 3 cameras, each positioned relative to the track for optimal 3D reconstruction of the rail profile. The set of cameras is translated across the 100 m length of tangent (straight) track to yield a total of 1995 camera views. Photorealistic lighting of each of the 40 scenes is achieved with the implementation of high-definition, high dynamic range (HDR) environmental textures. Additional variation is introduced in the form of camera focal lengths, camera location and rotation parameters and shader modifications for materials. Representative track geometry provides random and unique vertical alignment data for the rail profile for every scene. This primary, synthetic dataset is augmented by a smaller photograph collection consisting of 320 annotated photographs for improved semantic segmentation performance. The combination of diffuse and specular properties increases the ambiguity and complexity of the data distribution. RailEnV-PASMVS represents an application specific dataset for railway engineering, against the backdrop of existing datasets available in the field of computer vision, providing the precision required for novel research applications in the field of transportation engineering. The novelty of the RailEnV-PASMVS dataset is demonstrated with two use cases, resolving shortcomings of the existing PASMVS dataset., Graphical abstract Image, graphical abstract

This paper presents a novel approach to using modern graphics card capabilities for predicting the hydroacoustic target echo strength of an object. The key feature of this method is to inject code in the fragment shader stage of the OpenGL graphics pipeline, in effect transforming the optics reflection problem of computer graphics to an acoustic reflection problem. This is exemplified with the simple situation of an acoustically hard body and the monostatic detection case. Numerical accuracy is compared with Kirchhoff theory for simple bodies. Some suggestions for further work are proposed.