Search

Natural language offers a highly intuitive interface for image editing. In this paper, we introduce the first solution for performing local (region-based) edits in generic natural images, based on a natural language description along with an ROI mask. We achieve our goal by leveraging and combining a pretrained language-image model (CLIP), to steer the edit towards a user-provided text prompt, with a denoising diffusion probabilistic model (DDPM) to generate natural-looking results. To seamlessly fuse the edited region with the unchanged parts of the image, we spatially blend noised versions of the input image with the local text-guided diffusion latent at a progression of noise levels. In addition, we show that adding augmentations to the diffusion process mitigates adversarial results. We compare against several baselines and related methods, both qualitatively and quantitatively, and show that our method outperforms these solutions in terms of overall realism, ability to preserve the background and matching the text. Finally, we show several text-driven editing applications, including adding a new object to an image, removing/replacing/altering existing objects, background replacement, and image extrapolation. Code is available at: https://omriavrahami.com/blended-diffusion-page/, CVPR 2022. Code is available at: https://omriavrahami.com/blended-diffusion-page/

Learning 3D generative models from a dataset of monocular images enables self-supervised 3D reasoning and controllable synthesis. State-of-the-art 3D generative models are GANs which use neural 3D volumetric representations for synthesis. Images are synthesized by rendering the volumes from a given camera. These models can disentangle the 3D scene from the camera viewpoint in any generated image. However, most models do not disentangle other factors of image formation, such as geometry and appearance. In this paper, we design a 3D GAN which can learn a disentangled model of objects, just from monocular observations. Our model can disentangle the geometry and appearance variations in the scene, i.e., we can independently sample from the geometry and appearance spaces of the generative model. This is achieved using a novel non-rigid deformable scene formulation. A 3D volume which represents an object instance is computed as a non-rigidly deformed canonical 3D volume. Our method learns the canonical volume, as well as its deformations, jointly during training. This formulation also helps us improve the disentanglement between the 3D scene and the camera viewpoints using a novel pose regularization loss defined on the 3D deformation field. In addition, we further model the inverse deformations, enabling the computation of dense correspondences between images generated by our model. Finally, we design an approach to embed real images into the latent space of our disentangled generative model, enabling editing of real images., CVPR 2022

In this article, airborne broadband (maximum 400 MHz bandwidth) C-band circularly polarized synthetic aperture radar (SAR) is proposed and developed for further study on airborne compact polarized synthetic aperture radar (CP SAR) system using circular polarization. This article explains the scattering characteristic of circular polarization as the concept of circularly polarized SAR, system configuration, RF system and antenna, ground test, and flight test of circularly polarized SAR in Hinotori-C2 (Firebird-C2) mission onboard CN235MPA aircraft on 14 and 15 March 2018 at South Celebes, Indonesia. The result of the flight test depicts multipath full polarimetric circularly polarized images that show good performance of circularly polarized SAR and matched well to the result of the ground test of a multipolarized single pulse. The circularly polarized scattering clarification using trihedral, cylindrical, and omni corner reflectors (TCR, CCR, and OCR), assessment of circularly polarized image analysis, and image classification using the conventional axial ratio (AR), ellipticity (ε), and polarization ratio (ρ) are discussed. The proposed circularly polarized SAR will enrich the existing CP SAR systems and could be employed in further study of CP SAR calibration technique, also applications development for the environment and disaster monitoring using CP SAR.

Steerable needles that are able to follow curvilinear trajectories and steer around anatomical obstacles are a promising solution for many interventional procedures. In the lung, these needles can be deployed from the tip of a conventional bronchoscope to reach lung lesions for diagnosis. The reach of such a device depends on several design parameters including the bronchoscope diameter, the angle of the piercing device relative to the medial axis of the airway, and the needle’s minimum radius of curvature while steering. Assessing the effect of these parameters on the overall system’s clinical utility is important in informing future design choices and understanding the capabilities and limitations of the system. In this paper, we analyze the effect of various settings for these three robot parameters on the percentage of the lung that the robot can reach. We combine Monte Carlo random sampling of piercing configurations with a Rapidly-exploring Random Trees based steerable needle motion planner in simulated human lung environments to asymptotically accurately estimate the volume of sites in the lung reachable by the robot. We highlight the importance of each parameter on the overall system’s reachable workspace in an effort to motivate future device innovation and highlight design trade-offs.

Micro-coil based magnetic stimulation has been shown to be an effective form of neural modulation [1]. Microcoil devices have advantages over the more commonly used micro-electrodes due to the induced electric fields being spatially asymmetric, allowing for neuron orientation-based selectivity. Furthermore, the micro-coils do not require direct contact with biological tissue, allowing for complete encapsulation of the devices to maintain their stimulation efficiency over time and avoid electrochemical effects such as corrosion, hydrolysis, and other sources of damage to the tissue and probe. However, previous micro-coil based neural probes lack reconfigurable hardware causing their stimulation sites to be fixed post-insertion. The proposed stimulator integrates micro-coils, CMOS control circuitry, and additional nanofabrication processing to fabricate a neural probe with spatially programmable micro-coil based neurostimulation. In-vitro testing with mouse olfactory bulb slices was performed using a commercial MEA to record changes in neural spike behavior during micro-coil based neurostimulation.

A 16-channel, 12-bit, 2 MS/s Digitizer ASIC, called ColdADC, intended as the digitizer for the DUNE Single-Phase Far Detector, is presented. ColdADC has an input buffer per channel and two 16 MS/s Pipelined ADCs that each digitizes the output of 8 preamplifiers. The ASIC was developed using specialized design techniques for long-term reliability in cryogenic environments and a customized cryogenic standard cell library. ColdADC, with a die area of approximately 52.4 mm2 and fabricated in 65 nm CMOS technology, achieves 186 µV-rms noise performance and 10.5-bit ENOB at a temperature of 77 K, while dissipating 425 mW (26.6 mW per channel). The prototype has been successfully interoperated with the intended preamplifier at cryogenic temperatures.

Results from testing of position-sensitive capacitive Frisch-grid (PSCFG) TIBr gamma-ray detectors are presented. Due to its high atomic number, high density, and low Fano factor, TIBr offers excellent energy resolution and high detection efficiency over a wide energy range, thus providing significant advantages over other detector materials commonly used in hand-held instruments. Using high-fidelity 3-D position sensing enables the response non-uniformity caused by defects in the TIBr crystal to be corrected, thereby offering an approach to overcome one of the technical barriers limiting the use of this promising semiconductor material. By utilizing the 3-D position information, temporal and spatial variations of the charge collection efficiency are presented, which provide microscopic characterization of the PSCFG devices. Using the 3-D response correction technique, the best energy resolution measured is

Deoxyribonucleic acid (DNA) has been shown to interact with poly(vinylidene fluoride) (PVDF), resulting in increased electroactive β-phase content and piezoelectric response under certain casting conditions. While the use of DNA as a self-poling agent has the potential to eliminate the need for additional stretching and poling steps in piezoelectric PVDF films, the mechanism through which DNA poles PVDF is not yet well understood, hindering the optimization of the process. Here, we performed a study to screen the effects of drying temperature, amount of nucleic acid additive, weight percentage of PVDF, spin-casting speed, and molecular weight of PVDF on spin-cast PVDF-DNA hybrid films. The phase composition and piezoelectric response of the resulting films were quantified using Fourier-transform infrared spectroscopy (FTIR) and a piezoelectric meter, respectively. We found that DNA has a significant influence on the PVDF films' β-phase content; however, this effect is masked at low drying temperatures. While DNA facilitates the formation of the electroactive β-phase of PVDF, we could not find evidence that DNA enhances PVDF's piezoelectric response. These results conflict with previous literature, which reported that DNA aligns the dipoles of PVDF such that the films exhibited a significant piezoelectric response. Overall, this study identifies that nucleic acid additives - under certain casting conditions - have an important effect on PVDF film phase composition.

Present retinal neuroprostheses have limited performance capabilities due to indiscriminate activation of different neural pathways. Based on our success in differentially activating ON and OFF cells using high frequency stimuli in a healthy retina, in this study we explored whether we could achieve similar differential activation between these two cell types but in degenerate retina. We found that after blocking the synaptic network, ON retinal ganglion cells (RGCs) could be differentially activated at higher frequencies (4 - 6 kHz) and amplitudes (200 - 240 µA), and OFF RGCs at relatively lower amplitudes (80 - 160 µA) across all tested frequencies. We further found that both cell types could be controlled by quickly modulating the frequency using short stimulation bursts. This work takes us one step closer to reducing the likelihood of indiscriminate activation of RGCs by accurately controlling the activation of functionally-distinct neural pathways.

The Thulium fiber laser (TFL) is being explored as a potential alternative to the gold standard Holmium:YAG laser for infrared laser ablation of kidney stones. Laser-induced vapor bubbles contribute to both the ablation mechanism and stone retropulsion. In this preliminary study, a biocompatible surfactant with concentrations of 1-5% was used to enhance the vapor bubble dimensions during the laser pulse. Bubble dimensions using surfactant increased on average by 25% compared with water only (control). With further development, introduction of the surfactant into the saline irrigation flow typically delivered through the working channel of the ureteroscope during laser lithotripsy, may contribute to more efficient stone ablation.Clinical Relevance-This preliminary study demonstrates that the dimensions of laser-induced vapor bubbles created during infrared laser lithotripsy can be enhanced by up to 25%, for potential clinical translation into more efficient lithotripsy and use in the "popcorn" ablation method.

Retinal microprostheses strive to evoke a sense of vision in individuals blinded by outer retinal degenerative diseases, by electrically stimulating the surviving retina. It is widely suspected that a stimulation strategy that can selectively activate different retinal ganglion cell types will improve the quality of evoked phosphenes. Previous efforts towards this goal demonstrated the potential for selective ON and OFF brisk-transient cell activation using high-rate (2000 pulses per second, PPS) stimulation. Here, we build upon this earlier work by testing an additional rate of stimulation and additional cell populations. We find considerable variability in responses both within and across individual cell types, but show that the sensitivity of a ganglion cell to repetitive stimulation is highly correlated to its single-pulse threshold. Consistent with this, we found thresholds for both stimuli to be correlated to soma size, and thus likely mediated by the properties of the axon initial segment. The ultimate efficacy of high-rate stimulation will likely depend on several factors, chief among which are (a) the residual ganglion types, and (b) the stimulation frequency.

Recent advances in machine learning and computer graphics have made it easier to convincingly manipulate video and audio. These so-called deep-fake videos range from complete full-face synthesis and replacement (face-swap), to complete mouth and audio synthesis and replacement (lip-sync), and partial word-based audio and mouth synthesis and replacement. Detection of deep fakes with only a small spatial and temporal manipulation is particularly challenging. We describe a technique to detect such manipulated videos by exploiting the fact that the dynamics of the mouth shape - visemes - are occasionally inconsistent with a spoken phoneme. We focus on the visemes associated with words having the sound M (mama), B (baba), or P (papa) in which the mouth must completely close in order to pronounce these phonemes. We observe that this is not the case in many deep-fake videos. Such phoneme-viseme mismatches can, therefore, be used to detect even spatially small and temporally localized manipulations. We demonstrate the efficacy and robustness of this approach to detect different types of deep-fake videos, including in-the-wild deep fakes.

The accurate knowledge of Heat Transfer Coefficients (which shows the amount of heat exchanged between an object and its environment) is essential for the design of precise heat transfer operations. There are many numerical methods available (usually based on heuristics, like Genetic Algorithms, Particle Swarm Optimization, Fireworks Algorithm, etc.) for solving the Inverse Heat Conduction Problem, but these usually have a high computational demand. As an alternative, it is possible to design an Artificial Neural Network model efficiently predicting the shape of the HTC function based on the temperature signals recorded during real heat treatment processes. This paper presents a feed-forward network and examines various aspects (learning time, training loss, validation loss) of the model using three different hardware configurations (single CPU, single GPU, multiple GPUs).

There are numerous applications based on the simulation of heat transfer. A special kind of these tasks is the Inverse Heat Conduction Problem, where the thermal characteristics of the workpiece and the surrounding environment are known, and the objective of the search is to determine the Heat Transfer Coefficient. The most commonly-used methods for solving this problem are based on a discretized numerical analysis (finite-element or finite-difference based methods) combined with some kind of heuristics to find the optimal value. There are various computer-assisted numerical methods available, but these usually have unacceptably high computational demand. Using parallel algorithms and architectures (especially the massively parallel graphics accelerators) makes it possible to speed up the heat transfer simulation sub-procedure of the search. This paper presents a novel CUDA based data-parallel solution which gives the same accuracy as the traditional methods but with $32 \times $ smaller runtime. This method is able to support any kind of heuristic algorithms (Genetic Algorithm, Particle Swarm Optimization, Fireworks Algorithms, Gradient-based methods) in order to find the appropriate HTC value.

Reconstruction of the heat transfer coefficients can be solved in several ways and several configurations. The Inverse Heat Transfer Problems (IHCP) collect all type of algorithms which try to solve the Heat Transfer Problems (calculating the heat transfer coefficients) using inverse way. The bio-inspired solutions as the numerical method could solve this problem. Due to some limitations of the bio-inspired solutions, the appropriate results can be reached an easier way by hybrid methods. A hybrid method is concatenating the Fireworks Algorithm (FWA) and a gradient method at the end of the FWA calculation. The objective function has been minimized by the deviation of a measured and the calculated temperatures.

This research presents the design and implementation of hardware for face detection based on the system of the Viola-Jones Algorithm in the Field Programmable Gate Array (FPGA) using VHSIC Hardware Description Language (VHDL); VHSIC (Very High Speed Integrated Circuit). The data used in this study is based on real conditions and uses sample images where in this research we use the images sample with dimensions 450×375 pixels which is like the resolution of webcam that using for the Xilinx Zybo (Zynq Board). In this research, the proposed architecture is using VHDL and implemented in Xilinx Zybo Zynq-7000 ARM/FPGA SoC Trainer Board. To measure the performance of the FPGA-based face detection system from the Viola-Jones Algorithm, the number of frames processed per second (FPS) must be measured, where the processes must be designed which are divided into two types of partitioning, there are software and hardware designs. This is accomplished by measuring the time elapsed from capturing an image to displaying the detection results. More specifically this is the time elapsed from the start of a frame capture to the end of drawing face bounding boxes on to the captured image. An experiment was conducted to gather the minimum, maximum and average FPS of FPGA face detection system of the Viola-Jones Algorithm. From the process of this research we’ve been got hardware latency just for the 3rd modules implemented is 125,207 millisecond, whereas software implemented (run on ARM Cortex 9 / Hardcore Zybo Processor) is 3642,993 millisecond, so speedup by implementing these 3 modules as a hardware accelerator is 29,095 rounds.