Search

We present Prompt Diffusion, a framework for enabling in-context learning in diffusion-based generative models. Given a pair of task-specific example images, such as depth from/to image and scribble from/to image, and a text guidance, our model automatically understands the underlying task and performs the same task on a new query image following the text guidance. To achieve this, we propose a vision-language prompt that can model a wide range of vision-language tasks and a diffusion model that takes it as input. The diffusion model is trained jointly over six different tasks using these prompts. The resulting Prompt Diffusion model is the first diffusion-based vision-language foundation model capable of in-context learning. It demonstrates high-quality in-context generation on the trained tasks and generalizes effectively to new, unseen vision tasks with their respective prompts. Our model also shows compelling text-guided image editing results. Our framework, with code publicly available at https://github.com/Zhendong-Wang/Prompt-Diffusion, aims to facilitate research into in-context learning for computer vision.

We present a differentiable joint pruning and quantization (DJPQ) scheme. We frame neural network compression as a joint gradient-based optimization problem, trading off between model pruning and quantization automatically for hardware efficiency. DJPQ incorporates variational information bottleneck based structured pruning and mixed-bit precision quantization into a single differentiable loss function. In contrast to previous works which consider pruning and quantization separately, our method enables users to find the optimal trade-off between both in a single training procedure. To utilize the method for more efficient hardware inference, we extend DJPQ to integrate structured pruning with power-of-two bit-restricted quantization. We show that DJPQ significantly reduces the number of Bit-Operations (BOPs) for several networks while maintaining the top-1 accuracy of original floating-point models (e.g., 53x BOPs reduction in ResNet18 on ImageNet, 43x in MobileNetV2). Compared to the conventional two-stage approach, which optimizes pruning and quantization independently, our scheme outperforms in terms of both accuracy and BOPs. Even when considering bit-restricted quantization, DJPQ achieves larger compression ratios and better accuracy than the two-stage approach., Comment: Accepted to ECCV 2020

Lithium ion-doped polyglycerol sebacate scaffold(PGS)-Li was synthesized by adding lithium ions to polyglycerol sebacate(PGS) during its crosslinking process due to the specific effects of lithium ions on periodontal ligament cells, cementoblasts and the eminent performance of PGS. The molecular mass, composition, structure, porosity, thermal properties, and hydrophilicity of the composite were characterized by gel permeation chromatography(GPC), Fourier transform infrared spectroscopy(FTIR), inductively coupled plasma optical emission spectrometer(ICP-OES), scanning electron microscopy(SEM), X-ray photoelectron spectroscopy(XPS), thermogravimetric analyzer(TGA) and contact angle measurments, and the degradation of the material was evaluated by in vitro degradation experiments. The biological activity of PGS-Li scaffold was detected by calcein-AM staining and cytotoxicity test. The results indicate that PGS-Li scaffold has been successfully synthesized, which has similar composition and structure to PGS, but slightly larger molecular weight. In addition, the porosity and pore size of PGS-Li scaffold basically meet the requirements of engineering scaffold materials and the seaffold shows better performance in terms of hydrophilicity and thermal stability than PGS. In vitro degradation experimental results show that the degradation rate of PGS-Li scaffold is higher than that of PGS. What’s more, the results of cytotoxicity test and cell staining show that there is no significant difference in the proliferation and cell morphology of cementoblasts.