Search

Self-supervised learning (SSL) is a powerful tool in machine learning, but understanding the learned representations and their underlying mechanisms remains a challenge. This paper presents an in-depth empirical analysis of SSL-trained representations, encompassing diverse models, architectures, and hyperparameters. Our study reveals an intriguing aspect of the SSL training process: it inherently facilitates the clustering of samples with respect to semantic labels, which is surprisingly driven by the SSL objective's regularization term. This clustering process not only enhances downstream classification but also compresses the data information. Furthermore, we establish that SSL-trained representations align more closely with semantic classes rather than random classes. Remarkably, we show that learned representations align with semantic classes across various hierarchical levels, and this alignment increases during training and when moving deeper into the network. Our findings provide valuable insights into SSL's representation learning mechanisms and their impact on performance across different sets of classes.

Accessing the low-energy non-equilibrium dynamics of materials and their polaritons with simultaneous high spatial and temporal resolution has been a bold frontier of electron microscopy in recent years. One of the main challenges lies in the ability to retrieve extremely weak signals while simultaneously disentangling amplitude and phase information. Here, we present Free-Electron Ramsey Imaging (FERI), a microscopy approach based on light-induced electron modulation that enables coherent amplification of optical near-fields in electron imaging. We provide simultaneous time-, space-, and phase-resolved measurements of a micro-drum made from a hexagonal boron nitride membrane visualizing the sub-cycle dynamics of 2D polariton wavepackets therein. The phase-resolved measurements reveals vortex-anti-vortex singularities on the polariton wavefronts, together with an intriguing phenomenon of a traveling wave mimicking the amplitude profile of a standing wave. Our experiments show a 20-fold coherent amplification of the near-field signal compared to conventional electron near-field imaging, resolving peak field intensities in the order of ~W/cm2, corresponding to field amplitudes of a few kV/m. As a result, our work paves the way for spatio-temporal electron microscopy of biological specimens and quantum materials, exciting yet delicate samples that are currently difficult to investigate.

The complex range of interactions between electrons and electromagnetic fields gave rise to countless scientific and technological advances. A prime example is photon-induced nearfield electron microscopy (PINEM), enabling the detection of confined electric fields in illuminated nanostructures with unprecedented spatial resolution. However, PINEM is limited by its dependence on strong fields, making it unsuitable for sensitive samples, and its inability to resolve complex phasor information. Here, we leverage the nonlinear, over-constrained nature of PINEM to present an algorithmic microscopy approach, achieving far superior nearfield imaging capabilities. Our algorithm relies on free-electron Ramsey-type interferometry to produce orders-of-magnitude improvement in sensitivity and ambiguity-immune nearfield phase reconstruction, both of which are optimal when the electron exhibits a fully quantum behavior. Our results demonstrate the potential of combining algorithmic approaches with novel modalities in electron microscopy, and may lead to various applications from imaging sensitive biological samples to performing full-field tomography of confined light.

We consider gluons scattering in Type IIB string theory on AdS$_5\times S^5/\mathbb{Z}_2$ in the presence of D7 branes, which is dual to the flavor multiplet correlator in a certain 4d $\mathcal{N}=2$ $USp(2N)$ gauge theory with $SO(8)$ flavor symmetry and complexified coupling $\tau$. We compute this holographic correlator in the large $N$ and finite $\tau$ expansion using constraints from derivatives of the mass deformed sphere free energy, which we compute to all orders in $1/N$ and finite $\tau$ using supersymmetric localization. In particular, we fix the $F^4$ higher derivative correction to gluon scattering on AdS at finite string coupling $\tau_s=\tau$ in terms of Jacobi theta functions, which feature the expected relations between the $SL(2,\mathbb{Z})$ duality and the $SO(8)$ triality of the CFT, and match it to the known flat space term. We also use the flat space limit to compute $D^2F^4$ corrections of the correlator at finite $\tau$ in terms of a non-holomorphic Eisenstein series. At weak string coupling, we find that the AdS correlator takes a form which is remarkably similar to that of the flat space Veneziano amplitude., Comment: 26 pages plus appendices, v6 typo fixed

MLaaS Service Providers (SPs) holding a Neural Network would like to keep the Neural Network weights secret. On the other hand, users wish to utilize the SPs' Neural Network for inference without revealing their data. Multi-Party Computation (MPC) offers a solution to achieve this. Computations in MPC involve communication, as the parties send data back and forth. Non-linear operations are usually the main bottleneck requiring the bulk of communication bandwidth. In this paper, we focus on ResNets, which serve as the backbone for many Computer Vision tasks, and we aim to reduce their non-linear components, specifically, the number of ReLUs. Our key insight is that spatially close pixels exhibit correlated ReLU responses. Building on this insight, we replace the per-pixel ReLU operation with a ReLU operation per patch. We term this approach 'Block-ReLU'. Since different layers in a Neural Network correspond to different feature hierarchies, it makes sense to allow patch-size flexibility for the various layers of the Neural Network. We devise an algorithm to choose the optimal set of patch sizes through a novel reduction of the problem to the Knapsack Problem. We demonstrate our approach in the semi-honest secure 3-party setting for four problems: Classifying ImageNet using ResNet50 backbone, classifying CIFAR100 using ResNet18 backbone, Semantic Segmentation of ADE20K using MobileNetV2 backbone, and Semantic Segmentation of Pascal VOC 2012 using ResNet50 backbone. Our approach achieves competitive performance compared to a handful of competitors. Our source code is publicly available: https://github.com/yg320/secure_inference.

We consider the long-standing question of finding a parameter of a class of probability distributions that characterizes its PAC learnability. We provide a rather surprising answer - no such parameter exists. Our techniques allow us to show similar results for several general notions of characterizing learnability and for several learning tasks. We show that there is no notion of dimension that characterizes the sample complexity of learning distribution classes. We then consider the weaker requirement of only characterizing learnability (rather than the quantitative sample complexity function). We propose some natural requirements for such a characterization and go on to show that there exists no characterization of learnability that satisfies these requirements for classes of distributions. Furthermore, we show that our results hold for various other learning problems. In particular, we show that there is no notion of dimension characterizing (or characterization of learnability) for any of the tasks: classification learning for distribution classes, learning of binary classifications w.r.t. a restricted set of marginal distributions, and learnability of classes of real-valued functions with continuous losses.

We propose SampleDepth, a Convolutional Neural Network (CNN), that is suited for an adaptive LiDAR. Typically,LiDAR sampling strategy is pre-defined, constant and independent of the observed scene. Instead of letting a LiDAR sample the scene in this agnostic fashion, SampleDepth determines, adaptively, where it is best to sample the current frame. To do that, SampleDepth uses depth samples from previous time steps to predict a sampling mask for the current frame. Crucially, SampleDepth is trained to optimize the performance of a depth completion downstream task. SampleDepth is evaluated on two different depth completion networks and two LiDAR datasets, KITTI Depth Completion and the newly introduced synthetic dataset, SHIFT. We show that SampleDepth is effective and suitable for different depth completion downstream tasks.

Physical design refers to mathematical optimization of a desired objective (e.g. strong light--matter interactions, or complete quantum state transfer) subject to the governing dynamical equations, such as Maxwell's or Schrodinger's differential equations. Computing an optimal design is challenging: generically, these problems are highly nonconvex and finding global optima is NP hard. Here we show that for linear-differential-equation dynamics (as in linear electromagnetism, elasticity, quantum mechanics, etc.), the physical-design optimization problem can be transformed to a sparse-matrix, quadratically constrained quadratic program (QCQP). Sparse QCQPs can be tackled with convex optimization techniques (such as semidefinite programming) that have thrived for identifying global bounds and high-performance designs in other areas of science and engineering, but seemed inapplicable to the design problems of wave physics. We apply our formulation to prototypical photonic design problems, showing the possibility to compute fundamental limits for large-area metasurfaces, as well as the identification of designs approaching global optimality. Looking forward, our approach highlights the promise of developing bespoke algorithms tailored to specific physical design problems., Comment: 9 pages, 4 figures, plus references and Supplementary Materials

We study a model of dynamic combinatorial assignment of indivisible objects without money. We introduce a new solution concept called ``dynamic approximate competitive equilibrium from equal incomes'' (DACEEI), which stipulates that markets must approximately clear in almost all time periods. A naive repeated application of approximate competitive equilibrium from equal incomes (Budish, 2011) does not yield a desirable outcome because the approximation error in market-clearing compounds quickly over time. We therefore develop a new version of the static approximate competitive equilibrium from carefully constructed random budgets which ensures that, in expectation, markets clear exactly. We then use it to design the ``online combinatorial assignment mechanism'' (OCAM) which implements a DACEEI with high probability. The OCAM is (i) group-strategyproof up to one object (ii) envy-free up to one object for almost all agents (iii) approximately market-clearing in almost all periods with high probability when the market is large and arrivals are random. Applications include refugee resettlement, daycare assignment, and airport slot allocation.

Cavity quantum electrodynamics (QED), wherein a quantum emitter is coupled to electromagnetic cavity modes, is a powerful platform for implementing quantum sensors, memories, and networks. However, due to the fundamental tradeoff between gate fidelity and execution time, as well as limited scalability, the use of cavity-QED for quantum computation was overtaken by other architectures. Here, we introduce a new element into cavity-QED - a free charged particle, acting as a flying qubit. Using free electrons as a specific example, we demonstrate that our approach enables ultrafast, deterministic and universal discrete-variable quantum computation in a cavity-QED-based architecture, with potentially improved scalability. Our proposal hinges on a novel excitation blockade mechanism in a resonant interaction between a free-electron and a cavity polariton. This nonlinear interaction is faster by several orders of magnitude with respect to current photon-based cavity-QED gates, enjoys wide tunability and can demonstrate fidelities close to unity. Furthermore, our scheme is ubiquitous to any cavity nonlinearity, either due to light-matter coupling as in the Jaynes-Cummings model or due to photon-photon interactions as in a Kerr-type many-body system. In addition to promising advancements in cavity-QED quantum computation, our approach paves the way towards ultrafast and deterministic generation of highly-entangled photonic graph states and is applicable to other quantum technologies involving cavity-QED.

The problem of foreground material signature extraction in an intimate (nonlinear) mixing setting is considered. It is possible for a foreground material signature to appear in combination with multiple background material signatures. We explore a framework for foreground material signature extraction based on a patch model that accounts for such background variation. We identify data conditions under which a foreground material signature can be extracted up to scaling and elementwise-inverse variations. We present algorithms based on volume minimization and endpoint member identification to recover foreground material signatures under these conditions. Numerical experiments on real and synthetic data illustrate the efficacy of the proposed algorithms., Comment: 41 pages, 5 figures

Current solutions to quantum vulnerabilities of widely used cryptographic schemes involve migrating users to post-quantum schemes before quantum attacks become feasible. This work deals with protecting quantum procrastinators: users that failed to migrate to post-quantum cryptography in time. To address this problem in the context of digital signatures, we introduce a technique called signature lifting, that allows us to lift a deployed pre-quantum signature scheme satisfying a certain property to a post-quantum signature scheme that uses the same keys. Informally, the said property is that a post-quantum one-way function is used "somewhere along the way" to derive the public-key from the secret-key. Our constructions of signature lifting relies heavily on the post-quantum digital signature scheme Picnic (Chase et al., CCS'17). Our main case-study is cryptocurrencies, where this property holds in two scenarios: when the public-key is generated via a key-derivation function or when the public-key hash is posted instead of the public-key itself. We propose a modification, based on signature lifting, that can be applied in many cryptocurrencies for securely spending pre-quantum coins in presence of quantum adversaries. Our construction improves upon existing constructions in two major ways: it is not limited to pre-quantum coins whose ECDSA public-key has been kept secret (and in particular, it handles all coins that are stored in addresses generated by HD wallets), and it does not require access to post-quantum coins or using side payments to pay for posting the transaction., Comment: Minor revision

We show that it is theoretically possible to use higher energy levels for storing and controlling two qubits within a superconducting transmon. This is done by identifying energy levels as product states between multiple effecitve qubits. As a proof of concept we realise a complete set of gates necessary for universal computing by numerically optimising control pulses for single qubit gates on each of the qubits, entangling gates between the two qubits in one transmon, and an entangling gate between two qubits from two coupled transmons. The optimisation considers parameters which could make it possible to validate this experimentally. With these control pulses it is in principle possible to double the number of available qubits without any overhead in hardware. The additional qubits could be used in algorithms which need many short-living qubits such as syndrom qubits in error correction or by embedding effecitve higher connectivity in qubit networks., Comment: 14 pages, 12 figures

The AGN STORM 2 collaboration targeted the Seyfert 1 galaxy Mrk 817 for a year-long multiwavelength, coordinated reverberation mapping campaign including HST, Swift, XMM-Newton, NICER, and ground-based observatories. Early observations with NICER and XMM revealed an X-ray state ten times fainter than historical observations, consistent with the presence of a new dust-free, ionized obscurer. The following analysis of NICER spectra attributes variability in the observed X-ray flux to changes in both the column density of the obscurer by at least one order of magnitude ($N_\mathrm{H}$ ranges from $2.85\substack{+0.48\\ -0.33} \times 10^{22}\text{ cm}^{-2}$ to $25.6\substack{+3.0\\ -3.5} \times 10^{22} \text{ cm}^{-2}$) and the intrinsic continuum brightness (the unobscured flux ranges from $10^{-11.8}$ to $10^{-10.5}$ erg s$^{-1}$ cm$^{-2}$ ). While the X-ray flux generally remains in a faint state, there is one large flare during which Mrk 817 returns to its historical mean flux. The obscuring gas is still present at lower column density during the flare but it also becomes highly ionized, increasing its transparency. Correlation between the column density of the X-ray obscurer and the strength of UV broad absorption lines suggests that the X-ray and UV continua are both affected by the same obscuration, consistent with a clumpy disk wind launched from the inner broad line region., Comment: 19 pages, 9 figures, Accepted to ApJ

We present reverberation mapping measurements for the prominent ultraviolet broad emission lines of the active galactic nucleus Mrk817 using 165 spectra obtained with the Cosmic Origins Spectrograph on the Hubble Space Telescope. Our ultraviolet observations are accompanied by X-ray, optical, and near-infrared observations as part of the AGN Space Telescope and Optical Reverberation Mapping Program 2 (AGN STORM 2). Using the cross-correlation lag analysis method, we find significant correlated variations in the continuum and emission-line light curves. We measure rest-frame delayed responses between the far-ultraviolet continuum at 1180 A and Ly$\alpha$ $\lambda1215$ A ($10.4_{-1.4}^{+1.6}$ days), N V $\lambda1240$ A ($15.5_{-4.8}^{+1.0}$days), SiIV + OIV] $\lambda1397$ A ($8.2_{-1.4}^{+1.4}$ days), CIV $\lambda1549$ A ($11.8_{-2.8}^{+3.0}$ days), and HeII $\lambda1640$ A ($9.0_{-1.9}^{+4.5}$ days) using segments of the emission-line profile that are unaffected by absorption and blending, which results in sampling different velocity ranges for each line. However, we find that the emission-line responses to continuum variations are more complex than a simple smoothed, shifted, and scaled version of the continuum light curve. We also measure velocity-resolved lags for the Ly$\alpha$, and CIV emission lines. The lag profile in the blue wing of Ly$\alpha$ is consistent with virial motion, with longer lags dominating at lower velocities, and shorter lags at higher velocities. The CIV lag profile shows the signature of a thick rotating disk, with the shortest lags in the wings, local peaks at $\pm$ 1500 $\rm km\,s^{-1}$, and a local minimum at line center. The other emission lines are dominated by broad absorption lines and blending with adjacent emission lines. These require detailed models, and will be presented in future work., Comment: Submitted to ApJ. 25 pages, 8 figures, and 6 tables

We present the first results from a 100-day Swift, NICER and ground-based X-ray/UV/optical reverberation mapping campaign of the Narrow-Line Seyfert 1 Mrk 335, when it was in an unprecedented low X-ray flux state. Despite dramatic suppression of the X-ray variability, we still observe UV/optical lags as expected from disk reverberation. Moreover, the UV/optical lags are consistent with archival observations when the X-ray luminosity was >10 times higher. Interestingly, both low- and high-flux states reveal UV/optical lags that are 6-11 times longer than expected from a thin disk. These long lags are often interpreted as due to contamination from the broad line region, however the u band excess lag (containing the Balmer jump from the diffuse continuum) is less prevalent than in other AGN. The Swift campaign showed a low X-ray-to-optical correlation (similar to previous campaigns), but NICER and ground-based monitoring continued for another two weeks, during which the optical rose to the highest level of the campaign, followed ~10 days later by a sharp rise in X-rays. While the low X-ray countrate and relatively large systematic uncertainties in the NICER background make this measurement challenging, if the optical does lead X-rays in this flare, this indicates a departure from the zeroth-order reprocessing picture. If the optical flare is due to an increase in mass accretion rate, this occurs on much shorter than the viscous timescale. Alternatively, the optical could be responding to an intrinsic rise in X-rays that is initially hidden from our line-of-sight., Comment: Accepted for publication in the Astrophysical Journal. 15 pages, 8 figures, 3 tables

Finetuning a pretrained model has become a standard approach for training neural networks on novel tasks, resulting in fast convergence and improved performance. In this work, we study an alternative finetuning method, where instead of finetuning all the weights of the network, we only train a carefully chosen subset of layers, keeping the rest of the weights frozen at their initial (pretrained) values. We demonstrate that \emph{subset finetuning} (or SubTuning) often achieves accuracy comparable to full finetuning of the model, and even surpasses the performance of full finetuning when training data is scarce. Therefore, SubTuning allows deploying new tasks at minimal computational cost, while enjoying the benefits of finetuning the entire model. This yields a simple and effective method for multi-task learning, where different tasks do not interfere with one another, and yet share most of the resources at inference time. We demonstrate the efficiency of SubTuning across multiple tasks, using different network architectures and pretraining methods.

We introduce an approach for solving PDEs over manifolds using physics informed neural networks whose architecture aligns with spectral methods. The networks are trained to take in as input samples of an initial condition, a time stamp and point(s) on the manifold and then output the solution's value at the given time and point(s). We provide proofs of our method for the heat equation on the interval and examples of unique network architectures that are adapted to nonlinear equations on the sphere and the torus. We also show that our spectral-inspired neural network architectures outperform the standard physics informed architectures. Our extensive experimental results include generalization studies where the testing dataset of initial conditions is randomly sampled from a significantly larger space than the training set., Comment: 25 pages

We initiate a study of computable online (c-online) learning, which we analyze under varying requirements for "optimality" in terms of the mistake bound. Our main contribution is to give a necessary and sufficient condition for optimal c-online learning and show that the Littlestone dimension no longer characterizes the optimal mistake bound of c-online learning. Furthermore, we introduce anytime optimal (a-optimal) online learning, a more natural conceptualization of "optimality" and a generalization of Littlestone's Standard Optimal Algorithm. We show the existence of a computational separation between a-optimal and optimal online learning, proving that a-optimal online learning is computationally more difficult. Finally, we consider online learning with no requirements for optimality, and show, under a weaker notion of computability, that the finiteness of the Littlestone dimension no longer characterizes whether a class is c-online learnable with finite mistake bound. A potential avenue for strengthening this result is suggested by exploring the relationship between c-online and CPAC learning, where we show that c-online learning is as difficult as improper CPAC learning., Comment: To appear in the 34th International Conference on Algorithmic Learning Theory (ALT), 2023

We study the application of a square perturbing lattice to the naturally forming hexagonal arrays of dipolar droplets in a dipolar Bose-Einstein condensate. We find that the application of the lattice causes spontaneous pattern formation and leads to frustration in some regimes. For certain parameters, the ground state has neither the symmetry of the intrinsic hexagonal supersolid nor the symmetry of the square lattice. These results may give another axis on which to explore dipolar Bose-Einstein condensates and to probe the nature of supersolidity.

Growing soft materials which follow a 3D path in space are critical to applications such as search and rescue and minimally invasive surgery. Here, we present a concept for a single-input growing multi-stable soft material, based on a constrained straw-like structure. This class of materials are capable of maneuvering and transforming their configuration by elongation while executing multiple turns. This is achieved by sequenced actuation of bi-stable frusta with predefined constraints. Internal viscous flow and variations in the stability threshold of the individual cells enable sequencing and control of the robot's movement so as to follow a desired 3D path as the structure grows. We derive a theoretical description of the shape and dynamics resulting from a particular set of constraints. To validate the model and demonstrate the suggested concept, we present experiments of maneuvering in models of residential and biological environments. In addition to performing complex 3D maneuvers, the tubular structure of these robots may also be used as a conduit to reach inaccessible regions, which is demonstrated experimentally., Comment: 10 pages and 3 figures

Reading text in real-world scenarios often requires understanding the context surrounding it, especially when dealing with poor-quality text. However, current scene text recognizers are unaware of the bigger picture as they operate on cropped text images. In this study, we harness the representative capabilities of modern vision-language models, such as CLIP, to provide scene-level information to the crop-based recognizer. We achieve this by fusing a rich representation of the entire image, obtained from the vision-language model, with the recognizer word-level features via a gated cross-attention mechanism. This component gradually shifts to the context-enhanced representation, allowing for stable fine-tuning of a pretrained recognizer. We demonstrate the effectiveness of our model-agnostic framework, CLIPTER (CLIP TExt Recognition), on leading text recognition architectures and achieve state-of-the-art results across multiple benchmarks. Furthermore, our analysis highlights improved robustness to out-of-vocabulary words and enhanced generalization in low-data regimes., Comment: Accepted for publication by ICCV 2023

Visual Question Answering (VQA) and Image Captioning (CAP), which are among the most popular vision-language tasks, have analogous scene-text versions that require reasoning from the text in the image. Despite their obvious resemblance, the two are treated independently and, as we show, yield task-specific methods that can either see or read, but not both. In this work, we conduct an in-depth analysis of this phenomenon and propose UniTNT, a Unified Text-Non-Text approach, which grants existing multimodal architectures scene-text understanding capabilities. Specifically, we treat scene-text information as an additional modality, fusing it with any pretrained encoder-decoder-based architecture via designated modules. Thorough experiments reveal that UniTNT leads to the first single model that successfully handles both task types. Moreover, we show that scene-text understanding capabilities can boost vision-language models' performance on general VQA and CAP by up to 2.69% and 0.6 CIDEr, respectively.

Multiplication layers are a key component in various influential neural network modules, including self-attention and hypernetwork layers. In this paper, we investigate the approximation capabilities of deep neural networks with intermediate neurons connected by simple multiplication operations. We consider two classes of target functions: generalized bandlimited functions, which are frequently used to model real-world signals with finite bandwidth, and Sobolev-Type balls, which are embedded in the Sobolev Space $\mathcal{W}^{r,2}$. Our results demonstrate that multiplicative neural networks can approximate these functions with significantly fewer layers and neurons compared to standard ReLU neural networks, with respect to both input dimension and approximation error. These findings suggest that multiplicative gates can outperform standard feed-forward layers and have potential for improving neural network design.

Topological Interlocking Structures (TIS) have been increasingly studied in the past two decades. However, some fundamental questions concerning the effects of Young's modulus and the friction coefficient on the structural mechanics of the most common type of TIS application - centrally loaded slabs - are not yet clear. Here, we present a first-of-its-kind parametric study based on the Level-Set-Discrete-element-Method that aims to clarify how these two parameters affect multiple aspects of the behavior and failure of centrally-loaded TIS slabs. This includes the evolution of the structural response up to and including failure, the foremost structural response parameters, and the residual carrying capacity. We find that the structural response parameters in TIS slabs scale linearly with Young's modulus, that they saturate with the friction coefficient, and that the saturated response provides an upper-bound on the capacity of centrally loaded TIS slabs reported in the literature. This, together with additional findings, insights, and observations, comprise a novel contribution to our understanding of the interlocked structural form.

The diagnosis of primary bone tumors is challenging, as the initial complaints are often non-specific. Early detection of bone cancer is crucial for a favorable prognosis. Incidentally, lesions may be found on radiographs obtained for other reasons. However, these early indications are often missed. In this work, we propose an automatic algorithm to detect bone lesions in conventional radiographs to facilitate early diagnosis. Detecting lesions in such radiographs is challenging: first, the prevalence of bone cancer is very low; any method must show high precision to avoid a prohibitive number of false alarms. Second, radiographs taken in health maintenance organizations (HMOs) or emergency departments (EDs) suffer from inherent diversity due to different X-ray machines, technicians and imaging protocols. This diversity poses a major challenge to any automatic analysis method. We propose to train an off-the-shelf object detection algorithm to detect lesions in radiographs. The novelty of our approach stems from a dedicated preprocessing stage that directly addresses the diversity of the data. The preprocessing consists of self-supervised region-of-interest detection using vision transformer (ViT), and a foreground-based histogram equalization for contrast enhancement to relevant regions only. We evaluate our method via a retrospective study that analyzes bone tumors on radiographs acquired from January 2003 to December 2018 under diverse acquisition protocols. Our method obtains 82.43% sensitivity at 1.5% false-positive rate and surpasses existing preprocessing methods. For lesion detection, our method achieves 82.5% accuracy and an IoU of 0.69. The proposed preprocessing method enables to effectively cope with the inherent diversity of radiographs acquired in HMOs and EDs., Comment: Citation: Tal Zimbalist, Ronnie Rosen, Keren Peri-Hanania, Yaron Caspi, Bar Rinott, Carmel Zeltser-Dekel, Eyal Bercovich, Yonina C. Eldar, Shai Bagon, Detecting bone lesions in X-ray under diverse acquisition conditions, J. Med. Imag. 11(2), 024502 (2024), doi: 10.1117/1.JMI.11.2.024502

Topologically interlocked materials and structures, which are assemblies of unbonded interlocking building blocks, are promising concepts for versatile structural applications. They have been shown to exhibit exceptional mechanical properties, including outstanding combinations of stiffness, strength, and toughness, beyond those achievable with common engineering materials. Recent work has established a theoretical upper limit for the strength and toughness of beam-like topologically interlocked structures. However, this theoretical limit is only achievable for structures with unrealistically high friction coefficients; therefore, it remains unknown whether it is achievable in actual structures. Here, we demonstrate that a hierarchical approach for topological interlocking, inspired by biological systems, overcomes these limitations and provides a path toward optimized mechanical performance. We consider beam-like topologically interlocked structures that present a sinusoidal surface morphology with controllable amplitude and wavelength and examine the properties of the structures using numerical simulations. The results show that the presence of surface morphologies increases the effective frictional strength of the interfaces and, if well-designed, enables us to reach the theoretical limit of the structural carrying capacity with realistic friction coefficients. Furthermore, we observe that the contribution of the surface morphology to the effective friction coefficient of the interface is well described by a criterion combining the surface curvature and surface gradient. Our study demonstrates the ability to architecture the surface morphology in beam-like topological interlocked structures to significantly enhance its structural performance.

Text-guided image editing can have a transformative impact in supporting creative applications. A key challenge is to generate edits that are faithful to input text prompts, while consistent with input images. We present Imagen Editor, a cascaded diffusion model built, by fine-tuning Imagen on text-guided image inpainting. Imagen Editor's edits are faithful to the text prompts, which is accomplished by using object detectors to propose inpainting masks during training. In addition, Imagen Editor captures fine details in the input image by conditioning the cascaded pipeline on the original high resolution image. To improve qualitative and quantitative evaluation, we introduce EditBench, a systematic benchmark for text-guided image inpainting. EditBench evaluates inpainting edits on natural and generated images exploring objects, attributes, and scenes. Through extensive human evaluation on EditBench, we find that object-masking during training leads to across-the-board improvements in text-image alignment -- such that Imagen Editor is preferred over DALL-E 2 and Stable Diffusion -- and, as a cohort, these models are better at object-rendering than text-rendering, and handle material/color/size attributes better than count/shape attributes., Comment: CVPR 2023 Camera Ready

Image segmentation is a fundamental task in computer vision. Data annotation for training supervised methods can be labor-intensive, motivating unsupervised methods. Current approaches often rely on extracting deep features from pre-trained networks to construct a graph, and classical clustering methods like k-means and normalized-cuts are then applied as a post-processing step. However, this approach reduces the high-dimensional information encoded in the features to pair-wise scalar affinities. To address this limitation, this study introduces a lightweight Graph Neural Network (GNN) to replace classical clustering methods while optimizing for the same clustering objective function. Unlike existing methods, our GNN takes both the pair-wise affinities between local image features and the raw features as input. This direct connection between the raw features and the clustering objective enables us to implicitly perform classification of the clusters between different graphs, resulting in part semantic segmentation without the need for additional post-processing steps. We demonstrate how classical clustering objectives can be formulated as self-supervised loss functions for training an image segmentation GNN. Furthermore, we employ the Correlation-Clustering (CC) objective to perform clustering without defining the number of clusters, allowing for k-less clustering. We apply the proposed method for object localization, segmentation, and semantic part segmentation tasks, surpassing state-of-the-art performance on multiple benchmarks.