Search

Recent works demonstrate the effectiveness of diffusion models as unsupervised solvers for inverse imaging problems. Sparse-view computed tomography (CT) has greatly benefited from these advancements, achieving improved generalization without reliance on measurement parameters. However, this comes at the cost of potential hallucinations, especially when handling out-of-distribution (OOD) data. To ensure reliability, it is essential to study OOD detection for CT reconstruction across both clinical and industrial applications. This need further extends to enabling the OOD detector to function effectively as an anomaly inspection tool. In this paper, we explore the use of a diffusion model, trained to capture the target distribution for CT reconstruction, as an in-distribution prior. Building on recent research, we employ the model to reconstruct partially diffused input images and assess OOD-ness through multiple reconstruction errors. Adapting this approach for sparse-view CT requires redefining the notions of "input" and "reconstruction error". Here, we use filtered backprojection (FBP) reconstructions as input and investigate various definitions of reconstruction error. Our proof-of-concept experiments on the MNIST dataset highlight both successes and failures, demonstrating the potential and limitations of integrating such an OOD detector into a CT reconstruction system. Our findings suggest that effective OOD detection can be achieved by comparing measurements with forward-projected reconstructions, provided that reconstructions from noisy FBP inputs are conditioned on the measurements. However, conditioning can sometimes lead the OOD detector to inadvertently reconstruct OOD images well. To counter this, we introduce a weighting approach that improves robustness against highly informative OOD measurements, albeit with a trade-off in performance in certain cases.

Establishing a unified theory of cognition has been a major goal of psychology. While there have been previous attempts to instantiate such theories by building computational models, we currently do not have one model that captures the human mind in its entirety. Here we introduce Centaur, a computational model that can predict and simulate human behavior in any experiment expressible in natural language. We derived Centaur by finetuning a state-of-the-art language model on a novel, large-scale data set called Psych-101. Psych-101 reaches an unprecedented scale, covering trial-by-trial data from over 60,000 participants performing over 10,000,000 choices in 160 experiments. Centaur not only captures the behavior of held-out participants better than existing cognitive models, but also generalizes to new cover stories, structural task modifications, and entirely new domains. Furthermore, we find that the model's internal representations become more aligned with human neural activity after finetuning. Taken together, Centaur is the first real candidate for a unified model of human cognition. We anticipate that it will have a disruptive impact on the cognitive sciences, challenging the existing paradigm for developing computational models.

We investigate wavelet-like localized solutions in nonlinear waveguides, enabled by complementary propagation constants embedded in domains of anomalous dispersion. They are carrier-envelope-phase stable and independent of fine details of the dispersion profile. Their spectra extend over vast ranges and they are also robust against nonlinear perturbations, providing properties highly demanded in ultrafast science. In the limits of low and high energies, approximate representations are given by modified Morlet wavelets and Gaussian pulses, respectively. We demonstrate their special features by exploiting general principles of quantum scattering theory and solid-state physics, realizing interlocked chains of wavelets analog to photonic time crystals.

In-context learning, the ability to adapt based on a few examples in the input prompt, is a ubiquitous feature of large language models (LLMs). However, as LLMs' in-context learning abilities continue to improve, understanding this phenomenon mechanistically becomes increasingly important. In particular, it is not well-understood how LLMs learn to solve specific classes of problems, such as reinforcement learning (RL) problems, in-context. Through three different tasks, we first show that Llama $3$ $70$B can solve simple RL problems in-context. We then analyze the residual stream of Llama using Sparse Autoencoders (SAEs) and find representations that closely match temporal difference (TD) errors. Notably, these representations emerge despite the model only being trained to predict the next token. We verify that these representations are indeed causally involved in the computation of TD errors and $Q$-values by performing carefully designed interventions on them. Taken together, our work establishes a methodology for studying and manipulating in-context learning with SAEs, paving the way for a more mechanistic understanding.

The polarization states of terahertz (THz) radiation generated in a photo-ionized gas driven by strong two- or multi-frequency fields with locally controlled polarization are studied. We reveal a universal property of the resulting THz waveforms: the ellipticity of their polarization state increases linearly with the frequency. This ``linear chirp of ellipticity'' makes plasma-based THz generation unique among other THz sources. However, it also puts some constraints on the polarization properties of the generated THz radiation. We derive a general expression for the THz ellipticity and demonstrate how the polarization states of the generated THz waveforms can be manipulated and controlled by the polarization of the pump pulses.

We study solitary wave solutions for the nonlinear Schr\"odinger equation perturbed by the effects of third-, and fourth-order dispersion, maintaining a wavenumber gap between the solitary waves and the propagation constant. We numerically construct members of a family of such solitary waves, including Kruglov and Harvey's exact solution, using the spectral renormalization method and establish empirical relations between the pulse parameters. A deeper insight into the properties of solitary waves and solitons can be obtained through collisions. Therefore we perform pulse propagation simulations demonstrating different collision regimes. Depending on the pulses initial phase difference, this can lead to the formation of short-lived two-pulse bound states. While these collisions are generally inelastic, singular phase values exist at which they are elastic. Finally, we detail the properties of Kruglov and Harvey's soliton solution under loss, verifying earlier predictions of perturbation theory and suggesting a convergence to the soliton solution of the standard nonlinear Schr\"odinger equation in the limit of large propagation distances.

In this work, we develop adaptive schemes using goal-oriented error control for a highly nonlinear flow temperature model with temperature dependent density. The dual-weighted residual method for computing error indicators to steer mesh refinement and solver control is employed. The error indicators are used to employ adaptive algorithms, which are substantiated with several numerical tests. Therein, error reductions and effectivity indices are consulted to establish the robustness and efficiency of our framework.

In three-dimensional models obtained by photogrammetry of existing structures, all of the shapes that the eye can select cannot always find their equivalents in the geometric components of the model. However, the matching of meaningful parts and assemblages with the records acquired with rapid and detailed documentation methods will provide an advantage for the creation of information models of existing structures. While aiming to produce answers to this problem and in order to overcome the difficulties of pattern recognition in three-dimensional models, we used two-dimensional samples obtained by projection. Processing techniques such as ambient occlusion, curvature and normal maps are commonly used in modern computer graphics applications that enable the representation of three-dimensional surface properties in two-dimensional data sets. The method we propose is based on the recognition of patterns through these mappings instead of the usual light-based visualization. The first stage of the application is photogrammetric capture of a few examples of Zeugma mosaics and three-dimensional digital modeling of a set of Seljuk era brick walls based on knowledge obtained through architectural history literature. The second stage covers the creation of digital models byprocessing the surface representation obtained from this data using Alice Vision, OpenCV-Python, and Autodesk Maya to include information on aspects of the making of the walls. What is envisioned for the next stages is that the mapping data contributes and supports the knowledge for rule-based design and making processesof cultural heritage., Comment: 9 pages, in Turkish language. 6 figures. In: MSTAS 2019 - (XIII. Computational Design in Architecture National Symposium) pp. 313-326., Kocaeli, Turkey (2019)

This study aims to present a material based, second order design method that makes the rapid creation of bridging structures in order to connect two or more horizontal planes which are currently separated and three design instances created via such method. The first element of the presented method is a generative system that creates circulation surfaces through the interpolation of the curves defined on the current surfaces, and also creates the structural volumes via rheotomic (minimal) surfaces. The second constituent is the instantiation of the exposed variable, connected to a displacement algorithm inspired by allometry based on user input, contextual data, or simulation results. The method was created with applicability through additive manufacturing in consideration. The design process - similar to manufacturing - proceeds in a vertical manner, in order to reduce the generation of support geometry as much as possible. The system includes a raster data input viable for simulation results, feeding the accumulation variable in order to modify material amount or quality with the aim of improving structural performance where stress is greater. Through the application of the method, three physical models which connect different horizontal planes were obtained. The models can be evaluated with digital of physical simulations, and results can be utilized in an iterative manner, improving the results by each recursion., Comment: 8 pages, in Turkish language. 9 figures. XII. Computational Design Research in Architecture Symposium, June 21-22, 2018 Isparta, Turkey

Generative systems are becoming a crucial part of current design practice. There exist gaps however, between the digital processes, field data and designer's input. To solve this problem, multiple processes were developed in order to generate emergent and self-organizing design solutions that combine the designer's input with surface models acquired via photogrammetry and generative design tools. Different generative design methods were utilized for trials, including surface scattering based on UV coordinates, animation snapshots (similar to long exposure photography) and a particle swarm algorithm on arbitrary data, interpolated within GIS software. A large volume of adaptive forms that are complex, yet responsive to changes in parameters, user input, topography and/or various spatial data were acquired. Resulting outputs were rendered and projection mapped onto the original physical model and evaluated for further iterations., Comment: 10 pages, 10 figures. 4th International Symposium - Formal Methods in Architecture, April 3-5, 2017 Porto, Portugal

There has been a growing adoption of computer vision tools and technologies in architectural design workflows over the past decade. Notable use cases include point cloud generation, visual content analysis, and spatial awareness for robotic fabrication. Multiple image classification, object detection, and semantic pixel segmentation models have become popular for the extraction of high-level symbolic descriptions and semantic content from two-dimensional images and videos. However, a major challenge in this regard has been the extraction of high-level architectural structures (walls, floors, ceilings windows etc.) from diverse imagery where parts of these elements are occluded by furniture, people, or other non-architectural elements. This project aims to tackle this problem by proposing models that are capable of extracting architecturally meaningful semantic descriptions from two-dimensional scenes of populated interior spaces. 1000 virtual classrooms are parametrically generated, randomized along key spatial parameters such as length, width, height, and door/window positions. The positions of cameras, and non-architectural visual obstructions (furniture/objects) are also randomized. A Generative Adversarial Network (GAN) for image-to-image translation (Pix2Pix) is trained on synthetically generated rendered images of these enclosures, along with corresponding image abstractions representing high-level architectural structure. The model is then tested on unseen synthetic imagery of new enclosures, and outputs are compared to ground truth using pixel-wise comparison for evaluation. A similar model evaluation is also carried out on photographs of existing indoor enclosures, to measure its performance in real-world settings., Comment: Written for 6.869, Advances in Computer Vision at MIT, Spring 2022. 9 pages, 13 figures

Remote collaboration is a common reality of spatial design processes, but tools for computer aided design were made for single users. Via TeamCAD, we introduce a user experience where online remote collaboration experience is more like working on a table. Using speech and gesture recognition based on state of the art machine learning through webcam and microphone input, TeamCAD plugs into existing software through API's, keybindings, and mouse input. We share results from user studies conducted on graduate students from . Our user tests were run on Blender animation software, making simultaneous use of both modalities for given tasks. We mitigated challenges in terms of robustness and latency in readily available voice recognition models. Our prototype has proven to be an intuitive interface, providing a suitable denominator for collaborators with or without previous experience in three-dimensional modeling applications., Comment: Created for 6.835, Intelligent Multimodal User Interfaces at MIT, Spring 2022. Submitted to CHI 2024. 10 pages, 6 figures, 1 table

Visual design relies on seeing things in different ways, acting on them, and seeing results to act again. Parametric design tools are often not robust to design changes that result from sketching over the visualization of their output. We propose a sketch to 3d workflow as an experiment medium for evaluating neural networks and their latent spaces as a representation that is robust to overlay sketching., Comment: Written for 4.453 Creative Machine Learning at MIT, Spring 2023. 9 pages, 9 figures

Sentiment analysis methods are rapidly being adopted by the field of Urban Design and Planning, for the crowdsourced evaluation of urban environments. However, most models used within this domain are able to identify positive or negative sentiment associated with a textual appraisal as a whole, without inferring information about specific urban aspects contained within it, or the sentiment associated with them. While Aspect Based Sentiment Analysis (ABSA) is becoming increasingly popular, most existing ABSA models are trained on non-urban themes such as restaurants, electronics, consumer goods and the like. This body of research develops an ABSA model capable of extracting urban aspects contained within geo-located textual urban appraisals, along with corresponding aspect sentiment classification. We annotate a dataset of 2500 crowdsourced reviews of public parks, and train a Bidirectional Encoder Representations from Transformers (BERT) model with Local Context Focus (LCF) on this data. Our model achieves significant improvement in prediction accuracy on urban reviews, for both Aspect Term Extraction (ATE) and Aspect Sentiment Classification (ASC) tasks. For demonstrative analysis, positive and negative urban aspects across Boston are spatially visualized. We hope that this model is useful for designers and planners for fine-grained urban sentiment evaluation., Comment: Created for 6.8610, Quantitative Methods for Natural Language Processing at MIT Fall 2022. 5 pages, 4 figures

Humans represent scenes and objects in rich feature spaces, carrying information that allows us to generalise about category memberships and abstract functions with few examples. What determines whether a neural network model generalises like a human? We tested how well the representations of $86$ pretrained neural network models mapped to human learning trajectories across two tasks where humans had to learn continuous relationships and categories of natural images. In these tasks, both human participants and neural networks successfully identified the relevant stimulus features within a few trials, demonstrating effective generalisation. We found that while training dataset size was a core determinant of alignment with human choices, contrastive training with multi-modal data (text and imagery) was a common feature of currently publicly available models that predicted human generalisation. Intrinsic dimensionality of representations had different effects on alignment for different model types. Lastly, we tested three sets of human-aligned representations and found no consistent improvements in predictive accuracy compared to the baselines. In conclusion, pretrained neural networks can serve to extract representations for cognitive models, as they appear to capture some fundamental aspects of cognition that are transferable across tasks. Both our paradigms and modelling approach offer a novel way to quantify alignment between neural networks and humans and extend cognitive science into more naturalistic domains.

In this work, we consider the time-harmonic Maxwell's equations and their numerical solution with a domain decomposition method. As an innovative feature, we propose a feedforward neural network-enhanced approximation of the interface conditions between the subdomains. The advantage is that the interface condition can be updated without recomputing the Maxwell system at each step. The main part consists of a detailed description of the construction of the neural network for domain decomposition and the training process. To substantiate this proof of concept, we investigate a few subdomains in some numerical experiments with low frequencies. Therein the new approach is compared to a classical domain decomposition method. Moreover, we highlight current challenges of training and testing with different wave numbers and we provide information on the behaviour of the neural-network, such as convergence of the loss function, and different activation functions., Comment: 17 pages, 11 figures, 2 tables

We present a detailed overview of the physics of two-color soliton molecules in nonlinear waveguides, i.e. bound states of localized optical pulses which are held together due to an incoherent interaction mechanism. The mutual confinement, or trapping, of the subpulses, which leads to a stable propagation of the pulse compound, is enabled by the nonlinear Kerr effect. Special attention is paid to the description of the binding mechanism in terms of attractive potential wells, induced by the refractive index changes of the subpulses, exerted on one another through cross-phase modulation. Specifically, we discuss nonlinear-photonics meta atoms, given by pulse compounds consisting of a strong trapping pulse and a weak trapped pulse, for which trapped states of low intensity are determined by a Schr\"odinger-type eigenproblem. We discuss the rich dynamical behavior of such meta-atoms, demonstrating that an increase of the group-velocity mismatch of both subpulses leads to an ionization-like trapping-to-escape transition. We further demonstrate that if both constituent pulses are of similar amplitude, molecule-like bound-states are formed. We show that z-periodic amplitude variations permit a coupling of these pulse compound to dispersive waves, resulting in the resonant emission of Kushi-comb-like multi-frequency radiation.

We study incoherently coupled two-frequency pulse compounds in waveguides with single zero-dispersion and zero-nonlinearity points. In such waveguides, supported by a negative nonlinearity, soliton dynamics can be obtained even in domains of normal dispersion. We demonstrate trapping of weak pulses by solitary-wave wells, forming nonlinear-photonics meta-atoms, and molecule-like bound-states of pulses. We study the impact of Raman effect on these pulse compounds, finding that, depending on the precise subpulse configuration, they decelerate, accelerate, or are completely unaffected. Our results extend the range of systems in which two-frequency pulse compounds can be expected to exist and demonstrate further unique and unexpected behavior.

We numerically explore synthetic crystal diamond for realizing novel light sources in ranges which are up to now difficult to achieve with other materials, such as sub-10-fs pulse durations and challenging spectral ranges. We assess the performance of on-chip diamond waveguides for controlling light generation by means of nonlinear soliton dynamics. Tailoring the cross-section of such diamond waveguides allows to design dispersion profiles with custom zero-dispersion points and anomalous dispersion ranges exceeding an octave. Various propagation dynamics, including supercontinuum generation by soliton fission, can be realized in diamond photonics. In stark contrast to usual silica-based optical fibers, where such processes occur on the scale of meters, in diamond millimeter-scale propagation distances are sufficient. Unperturbed soliton-dynamics prior to soliton fission allow to identify a pulse self-compression scenario that promises record-breaking compression factors on chip-size propagation lengths.