First principles and machine learning methods in molecules, fluids, and solids

Condensed matter physics spans molecules, fluids, and solids. Mathematical models have been developed to simulate physical systems, but exact solutions are special leaving the many-body Schr\"odinger equation analytically intractable. Therefore, approximations have to be constructed to make those difficult-to-solve problems tractable. This thesis adapts different levels of approximation, developing an exact formalism for a quantum operator, investigating the origins of intriguing phenomena in solids using first principle methods, and heuristic data-driven machine learning to guide biomedical developments. The quantum Monte Carlo method is used to study molecules where the formalism for force constants matrix is created for the first time using variational and diffusion Monte Carlo, which are the most accurate technique in electronic structure calculation but more resource demanding. We make the atomic relaxation and vibrational frequency calculation possible via the direct approach. The performance has been tested on a diverse set of case studies, with varying symmetries and mass distributions, and shows the established formalism outperforms leading computational methods over an order of magnitude in terms of the accuracy of vibrational frequency relative to experiment. This opens the way for self-contained quantum Monte Carlo simulations from relaxing atomic positions to calculating vibrational frequencies without being limited to the accuracy from density functional theory for structure relaxation. Density functional theory is applied to investigate the large and complex solids of perovskites and heterostructures efficiently. We explore the origins of Raman modes for transient photophysics in layered $2$D perovskites and discover the twisting of organic cations could influence the excitons in the inorganic layer. Understanding the coherent interplay of excitons, spins, and phonons facilitates the design of efficient light emission devices and solar cells. Inspired by an unexpected exchange bias effect in topological insulator superlattices, we introduce magnetic models to investigate the doping effect and predict the interfacial coupling between dopants induces exchange bias. We provide a new pathway for achieving the high-temperature quantum anomalous Hall effect in exchange bias stabilised magnetic systems by interface engineering. Finally, we show machine learning can deliver new insights for systems lacking theoretical models, and where data collection is expensive and difficult. We design a neural network model capable of handling missing data and estimating prediction uncertainties. We apply it to study the efficacy of biomedical stem cell fluids for cartilage repair and identify critical properties, optimal dose, and predicted therapeutic outcomes for personalised therapy. We provide references for scientists and clinicians to design better therapy strategies, and the technology can be adapted for addressing research problems across different subjects.