Search

Heterogeneity in the cell population of cancer tissues poses many challenges in cancer diagnosis and treatment. Studying the heterogeneity in cell populations from gene expression measurement data in the context of cancer research is a problem of paramount importance. In addition, reducing the computation time of the algorithms that deal with high volumes of data has its obvious merits. Parallelizable models using Markov chain Monte Carlo methods are typically slow. This paper shows a novel, computationally efficient, and parallelizable model to analyze heterogeneity in cancer tissues using GPUs. Because our model is parallelizable, the input data size does not affect the computation time much, provided the hardware resources are not exhausted. Our model uses qPCR (quantitative polymerase chain reaction) gene expression measurements to study heterogeneity in cancer tissue. We compute the cell proportion breakup by accelerating variational methods on a GPU. We test this model on synthetic and real-world gene expression data collected from fibroblasts and compare the performance of our algorithm with those of MCMC and Expectation Maximization. Our new model is computationally less complex and faster than existing Bayesian models for cancer tissue heterogeneity.

Crowd counting finds direct applications in real-world situations, making computational efficiency and performance crucial. However, most of the previous methods rely on a heavy backbone and a complex downstream architecture that restricts the deployment. To address this challenge and enhance the versatility of crowd-counting models, we introduce two lightweight models. These models maintain the same downstream architecture while incorporating two distinct backbones: MobileNet and MobileViT. We leverage Adjacent Feature Fusion to extract diverse scale features from a Pre-Trained Model (PTM) and subsequently combine these features seamlessly. This approach empowers our models to achieve improved performance while maintaining a compact and efficient design. With the comparison of our proposed models with previously available state-of-the-art (SOTA) methods on ShanghaiTech-A ShanghaiTech-B and UCF-CC-50 dataset, it achieves comparable results while being the most computationally efficient model. Finally, we present a comparative study, an extensive ablation study, along with pruning to show the effectiveness of our models.

Utilizing massive web-scale datasets has led to unprecedented performance gains in machine learning models, but also imposes outlandish compute requirements for their training. In order to improve training and data efficiency, we here push the limits of pruning large-scale multimodal datasets for training CLIP-style models. Today's most effective pruning method on ImageNet clusters data samples into separate concepts according to their embedding and prunes away the most prototypical samples. We scale this approach to LAION and improve it by noting that the pruning rate should be concept-specific and adapted to the complexity of the concept. Using a simple and intuitive complexity measure, we are able to reduce the training cost to a quarter of regular training. By filtering from the LAION dataset, we find that training on a smaller set of high-quality data can lead to higher performance with significantly lower training costs. More specifically, we are able to outperform the LAION-trained OpenCLIP-ViT-B32 model on ImageNet zero-shot accuracy by 1.1p.p. while only using 27.7% of the data and training compute. Despite a strong reduction in training cost, we also see improvements on ImageNet dist. shifts, retrieval tasks and VTAB. On the DataComp Medium benchmark, we achieve a new state-of-the-art Imagehttps://info.arxiv.org/help/prep#commentsNet zero-shot accuracy and a competitive average zero-shot accuracy on 38 evaluation tasks., Comment: Accepted at ICLR 2024, code available at https://github.com/amro-kamal/effective_pruning

Separating out the contributions of different scattering channels in strongly interacting metals is crucial in identifying the mechanisms that govern their properties. While momentum or current relaxation rates can be readily probed via \textit{dc} resistivity or optical/THz spectroscopy, distinguishing different kinds of inelastic scattering can be more challenging. Using nonlinear THz 2D coherent spectroscopy, we measure the rates of energy relaxation after THz excitation in the strongly interacting Fermi liquid, Sr$_2$RuO$_4$. Energy relaxation is a bound on the total scattering and specifically a measure of contributions to the electron self-energy that arise from {\it inelastic} coupling to a bath. We observe two distinct energy relaxation channels: a fast process that we interpret as energy loss to the phonon system and a much slower relaxation that we interpret as arising from a non-equilibrium phonon effects and subsequent heat loss through diffusion. Interestingly, even the faster energy relaxation rate is at least an order of magnitude slower than the overall momentum relaxation rate, consistent with strong electron interactions and the dominance of energy-conserving umklapp or interband electron-electron scattering in momentum relaxation. The slowest energy relaxation rate decays on a sub-GHz scale, consistent with the relaxation dynamics of non-equilibrium phonons. Our observations reveal the versatility of nonlinear THz spectroscopy to measure the energy relaxation dynamics in correlated metals. Our work also highlights the need for improved theoretical understanding of such processes in interacting metals., Comment: 6 pages, 3 figures. SM included

Deep neural networks are characterized by multiple symmetrical, equi-loss solutions that are redundant. Thus, the order of neurons in a layer and feature maps can be given arbitrary permutations, without affecting (or minimally affecting) their output. If we shuffle these neurons, or if we apply to them some perturbations (like fine-tuning) can we put them back in the original order i.e. re-synchronize? Is there a possible corruption threat? Answering these questions is important for applications like neural network white-box watermarking for ownership tracking and integrity verification. We advance a method to re-synchronize the order of permuted neurons. Our method is also effective if neurons are further altered by parameter pruning, quantization, and fine-tuning, showing robustness to integrity attacks. Additionally, we provide theoretical and practical evidence for the usual means to corrupt the integrity of the model, resulting in a solution to counter it. We test our approach on popular computer vision datasets and models, and we illustrate the threat and our countermeasure on a popular white-box watermarking method.

We explore the properties of neutral mesons within the context of a chirally imbalanced medium, employing the two-flavor Nambu--Jona-Lasinio model. The temperature dependence of the constituent quark mass at finite values of the chiral chemical potential (CCP) demonstrates the well-established phenomena of chiral catalysis at lower temperatures and inverse chiral catalysis at higher temperatures. The polarization functions in both the scalar ($\sigma$) and pseudo-scalar ($\pi^0$) channels have been evaluated using real time formalism of thermal field theory. These have been used to determine the masses and spectral functions of $ \sigma $ and $ \pi $ mesons. Detailed investigation of the analytic structure of the imaginary part of the polarization function for $\sigma$ and $\pi$ mesons results in the emergence of non-trivial Landau cut contributions due to the presence of chiral imbalance. The multiple solutions for the mass of the $\pi$ meson for specific values of CCP have been analysed on the basis of their residue at the pole. Furthermore, we have observed abrupt changes in the masses of both scalar and pseudo-scalar mesons at finite CCP values, particularly at higher temperatures. A decreasing trend in the Mott transition temperature is seen with the increase in CCP., Comment: Version published in Physical Review D

Density functional theory based calculations have been performed for solving the phonon Boltzmann transport equation to investigate the thermal transport properties of monolayer (ML) ZnO under in-plane isotropic biaxial tensile strain. The in-plane lattice thermal conductivity ($\kappa_{\text{L}}$) of ML-ZnO increases dramatically in response to the biaxial tensile strain ranging from 0% to 10%, conflicting with the general belief. The strain-induced stiffening of the ZA phonon mode and the resulting concomitant increase in group velocity and decrease in phonon population is found to play a significant role behind the unusual enhancement of $\kappa_{\text{L}}$. The mode resolved analysis shows the tensile strain driven competitive behavior between different phonon properties, mainly the group velocity and phonon lifetimes, being responsible for the observed unusual enhancement in $\kappa_{\text{L}}$. Additionally, the phonon scattering calculations show the importance of inclusion of 4-phonon scattering in the thermal transport calculations suggesting the significance of higher-order anharmonicity in ML-ZnO. A strikingly high 4-phonon scattering strength in ML-ZnO primarily results from the strong anharmonicity, quadratic ZA mode dispersion, large frequency gap in phonon dispersion, and reflection symmetry induced selection rule. The incorporation of 4-phonon scattering significantly alters the transport characteristics of all the phonon modes, in general and ZA phonons, in particular. At large strains, a linear dispersion of the ZA mode and closure of the frequency gap is observed, which results in a significant reduction of 4-phonon scattering strength in ML-ZnO., Comment: 15 pages, 13 figures

In the calculations of lattice thermal conductivity ($\kappa_{\text{L}}$), vital contributions stemming from four-phonon scattering are often neglected. The significance of four-phonon scattering in the thermal transport properties of monolayer (ML) MoS$_{2}$ has been unraveled using first-principles calculations combined with the Boltzmann transport equation. If only three-phonon scattering processes are considered then the $\kappa_{\text{L}}$ is found to be significantly overestimated ($\sim$ 115.8 Wm$^{-1}$K$^{-1}$ at 300 K). With the incorporation of the four-phonon scattering processes, the $\kappa_{\text{L}}$ reduces to 24.6 Wm$^{-1}$K$^{-1}$, which is found to be closer to the experimentally measured $\kappa_{\text{L}}$ of 34.5 Wm$^{-1}$K$^{-1}$. Four-phonon scattering significantly impacts the carrier lifetime ($\tau$) of the low-energy out-of-plane acoustic mode (ZA) phonons and thereby, suppresses its contribution in $\kappa_{\text{L}}$ from 64% (for three-phonon scattering) to 16% (for both three- and four-phonon scatterings). The unusually high four-phonon scattering rate ($\tau_{4}^{-1}$) of the ZA phonons is found to result from the simultaneous effect of the acoustic-optical frequency gap, strong anharmonicity, and the reflection symmetry imposed selection rule. The strong coupling between the quadratic dispersion of the ZA mode and the $\tau_{4}^{-1}$ is discovered by the application of mechanical strain. The strain induced increase in the linearity of the ZA mode dispersion dramatically reduces the significance of the four-phonon scattering in the strained ML-MoS$_{2}$, both qualitatively and quantitatively. These conclusions will provide significant insights into the thermal transport phenomena in ML-MoS$_{2}$, as well as any other 2D material., Comment: 14 pages, 13 figures

This paper lays down the research agenda for a domain-specific foundation model for operating systems (OSes). Our case for a foundation model revolves around the observations that several OS components such as CPU, memory, and network subsystems are interrelated and that OS traces offer the ideal dataset for a foundation model to grasp the intricacies of diverse OS components and their behavior in varying environments and workloads. We discuss a wide range of possibilities that then arise, from employing foundation models as policy agents to utilizing them as generators and predictors to assist traditional OS control algorithms. Our hope is that this paper spurs further research into OS foundation models and creating the next generation of operating systems for the evolving computing landscape., Comment: Machine Learning for Systems Workshop at 37th NeurIPS Conference, 2023, New Orleans, LA, USA

Low-Rank Adaptation (LoRA) has recently gained attention for fine-tuning foundation models by incorporating trainable low-rank matrices, thereby reducing the number of trainable parameters. While LoRA offers numerous advantages, its applicability for real-time serving to a diverse and global user base is constrained by its incapability to handle multiple task-specific adapters efficiently. This imposes a performance bottleneck in scenarios requiring personalized, task-specific adaptations for each incoming request. To mitigate this constraint, we introduce Fast LoRA (FLoRA), a framework in which each input example in a minibatch can be associated with its unique low-rank adaptation weights, allowing for efficient batching of heterogeneous requests. We empirically demonstrate that FLoRA retains the performance merits of LoRA, showcasing competitive results on the MultiPL-E code generation benchmark spanning over 8 languages and a multilingual speech recognition task across 6 languages., Comment: 16 pages, 3 figures

Employing the non-linear Walecka model we investigate the characteristics of nuclear matter under the influence of a background magnetic field at a finite temperature and baryon chemical potential. In the presence of the magnetic field the spinodal lines and the critical end point (CEP) undergo changes in the $T-\mu_B$ plane. The squared speed of sound exhibits anisotropic behavior, dividing into parallel and perpendicular components. Additionally, the presence of a magnetic field induces anisotropy in the isothermal compressibility. It is found that the parallel component is smaller than the perpendicular one for all values of temperature, chemical potential and magnetic field indicating that the equation of state is stiffer along the magnetic field direction.

With the goal of understanding the visual concepts that CLIP associates with text prompts, we show that the latent space of CLIP can be visualized solely in terms of linear transformations on simple geometric primitives like circles and straight lines. Although existing approaches achieve this by sketch-synthesis-through-optimization, they do so on the space of B\'ezier curves, which exhibit a wastefully large set of structures that they can evolve into, as most of them are non-essential for generating meaningful sketches. We present CLIPDrawX, an algorithm that provides significantly better visualizations for CLIP text embeddings, using only simple primitive shapes like straight lines and circles. This constrains the set of possible outputs to linear transformations on these primitives, thereby exhibiting an inherently simpler mathematical form. The synthesis process of CLIPDrawX can be tracked end-to-end, with each visual concept being explained exclusively in terms of primitives. Implementation will be released upon acceptance. Project Page: $\href{https://clipdrawx.github.io/}{\text{https://clipdrawx.github.io/}}$.

Spatial statistics is traditionally based on stationary models on $\mathbb{R^d}$ like Mat\'ern fields. The adaptation of traditional spatial statistical methods, originally designed for stationary models in Euclidean spaces, to effectively model phenomena on linear networks such as stream systems and urban road networks is challenging. The current study aims to analyze the incidence of traffic accidents on road networks using three different methodologies and compare the model performance for each methodology. Initially, we analyzed the application of spatial triangulation precisely on road networks instead of traditional continuous regions. However, this approach posed challenges in areas with complex boundaries, leading to the emergence of artificial spatial dependencies. To address this, we applied an alternative computational method to construct nonstationary barrier models. Finally, we explored a recently proposed class of Gaussian processes on compact metric graphs, the Whittle-Mat\'ern fields, defined by a fractional SPDE on the metric graph. The latter fields are a natural extension of Gaussian fields with Mat\'ern covariance functions on Euclidean domains to non-Euclidean metric graph settings. A ten-year period (2010-2019) of daily traffic-accident records from Barcelona, Spain have been used to evaluate the three models referred above. While comparing model performance we observed that the Whittle-Mat\'ern fields defined directly on the network outperformed the network triangulation and barrier models. Due to their flexibility, the Whittle-Mat\'ern fields can be applied to a wide range of environmental problems on linear networks such as spatio-temporal modeling of water contamination in stream networks or modeling air quality or accidents on urban road networks., Comment: 24 pages, 9 figures

We study the azimuthal angle and transverse momentum dependence of dilepton production from hot and magnetized hadronic matter using $\rho^0$-meson dominance. The thermomagnetic spectral function of the $\rho^0$ is evaluated using the real time method of thermal field theory and Schwinger proper-time formulation. A continuous spectrum is obtained in which there is sizeable Landau cut contributions in the low invariant mass region as a consequence of finite background field. The emission rate of the dileptons is found to be significantly anisotropic in this region and the later effectively increases with the strength of the background field. In addition, we also evaluate the elliptic flow parameter ($v_2$) as a function of invariant mass for different values of magnetic field and temperature. We find that in low invariant mass region $v_2$ remains positive at lower values of $eB$ signifying that the production rate could be larger along the direction transverse to the background field. This behaviour is consistent with the angular dependence of dilepton production rate., Comment: Version published in European Physical Journal A

To safeguard machine learning systems that operate on textual data against out-of-distribution (OOD) inputs that could cause unpredictable behaviour, we explore the use of topological features of self-attention maps from transformer-based language models to detect when input text is out of distribution. Self-attention forms the core of transformer-based language models, dynamically assigning vectors to words based on context, thus in theory our methodology is applicable to any transformer-based language model with multihead self-attention. We evaluate our approach on BERT and compare it to a traditional OOD approach using CLS embeddings. Our results show that our approach outperforms CLS embeddings in distinguishing in-distribution samples from far-out-of-domain samples, but struggles with near or same-domain datasets., Comment: 8 pages, 6 figures, 3 tables, to be published in proceedings of the IJCAI-2024 AISafety Workshop

We compute the equation of state (EoS) of strange quark stars (SQSs) with the MIT Bag model using density dependent bag pressure, characterized by a Gaussian distribution function. The bag pressure's density dependence is controlled by three key parameters namely the asymptotic value ($B_{as}$), $\Delta B(=B_0 - B_{as})$, and $\beta$. We explore various parameter combinations ($B_{as}$, $\Delta B$, $\beta$) that adhere to the Bodmer-Witten conjecture, a criterion for the stability of SQSs. Our primary aim is to analyze the effects of these parameter variations on the structural properties of SQSs. However we find that none of the combinations can satisfy the NICER data for PSR J0030+0451 and the constraint on tidal deformability from GW170817. So it can be emphasized that this model cannot describe reasonable SQS configurations. We also extend our work to calculate structural properties of hybrid stars (HSs). With the density dependent bag model (DDBM), these astrophysical constraints are fulfilled by the HSs configurations within a very restricted range of the three parameters. The present work is the first to constrain the parameters of DDBM for both SQS and HSs using the recent astrophysical constraints on tidal deformabiity from GW170817 and that on mass-radius relationship from NICER data., Comment: Accepted for publication in Nuclear Physics A

With constant growth of civilization and modernization of cities all across the world since past few centuries smart traffic management of vehicles is one of the most sorted after problem by research community. It is a challenging problem in computer vision and artificial intelligence domain. Smart traffic management basically involves segmentation of vehicles, estimation of traffic density and tracking of vehicles. The vehicle segmentation from traffic videos helps realization of niche applications such as monitoring of speed and estimation of traffic. When occlusions, background with clutters and traffic with density variations are present, this problem becomes more intractable in nature. Keeping this motivation in this research work, we investigate Faster R-CNN based deep learning method towards segmentation of vehicles. This problem is addressed in four steps viz minimization with adaptive background model, Faster R-CNN based subnet operation, Faster R-CNN initial refinement and result optimization with extended topological active nets. The computational framework uses ideas of adaptive background modeling. It also addresses shadow and illumination related issues. Higher segmentation accuracy is achieved through topological active net deformable models. The topological and extended topological active nets help to achieve stated deformations. Mesh deformation is achieved with minimization of energy. The segmentation accuracy is improved with modified version of extended topological active net. The experimental results demonstrate superiority of this computational framework, Comment: Book Chapter

Photon counting radiation detectors have become an integral part of medical imaging modalities such as Positron Emission Tomography or Computed Tomography. One of the most promising detectors is the wide bandgap room temperature semiconductor detectors, which depends on the interaction gamma/x-ray photons with the detector material involves Compton scattering which leads to multiple interaction photon events (MIPEs) of a single photon. For semiconductor detectors like CdZnTeSe (CZTS), which have a high overlap of detected energies between Compton and photoelectric events, it is nearly impossible to distinguish between Compton scattered events from photoelectric events using conventional readout electronics or signal processing algorithms. Herein, we report a deep learning classifier CoPhNet that distinguishes between Compton scattering and photoelectric interactions of gamma/x-ray photons with CdZnTeSe (CZTS) semiconductor detectors. Our CoPhNet model was trained using simulated data to resemble actual CZTS detector pulses and validated using both simulated and experimental data. These results demonstrated that our CoPhNet model can achieve high classification accuracy over the simulated test set. It also holds its performance robustness under operating parameter shifts such as Signal-Noise-Ratio (SNR) and incident energy. Our work thus laid solid foundation for developing next-generation high energy gamma-rays detectors for better biomedical imaging., Comment: 17 pages

ChipNeMo aims to explore the applications of large language models (LLMs) for industrial chip design. Instead of directly deploying off-the-shelf commercial or open-source LLMs, we instead adopt the following domain adaptation techniques: domain-adaptive tokenization, domain-adaptive continued pretraining, model alignment with domain-specific instructions, and domain-adapted retrieval models. We evaluate these methods on three selected LLM applications for chip design: an engineering assistant chatbot, EDA script generation, and bug summarization and analysis. Our evaluations demonstrate that domain-adaptive pretraining of language models, can lead to superior performance in domain related downstream tasks compared to their base LLaMA2 counterparts, without degradations in generic capabilities. In particular, our largest model, ChipNeMo-70B, outperforms the highly capable GPT-4 on two of our use cases, namely engineering assistant chatbot and EDA scripts generation, while exhibiting competitive performance on bug summarization and analysis. These results underscore the potential of domain-specific customization for enhancing the effectiveness of large language models in specialized applications., Comment: Updated results for ChipNeMo-70B model

While large language models (LLMs) equipped with techniques like chain-of-thought prompting have demonstrated impressive capabilities, they still fall short in their ability to reason robustly in complex settings. However, evaluating LLM reasoning is challenging because system capabilities continue to grow while benchmark datasets for tasks like logical deduction have remained static. We introduce MuSR, a dataset for evaluating language models on multistep soft reasoning tasks specified in a natural language narrative. This dataset has two crucial features. First, it is created through a novel neurosymbolic synthetic-to-natural generation algorithm, enabling the construction of complex reasoning instances that challenge GPT-4 (e.g., murder mysteries roughly 1000 words in length) and which can be scaled further as more capable LLMs are released. Second, our dataset instances are free text narratives corresponding to real-world domains of reasoning; this makes it simultaneously much more challenging than other synthetically-crafted benchmarks while remaining realistic and tractable for human annotators to solve with high accuracy. We evaluate a range of LLMs and prompting techniques on this dataset and characterize the gaps that remain for techniques like chain-of-thought to perform robust reasoning.

Recent advances in fine-grained representation learning leverage local-to-global (emergent) relationships for achieving state-of-the-art results. The relational representations relied upon by such methods, however, are abstract. We aim to deconstruct this abstraction by expressing them as interpretable graphs over image views. We begin by theoretically showing that abstract relational representations are nothing but a way of recovering transitive relationships among local views. Based on this, we design Transitivity Recovering Decompositions (TRD), a graph-space search algorithm that identifies interpretable equivalents of abstract emergent relationships at both instance and class levels, and with no post-hoc computations. We additionally show that TRD is provably robust to noisy views, with empirical evidence also supporting this finding. The latter allows TRD to perform at par or even better than the state-of-the-art, while being fully interpretable. Implementation is available at https://github.com/abhrac/trd., Comment: Neural Information Processing Systems (NeurIPS) 2023

This article is intended as a guide for new graduate students in the field of computational science. With the increasing influx of students from diverse backgrounds joining the ever-popular field, this short guide aims to help students navigate through the various computational techniques that they are likely to encounter during their studies. These techniques span from Bash scripting and scientific programming to machine learning, among other areas. This paper is divided into ten sections, each introducing a different computational method. To enhance readability, we have adopted a casual and instructive tone, and included code snippets where relevant. Please note that due to the introductory nature of this article, it is not intended to be exhaustive; instead, we direct readers to a list of references to expand their knowledge of the techniques discussed within the paper. It is likely that this article will continue to evolve with time, and as such, we advise readers to seek the latest version. Finally, readers should note this article serves as an extension to our student-led seminar series, with additional resources and videos available at \url{https://computationaltoolkit.github.io/} for reference., Comment: 45 pages, 3 figures

We introduce Cosmos, a framework for object-centric world modeling that is designed for compositional generalization (CompGen), i.e., high performance on unseen input scenes obtained through the composition of known visual "atoms." The central insight behind Cosmos is the use of a novel form of neurosymbolic grounding. Specifically, the framework introduces two new tools: (i) neurosymbolic scene encodings, which represent each entity in a scene using a real vector computed using a neural encoder, as well as a vector of composable symbols describing attributes of the entity, and (ii) a neurosymbolic attention mechanism that binds these entities to learned rules of interaction. Cosmos is end-to-end differentiable; also, unlike traditional neurosymbolic methods that require representations to be manually mapped to symbols, it computes an entity's symbolic attributes using vision-language foundation models. Through an evaluation that considers two different forms of CompGen on an established blocks-pushing domain, we show that the framework establishes a new state-of-the-art for CompGen in world modeling. Artifacts are available at: https://trishullab.github.io/cosmos-web/, Comment: Uploading ICLR,2024 Camera Ready Version

Differential Privacy (DP) is a well-established framework to quantify privacy loss incurred by any algorithm. Traditional formulations impose a uniform privacy requirement for all users, which is often inconsistent with real-world scenarios in which users dictate their privacy preferences individually. This work considers the problem of mean estimation, where each user can impose their own distinct privacy level. The algorithm we propose is shown to be minimax optimal and has a near-linear run-time. Our results elicit an interesting saturation phenomenon that occurs. Namely, the privacy requirements of the most stringent users dictate the overall error rates. As a consequence, users with less but differing privacy requirements are all given more privacy than they require, in equal amounts. In other words, these privacy-indifferent users are given a nontrivial degree of privacy for free, without any sacrifice in the performance of the estimator., Comment: A preliminary conference version was published at ISIT 2023 and uploaded to arxiv (arXiv:2305.09668). This version significantly expands on the previous article and is being submitted to a journal

In this paper we prove a blow-up criterion for the compressible Navier-Stokes-Fourier system for general thermal and caloric equations of state with inhomogeneous boundary conditions for the velocity and the temperature. Assuming only that Gibb's equation and the thermodynamic stability hold, we show that solutions in a certain regularity class remain regular under the condition that the density, the temperature and the modulus of the velocity are bounded.

Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable abilities to follow diverse human instructions and perform a wide range of tasks. However, when probing language models using a range of basic table-understanding tasks, we observe that today's language models are still sub-optimal in many table-related tasks, likely because they are pre-trained predominantly on \emph{one-dimensional} natural-language texts, whereas relational tables are \emph{two-dimensional} objects. In this work, we propose a new "\emph{table-tuning}" paradigm, where we continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using diverse table-tasks synthesized from real tables as training data, with the goal of enhancing language models' ability to understand tables and perform table tasks. We show that our resulting Table-GPT models demonstrate (1) better \emph{table-understanding} capabilities, by consistently outperforming the vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout unseen tasks, and (2) strong \emph{generalizability}, in its ability to respond to diverse human instructions to perform new table-tasks, in a manner similar to GPT-3.5 and ChatGPT.