Search

Foundation models have revolutionized tasks in computer vision and natural language processing. However, in the realm of tabular data, tree-based models like XGBoost continue to dominate. TabPFN, a transformer model tailored for tabular data, mirrors recent foundation models in its exceptional in-context learning capability, being competitive with XGBoost's performance without the need for task-specific training or hyperparameter tuning. Despite its promise, TabPFN's applicability is hindered by its data size constraint, limiting its use in real-world scenarios. To address this, we present in-context data distillation (ICD), a novel methodology that effectively eliminates these constraints by optimizing TabPFN's context. ICD efficiently enables TabPFN to handle significantly larger datasets with a fixed memory budget, improving TabPFN's quadratic memory complexity but at the cost of a linear number of tuning steps. Notably, TabPFN, enhanced with ICD, demonstrates very strong performance against established tree-based models and modern deep learning methods on 48 large tabular datasets from OpenML.

The goal of multimodal alignment is to learn a single latent space that is shared between multimodal inputs. The most powerful models in this space have been trained using massive datasets of paired inputs and large-scale computational resources, making them prohibitively expensive to train in many practical scenarios. We surmise that existing unimodal encoders pre-trained on large amounts of unimodal data should provide an effective bootstrap to create multimodal models from unimodal ones at much lower costs. We therefore propose FuseMix, a multimodal augmentation scheme that operates on the latent spaces of arbitrary pre-trained unimodal encoders. Using FuseMix for multimodal alignment, we achieve competitive performance -- and in certain cases outperform state-of-the art methods -- in both image-text and audio-text retrieval, with orders of magnitude less compute and data: for example, we outperform CLIP on the Flickr30K text-to-image retrieval task with $\sim \! 600\times$ fewer GPU days and $\sim \! 80\times$ fewer image-text pairs. Additionally, we show how our method can be applied to convert pre-trained text-to-image generative models into audio-to-image ones. Code is available at: https://github.com/layer6ai-labs/fusemix., Comment: CVPR 2024 (Highlight)

Bearing fault identification and analysis is an important research area in the field of machinery fault diagnosis. Aiming at the common faults of rolling bearings, we propose a data-driven diagnostic algorithm based on the characteristics of bearing vibrations called multi-size kernel based adaptive convolutional neural network (MSKACNN). Using raw bearing vibration signals as the inputs, MSKACNN provides vibration feature learning and signal classification capabilities to identify and analyze bearing faults. Ball mixing is a ball bearing production quality problem that is difficult to identify using traditional frequency domain analysis methods since it requires high frequency resolutions of the measurement signals and results in a long analyzing time. The proposed MSKACNN is shown to improve the efficiency and accuracy of ball mixing diagnosis. To further demonstrate the effectiveness of MSKACNN in bearing fault identification, a bearing vibration data acquisition system was developed, and vibration signal acquisition was performed on rolling bearings under five different fault conditions including ball mixing. The resulting datasets were used to analyze the performance of our proposed model. To validate the adaptive ability of MSKACNN, fault test data from the Case Western Reserve University Bearing Data Center were also used. Test results show that MSKACNN can identify the different bearing conditions with high accuracy with high generalization ability. We presented an implementation of the MSKACNN as a lightweight module for a real-time bearing fault diagnosis system that is suitable for production., Comment: 21 pages, 16 figures

In text-video retrieval, the objective is to learn a cross-modal similarity function between a text and a video that ranks relevant text-video pairs higher than irrelevant pairs. However, videos inherently express a much wider gamut of information than texts. Instead, texts often capture sub-regions of entire videos and are most semantically similar to certain frames within videos. Therefore, for a given text, a retrieval model should focus on the text's most semantically similar video sub-regions to make a more relevant comparison. Yet, most existing works aggregate entire videos without directly considering text. Common text-agnostic aggregations schemes include mean-pooling or self-attention over the frames, but these are likely to encode misleading visual information not described in the given text. To address this, we propose a cross-modal attention model called X-Pool that reasons between a text and the frames of a video. Our core mechanism is a scaled dot product attention for a text to attend to its most semantically similar frames. We then generate an aggregated video representation conditioned on the text's attention weights over the frames. We evaluate our method on three benchmark datasets of MSR-VTT, MSVD and LSMDC, achieving new state-of-the-art results by up to 12% in relative improvement in Recall@1. Our findings thereby highlight the importance of joint text-video reasoning to extract important visual cues according to text. Full code and demo can be found at: https://layer6ai-labs.github.io/xpool, Comment: CVPR 2022

Utilization of sludge pyrochar (SP) is the terminal step to loop the entire harmless disposal process of sewage sludge with pyrolysis. A new, easily recyclable, and safe adsorbent with well-immobilized heavy metals (HMs) was prepared from SP for ciprofloxacin (CIP) adsorption. The operational conditions for the adsorbent preparation were systematically optimized based on recycling rate and adsorption performance. Additionally, the adsorption conditions, adsorption kinetics, isotherms, and regeneration of adsorbents were further investigated in the present study. The results showed that easily recyclable and safe adsorbents were successfully prepared at 1,100 °C under N2 atmospheric conditions (SPA-N-1100) with a maximum CIP adsorption capacity of 10.42 mg/g. SPA-N-1100 exhibited good CIP adsorption performance at an adsorption temperature of 45 °C and pH between 8.0 to 9.0. The adsorbents were regenerated by thermal desorption at 450 °C with a thorough decomposition of CIP. The adsorption mechanism was mainly dominated by its special porous microspheres-accumulation structure and surface species (e.g., FeP and graphite). Moreover, HMs in the adsorbents were well immobilized in SPA-N-1100 by the generation of new metal mineral phases and encapsulation of melting minerals, which had an ultralow potential for ecological risk during application.

In view of the risks induced by the inhibitory effects of applying impracticably large amounts of sewage sludge biochar (SSB) to the alkaline soil, this field study investigated the influence of moderate biochar amendments (0, 1500, 4500, and 9000 kg/hm2) on corn growth, alkaline soil properties, and the uptake of potentially toxic elements (PTEs). The results showed that applying more SSB would decrease the ammonium nitrogen concentration and increase the available phosphorus and potassium concentrations, which inhibited corn plant growth because of high background nutrient levels of the alkaline soil. When the alkaline soil was amended with 1500 kg/hm2 SSB, the dry weight of 100 niblets increased from 32.11 g in the control to 35.07 g. There was no significant variation in the total concentration of PTEs in the soil. The concentrations of Mn, Ni, Cu, and Zn in niblets decreased from 5.54, 0.83, 2.26, and 27.15 mg/kg in the control to 4.47, 0.62, 1.30, and 23.45 mg/kg, respectively. Accordingly, the health risk from corn consumption was significantly reduced. Furthermore, the combination of SSB and fertilizer improved corn growth and reduced the risk of consumption of PTEs. Therefore, considering the increase in corn fruit yield and the decrease in consumption risk, applying 1500 kg/hm2 of biochar to alkaline soils is a realistically achievable rate, which can broaden the utilization of SSB for remediation of different types of soil.

Localizing actions in video is a core task in computer vision. The weakly supervised temporal localization problem investigates whether this task can be adequately solved with only video-level labels, significantly reducing the amount of expensive and error-prone annotation that is required. A common approach is to train a frame-level classifier where frames with the highest class probability are selected to make a video-level prediction. Frame level activations are then used for localization. However, the absence of frame-level annotations cause the classifier to impart class bias on every frame. To address this, we propose the Action Selection Learning (ASL) approach to capture the general concept of action, a property we refer to as "actionness". Under ASL, the model is trained with a novel class-agnostic task to predict which frames will be selected by the classifier. Empirically, we show that ASL outperforms leading baselines on two popular benchmarks THUMOS-14 and ActivityNet-1.2, with 10.3% and 5.7% relative improvement respectively. We further analyze the properties of ASL and demonstrate the importance of actionness. Full code for this work is available here: https://github.com/layer6ai-labs/ASL., Comment: CVPR 2021

Potentially toxic elements in municipal sewage sludge can be effectively immobilized during biochar production via pyrolysis. However, the bioavailability of these elements when biochar is applied in soilless cultivation to improve substrate quality has yet to be sufficiently established. In this study, we investigated the chemical speciation and cucumber plant uptake of potentially toxic elements in soilless cultivation when the growth substrate was amended with sewage sludge biochar (0, 5, 10, 15, and 20 wt.%). It was found that the addition of 10 wt.% biochar was optimal with respect to obtaining a high cucumber biomass and achieving low environmental risk considering the occurrence of hormesis. When the substrate was amended with 10 wt.% biochar, cucumber fruit contained lower concentrations of As, Cr, and Zn and smaller bioavailable fractions of As, Cd, Cr, Ni, Cu, and Zn compared with the fruit of control plants, thereby meeting national safety requirements (standard GB 2762-2012, China). Most of the As and Cd taken up by cucumbers accumulated in the leaves and fruit, whereas Cr was found primarily in the roots, and most Ni, Cu, and Zn was detected in the fruit. Importantly, only small proportions of the potentially toxic elements in biochar were taken up by cucumber plants (As: 0.0075%; Cd: 0.038%; Ni: 0.0064%; Cu: 0.0016%; and Zn: 0.0015%). Given that the As, Cd, Ni, and Zn speciation in sewage sludge biochar was effectively immobilized after cultivation, the findings of this study indicate that sewage sludge biochar is a suitable substrate amendment in terms of the risk posed by potentially toxic elements.

Industrial sludge is a type of solid waste that is known for its high ash content and heavy metal concentrations. In this study, industrial sludge was used for co-pyrolysis with rice straw at different temperatures (400, 500, and 600 °C) and mixture proportions (1:0, 9:1, 4:1, and 3:2, w/w). The relationship between heavy metal speciation and char properties was characterized in detail to investigate the effects of co-pyrolysis. Results show that the addition of rice straw can decrease the ash content and increase thermal stability and pore structure of char due to its high organic content. Simultaneously, co-pyrolysis decreases heavy metal concentrations and enhances the transformation of Cr, Zn, and Cd to a more stable fraction, with higher alkalinity, aromaticity, and specific surface area of char. Moreover, the leaching toxicity of Zn with high content is decreased to below threshold values, and the potential ecological risk of char is reduced to the lowest level under the condition of 600 °C with a mixing proportion of 4:1. This work demonstrates that co-pyrolysis provides an effective industrial sludge treatment for improving char characteristics and immobilizing heavy metals, indicating its superior application potential.

Hydrothermal pretreatment was used for dewatering food waste digestate residue, and biochar/biogas were generated from the separated solid and liquid phases via pyrolysis and anaerobic digestion, respectively. Increasing hydrothermal pretreatment temperature (110–200 °C) clearly improved dewaterability, whereas enhancing treatment duration (30–90 min) had little impact. The optimal condition of 160 °C/30 min gave the best dewatering performance with relative lower energy consumption and was chosen for pilot-scale verification achieving 61.7 wt% dry weight content after mechanical squeezing. Moreover, the filtrate and filter cake obtained at optimal condition were applied for biogas and biochar production in lab scale. The methane yield of the filtrate was 335 mL/g COD. Pyrolysis temperature of 500 °C gave better biochar performance and was verified in a pilot scale test. Additionally, the heavy metals in digestate were effectively immobilized during hydrothermal dewatering and pyrolysis processes. In the end, mass/energy balance calculated using pilot-scale data presented the combined systems provided a promising strategy for accomplishing energy recovery and resource reuse of digestate residue.

Purpose: Pyrolysis is the most effective method to completely remove antibiotics and immobilize heavy metals from livestock feces. However, the effect of the pyrolysis temperature on antibiotic removal at laboratory and pilot scales is still unclear. Materials and methods: The pyrolysis technique was used to convert pig manure (PM) and chicken manure (CM) into biochar at different temperatures from 300 to 700 °C in a laboratory-scale test. The performance of antibiotic removal and heavy metal immobilization in livestock feces was studied, and the optimal temperature of 600 °C was selected for the pilot-scale verification. Results and discussion: The results showed that the removal of the antibiotics tylosin (TYL), tetracycline (TC), chlortetracycline (CTC), doxycycline (DOXY), sulfamethazine (SMZ), sulfadiazine (SDZ), and sulfamethoxazole (SMX) was satisfactory, and all seven typical antibiotics were completely removed at 600 °C. In addition, the heavy metals zinc (Zn), copper (Cu), chromium (Cr), lead (Pb), nickel (Ni), cadmium (Cd), and arsenic (As) in the manure were well immobilized, and higher temperatures (above 600 °C) favored their immobilization. The results of heavy metal immobilization and antibiotic removal of the pilot-scale test were similar to those of the laboratory-scale test. Conclusions: Laboratory- and pilot-scale experiments showed that the current study provided a safe method and technology for treating and recycling livestock feces into biochar via the pyrolysis process.

We present our winning solution to the Open Images 2019 Visual Relationship challenge. This is the largest challenge of its kind to date with nearly 9 million training images. Challenge task consists of detecting objects and identifying relationships between them in complex scenes. Our solution has three stages, first object detection model is fine-tuned for the challenge classes using a novel weight transfer approach. Then, spatio-semantic and visual relationship models are trained on candidate object pairs. Finally, features and model predictions are combined to generate the final relationship prediction. Throughout the challenge we focused on minimizing the hardware requirements of our architecture. Specifically, our weight transfer approach enables much faster optimization, allowing the entire architecture to be trained on a single GPU in under two days. In addition to efficient optimization, our approach also achieves superior accuracy winning first place out of over 200 teams, and outperforming the second place team by over $5\%$ on the held-out private leaderboard.

We present a novel Cross-Class Relevance Learning approach for the task of temporal concept localization. Most localization architectures rely on feature extraction layers followed by a classification layer which outputs class probabilities for each segment. However, in many real-world applications classes can exhibit complex relationships that are difficult to model with this architecture. In contrast, we propose to incorporate target class and class-related features as input, and learn a pairwise binary model to predict general segment to class relevance. This facilitates learning of shared information between classes, and allows for arbitrary class-specific feature engineering. We apply this approach to the 3rd YouTube-8M Video Understanding Challenge together with other leading models, and achieve first place out of over 280 teams. In this paper we describe our approach and show some empirical results.

We present our solution to Landmark Image Retrieval Challenge 2019. This challenge was based on the large Google Landmarks Dataset V2[9]. The goal was to retrieve all database images containing the same landmark for every provided query image. Our solution is a combination of global and local models to form an initial KNN graph. We then use a novel extension of the recently proposed graph traversal method EGT [1] referred to as semi-supervised EGT to refine the graph and retrieve better candidates.