Search

We introduce a lightweight LLM-based framework designed to enhance the autonomy and robustness of domestic robots, targeting onboard embodied intelligence. By addressing challenges such as kinematic constraints and dynamic environments, our approach reduces reliance on large-scale data and incorporates a robot-agnostic pipeline. Our framework, InteLiPlan, ensures that the LLM model's decision-making capabilities are effectively aligned with robotic functions, enhancing operational robustness and adaptability, while our human-in-the-loop mechanism allows for real-time human intervention in the case where the system fails. We evaluate our method in both simulation and on the real Toyota HSR robot. The results show that our method achieves a 93% success rate in the fetch me task completion with system failure recovery, outperforming the baseline method in a domestic environment. InteLiPlan achieves comparable performance to the state-of-the-art large-scale LLM-based robotics planner, while guaranteeing real-time onboard computing with embodied intelligence.

We present the results of a four-year velocity-resolved reverberation mapping (RM) campaign of the changing-look active galactic nucleus (CL-AGN) NGC 4151 during its outburst phase. By measuring the time lags of the \ha, \hb, \hg, \hei, and \heii\ emission lines, we confirm a stratified broad-line region (BLR) structure that aligns with predictions from photoionization models. Intriguingly, we observed an ``anti-breathing" phenomenon, where the lags of broad emission lines decreased with increasing luminosity, contrary to the typical expectation. This anomaly may be attributed to the influence of the ultraviolet-optical lag or non-virialized motions in the BLR gas. Velocity-resolved RM and ionization mapping analyses revealed rapid and significant changes in the BLR geometry and kinematics on timescales within one year, which cannot be interpreted by any single mechanism, such as an inhomogeneous BLR, variations in radiation pressure, or changes in the illuminated ionizing field. Additionally, the \hb\ lags of NGC 4151 and other CL-AGNs agree with the radius-luminosity relationship established for AGNs with low accretion rates, implying that the CL phenomenon is more likely driven by intrinsic changes in the accretion rate rather than obscuration. These findings provide new insights into the complex internal processes of CL-AGNs and highlight the importance of long-term, multi-line RM for understanding BLR structures, geometry, and kinematics., Comment: 15 pages, 7 figures, 3 tables, Comments welcome! Submitted to ApJ

Resolving the inner structures of active galactic nuclei (AGNs) provides the "standard ruler" to measure the parallax distances of the Universe and a powerful way to weigh supermassive black holes (SMBHs). Thanks to time-domain observations, it is possible to use the reverberation mapping (RM) technique to measure time delays between different light curves that probe the structures of the SMBH accretion disks and broad line regions (BLRs), which are otherwise often too compact to be spatially resolved for most AGNs. Despite decades of RM studies, the critical physical process that controls the structures of SMBH accretion disk and BLR and their temporal evolution remains unclear. Here we report the variation of the SMBH accretion disk structure of NGC 4151 in response to changes in luminosity within 6 years. In the high-flux state, the time delays measured from our continuum RM with high-cadence (2 days) spectroscopy are 3.8 times larger than that in the low-flux state and 15 times longer than the classical standard thin disk (SSD) prediction. This result provides the first piece of direct evidence that the SMBH disk structure "breathes" in highly-variable AGN manifestations. The time-delay change severely challenges the popular X-ray reprocessing of the SSD model, with or without BLR contributions. More importantly, the continuum time delays can be comparable with the time delay between the broad Hb line and the nearby optical continuum, and the latter is commonly used to calculate the BLR sizes. Hence, the BLR sizes are significantly underestimated if the continuum time delays are not properly considered. This underestimation introduces up to 0.3 dex systematic uncertainties on RM SMBH masses and BLR parallax distances. Our findings underscore that simultaneous continuum and BLR RM studies are vital for better deciphering the SMBH mass growth and the cosmological expansion history., Comment: 18 pages, 8 figures, comments welcome

This paper compared physics-informed neural network (PINN), conventional neural network (NN) and traditional numerical discretization methods on solving differential equations (DEs) through literature investigation and experimental validation. We focused on the soft-constrained PINN approach and formalized its mathematical framework and computational flow for solving Ordinary DEs and Partial DEs (ODEs/PDEs). The working mechanism and its accuracy and efficiency were experimentally verified by solving typical linear and non-linear oscillator ODEs. We demonstrate that the DeepXDE-based implementation of PINN is not only light code and efficient in training, but also flexible across CPU/GPU platforms. PINN greatly reduces the need for labeled data: when the nonlinearity of the ODE is weak, a very small amount of supervised training data plus a few unsupervised collocation points are sufficient to predict the solution; in the minimalist case, only one or two training points (with initial values) are needed for first- or second-order ODEs, respectively. We also find that, with the aid of collocation points and the use of physical information, PINN has the ability to extrapolate data outside the time domain of the training set, and especially is robust to noisy data, thus with enhanced generalization capabilities. Training is accelerated when the gains obtained along with the reduction in the amount of data outweigh the delay caused by the increase in the loss function terms. The soft-constrained PINN can easily impose a physical law (e.g., conservation of energy) constraint by adding a regularization term to the total loss function, thus improving the solution performance to ODEs that obey this physical law. Furthermore, PINN can also be used for stiff ODEs, PDEs, and other types of DEs, and is becoming a favorable catalyst for the era of Digital Twins., Comment: 24 pages, 7 figures, 2 tables, etc. Ready for submission

Broad emission lines of active galactic nuclei (AGNs) originate from the broad-line region (BLR), consisting of dense gas clouds in orbit around an accreting supermassive black hole. Understanding the geometry and kinematics of the region is crucial for gaining insights into the physics and evolution of AGNs. Conventional velocity-resolved reverberation mapping may face challenges in disentangling the degeneracy between intricate motion and geometry of this region. To address this challenge, new key constraints are required. Here, we report the discovery of an asymmetric BLR using a novel technique: velocity-resolved ionization mapping, which can map the distance of emitting gas clouds by measuring Hydrogen line ratios at different velocities. By analyzing spectroscopic monitoring data, we find that the Balmer decrement is anticorrelated with the continuum and correlated with the lags across broad emission line velocities. Some line ratio profiles deviate from the expectations for a symmetrically virialized BLR, suggesting that the red-shifted and blue-shifted gas clouds may not be equidistant from the supermassive black hole (SMBH). This asymmetric geometry might represent a formation imprint, provide new perspectives on the evolution of AGNs, and influence SMBH mass measurements., Comment: 20 pages, 10 figures, Accepted by ApJ

We present a novel multimodal neural network for classifying astronomical sources in multiband ground-based observations, from optical to near infrared, to separate sources in stars, galaxies and quasars. Our approach combines a convolutional neural network branch for learning morphological features from $r$-band images with an artificial neural network branch for extracting spectral energy distribution (SED) information. Specifically, we have used 9-band optical ($ugri$) and NIR ($ZYHJK_s$) data from the Kilo-Degree Survey (KiDS) Data Release 5. The two branches of the network are concatenated and feed into fully-connected layers for final classification. We train the network on a spectroscopically confirmed sample from the Sloan Digital Sky Survey cross-matched with KiDS. The trained model achieves 98.76\% overall accuracy on an independent testing dataset, with F1 scores exceeding 95\% for each class. Raising the output probability threshold, we obtain higher purity at the cost of a lower completeness. We have also validated the network using external catalogs cross-matched with KiDS, correctly classifying 99.74\% of a pure star sample selected from Gaia parallaxes and proper motions, and 99.74\% of an external galaxy sample from the Galaxy and Mass Assembly survey, adjusted for low-redshift contamination. We apply the trained network to 27,334,751 KiDS DR5 sources with $r \leqslant 23$ mag to generate a new classification catalog. This multimodal neural network successfully leverages both morphological and SED information to enable efficient and robust classification of stars, quasars, and galaxies in large photometric surveys., Comment: 18 pages, 12 figures, 2 tables, Submitted to ApJS

Current state-of-the-art spatial reasoning-enhanced VLMs are trained to excel at spatial visual question answering (VQA). However, we believe that higher-level 3D-aware tasks, such as articulating dynamic scene changes and motion planning, require a fundamental and explicit 3D understanding beyond current spatial VQA datasets. In this work, we present SpatialPIN, a framework designed to enhance the spatial reasoning capabilities of VLMs through prompting and interacting with priors from multiple 3D foundation models in a zero-shot, training-free manner. Extensive experiments demonstrate that our spatial reasoning-imbued VLM performs well on various forms of spatial VQA and can extend to help in various downstream robotics tasks such as pick and stack and trajectory planning., Comment: NeurIPS 2024; Project Page: https://dannymcy.github.io/zeroshot_task_hallucination/

Bugs exist in hardware, such as CPU. Unlike software bugs, these hardware bugs need to be detected before deployment. Previous fuzzing work in CPU bug detection has several disadvantages, e.g., the length of RTL input instructions keeps growing, and longer inputs are ineffective for fuzzing. In this paper, we propose INSTILLER (Instruction Distiller), an RTL fuzzer based on ant colony optimization (ACO). First, to keep the input instruction length short and efficient in fuzzing, it distills input instructions with a variant of ACO (VACO). Next, related work cannot simulate realistic interruptions well in fuzzing, and INSTILLER solves the problem of inserting interruptions and exceptions in generating the inputs. Third, to further improve the fuzzing performance of INSTILLER, we propose hardware-based seed selection and mutation strategies. We implement a prototype and conduct extensive experiments against state-of-the-art fuzzing work in real-world target CPU cores. In experiments, INSTILLER has 29.4% more coverage than DiFuzzRTL. In addition, 17.0% more mismatches are detected by INSTILLER. With the VACO algorithm, INSTILLER generates 79.3% shorter input instructions than DiFuzzRTL, demonstrating its effectiveness in distilling the input instructions. In addition, the distillation leads to a 6.7% increase in execution speed on average.

Coverage-guided gray-box fuzzing (CGF) is an efficient software testing technique. There are usually multiple objectives to optimize in CGF. However, existing CGF methods cannot successfully find the optimal values for multiple objectives simultaneously. In this paper, we propose a gray-box fuzzer for multi-objective optimization (MOO) called MobFuzz. We model the multi-objective optimization process as a multi-player multi-armed bandit (MPMAB). First, it adaptively selects the objective combination that contains the most appropriate objectives for the current situation. Second, our model deals with the power schedule, which adaptively allocates energy to the seeds under the chosen objective combination. In MobFuzz, we propose an evolutionary algorithm called NIC to optimize our chosen objectives simultaneously without incurring additional performance overhead. To prove the effectiveness of MobFuzz, we conduct experiments on 12 real-world programs and the MAGMA data set. Experiment results show that multi-objective optimization in MobFuzz outperforms single-objective fuzzing in the baseline fuzzers. In contrast to them, MobFuzz can select the optimal objective combination and increase the values of multiple objectives up to 107%, with at most a 55% reduction in the energy consumption. Moreover, MobFuzz has up to 6% more program coverage and finds 3x more unique bugs than the baseline fuzzers. The NIC algorithm has at least a 2x improvement with a performance overhead of approximately 3%.

Deploying neural networks on microcontroller units (MCUs) presents substantial challenges due to their constrained computation and memory resources. Previous researches have explored patch-based inference as a strategy to conserve memory without sacrificing model accuracy. However, this technique suffers from severe redundant computation overhead, leading to a substantial increase in execution latency. A feasible solution to address this issue is mixed-precision quantization, but it faces the challenges of accuracy degradation and a time-consuming search time. In this paper, we propose QuantMCU, a novel patch-based inference method that utilizes value-driven mixed-precision quantization to reduce redundant computation. We first utilize value-driven patch classification (VDPC) to maintain the model accuracy. VDPC classifies patches into two classes based on whether they contain outlier values. For patches containing outlier values, we apply 8-bit quantization to the feature maps on the dataflow branches that follow. In addition, for patches without outlier values, we utilize value-driven quantization search (VDQS) on the feature maps of their following dataflow branches to reduce search time. Specifically, VDQS introduces a novel quantization search metric that takes into account both computation and accuracy, and it employs entropy as an accuracy representation to avoid additional training. VDQS also adopts an iterative approach to determine the bitwidth of each feature map to further accelerate the search process. Experimental results on real-world MCU devices show that QuantMCU can reduce computation by 2.2x on average while maintaining comparable model accuracy compared to the state-of-the-art patch-based inference methods., Comment: Accepted by the 27th Design, Automation and Test in Europe Conference (DATE 2024)

Long-term spectroscopic monitoring campaigns on active galactic nuclei (AGNs) provide a wealth of information about its interior structure and kinematics. However, a number of the observations suffer from the contamination of second-order spectra (SOS) which will introduce some undesirable uncertainties at the red side of the spectra. In this paper, we test the effect of SOS and propose a method to correct it in the time domain spectroscopic data using the simultaneously observed comparison stars. Based on the reverberation mapping (RM) data of NGC 5548 in 2019, one of the most intensively monitored AGNs by the Lijiang 2.4 m telescope, we find that the scientific object, comparison star, and spectrophotometric standard star can jointly introduce up to similar to 30% SOS for Grism 14. This irregular but smooth SOS significantly affects the flux density and profile of the emission line, while having little effect on the light curve. After applying our method to each spectrum, we find that the SOS can be corrected effectively. The deviation between corrected and intrinsic spectra is similar to 2%, and the impact of SOS on time lag is very minor. This method makes it possible to obtain the H alpha RM measurements from archival data provided that the spectral shape of the AGN under investigation does not have a large change.

The introduction of neural radiance fields has greatly improved the effectiveness of view synthesis for monocular videos. However, existing algorithms face difficulties when dealing with uncontrolled or lengthy scenarios, and require extensive training time specific to each new scenario. To tackle these limitations, we propose DynPoint, an algorithm designed to facilitate the rapid synthesis of novel views for unconstrained monocular videos. Rather than encoding the entirety of the scenario information into a latent representation, DynPoint concentrates on predicting the explicit 3D correspondence between neighboring frames to realize information aggregation. Specifically, this correspondence prediction is achieved through the estimation of consistent depth and scene flow information across frames. Subsequently, the acquired correspondence is utilized to aggregate information from multiple reference frames to a target frame, by constructing hierarchical neural point clouds. The resulting framework enables swift and accurate view synthesis for desired views of target frames. The experimental results obtained demonstrate the considerable acceleration of training time achieved - typically an order of magnitude - by our proposed method while yielding comparable outcomes compared to prior approaches. Furthermore, our method exhibits strong robustness in handling long-duration videos without learning a canonical representation of video content.

The stability of five-dimensional singly rotating Myers-Perry Black Holes against massive scalar perturbations is studied. Both the quasibound states and quasinormal modes of the massive scalar field are considered. For the quasibound states, we use an analytical method to discuss the effective potential felt by the scalar field, and found that there is no potential well outside the event horizon. Thus, singly rotating Myers-Perry Black Holes are stable against the perturbation of quasibound states of massive scalar fields. Then, We use continued fraction method based on solving a seven-term recurrence relations to compute the spectra of the quasinormal modes. For different values of the black hole rotation parameter $a$, scalar mass parameter $\mu$ and angular quantum numbers, all found quasinormal modes are damped. So singly rotating Myers-Perry Black Holes are also stable against the perturbation of quasinormal modes of massive scalar fields. Besides, when the scalar mass $\mu$ becomes relatively large, the long-living quasiresonances are also found as in other rotating black hole models. Our results complement previous arguments on the stability of five-dimensional singly rotating Myers-Perry black holes against massive scalar perturbations., Comment: references added

Directed grey-box fuzzing (DGF) is a target-guided fuzzing intended for testing specific targets (e.g., the potential buggy code). Despite numerous techniques proposed to enhance directedness, the existing DGF techniques still face challenges, such as taking into account the difficulty of reaching different basic blocks when designing the fitness metric, and promoting the effectiveness of symbolic execution (SE) when solving the complex constraints in the path to the target. In this paper, we propose a directed hybrid fuzzer called HyperGo. To address the challenges, we introduce the concept of path probability and combine the probability with distance to form an adaptive fitness metric called probability-based distance. By combining the two factors, probability-based distance can adaptively guide DGF toward paths that are closer to the target and have more easy-to-satisfy path constraints. Then, we put forward an Optimized Symbolic Execution Complementary (OSEC) scheme to combine DGF and SE in a complementary manner. The OSEC would prune the unreachable branches and unsolvable branches, and prioritize symbolic execution of the seeds whose paths are closer to the target and have more branches that are difficult to be covered by DGF. We evaluated HyperGo on 2 benchmarks consisting of 21 programs with a total of 100 target sites. The experimental results show that HyperGo achieves 38.47$\times$, 30.89$\times$, 28.52$\times$, 106.09$\times$ and 143.22$\times$ speedup compared to AFLGo, AFLGoSy, BEACON, WindRanger, and ParmeSan, respectively in reaching target sites, and 3.44$\times$, 3.63$\times$, 4.10$\times$, 3.26$\times$, and 3.00$\times$ speedup in exposing known vulnerabilities. Moreover, HyperGo discovered 37 undisclosed vulnerabilities from 7 real-world programs., Comment: 16 pages

We consider the stability of six-dimensional singly rotating Myers-Perry black holes under massive scalar perturbations. Using Leaver's continued fraction method, we compute the quasinormal modes of the massive scalar fields. All modes found are damped under the quasinormal boundary conditions. It is also found that long-living modes called quasiresonances exist for large scalar masses as in the four-dimensional Kerr black hole case. Our numerical results provide a direct and complement evidence for the stability of six-dimensional MP black holes under massive scalar perturbation., Comment: 11 pages,9 figures

This paper proposes Shoggoth, an efficient edge-cloud collaborative architecture, for boosting inference performance on real-time video of changing scenes. Shoggoth uses online knowledge distillation to improve the accuracy of models suffering from data drift and offloads the labeling process to the cloud, alleviating constrained resources of edge devices. At the edge, we design adaptive training using small batches to adapt models under limited computing power, and adaptive sampling of training frames for robustness and reducing bandwidth. The evaluations on the realistic dataset show 15%-20% model accuracy improvement compared to the edge-only strategy and fewer network costs than the cloud-only strategy., Comment: Accepted by 60th ACM/IEEE Design Automation Conference (DAC2023)

A truly generalizable approach to rigid segmentation and motion estimation is fundamental to 3D understanding of articulated objects and moving scenes. In view of the closely intertwined relationship between segmentation and motion estimates, we present an SE(3) equivariant architecture and a training strategy to tackle this task in an unsupervised manner. Our architecture is composed of two interconnected, lightweight heads. These heads predict segmentation masks using point-level invariant features and estimate motion from SE(3) equivariant features, all without the need for category information. Our training strategy is unified and can be implemented online, which jointly optimizes the predicted segmentation and motion by leveraging the interrelationships among scene flow, segmentation mask, and rigid transformations. We conduct experiments on four datasets to demonstrate the superiority of our method. The results show that our method excels in both model performance and computational efficiency, with only 0.25M parameters and 0.92G FLOPs. To the best of our knowledge, this is the first work designed for category-agnostic part-level SE(3) equivariance in dynamic point clouds., Comment: To appear at NeurIPS 2023

Deep learning is experiencing a rise in foundation models that are expected to lead in various fields. The massive number of parameters necessitates the use of tensor model parallelism (TMP) in foundation model training. However, TMP requires frequent communication operations which significantly reduces the training efficiency. In this paper, we present Oases, an automated TMP method with overlapped communication to accelerate foundation model training. Oases proposes a fine-grained training schedule to maximize overlapping communication and computation operations that have data dependence. Additionally, we design the Oases planner that searches for the best model parallel strategy to achieve further accelerations. Unlike existing methods, Oases planner is specifically tailored to model the cost of overlapped communication-computation operations. We evaluate Oases on various model settings and train environments, and compare Oases to four stat-of-the-art implementations. Experimental results demonstrate that Oases achieves speedups of 1.01--1.48X over the fastest baseline, and speedups of up to 1.9X over Megatron-LM.

The MultiCoNER II task aims to detect complex, ambiguous, and fine-grained named entities in low-context situations and noisy scenarios like the presence of spelling mistakes and typos for multiple languages. The task poses significant challenges due to the scarcity of contextual information, the high granularity of the entities(up to 33 classes), and the interference of noisy data. To address these issues, our team {\bf PAI} proposes a universal Named Entity Recognition (NER) system that integrates external entity information to improve performance. Specifically, our system retrieves entities with properties from the knowledge base (i.e. Wikipedia) for a given text, then concatenates entity information with the input sentence and feeds it into Transformer-based models. Finally, our system wins 2 first places, 4 second places, and 1 third place out of 13 tracks. The code is publicly available at \url{https://github.com/diqiuzhuanzhuan/semeval-2023}., Comment: win 2 first places, 4 second places, and 1 third place out of 13 tracks

Recent works in robotic manipulation through reinforcement learning (RL) or imitation learning (IL) have shown potential for tackling a range of tasks e.g., opening a drawer or a cupboard. However, these techniques generalize poorly to unseen objects. We conjecture that this is due to the high-dimensional action space for joint control. In this paper, we take an alternative approach and separate the task of learning 'what to do' from 'how to do it' i.e., whole-body control. We pose the RL problem as one of determining the skill dynamics for a disembodied virtual manipulator interacting with articulated objects. The whole-body robotic kinematic control is optimized to execute the high-dimensional joint motion to reach the goals in the workspace. It does so by solving a quadratic programming (QP) model with robotic singularity and kinematic constraints. Our experiments on manipulating complex articulated objects show that the proposed approach is more generalizable to unseen objects with large intra-class variations, outperforming previous approaches. The evaluation results indicate that our approach generates more compliant robotic motion and outperforms the pure RL and IL baselines in task success rates. Additional information and videos are available at https://kl-research.github.io/decoupskill, Comment: Accepted to IEEE International Conference on Robotics and Automation (ICRA) 2023

In this paper, a novel robotic grasping system is established to automatically pick up objects in cluttered scenes. A composite robotic hand composed of a suction cup and a gripper is designed for grasping the object stably. The suction cup is used for lifting the object from the clutter first and the gripper for grasping the object accordingly. We utilize the affordance map to provide pixel-wise lifting point candidates for the suction cup. To obtain a good affordance map, the active exploration mechanism is introduced to the system. An effective metric is designed to calculate the reward for the current affordance map, and a deep Q-Network (DQN) is employed to guide the robotic hand to actively explore the environment until the generated affordance map is suitable for grasping. Experimental results have demonstrated that the proposed robotic grasping system is able to greatly increase the success rate of the robotic grasping in cluttered scenes., Comment: has been accepted by IEEE/RSJ International Conference on Intelligent Robots and Systems 2019

To cluster, classify and represent are three fundamental objectives of learning from high-dimensional data with intrinsic structure. To this end, this paper introduces three interpretable approaches, i.e., segmentation (clustering) via the Minimum Lossy Coding Length criterion, classification via the Minimum Incremental Coding Length criterion and representation via the Maximal Coding Rate Reduction criterion. These are derived based on the lossy data coding and compression framework from the principle of rate distortion in information theory. These algorithms are particularly suitable for dealing with finite-sample data (allowed to be sparse or almost degenerate) of mixed Gaussian distributions or subspaces. The theoretical value and attractive features of these methods are summarized by comparison with other learning methods or evaluation criteria. This summary note aims to provide a theoretical guide to researchers (also engineers) interested in understanding 'white-box' machine (deep) learning methods.

Background : Proton flash therapy is an emergency research topic in radiation therapy since the Varian announced the promising results from the first in human clinical trial of Flash therapy recently. However, it still needs a lot of researches on this topic, not only to understand the mechanism of the radiobiological effects but also to develop an appropriate dose monitoring system. Purpose : In this study we setup an experimental station for small animal proton Flash irradiation in a clinical machine. The dose monitoring system is able to provide real-time irradiation dose and irradiation time structure. Methods : The dose monitoring system includes homebrewed transmission ionization chamber (TIC), plastic scintillator based beam position monitor, and Poor Man Faraday Cup (FC). Both TIC and FC are equipped with a homebrewed fast reading current integral electronics device. The imaging guidance system comprises a moveable CT, laser, as well as attaching a bead on the body surface of the mouse can accurately guide the testing small animal in position. Results : The dose monitoring system can provide the time structure of delivered dose rate within 1 ms time resolution. Experimental testing results show that the highest dose in one pulse of 230 MeV proton that can be delivered to the target is about 20 Gy during 199 ms pulse period at 100 Gy/s dose rate. Conclusion : A proton research irradiation platform dedicated for studying small animal Flash biological effects has been established at Chang Gung Memorial Hospital. The final setup data represent a reference for the beam users to plan the experiments as well as for the improvement of the facility.