Search

High-quality video-text preference data is crucial for Multimodal Large Language Models (MLLMs) alignment. However, existing preference data is very scarce. Obtaining VQA preference data for preference training is costly, and manually annotating responses is highly unreliable, which could result in low-quality pairs. Meanwhile, AI-generated responses controlled by temperature adjustment lack diversity. To address these issues, we propose a high-quality VQA preference dataset, called \textit{\textbf{M}ultiple \textbf{M}ultimodal \textbf{A}rtificial \textbf{I}ntelligence \textbf{P}reference Datasets in \textbf{V}QA} (\textbf{MMAIP-V}), which is constructed by sampling from the response distribution set and using an external scoring function for response evaluation. Furthermore, to fully leverage the preference knowledge in MMAIP-V and ensure sufficient optimization, we propose \textit{\textbf{Iter}ative \textbf{W}eak-to-\textbf{S}trong \textbf{R}einforcement \textbf{L}earning from \textbf{AI} \textbf{F}eedback for video MLLMs} (\textbf{Iter-W2S-RLAIF}), a framework that gradually enhances MLLMs' alignment capabilities by iteratively updating the reference model and performing parameter extrapolation. Finally, we propose an unbiased and information-complete evaluation scheme in VQA evaluation. Experiments demonstrate that MMAIP-V is beneficial for MLLMs in preference learning and Iter-W2S-RLAIF fully exploits the alignment information in MMAIP-V. We believe that the proposed automatic VQA preference data generation pipeline based on AI feedback can greatly promote future work in the MLLMs alignment. \textbf{Code and dataset are available} \href{https://anonymous.4open.science/r/MMAIP-V_Iter-W2S-RLAIF-702F}{MMAIP-V\_Iter-W2S-RLAIF-702F}.

This paper examines the pivotal role of dropout techniques in mitigating overfitting in language model training. It conducts a comprehensive investigation into the influence of variable dropout rates on both individual layers and residual connections within the context of language modeling. Our study conducts training of a decoder implementation on the classic Tiny Shakespeare data to examine the effects of the adjustments on training efficiency and validation error. Results not only confirm the benefits of dropout for regularization and residuals for convergence, but also reveal their interesting interactions. There exists an important trade-off between the depth of residual connections and the dropout on these connections for optimal deep neural network convergence and generalization., Comment: 5 pages, 4 figures

Enhancing the conformity of large language models (LLMs) to human preferences remains an ongoing research challenge. Recently, offline approaches such as Direct Preference Optimization (DPO) have gained prominence as attractive options due to offering effective improvement in simple, efficient, and stable without interactions with reward models. However, these offline preference optimization methods highly rely on the quality of pairwise preference samples. Meanwhile, numerous iterative methods require additional training of reward models to select positive and negative samples from the model's own generated responses for preference learning. Furthermore, as LLMs' capabilities advance, it is quite challenging to continuously construct high-quality positive and negative preference instances from the model's outputs due to the lack of diversity. To tackle these challenges, we propose TSO, or Self-Training with Scaled Preference Optimization, a framework for preference optimization that conducts self-training preference learning without training an additional reward model. TSO enhances the diversity of responses by constructing a model matrix and incorporating human preference responses. Furthermore, TSO introduces corrections for model preference errors through human and AI feedback. Finally, TSO adopts iterative and dual clip reward strategies to update the reference model and its responses, adaptively adjusting preference data and balancing the optimization process. Experimental results demonstrate that TSO outperforms existing mainstream methods on various alignment evaluation benchmarks, providing practical insight into preference data construction and model training strategies in the alignment domain.

We introduce the emPDF (Empirical Distribution Function), a novel dynamical modeling method that infers the gravitational potential from kinematic tracers with optimal statistical efficiency under the minimal assumption of steady state. emPDF determines the best-fit potential by maximizing the similarity between instantaneous kinematics and the time-averaged phase-space distribution function (DF), which is empirically constructed from observation upon the theoretical foundation of oPDF (Han et al. 2016). This approach eliminates the need for presumed functional forms of DFs or orbit libraries required by conventional DF- or orbit-based methods. emPDF stands out for its flexibility, efficiency, and capability in handling observational effects, making it preferable to the popular Jeans equation or other minimal assumption methods, especially for the Milky Way (MW) outer halo where tracers often have limited sample size and poor data quality. We apply emPDF to infer the MW mass profile using Gaia DR3 data of satellite galaxies and globular clusters, obtaining consistent measurements with the constraints from simulation-informed DF fitting (Li et al. 2020). While the simulation-informed DF offers superior precision owing to the additional information extracted from simulations, emPDF is independent of such supplementary knowledge and applicable to general tracer populations. We provide tabulated measurements of the mass profile from emPDF, along with updated measurements from simulation-informed DF., Comment: 18 pages, 10 figures. Submitted to MNRAS. Comments are welcome

Satellite galaxies can be used to indicate the dynamical mass of galaxy groups and clusters. In this study, we apply the axis-symmetric Jeans Anisotropic Multi-Gaussian Expansion JAM modeling to satellite galaxies in 28 galaxy clusters selected from the TNG300-1 simulation with halo mass of $\log_{10}M_{200}/M_\odot>14.3$. If using true bound satellites as tracers, the best constrained total mass within the half-mass radius of satellites, $M(

Reinforcement Learning from Human Feedback (RLHF) facilitates the alignment of large language models (LLMs) with human preferences, thereby enhancing the quality of responses generated. A critical component of RLHF is the reward model, which is trained on preference data and outputs a scalar reward during the inference stage. However, the collection of preference data still lacks thorough investigation. Recent studies indicate that preference data is collected either by AI or humans, where chosen and rejected instances are identified among pairwise responses. We question whether this process effectively filters out noise and ensures sufficient diversity in collected data. To address these concerns, for the first time, we propose a comprehensive framework for preference data collection, decomposing the process into four incremental steps: Prompt Generation, Response Generation, Response Filtering, and Human Labeling. This structured approach ensures the collection of high-quality preferences while reducing reliance on human labor. We conducted comprehensive experiments based on the data collected at different stages, demonstrating the effectiveness of the proposed data collection method.

It is an interesting question Can and How Large Language Models (LLMs) understand non-language network data, and help us detect unknown malicious flows. This paper takes Carpet Bombing as a case study and shows how to exploit LLMs' powerful capability in the networking area. Carpet Bombing is a new DDoS attack that has dramatically increased in recent years, significantly threatening network infrastructures. It targets multiple victim IPs within subnets, causing congestion on access links and disrupting network services for a vast number of users. Characterized by low-rates, multi-vectors, these attacks challenge traditional DDoS defenses. We propose DoLLM, a DDoS detection model utilizes open-source LLMs as backbone. By reorganizing non-contextual network flows into Flow-Sequences and projecting them into LLMs semantic space as token embeddings, DoLLM leverages LLMs' contextual understanding to extract flow representations in overall network context. The representations are used to improve the DDoS detection performance. We evaluate DoLLM with public datasets CIC-DDoS2019 and real NetFlow trace from Top-3 countrywide ISP. The tests have proven that DoLLM possesses strong detection capabilities. Its F1 score increased by up to 33.3% in zero-shot scenarios and by at least 20.6% in real ISP traces.

In this paper, we set out to construct a set of reference mock galaxy redshift surveys (MGRSs) for the future Chinese Space-station Survey Telescope (CSST) observation, where subsequent survey selection effects can be added and evaluated. This set of MGRSs is generated using the dark matter subhalos extracted from a high-resolution Jiutian $N$-body simulation of the standard $\Lambda$CDM cosmogony with $\Omega_m=0.3111$, $\Omega_{\Lambda}=0.6889$, and $\sigma_8=0.8102$. The simulation has a boxsize of $1~h^{-1} {\rm Gpc}$, and consists of $6144^3$ particles with mass resolution $3.723 \times 10^{8} h^{-1} M_\odot $. In order to take into account the effect of redshift evolution, we first use all 128 snapshots in the Jiutian simulation to generate a light-cone halo/subhalo catalog. Next, galaxy luminosities are assigned to the main and subhalo populations using the subhalo abundance matching (SHAM) method with the DESI $z$-band luminosity functions at different redshifts. Multi-band photometries, as well as images, are then assigned to each mock galaxy using a 3-dimensional parameter space nearest neighbor sampling of the DESI LS observational galaxies and groups. Finally, the CSST and DESI LS survey geometry and magnitude limit cuts are applied to generate the required MGRSs. As we have checked, this set of MGRSs can generally reproduce the observed galaxy luminosity/mass functions within 0.1 dex for galaxies with $L > 10^8 L_\odot$ (or $M_* > 10^{8.5} M_\odot$) and within 1-$\sigma$ level for galaxies with $L < 10^8L_\odot$ (or $M_* < 10^{8.5} M_\odot$). Together with the CSST slitless spectra and redshifts for our DESI LS seed galaxies that are under construction, we will set out to test various slitless observational selection effects in subsequent probes., Comment: 13 pages, 9 figures, accepted for publication in MNRAS

Linear programming (LP) problems are pervasive in real-life applications. However, despite their apparent simplicity, an untrained user may find it difficult to determine the linear model of their specific problem. We envisage the creation of a goal-oriented conversational agent that will engage in conversation with the user to elicit all information required so that a subsequent agent can generate the linear model. In this paper, we present an approach for the generation of sample dialogues that can be used to develop and train such a conversational agent. Using prompt engineering, we develop two agents that "talk" to each other, one acting as the conversational agent, and the other acting as the user. Using a set of text descriptions of linear problems from NL4Opt available to the user only, the agent and the user engage in conversation until the agent has retrieved all key information from the original problem description. We also propose an extrinsic evaluation of the dialogues by assessing how well the summaries generated by the dialogues match the original problem descriptions. We conduct human and automatic evaluations, including an evaluation approach that uses GPT-4 to mimic the human evaluation metrics. The evaluation results show an overall good quality of the dialogues, though research is still needed to improve the quality of the GPT-4 evaluation metrics. The resulting dialogues, including the human annotations of a subset, are available to the research community. The conversational agent used for the generation of the dialogues can be used as a baseline., Comment: GEM Workshop @ EMNLP 2023

In this investigation, we leverage the combination of Dark Energy Spectroscopic Instrument Legacy imaging Surveys Data Release 9 (DESI LS DR9), Survey Validation 3 (SV3), and Year 1 (Y1) data sets to estimate the conditional luminosity and stellar mass functions (CLFs & CSMFs) of galaxies across various halo mass bins and redshift ranges. To support our analysis, we utilize a realistic DESI Mock Galaxy Redshift Survey (MGRS) generated from a high-resolution Jiutian simulation. An extended halo-based group finder is applied to both MGRS catalogs and DESI observation. By comparing the r and z-band luminosity functions (LFs) and stellar mass functions (SMFs) derived using both photometric and spectroscopic data, we quantified the impact of photometric redshift (photo-z) errors on the galaxy LFs and SMFs, especially in the low redshift bin at low luminosity/mass end. By conducting prior evaluations of the group finder using MGRS, we successfully obtain a set of CLF and CSMF measurements from observational data. We find that at low redshift the faint end slopes of CLFs and CSMFs below $10^{9}h^{-2}L_{\odot}$ (or $h^{-2}M_{\odot}$) evince a compelling concordance with the subhalo mass functions. After correcting the cosmic variance effect of our local Universe following arXiv:1809.00523, the faint end slopes of the LFs/SMFs turn out to be also in good agreement with the slope of the halo mass function., Comment: 28 pages, 13 figures, Accepted for publication in ApJ

Although chain-of-thought (CoT) prompting combined with language models has achieved encouraging results on complex reasoning tasks, the naive greedy decoding used in CoT prompting usually causes the repetitiveness and local optimality. To address this shortcoming, ensemble-optimization tries to obtain multiple reasoning paths to get the final answer assembly. However, current ensemble-optimization methods either simply employ rule-based post-processing such as \textit{self-consistency}, or train an additional model based on several task-related human annotations to select the best one among multiple reasoning paths, yet fail to generalize to realistic settings where the type of input questions is unknown or the answer format of reasoning paths is unknown. To avoid their limitations, we propose \textbf{Self-Agreement}, a generalizable ensemble-optimization method applying in almost all scenarios where the type of input questions and the answer format of reasoning paths may be known or unknown. Self-agreement firstly samples from language model's decoder to generate a \textit{diverse} set of reasoning paths, and subsequently prompts the language model \textit{one more time} to determine the optimal answer by selecting the most \textit{agreed} answer among the sampled reasoning paths. Self-agreement simultaneously achieves remarkable performance on six public reasoning benchmarks and superior generalization capabilities., Comment: Accepted by Findings of ACL 2024

The Conditional Colour-Magnitude Distribution (CCMD) is a comprehensive formalism of the colour-magnitude-halo mass relation of galaxies. With joint modelling of a large sample of SDSS galaxies in fine bins of galaxy colour and luminosity, Xu et al. inferred parameters of a CCMD model that well reproduces colour- and luminosity-dependent abundance and clustering of present-day galaxies. In this work, we provide a test and investigation of the CCMD model by studying the colour and luminosity distribution of galaxies in galaxy groups. An apples-to-apples comparison of group galaxies is achieved by applying the same galaxy group finder to identify groups from the CCMD galaxy mocks and from the SDSS data, avoiding any systematic effect of group finding and mass assignment on the comparison. We find an overall nice agreement in the conditional luminosity function (CLF), the conditional colour function (CCF), and the CCMD of galaxies in galaxy groups inferred from CCMD mock and SDSS data. We also discuss the subtle differences revealed by the comparison. In addition, using two external catalogues constructed to only include central galaxies with halo mass measured through weak lensing, we find that their colour-magnitude distribution shows two distinct and orthogonal components, in line with the prediction of the CCMD model. Our results suggest that the CCMD model provides a good description of halo mass dependent galaxy colour and luminosity distribution. The halo and CCMD mock catalogues are made publicly available to facilitate other investigations., Comment: Match with published version. The halo catalogues and CCMD mocks are publicly available at https://www.astro.utah.edu/~zhengzheng/data.html

In this paper, the limitations of YOLOv5s model on small target detection task are deeply studied and improved. The performance of the model is successfully enhanced by introducing GhostNet-based convolutional module, RepGFPN-based Neck module optimization, CA and Transformer's attention mechanism, and loss function improvement using NWD. The experimental results validate the positive impact of these improvement strategies on model precision, recall and mAP. In particular, the improved model shows significant superiority in dealing with complex backgrounds and tiny targets in real-world application tests. This study provides an effective optimization strategy for the YOLOv5s model on small target detection, and lays a solid foundation for future related research and applications., Comment: 8 pages 10 figures

As global attention on renewable and clean energy grows, the research and implementation of microgrids become paramount. This paper delves into the methodology of exploring the relationship between the operational and environmental costs of microgrids through multi-objective optimization models. By integrating various optimization algorithms like Genetic Algorithm, Simulated Annealing, Ant Colony Optimization, and Particle Swarm Optimization, we propose an integrated approach for microgrid optimization. Simulation results depict that these algorithms provide different dispatch results under economic and environmental dispatch, revealing distinct roles of diesel generators and micro gas turbines in microgrids. Overall, this study offers in-depth insights and practical guidance for microgrid design and operation., Comment: 11 pages, 6 fiures

We use the eROSITA Final Equatorial-Depth Survey (eFEDS) to measure the rest-frame 0.1-2.4 keV band X-ray luminosities of $\sim$ 600,000 DESI groups using two different algorithms in the overlap region of the two observations. These groups span a large redshift range of $0.0 \le z_g \le 1.0$ and group mass range of $10^{10.76}h^{-1}M_{\odot} \le M_h \le 10^{15.0}h^{-1}M_{\odot}$. (1) Using the blind detection pipeline of eFEDS, we find that 10932 X-ray emission peaks can be cross matched with our groups, $\sim 38 \%$ of which have signal-to-noise ratio $\rm{S}/\rm{N} \geq 3$ in X-ray detection. Comparing to the numbers reported in previous studies, this matched sample size is a factor of $\sim 6$ larger. (2) By stacking X-ray maps around groups with similar masses and redshifts, we measure the average X-ray luminosity of groups as a function of halo mass in five redshift bins. We find, in a wide halo mass range, the X-ray luminosity, $L_{\rm X}$, is roughly linearly proportional to $M_{h}$, and is quite independent to the redshift of the groups. (3) We use a Poisson distribution to model the X-ray luminosities obtained using two different algorithms and obtain best-fit $L_{\rm X}=10^{28.46\pm0.03}M_{h}^{1.024\pm0.002}$ and $L_{\rm X}=10^{26.73 \pm 0.04}M_{h}^{1.140 \pm 0.003}$ scaling relations, respectively. The best-fit slopes are flatter than the results previously obtained, but closer to a self-similar prediction., Comment: 15 pages, 13 figures, Monthly Notices of the Royal Astronomical Society, Volume 523, Issue 4, pp.4909-4922

Herein, we present a deep-learning technique for reconstructing the dark-matter density field from the redshift-space distribution of dark-matter halos. We built a UNet-architecture neural network and trained it using the COmoving Lagrangian Acceleration fast simulation, which is an approximation of the N-body simulation with $512^3$ particles in a box size of 500 Mpc $h^{-1}$. Further, we tested the resulting UNet model not only with training-like test samples but also with standard N-body simulations, such as the Jiutian simulation with $6144^3$ particles in a box size of 1000 Mpc $h^{-1}$ and the ELUCID simulation, which has a different cosmology. The real-space dark-matter density fields in the three simulations can be reconstructed reliably with only a small reduction of the cross-correlation power spectrum at 1% and 10% levels at $k=0.1$ and $0.3~h\mathrm{Mpc^{-1}}$, respectively. The reconstruction clearly helps to correct for redshift-space distortions and is unaffected by the different cosmologies between the training (Planck2018) and test samples (WMAP5). Furthermore, we tested the application of the UNet-reconstructed density field to obtain the velocity \& tidal field and found that this approach provides better results compared to the traditional approach based on the linear bias model, showing a 12.2% improvement in the correlation slope and a 21.1% reduction in the scatter between the predicted and true velocities. Thus, our method is highly efficient and has excellent extrapolation reliability beyond the training set. This provides an ideal solution for determining the three-dimensional underlying density field from the plentiful galaxy survey data., Comment: 14 pages, 16 figures

The distribution of baryons provides a significant way to understand the formation of galaxy clusters by revealing the details of its internal structure and changes over time. In this paper, we present theoretical studies on the scaled profiles of physical properties associated with the baryonic components, including gas density, temperature, metallicity, pressure and entropy as well as stellar mass, metallicity and satellite galaxy number density in galaxy clusters from $z=4$ to $z=0$ by tracking their progenitors. These mass-complete simulated galaxy clusters are coming from THE THREE HUNDRED with two runs: GIZMO-SIMBA and Gadget-X. Through comparisons between the two simulations, and with observed profiles which are generally available at low redshift, we find that (1) the agreements between the two runs and observations are mostly at outer radii $r \gtrsim 0.3r_{500}$, in line with the self-similarity assumption. While Gadget-X shows better agreements with the observed gas profiles in the central regions compared to GIZMO-SIMBA; (2) the evolution trends are generally consistent between the two simulations with slightly better consistency at outer radii. In detail, the gas density profile shows less discrepancy than the temperature and entropy profiles at high redshift. The differences in the cluster centre and gas properties imply different behaviours of the AGN models between Gadget-X and GIZMO-SIMBA, with the latter, maybe too strong for this cluster simulation. The high-redshift difference may be caused by the star formation and feedback models or hydrodynamics treatment, which requires observation constraints and understanding., Comment: 20 pages, 20 figures, accepted in MNRAS

Long-term user engagement (LTE) optimization in sequential recommender systems (SRS) is shown to be suited by reinforcement learning (RL) which finds a policy to maximize long-term rewards. Meanwhile, RL has its shortcomings, particularly requiring a large number of online samples for exploration, which is risky in real-world applications. One of the appealing ways to avoid the risk is to build a simulator and learn the optimal recommendation policy in the simulator. In LTE optimization, the simulator is to simulate multiple users' daily feedback for given recommendations. However, building a user simulator with no reality-gap, i.e., can predict user's feedback exactly, is unrealistic because the users' reaction patterns are complex and historical logs for each user are limited, which might mislead the simulator-based recommendation policy. In this paper, we present a practical simulator-based recommender policy training approach, Simulation-to-Recommendation (Sim2Rec) to handle the reality-gap problem for LTE optimization. Specifically, Sim2Rec introduces a simulator set to generate various possibilities of user behavior patterns, then trains an environment-parameter extractor to recognize users' behavior patterns in the simulators. Finally, a context-aware policy is trained to make the optimal decisions on all of the variants of the users based on the inferred environment-parameters. The policy is transferable to unseen environments (e.g., the real world) directly as it has learned to recognize all various user behavior patterns and to make the correct decisions based on the inferred environment-parameters. Experiments are conducted in synthetic environments and a real-world large-scale ride-hailing platform, DidiChuxing. The results show that Sim2Rec achieves significant performance improvement, and produces robust recommendations in unseen environments.

The Sunyaev Zel'dovich (SZ) effect is sensitive to the pressure of ionized gas inside galaxy clusters, which is in turn controlled largely by the gravitational potential of the cluster. Changing the concentration parameter describing the cluster mass distribution impacts the gravitational potential and thus the cluster SZ signal, with implications for cosmological and other analyses of SZ-selected clusters. We investigate the concentration-SZ relation in theory and simulations. We find that the impact of concentration on the inner SZ profile ($R \lesssim 0.75 R_{200c}$) can be captured with standard polytropic gas models. However, we find that such models do a poor job of reproducing the outer SZ profiles ($R \gtrsim 0.75 R_{200c}$) and the relation between the integrated SZ signal, $Y$, and concentration. This disagreement results from a sharp truncation of the gas pressure profile near the splashback radius, likely caused by virial shocks. We develop a simple description of the truncation that leads to a good match with the simulated SZ profiles out to several $R_{200c}$ for clusters of varying mass and concentration, and that also accurately predicts the concentration-$Y$ relationship. Finally, we determine how inference of the linear bias parameter and splashback radius for SZ-selected clusters can be biased by ignoring the concentration dependence of the SZ signal, finding that bias to the former is essentially negligible, while bias to the latter can be as much as 2\%., Comment: 16 pages, 11 figures; replaced to match version accepted by MNRAS. Revised version uses an updated model for nonthermal pressure support, but conclusions are not impacted

The star formation and quenching of central galaxies are regulated by the assembly histories of their host halos. In this work, we use the central stellar mass to halo mass ratio as a proxy of halo formation time, and we devise three different models, from the physical hydrodynamical simulation to the empirical statistical model, to demonstrate its robustness. With this proxy, we inferred the dependence of the central galaxy properties on the formation time of their host halos using the SDSS main galaxy sample, where central galaxies are identified with the halo-based group finder. We found that central galaxies living in late-formed halos have higher quiescent fractions and lower spiral fractions than their early-formed counterparts by $\lesssim$ 8%. Finally, we demonstrate that the group finding algorithm has a negligible impact on our results., Comment: 14 pages, 7 + 5 figures, MNRAS 522, 3188

Current mainstream object detection methods for large aerial images usually divide large images into patches and then exhaustively detect the objects of interest on all patches, no matter whether there exist objects or not. This paradigm, although effective, is inefficient because the detectors have to go through all patches, severely hindering the inference speed. This paper presents an Objectness Activation Network (OAN) to help detectors focus on fewer patches but achieve more efficient inference and more accurate results, enabling a simple and effective solution to object detection in large images. In brief, OAN is a light fully-convolutional network for judging whether each patch contains objects or not, which can be easily integrated into many object detectors and jointly trained with them end-to-end. We extensively evaluate our OAN with five advanced detectors. Using OAN, all five detectors acquire more than 30.0% speed-up on three large-scale aerial image datasets, meanwhile with consistent accuracy improvements. On extremely large Gaofen-2 images (29200$\times$27620 pixels), our OAN improves the detection speed by 70.5%. Moreover, we extend our OAN to driving-scene object detection and 4K video object detection, boosting the detection speed by 112.1% and 75.0%, respectively, without sacrificing the accuracy. Code is available at https://github.com/Ranchosky/OAN., Comment: This manuscript is the accepted version for SCIENCE CHINA Information Sciences

A fundamental question in any peer-to-peer ride-sharing system is how to, both effectively and efficiently, meet the request of passengers to balance the supply and demand in real time. On the passenger side, traditional approaches focus on pricing strategies by increasing the probability of users' call to adjust the distribution of demand. However, previous methods do not take into account the impact of changes in strategy on future supply and demand changes, which means drivers are repositioned to different destinations due to passengers' calls, which will affect the driver's income for a period of time in the future. Motivated by this observation, we make an attempt to optimize the distribution of demand to handle this problem by learning the long-term spatio-temporal values as a guideline for pricing strategy. In this study, we propose an offline deep reinforcement learning based method focusing on the demand side to improve the utilization of transportation resources and customer satisfaction. We adopt a spatio-temporal learning method to learn the value of different time and location, then incentivize the ride requests of passengers to adjust the distribution of demand to balance the supply and demand in the system. In particular, we model the problem as a Markov Decision Process (MDP).