Food information has a high reward value for human beings, and may be more so for those who are overweight or obese. Attention bias towards or away from food stimuli appears to be related to clinical symptoms in disordered eating and obesity. A popular task to assess attentional bias is the dot probe task. However, several studies have shown that the dot probe task has low reliability, making it a poor candidate to study individual differences or assess changes. A recent study (van Ens et al., 2019), however, seems to suggest that, by using a long stimulus presentation duration (3 seconds or longer), reaction time index may have acceptable internal and good test-retest reliability; in addition, although eye-tracking based first fixation measure had poor internal and test-retest reliability, the total dwell time measure had excellent internal and acceptable test-retest reliability. But several confounding factors and major limitations in this study may affect the validity and generalizability of the results: 1) it was mentioned that the color and complexity between food and nonfood stimuli were carefully matched, but it was not clear how; moreover, important physical properties such as luminance and contrast were not controlled for. Therefore, it was not clear whether any bias observed between food and nonfood images might be due to potential physical differences in luminance, contrast, etc. 2) Only 20 food stimuli (10 for high caloric density and 10 for low caloric density) were used, each presented for four times in the experiment. The limited number of images raises the question of generalizability across different food images; and repetitive presentations of images raises the possibility that any bias observed might not be due to attention per se but might relate to memory. 3) It was mentioned that appetizing stimuli were used to draw participants’ attention but it was not clear whether and how tastefulness was actually measured, which is important given individual differences in food preference. Moreover, caloric density was used, but it might not be a good index of healthfulness, since healthfulness, too, can be subjective. 4) reliability was measured only by Pearson’s correlation, which is a limited measure of reliability; for example, it does not take into account systematic differences between testing sessions. 5) the food and nonfood images were presented for 3 seconds, which is much longer than typically used in the dot probe task. It is then not clear how presentation duration might affect attentional bias and its reliability. Therefore, we plan to conduct a systematic investigation of the reliability of eye tracking based attention bias to food images in the dot-probe task using both reaction time and eye tracking measures. We plan to do so in two studies. In study 1, we resolve the confounds and limitations identified above with a few key innovations: 1) to control for differences in physical properties between food and nonfood stimuli, we use diffeomorphic transformed images based on the food images to serve as low-level control stimuli; 2) to improve generalizability and avoid the memory confound, we will increase the number of food images from 20 to 288, with each food image presented only once; 3) to control for individual differences in subjective perception, healthfulness and tastiness of the food items are rated by each participant individually, with the ratings then used to divide the food images into four groups (low healthfulness and low tastiness, low healthfulness and high tastiness, high healthfulness and low tastiness, and high healthfulness and high tastiness); 4) to provide a comprehensive evaluation, reliability is assessed by Pearson’s correlation coefficients/intraclass correlation coefficients (ICC), standard errors of measurement, and Cronbach’s alpha. In study 2, we also systematically investigate the role of presentation duration and the role of image repetitions in attention bias, by manipulating stimulus duration (100 ms, 500 ms, 1000 ms, vs. 3000 ms) and repetitions (1 time vs. 4 times).