Search

Recent developments in machine vision methods for automatic, quantitative analysis of social behavior have immensely improved both the scale and level of resolution with which we can dissect interactions between members of the same species. In this paper, we review these methods, with a particular focus on how biologists can apply them to their own work. We discuss several components of machine vision-based analyses: methods to record high-quality video for automated analyses, video-based tracking algorithms for estimating the positions of interacting animals, and machine learning methods for recognizing patterns of interactions. These methods are extremely general in their applicability, and we review a subset of successful applications of them to biological questions in several model systems with very different types of social behaviors.

SUMMARY Walking fruit flies, Drosophila melanogaster, use visual information to orient towards salient objects in their environment, presumably as a search strategy for finding food, shelter or other resources. Less is known, however, about the role of vision or other sensory modalities such as mechanoreception in the evaluation of objects once they have been reached. To study the role of vision and mechanoreception in exploration behavior, we developed a large arena in which we could track individual fruit flies as they walked through either simple or more topologically complex landscapes. When exploring a simple, flat environment lacking three-dimensional objects, flies used visual cues from the distant background to stabilize their walking trajectories. When exploring an arena containing an array of cones, differing in geometry, flies actively oriented towards, climbed onto, and explored the objects, spending most of their time on the tallest, steepest object. A fly's behavioral response to the geometry of an object depended upon the intrinsic properties of each object and not a relative assessment to other nearby objects. Furthermore, the preference was not due to a greater attraction towards tall, steep objects, but rather a change in locomotor behavior once a fly reached and explored the surface. Specifically, flies are much more likely to stop walking for long periods when they are perched on tall, steep objects. Both the vision system and the antennal chordotonal organs (Johnston's organs) provide sufficient information about the geometry of an object to elicit the observed change in locomotor behavior. Only when both these sensory systems were impaired did flies not show the behavioral preference for the tall, steep objects.