Visual search modelling

Modelling cognitive behaviors is a method that is often used to predict how someone would behave, but also to test one's hypotheses and construct theories by comparing the results of a model to the true behaviors observed in a human study.

Both goals are extremely interesting and can potentially yield great results that have many applications.

For example, at the SGL, we are working on a model of visual search that predicts a temporal sequence of gaze positions (a scanpath) which reproduces how a human would observe a scene in search of an object, from the apparition of a new stimulus to the detection of the target object.

Such a model would be useful to computer vision domains aiming at reducing the size of images or videos (image and video compression), for instance, because by predicting where people are most likely to look we can make the rest of the scene less detailed.

This is a work in progress that is possible thanks to our knowledge of the deployment of visual attention in scenes according to scene grammar, to our understanding of gaze movements and dynamics, and to our skills in using machine-learning tools, such as graph neural networks.

Related publications

to come