Natural Scene Perception

Consider for a moment the range of images that one might categorize as a picture of a forest. These images may have a wide variety of characteristics, many of which could overlap with other categories of images (e.g. beaches or mountains). However, when we see a forest, we can immediately categorize it without the need to first recognize individual trees and then infer the forest. Humans are remarkably adept at this seemingly difficult perceptual categorization process, and for good reason—recognizing the essence of a scene (often called “scene gist”), provides important context for subsequent more detailed analysis of our visual environment.

Using functional magnetic resonance imaging (fMRI) and distributed pattern analysis, we found that area V1, the parahippocampal place area (PPA), retrosplenial cortex (RSC) and lateral occipital complex (LOC) all contain information that distinguishes between natural scene categories. More importantly, correlations with behavior suggest that the information present in the PPA, RSC, and LOC may be used by humans to aid in categorizing natural scenes. Specifically, predictions based on fMRI signals in these areas are in remarkable agreement with behavioral scene categorization in two ways: they show similar confusions among categories; and they show a similar decrease in accuracy when scenes are presented up-down inverted. Together these results suggest that a network of regions, including the PPA, RSC and LOC participate in the human ability to categorize natural scenes.

We are currently continuing to explore this through eye tracking and behavioral experiments that aim to investigate the effect of natural scene contexts on human behavior and eye movement patterns. The focus of future research will also expand to include the investigation of scene perception across sensory modalities (e.g. audio-visual scene perception).