Smart Eyes: Attending and Recognizing Instances of Salient Events

The SEARISE project develops a trinocular active cognitive vision system, the Smart-Eyes, for detection, tracking and categorization of salient events and behaviours. Unlike other approaches in video surveillance, the system will have human-like capability to learn continuously from the visual input, self-adjust to ever changing visual environment, fixate salient events and follow their motion, categorize salient events dependent on the context. Inspired by the human visual system, a cyclopean camera will perform wide range monitoring of the visual field while active binocular stereo cameras will fixate and track salient objects, mimicking a focus of attention that switches between different interesting locations.

The core of this artificial cognitive visual system will be a dynamic hierarchical neural architecture – a computational model of visual processing in the brain. Information processing in Smart-Eyes will be highly efficient due to a multi-scale design: Controlled by the cortically plausible neural model, the active cameras will provide a multi-scale video record of salient events. The processing will self-organize to adapt to scale variations and to assign the majority of computational resources to the informative parts of the scene.

The Smart-Eyes system will be tested in real-life scenarios featuring the activity of people in different scales. In a long-range distance scenario, the system will be monitoring crowd behaviour of sport fans in a football arena. In a short range scenario, the system will be monitoring the behaviour of small groups of people and single individuals. The system’s capability for self-adaptation will be specifically demonstrated and quantified compared to systems with ‘classical’ architecture that are trained once and then used on a set of test scenes.

SEARISE Project has been supported by the FP7 Programme, under thematic objective Cognitive Systems and Robotics: www.cognitivesystems.eu