The eye-in-the-sky is about to get a brain. Researchers led by Professor Shaogang (Sean) Gong at Queen Mary University of London are developing a system to enhance the capabilities of CCTV. Termed the Samurai Project, and funded under the auspices of the EC, the new program would detect suspicious behavior in real time by monitoring a vast network of cameras. Feedback from system operators would help Samurai determine which behavior was abnormal, and which acceptable. The software is capable of tracking objects and people across several different camera views even under lighting changes (as when someone moves indoors). If ultimately successful, the project will go a long way to improving the usefulness of CCTV networks, allowing for intelligent, adaptive, and fast security surveillance. Check out the brief video from New Scientist after the break.

Samurai is indicative of a wider trend towards intelligent surveillance. Project Indect (funded by the EU) is looking to pour over online digital information, and software like Vitamin D Video is bringing smart video filtering to the private sector. We are likely to see a marked improvement in what passive elements, like cameras, are capable of when guided by learning software. In the short term this could mean the UK could get better use out of its nationwide (but London focused) CCTV system. Over the next few years transportation hubs, military bases, and other government run facilities will have the means to secure themselves against terrorist attacks. Just as importantly, intelligent software packages are likely to provide measured response suggestions for security personnel, so that someone suspected of graffiti wouldn’t be targeted in the same way as someone suspected of planting an improvised explosive device. Eventually, smart surveillance could allow those behind the cameras unprecedented insight into what all of us are doing, and possibly thinking.

There are so many security camera filtering programs (both publicly and privately funded) that it may be hard to see what makes Samurai unique. First,it’s name is one of the most convoluted acronyms I’ve seen recently: Suspicious and Abnormal behavior Monitoring Using a netwoRk of cAmeras for sItuational awareness enhancement. More importantly, Samurai will work with a variety of sensors (not just CCTV cameras) including mobile wearable audio and video recorders (attached to security teams) to enhance observation. It will focus on real-time images to and provide constant context-based data to said security personnel. System operators can also train Samurai using feedback so that it does not provide false alarms every time a custodian adjusts a trash can or a toddler throws a tantrum. If a threat is identified Samurai can track it through multiple camera POVs and lighting levels, relying on shapes and movement patterns to identify people.

Early November saw the Samurai Project team present an end-user workshop to demonstrate its current capabilities. The EC found Samurai promising enough to have guaranteed it funding through the end of 2011. This is certainly a Europe wide endeavor – research is also performed at the University of Verona and there are project partners in countries from France to Estonia. Still, a disproportionate number of those partners, and the head research team is located in the UK. Britain seems to be placing themselves at the forefront of government-funded smart surveillance technology.

As always when discussing surveillance, I have to wonder about the impact about such constant monitoring on our society. Is the EC worried about the appearance of being too similar to Big Brother? Well, the Samurai Project brochure seems needlessly fun and hip… so maybe.  Certainly the concepts of privacy have already shifted some with the advent of social networking. Future generations will have to become more comfortable with observation if they continue to fight against terrorism, drug trafficking, and random acts of violence. Undoubtedly, the boundaries of government or private intrusion into individual privacy will be the subject of an ongoing (and loud) debate. I think the choice in naming this project was somewhat prescient. From one point of view the samurai were a noble and honorable class of warriors that protected the public. From another view, they were feudal tyrants that as often oppressed their vassals as saved them. I’m sure that those in favor or against government monitoring feel the same way about smart security surveillance.

[photo credit: Samurai Project, EC]
[video credit: New Scientist via Prof. Gong at Queen Mary University, London]