Cross Camera Re-identification and recognition with Intelligent Video Tracking

How can Sentinel perform re-identification and track people across multiple cameras?

CROSS-CAMERA REIDENTIFICATION


Accuware Sentinel is probably the best intelligent video analytic technology, and can perform cross-camera reidentification and tracking.

What is “Cross Camera Re-Identification”?

In a venue monitored by multiple cameras, whose field of view may or may not be overlapping, a single individual can be seen by different cameras. The detection can happen at the same time or at different times, and it may be needed to track the location of that person across different cameras, and know where he is and has been. This process falls under the name of “Cross camera re-identification”.

How does it work?

Appearance Encoding Vectors

Sentinel assigns an “Appearance encoding vector” to each person detected. This vector is in fact a numerical vector that encodes the physical appearance of a person, including the colors of his clothes, his skin, height, hair, movements… The same person can have multiple vectors assigned: for example, if an individual leaves the field of view of a camera and re-enters again, he will be given a new vector.

Math helps!

If you want to know whether a person has already been seen by the system, you can simply compare the different Appearance Encoding Vectors for all the detected people. This is actually a mathematical process: you need to compute the euclidean distance between the two vectors. You can find at this link the complete technical explanation.

Is it accurate?

This question has multiple answers and it requires a complete explanation: the precision of the re-identification depends on several factors that affect how well Sentinel can recognize individuals. Let’s analyze them one by one:

Dimension of the population

Briefly: the less people Sentinel has to compare, the more precise the re-identification becomes. Why? Think about a security guard at the entrance of an entertainment park, whose role is to match the people that exit with the ones that had previously entered. This is an easy job if there are just a few people entering and leaving. It is easy to remember that the man with the red shirt entered the park before the lady with the yellow skirt. But if there are thousands of people the probability that someone looks similar to another person increases and it becomes harder to precisely match each individual. Sentinel, likewise, will find it hard to extract a unique and precise match for a single person.

People’s appearance

If everyone looks similar, it is hard to get the right match. Think again about the security guard: if there are just a few people entering the park, but they all look very similar and are dressed in the same way, it will not be an easy job to correctly match who leaves and who had entered.

Lighting conditions

The same color can appear extremely different under different lights. The human eye is well trained to filter lighting conditions and get a good perception of the real color, but it can be tricked too. If you enter a room where there is a red light bulb it will be hard for your eyes to perceive the real colors of objects. A white wall in a red room appears to be red. A red wall will look red, too! In this context, take a look at the following picture:

different colors under different lights

As you can see, three of the four bands are actually made of the same exact color: however, they look completely different depending on the lighting condition and reflections. And they can look similar to a different band of another color, under a different light.

Fortunately, it is possible to train Sentinel to mitigate the effects of lights. Artificial Intelligence and Deep Neural Networks (DNN) come to help, and Sentinel can learn up to a certain extent how to recognize different colors under different lights. This process is specific to each venue and installation and requires a good amount of engineering time: our tech support and R&D teams will work together with you to maximize the precision of Sentinel’s re-identification.

Conclusions

Sentinel can perform cross-camera reidentification with a level of precision that depends on several factors, as we have analyzed: it is therefore important to think about a Probabilistic Model. Given a specific individual, Sentinel is able to provide a list of “potential matches”, and it is possible to rank them using the Euclidean distance as a similitude metric.

Security applications

This becomes extremely important to save working time when looking for a specific person. It is no longer needed to go through hours of video footage and instead it is possible to get down to a short list of “suspects” that look like a specific person, and a single operator can manually check the detection list.

Analytics

In Analytical applications, where cross-camera tracking is needed to monitors the average paths and movements of people, this process can be automated by considering only the matches with a high rank of probability. When speaking about analytics it does not make sense to track 100% of the population: it is instead sufficient to monitor a significant sample of the population, which will be the part of the detections for which the probability of the re-identification is high enough to be considered safe. There may still be errors, but when it comes to thousands of detections the error is minimal and will be mitigated by the volume.

And what about Facial Recognition?

Facial recognition is an extremely accurate technology to re-identificate people, however it has several limitations:

  • Frontal detection: the person has to be detected by the camera with an angle up to 10-15°. This means that standard CCTV cameras will have an angle of detection too steep to provide a good face recognition. Of course, if a person is not looking towards the camera, face recognition will simply not work.
  • Face close-up: ideally the person has to be close enough to the camera for face recognition to work well. If the face is just 2-3 meters away from the camera, the precision of face recognition is going to be extremely low.
  • Image definition: of course face recognition requires an high quality of images, good resolution and clear lighting conditions.

As a matter of example, furthermore, a study published in January by the Massachusetts Institute of Technology (MIT) compared different facial recognition technologies: the study found that Amazon Rekognition, one of the most used facial recognition tools, had an error rate of 31 per cent when identifying the gender of images of women with dark skin. It is possible for our team to integrate Facial Recognition inside Sentinel as well: Accuware can provide customization services, to include additional features to Sentinel, based on your requests, including face recognition. Accuware R&D engineering team is available to discuss customization of the technology and the user interface, to let you get your own version of Sentinel, to address your specific requirements. We know that different projects have different requirements, and for this reason our technology is flexible and can be customized to fit your own needs.

Contact usContact us



WHY SENTINEL?