Appearance Similarity: A Face and License Plate Recognition Alternative

Increasing Effectiveness with Video Analysis

Increasingly, municipalities and corporations alike are getting more value out of their new and existing video surveillance (CCTV) systems by implementing AI-powered video analytics software. These solutions detect, classify, and identify objects (people, vehicles, animals, etc.), and then index them, thereby transforming video footage into impactful data that is searchable, actionable, and quantifiable. Video intelligence software can help security and law enforcement teams rapidly locate and track vehicles or persons-of-interest, such as a missing person or a suspect leaving a crime scene, by distilling information about persons or vehicles from hours of footage. Face and license plate recognition (LPR) functionalities are well known video analytics capabilities that support these efforts. However, not all face or license plate recognition software solutions are effective, versatile, or even legal enough to provide value across the scenarios and conditions found in a real-world video surveillance network. So, how can you get the results you value when conditions and regulations present a barrier? You leverage appearance similarity to complement your face and license plate recognition capabilities.

How Facial Recognition and License Plate Recognition Work

Face recognition — based on Deep Learning — extracts unique identity features from a face image, encodes them into a feature-vector that represents the unique face, and then compares the vector with feature-vectors extracted from images that have been added to a watchlist. The recognition process can occur automatically when a new object is detected, and a human operator can assess and confirm any resulting match. Operators may create watchlists of faces using external image sources or digital images extracted from video.

License plate recognition works in much the same way and allows for watchlist configuration — based on an uploaded list of vehicle plates or manually entered license plate numbers – as well. Operators can search for faces or plate numbers either in real-time (alerting when a face or plate that appears in video exists on a watchlist – or if it doesn’t exist on a watchlist in cases like the validation of authorized access), or after an event while conducting a forensic review of video footage. These capabilities improve situational awareness and response times in critical, time-sensitive situations.

Constrained or “In the Wild” Recognition

When it comes to selecting face or license plate recognition technology, organizations should consider whether they need LPR and face recognition for “in the wild” or constrained scenarios. Some video analytics solutions have face recognition and LPR functions that are purpose-built for constrained scenarios, such as identifying a face at a terminal or a license plate at the entrance or exit to a parking garage. In constrained recognition scenarios, the following factors are at play:

• A specialized camera is usually used for a single purpose and is positioned in a dedicated way.
• The camera is in a controlled (constrained) environment where the object’s behaviors are somewhat predictable, such as a vehicle slowing or stopping at an entrance or exit of a parking lot, or a person passing through a turnstile.
• The scene conditions (such as illumination) are controlled and optimal.

Other face recognition and LPR solutions are built for “in the wild” recognition, to more accurately locate a person or vehicle across a wide network of existing or non-dedicated surveillance cameras that are located indoors or outdoors, in challenging settings, on highway poles, streetlamps, buildings, etc. Depending on an organization’s video analysis objectives and constraints, they may need a solution designed especially for “in the wild” surveillance scenarios.

Appearance Similarity: A Recognition Technology Alternative

“In the wild” face recognition and LPR solutions can encounter challenging conditions, such as sub-optimal camera position, lighting, or weather, yet they can still provide important information to augment simple forensic and real-time video intelligence. However, when faces or license plates cannot be identified using facial recognition or LPR, appearance similarity is an important adjunct capability. In addition, privacy or legal considerations may prohibit facial recognition or LPR– making appearance similarity the only way to effectively identify a person or vehicle in video.

Appearance similarity helps locate objects of interest based on predefined object attributes such as gender, hats, clothing colors, etc. It can also be based on Deep Neural Network feature extraction – the process of extracting identity features from a person or vehicle — which is similar to facial recognition.

Operators using traditional forensic video analytics and where facial recognition or license plate recognition is not an option because of the way the cameras are set up or privacy regulations, can use search criteria mentioned above such as gender and clothing color. In such cases, a deep learning-based appearance similarity-matching engine is one of the most effective tools for finding occurrences of specific persons and vehicles in video. While appearance similarity filtering is not specific enough to find a specific individual on a university campus, it can still exponentially reduce time-to-target when it counts.

Advantages of comprehensive video content analytics

Recognition capabilities are an important part of video analysis, and organizations looking for a video analytics solution should strongly consider their needs and restrictions surrounding recognition technologies. Some comprehensive and scalable solutions offer appearance similarity search alongside face and license plate recognition, providing a robust solution for every situation. Some video analytics solutions also go the extra mile to offer deployment options that flexibly meet the recognition needs and restrictions from your industry or country. Make sure you explore the deployment options available for your chosen solution so you can translate video into maximum impact while upholding the responsible use of these powerful tools.

BriefCam is proud to offer GDPR-friendly products and flexible deployment of Face Recognition in our comprehensive platform. Learn how you can get the video analytics you need while supporting critical compliance to regulatory needs.

Editor’s note: This post was originally published in September 2020, and has been refreshed and updated for accuracy.