Anyone familiar with the ins and outs of hospital management understands that the security and operational challenges of hospitals are enormous. With thousands of patients, staff members, and visitors entering and circulating the hospital every day, there is much information to monitor and process. While many hospitals know that they need more effective methods for maintaining a safe and efficient environment, most hospitals lack the tools to manage their security and operational needs. One major setback many hospitals face is that they capture hours of video surveillance, but don’t have the resources to efficiently monitor and review the recordings. While this footage could be used to extract useful information and understand vulnerabilities and inefficiencies in the day-to-day hospital operations, petabytes of video data go to waste every day because there is no way of quickly and effectively reviewing all the footage. Video Synopsis® technology offers a smart and reliable video analytics solution for finding targets faster and enabling efficient monitoring. This makes it an ideal solution for maintaining hospital security and operations, facilitating hospitals to: Increase overall security Ensure patient and personnel safety Comply with procedures and regulations Mitigate false liability claims Improve operational management Optimize commercial operations The ability to easily manipulate video data with filters, makes it comfortable for anyone responsible for reviewing the data to find necessary information, detect anomalous behavior, and proactively prevent inefficiencies. We can see the profound impact of video analytics on hospitals by looking back at the Forbes interview with Director of Police, Security and Outside Services at Mass General Hospital, Bonnie Michelman, who anticipated how video analytics could influence hospitals, by combining video and analytics to identify risk situation, crowd control, unattended packages, abnormal traffic flow and suspicious loitering in restricted areas. Today, Mass General Hospital has implemented a video analytics strategy and is leveraging their video data to improve security and operations across the hospital. And it doesn’t stop at hospitals: As video analytics and Video Synopsis technology become more advanced, the types of organizations they can help will become more varied, leading to impactful results for many businesses and institutions.
ISC West 2017 was impressive, with many interesting products and new technologies – where GPU and deep learning took first place in prime time. Participating at NVIDIA’s booth and getting a first glimpse into exciting innovations, certainly highlighted the potential of video analytics powered by GPU and deep networks. With the ability to effectively process highly parallel computing tasks, such as video and graphics, today’s availability of GPU accelerates the process of deep learning, enabling real advancements, which until today could only be imagined. By applying GPU-based deep learning techniques to video analytics solutions, the market can expect faster processing of video alongside richer metadata. This combination will enhance the quality of object extraction and provide new applications that will be beneficial across many more verticals and use cases. Video analytics solutions, strengthened by deep learning, enable covering an entire scene for full object tracking at the highest level. Cost-efficient subscription-based cloud services for home and SMBs, trends over time, connecting to other applications, implementation with small appliances, cross camera search and family member identification are just some examples of possibilities the future holds. Furthermore, we can expect to see performance and accuracy improvements in: Video search, alerts and statistics Scene coverage for occupancy, crowd management and queue control Cross camera search and re-entry of objects Metadata aggregation over time and trending What’s clear from ISC West, is that expectations from deep learning are high – and rightfully so. GPU-based deep networks particularly facilitate making sense of actual video content. This means video can be searched by the content that is actually displayed in the video. Video remains the strongest sensor and metadata is still king. Leveraging deep learning capabilities, Video Synopsis® and analytics can deliver more value to various markets, from law enforcement and security to safe and smart cities, campuses and retail. Interested in learning more about GPU processing for video analytics? Check out CTO Tom Edlund’s latest blog post about cost efficient video processing using GPUs.
The video analytics industry is constantly producing new and innovative solutions to provide deeper insight and derive more useful applications from video. These new capabilities often require more compute power. GPUs (Graphic Processing Units), if used correctly, can accelerate video processing and reduce costs. Therefore, many platforms are now adding GPUs to offload the CPUs (Central Processing Units) processing. Recently, this trend has been accelerated by the wide adoption of Deep Neural Networks (DNNs) to analyze images and videos. GPUs were originally designed to optimize rendering-to-display devices and enabled the rapid advancement of the computer game industry. Today, GPUs are also used for General-purpose computing (GPGPU) using higher level frameworks such as OpenCL or CUDA. GPUs excel in repeated parallel computations of large data (aka Single Instruction Multiple Data, or SIMD) and are ideal for DNNs and many image processing algorithms. Also, GPUs have dedicated HW for video decoding and encoding. Considering these advantages, it may seem straightforward to run your video analytics on GPU and improve performance. This is true if most of your computations are done by large/deep DNNs or run full frame processing at high resolutions. However, video analytics are often highly optimized and limit the computations to small regions of interest and lower spatial and temporal resolutions, making it difficult to reach high utilization of the GPU. In such cases, you will need to invest more in your system architecture to fully utilize the GPU by using technologies such as batching and buffering. If you consider porting a video analytics engine from CPU to GPU you should ask yourself the following: Do I use DNNs for most of the computations today or am I planning to in the future? Are my computer vision algorithms GPU friendly (e.g. data size and parallel computations)? Where do I deploy my processing? Edge devices, laptops and workstations are ideal for GPUs, while its harder to reach cost effectiveness on local or cloud servers. Every organization’s video processing needs are unique, and the answers to these questions should help give you an idea as to whether moving to GPU processing might be worth considering for you. If you’re looking to gain deeper insight about running video processing on GPU, you are invited to join my talk on April 6th at 2:00pm at NVIDIA’s ISC West booth #20075.
Last week, I talked about how small and medium sized businesses (SMBs) face large challenges and whether they are getting value from their video recordings. This week, I want to ask the same question for homeowners. Are they getting their videos worth in the home? Ask anyone in the industry, and they’ll tell you that IP cameras are one of the fastest growing categories in consumer electronics, both as a stand-alone single point camera solution, or part of a monitored home security offering. SDM presents great perspectives on the role of IP cameras in the residential market in this article. The list of reasons why consumers buy a security camera is long and wide, but a common theme for the purchase is peace of mind. A security camera should be able to fulfill this promise because it allows people to: Know what’s going on in their home Gain a sense of security Be alerted when something goes wrong And, with mobile phones, check in from anywhere, anytime… But do the cameras really fulfill all of these promises? Unfortunately, I say they don’t…. If you watch almost any commercial for a home security camera, the focus of the marketing message is on the high level critical value propositions stated above, plus more. Things get complicated when the camera technology cannot fully deliver on either the promises made by the manufacturer, or the expectations of the consumer. Motion detection capabilities in a camera do exactly what they are supposed to do…. detect motion. Unfortunately, the camera does not have the ability to understand the relevance of the motion it’s detecting, so it sends an alert accordingly. It could be the family, the pets, trees blowing in the wind, light changes, etc. They detect and alert on ALL motion, not just the motion the owner wants to know about. Upon setting up a security camera in a home, most consumers are excited to get that first alert come across the phone indicating the camera detected motion. More often than not, Mom and Dad are telling the kids to go wave at the camera. Bzzzz! Your phone notifies you motion was detected. Hurrah!! Several days, or weeks later, after getting 20, 30, and sometimes 40 or more push notifications that your family and pets have walked in front of the camera, the homeowner gets frustrated, and ends up turning the alert capability off. I saw this exact consumer behavior pattern in my prior platform, and have heard the same from other platform providers. Once notifications are disabled, one of the very core reasons a consumer bought the camera to begin with goes away, and the expected value is no longer there. Rather than detect only motion, what if the camera could detect people, animals, vehicles, or more importantly, trusted family members? What if you only got alerts when the dog got on the couch, but not when your kids walked through the living room? What if you got an alert when a burglar was actually in your home rather than a trusted family member? What if you could see a 1 minute video summary of all of the activity in the home, and if you saw something of interest, you could stop and go to the original video from earlier in the day. These capabilities alone would fulfill the implied benefits and expected value consumers expect from a camera.
Small and medium businesses (SMBs) are faced with large challenges year after year. The three main business challenges for SMBs in 2017 are the same as in 2016, but in different order. Improve workforce productivity (2nd in 2016) Attract and retain new customers (3rd in 2016) Improve quality of products and processes (1st in 2016) On top of these 3 challenges, there is always the matter of security. Call it loss prevention, theft, shoplifting or retail shrinkage, this continually burdens all types of SMB owners. Additionally, statistics indicate every 1 in 11 people is a shoplifter, which is a huge concern for any business owner. Usually different problems require different solutions. However, with video analytics capabilities today, there’s one solution to solve all those concerns. Video data. By making use of video the right way, SMBs can manage business challenges and security issues easily and efficiently. As we’ve mentioned before, the #1 sensor is video, and for SMBs this means getting valuable information, which no other standard sensor can provide. Any business owner knows that the more information they have (providing they can actually make sense of it), the better value they can create for customers, and the more effectively they can manage their business. With the proliferation of surveillance cameras, more and more video data is available for SMBs. The problem however, is that typically, the video from surveillance cameras is either never viewed, or only viewed close to the time of an incident. And in general, the information generated from the video is never realized, data is forgotten, archived (which really means never to be found again), or simply deleted. If video essentially means data, i.e., valuable information, then why would any business owner ignore it? SMBs – read my lips: You can increase sales and profit by using video from your already-installed cameras. You can ensure a secure business by quickly applying Video Synopsis. You can gain insights from video data to run your business more efficiently. You can better understand customer behavior, why they do or don’t purchase. You can monitor employee productivity. You can understand optimized product layout. You can count how many customers walk into your business and see what they do. You can optimize your marketing efforts. And finally, you can sleep better. Next week, we'll take a look at whether homeowners are getting their video's worth from the cameras they install in their homes.
I have heard from many investigators that the most painful part of their jobs is reviewing video. Often investigators are working with too little information and viewing too much video to be effective in finding anything. Reviewing video no longer has to be an experience that leads to you tearing out your hair. It does not need to be like searching for a needle in a haystack, a futile waste of time. So what wastes time reviewing video? Looking at things that are not what you’re looking to find. I hope you weren’t expecting a more sophisticated answer, because there is none. It’s as simple as that. If you have to go through a large amount of video, it can help immensely to review a subset of the relevant things in that video. Less video data simply means less review time. So how can you reduce the video you must review to find your target? This is achieved by eliminating objects that are not relevant to what you want to find, while still keeping the objects relevant to your search. This is a simple concept, but takes many years of effort and significant technology to do well. Are there other means of reducing the video you need to review? Motion search is clearly not the solution for eliminating what you don’t want to see. While this may slightly reduce the time of reviewing video, this is in effect putting a bandage on an open wound since it still overwhelms an investigator with a long series of individual events for review. Effective review requires more capable tools. This is where the ability to add layering of analytical filter parameters (such as size, direction, color, path, speed), yields reduction in the number of results for review, leading to shorter review time. So really it’s what you don’t want to see that has to be taken into account when reviewing and searching video. This of course varies tremendously based on the person, vehicle or object you’re looking to find, the nature of the investigation or situation, and what you know about the suspect or event. That’s why it’s critical to leave the choice of eliminating objects in the hands of whoever is viewing the video and provide that person with tools that enable quick and interactive work with the search results. No matter how you slice it, if you can use what you know to reduce the results, you can cut the time you need to spend reviewing video. This helps find what you’re looking for, and in less time.
I spent that past couple of weeks at the leading VMS players’ annual events for partners, integrators and end users. Much was said and discussed on the increasing need for reliable video analytics solutions, and finally, thanks to today’s video quality and management capabilities video content can be leveraged for the benefit of all types of verticals and use cases. From security and safety, to retail, operations, business insights and even quality-of-life, everyone agreed that great value can be derived from video. It is, after all, considered to be the richest sensor. Still, what I really took away from these events was the inspiring notion that the promise of video analytics is so big; and we have so much more to discover and benefit from. Essentially for most people, video is video, is video. However, when we take a deeper look inside the video, we see that we're only scratching the surface by looking at what video presents to us overtly. To gain more from video, in ways we could only have imagined in the past, video should be used for its metadata (information about video data). Metadata provides a whole new meaning to searching, tracking and understanding object behavior within a video. In fact, you can even learn and monitor human behavior by using metadata. Metadata also enables video analytical capabilities in complex and high-activity scenes. Finally, with metadata, all types of details can be organized, whether counting cars crossing a junction or seeing where shoppers dwell in a store. With the right tools, valuable insights can be gained through metadata. Yes, metadata is king. As a result of the significant advancements in video analytics, metadata today presents information with very rich and detailed content – something organizations of all types want to hold on to. Furthermore, metadata is easier to store and save than actual video files. Metadata takes up less bandwidth, thus, extracting information from the video in the form of metadata becomes quick and simple. The metadata can then be cost-effectively uploaded to the cloud and aggregated for generating reports and identifying trends over time. Essentially, metadata is the key for further unlocking the potential of video analytics. Cutting-edge technologies such as deep learning, coupled with the power of metadata, ensure that video analytics continues to intrigue the industry and present new and exciting capacities, enabling more and more verticals to benefit from the promise of video analytics. All we need to remember is to take a better look into the video and not be fooled by what we simply see. Sometimes it’s what we don’t see at first that really matters.
Last week I presented Smart Synopsis – a highly powerful and intuitive video search engine that allows users to search for specific events by building complex queries. The matching events are rendered as a compact video by order of relevance. Processing the video once, and storing the metadata in an indexed database, enables search results that are returned instantly. Turns out, that within this metadata are the building blocks needed for most video related applications. For example: A single carefully selected thumbnail image of each object can be used to build numerous user interfaces with quick navigation through events, even on small screen devices. The full track of an object can be used to inject the relevant pixels to complex classifiers such as face recognition systems and optimize both cost and accuracy. Metadata from cameras can be used to connect devices in IoT environments. Big data engines can finally add input from the camera – the most data-rich sensor – to deliver significant business insights. In short, rich metadata is key to ensure that video analytics solutions (as smart as can be) are sustainable over time and worthy of their investment, offering the ability to grow, add applications and generate value from video across all levels of an organization.
When I joined BriefCam in 2011, we had an ambitious challenge: how to take the groundbreaking Video Synopsis® technology developed by Prof. Shmuel Peleg at the Hebrew University of Jerusalem to the next level. How to turn it from a technology that enables reviewing hours of video in minutes into a commercially-viable technology that could enable viewing weeks or even months of video in a short time. Enter Smart Synopsis®. The addition of the possibility to search for specific objects. This addition of sophisticated, and often complex search capabilities requires full object tracking and accurate segmentation, and is definitely a game changer. The importance of full tracking is twofold: it enables tracking all interesting objects in a scene, as well as the tracking the full trajectory of an object throughout a scene. This is particularly useful to: Understand the significance of behavior, such as the flow of customers in a store, where people tend to loiter (and the impact on security and sales), as well as interactions between people. Achieve a single alert per event. This is important since most tracking analytics are frame-based (they try to detect objects in a specific frame) with moderate localization (they only have a rough estimation of where the object is located – the bounding box), meaning that they may deliver multiple alerts per event or miss following events such as tailgaters. Accurate segmentation is also essential in the development of Smart Synopsis, in order ensure that synthetic video rendering looks fully realistic. This is achieved by deploying a tight-fitting mask in each frame (pixel level localization). Smart Synopsis delivers the full tracking and accurate segmentation that is essential for most mission-critical applications. It resolves object occlusions, and therefore can be used in high-level processes such as action recognition, object identification and object recognition. So, when basing video analytics on Smart Synopsis, finding what you are looking for becomes an even faster and more accurate process. Full context can be retained with the original, full-length video always remaining a click away, while reviewing the video synopses generated. Furthermore, Smart Synopsis integrates a methodology that uses mosaic images together with rich metadata for video indexing, thereby enabling the inclusion of objects from different times in static synopsis images. More on how things can get even smarter, next week.
I recently read an analysis from IHS indicating that the demise of server-based analytics is imminent. The analysis leaves one with an inevitable sense that all analytics will be done at the edge. They provide additional analysis that “pure server only based analytics was estimated to have shrunk by $39.4 million in one year… However, more steady transition, along with a continued dominance of server-based analytics in certain environments was still forecast.” And here’s a second however from IHS: “demand for high end server-based analytics is expected to be sustained.” True, technologies in the enterprise arena such as motion detection and applications like directional motion are being built into cameras and efficiently processed on the camera. In the residential space, this is often the case as well. More and more residential solutions are heading to the cloud and the ability to trigger recording or notification locally based on simple analytics provides many benefits. Yet even in the residential space, advanced analytics running in the cloud are dominant. Clearly, there are apparent advantages to edge analytics, offering better system scalability and access to uncompressed data. However again, in reality most analytics are still server-based. In fact, server-based video analytics offers high reliability, strong processing capabilities and sophisticated search options. Furthermore, server-based video analytics enables organizations to apply various types of analytical applications, providing value across all levels of the organization. So, I ask myself, will better edge processing capabilities and architecture “terminate” server-based video analytics? If you, for one, think this is the grand finale of server-based analytics, think again. Or read the round table discussion on the continuing role for server-based video analytics. Edge-based video analytics applications are normally limited to one application per camera due to insufficient processing power. While more powerful processors are going into camera and other edge devices, implemented in this fashion you are looking at a fixed capability device installed on the edge versus a more flexible infrastructure that can be centrally managed and expanded through additional servers or cloud resources. Server-based video analytics is not as limited by a device’s processing power, and in addition is camera agnostic. This means organizations can enjoy the ability to run multiple video analytics applications at the same time, and on any type of camera. Confused? Check out this great summary table of pros and cons for server-based vs. edge-based video analytics. Server-based, edge-based; what is evident is the growing need for smart and reliable video analytics solutions, way beyond what anyone once imagined video analytics could deliver. And this need will grow even more as the use of metadata becomes more common in security and video applications. I know metadata is a significant topic on its own, and will rightfully demand dedicated blog posts in the future. Stay tuned…