ABSTRACT
Abstract
An apparatus including an interface and a processor. The interface may be configured to receive pixel data generated by a capture device. The processor may be configured to generate video frames in response to the pixel data, perform computer vision operations on the video frames to detect objects, perform a classification of the objects detected based on characteristics of the objects, determine whether the classification of the objects corresponds to a user-defined event and a user-defined identity and generate encoded video frames from the video frames. The encoded video frames may comprise a first sample of the video frames selected at a first rate when the user-defined event is not detected and a second sample of the video frames selected at a second rate while the user-defined event is detected. The video frames comprising the user-defined identity without a second person may be excluded from the encoded video frames.
Description
This application relates to U.S. patent application Ser. No. 17/130,442, filed on Dec. 22, 2020, which relates to China Patent Application No. 202010845108.6, filed on Aug. 20, 2020. U.S. patent application Ser. No. 17/130,442 also relates to U.S. patent application Ser. No. 17/126,108, filed on Dec. 18, 2020. Each of the above applications are hereby incorporated by reference in its entirety.
FIELD OF THE INVENTION
The invention relates to computer vision generally and, more particularly, to a method and/or apparatus for implementing a person-of-interest centric timelapse video with AI input on home security camera to protect privacy.
BACKGROUND
A timelapse video mode for conventional internet-connected/cloud-enabled cameras is usually implemented by software operated on a cloud server. The software relies on using the resources of distributed processing of the cloud server (i.e., scalable computing). Timelapse video clips are displayed at a fixed frame rate for a fast-forward effect. For example, video frames are selected at fixed intervals to create the timelapse video (i.e., every thirtieth frame is selected from a thirty frames per second video to create the timelapse video).
Internet-connected/cloud-enabled cameras encode video data and then communicate the encoded video streams to the cloud servers. In order to transcode the encoded video streams into a timelapse video, the cloud server has to first decode the encoded video stream. The cloud servers spend lots of CPU cycles to decode the conventional video streams (i.e., compressed video using AVC or HEVC encoding), extract the video frames at fixed frame intervals, then transcode these frames into a timelapse video. Meanwhile, even with a timelapse video, users have difficulty finding important details captured by the internet-connected/cloud-enabled cameras. Since a timelapse video always uses a fixed frame rate that does not use all the video data originally captured by the internet-connected/cloud-enabled, video frames at a normal display speed are not available for the entire duration of time that something of interest to the user is in the original video captured. For example, when a security camera observes a potential event of interest, such as a person on the premises, the timelapse will be the same as when the security camera observes nothing of particular interest.
Conventional internet-connected/cloud-enabled cameras upload full video content to the cloud for easy access from a mobile phone. The video can be preserved for security purposes (i.e., used as evidence) or the cloud service provider might not delete uploaded content. Videos uploaded can remain available indefinitely. The uploaded video content can be a privacy concern. Uploaded video content can include video clips of family members. Uploaded video content can indicate behavioral patterns of people. Even if a user desired to upload the video when it was uploaded, the user might later regret having the video available. The privacy concern can make end-users uncomfortable or the video content could be used by hackers.
It would be desirable to implement a person-of-interest centric timelapse video with AI input on home security camera to protect privacy.
SUMMARY
The invention concerns an apparatus including an interface and a processor. The interface may be configured to receive pixel data generated by a capture device. The processor may be configured to generate video frames in response to the pixel data, perform computer vision operations on the video frames to detect objects, perform a classification of the objects detected based on characteristics of the objects, determine whether the classification of the objects corresponds to a user-defined event and a user-defined identity and generate encoded video frames from the video frames. The encoded video frames may comprise a first sample of the video frames selected at a first rate when the user-defined event is not detected and a second sample of the video frames selected at a second rate while the user-defined event is detected. The video frames comprising the user-defined identity without a second person may be excluded from the encoded video frames.
BRIEF DESCRIPTION OF THE FIGURES
Embodiments of the invention will be apparent from the following detailed description and the appended claims and drawings.
FIG. 1 is a diagram illustrating an example context of the present invention.
FIG. 2 is a diagram illustrating example internet-connected cameras implementing an example embodiment of the present invention.
FIG. 3 is a block diagram illustrating components of an apparatus configured to provide an event centric timelapse video with the assistance of a neural network.
FIG. 4 is a diagram illustrating an interconnected camera communicating with a cloud server and a video processing pipeline for generating a timelapse video.
FIG. 5 is a diagram illustrating a smart timelapse mode on an edge AI camera with CV analysis excluding privacy event video frames.
FIG. 6 is a diagram illustrating a smart timelapse mode on an edge AI camera with CV analysis including mixed event video frames.
FIG. 7 is a diagram illustrating event detection in a captured video frame.
FIG. 8 is a diagram illustrating an encoded video frame of a smart timelapse video stream with a privacy effect applied.
FIG. 9 is a diagram illustrating an application operating on a smart phone for controlling preferences for a timelapse video.
FIG. 10 is a diagram illustrating an application operating on a smart phone for controlling privacy settings for a timelapse video.
FIG. 11 is a flow diagram illustrating a method for implementing a person-of-interest centric timelapse video with AI input on home security camera to protect privacy.
FIG. 12 is a flow diagram illustrating a method for adding a distortion effect to protect privacy of a pre-defined identity.
FIG. 13 is a flow diagram illustrating a method for updating feature set information for identifying particular people using a local network connection.
FIG. 14 is a flow diagram illustrating a method for identifying a person to determine whether a privacy event has been detected.
DETAILED DESCRIPTION OF THE EMBODIMENTS
Embodiments of the present invention include providing a person-of-interest centric timelapse video with AI input on home security camera to protect privacy that may (i) implement event detection on an edge device, (ii) implement video encoding on an edge device, (iii) generate smart timelapse videos with varying framerates based on events/objects detected, (iv) detect objects/events using a convolutional neural network implemented on a processor, (v) adjust a timelapse framerate to capture all video frames when an event/object is detected, (vi) perform facial recognition and/or object classification local to the camera device, (vii) upload an encoded smart timelapse video to a cloud storage server, (viii) enable configuration of parameters for event/object detection and privacy concerns, (ix) exclude video frames that may be a privacy concern, (x) blur or otherwise conceal faces in an encoded video frame to preserve privacy and/or (xi) be implemented as one or more integrated circuits.
Embodiments of the present invention may be configured to generate a smart timelapse video. The smart timelapse video may be implemented by adjusting a video display speed automatically based on content detected in the captured video. The video display speed may be adjusted in response to a selection rate of video frames from the originally captured video to be used for the smart timelapse video. Embodiments of the present invention may be configured to generate encoded smart timelapse videos. The smart timelapse video may be generated on a cloud service (e.g., using scalable computing). The smart timelapse video may be generated on an edge device (e.g., an artificial intelligence (AI) camera). Generating the smart timelapse video on the edge device may comprise only using processing capabilities within the edge device (e.g., without outsourcing processing to an external device). In an example, the edge device may be an internet-connected and/or cloud-enabled camera that may implement a home security camera.
Embodiments of the present invention may be configured to protect the privacy of particular people when the smart timelapse video is generated. The video content (e.g., what appears in the smart timelapse video) may be automatically adjusted in response to the objects/events detected. Particular objects/events may be shown as captured in the smart timelapse video and other objects/events may be excluded and/or removed from the smart timelapse video stream. For example, the smart timelapse video stream may be generated to include the faces and behaviors of strangers but exclude the faces and behaviors of family members. Generally, the faces and/or behaviors excluded from the smart timelapse video stream may correspond to privacy concerns (e.g., identifying particular people, a person being uncomfortable being on video, preventing the storage of potentially embarrassing behaviors, etc.). The criteria for including or excluding video content may be varied according to the design criteria of a particular implementation.
The edge AI home security device/camera may be configured to implement artificial intelligence (AI) technology. Using AI technology, the edge AI camera may be a more powerful (e.g., by providing relevant data for the user) and a more power efficient solution than using a cloud server in many aspects. An edge AI camera may be configured to execute computer readable instructions locally (e.g., internally) on the device (e.g., without relying on external processing resources) to analyze the video content frame by frame. Based on the analysis, content in the video frames may be tagged with metadata information. The metadata information may be used to select video frames for a smart timelapse video. For example, the video frames may be classified by being tagged as having no interesting object/event and as having an interesting object/event. Computer vision (CV) operations may determine whether there is an interesting object/event (e.g., based on a pre-defined feature set).
When there is no interesting CV event (or type of object) in the video frames for a duration of N seconds (N=60/120/190/ . . . ), the edge AI camera may select one of the video frames captured during the no-event duration. The selected no-event video frame may be used for video encoding (e.g., a video encoder built in the edge AI camera device that performs video encoding internally). Selecting one video frame from the no-event duration N for encoding may be repeated for each N second duration with no event detected. Selecting one video frame from the no-event duration N for encoding may result in an encoded output that provides a condensed portion of the captured video (e.g., to effectively fast-forward through âmeaningless contentâ portions of video captured with a high display speed).
When there is an interesting CV event (or type of object) detected in the video frames for a duration of M seconds (M=5/15/30/ . . . ), the edge AI camera may adjust how many and/or the rate of selection of the CV event video frames for the event duration M. The event detected may be defined based on a pre-determined feature set. In an example, the object and/or event of interest may be considered to be detected when a person is detected, a car is detected, an animal detected, an amount of motion is detected, a particular face detected, etc. The selection of the video frames for the smart timelapse video may be adjusted to select all of the video frames from the M second duration of the CV event (e.g., for a 2 minute event captured at 60 frames per second, the entire 7200 frames may be selected). The selection of the CV event video frames may be adjusted to select video frames at a higher rate than for the no-event duration (e.g., select more frames, but not all frames) for the M second duration of the event (e.g., for a 2 minute event captured at 60 frames per second, the rate of selection may be changed to 30 frames and every other frame may be selecting resulting in 3600 video frames being selected).
The selected CV event video frames may be encoded (e.g., using the on-device video encoding of the AI edge camera to perform encoding locally). Selecting video frames at a higher rate for the event duration M for encoding may result in an encoded output that provides either a portion of the smart timelapse video with the normal display speed for the âmeaningful contentâ or a portion of the smart timelapse video with a slightly condensed (but not as high speed as the âmeaningless contentâ) display speed for the âmeaningful contentâ.
With the smart timelapse video generation implemented on an edge camera, the user may quickly browse through the captured video content for a long period of time (e.g., days/weeks/months), and the user may be confident that no interesting CV event will be missed. An interesting CV event may comprise detecting a person (e.g., a known person) by face detection and/or face recognition, detecting a car license plate (e.g., using a license plate reader), detecting a pet (e.g., a known animal) using animal/pet recognition, detecting motion (e.g., any motion detected above a pre-defined threshold), etc. The type of event and/or object detected that may be considered an interesting event may be varied according to the design criteria of a particular implementation.
Embodiments of the present invention may enable a user to specify the type of object/event that may be considered to be interesting. In one example, an app operating on a smartphone (e.g., a companion app for the edge AI camera) may be configured to adjust settings for the edge AI camera. In another example, the edge AI camera may be configured to provide a web interface (e.g., over a local area network) to enable a user to remotely select objects/events to be considered interesting events. In yet another example, the edge AI camera may connect to a cloud server, and the user may use a web interface to adjust settings stored on the cloud server that may then be sent to the edge AI camera to control the smart timelapse object type(s) of interest.
The duration of the smart timelapse video may be configured as regular intervals and/or since the last time the user has received a timelapse video. For example, if the user has missed 20 event notifications, the moment the users interacts with the app (e.g., swipes on a smartphone) to view the events, the event-focused timelapse may be presented to the user for viewing. While embodiments of the present invention may generally be performed local to an edge AI camera (e.g., implementing a processor configured to implement a convolutional neural network and/or video encoding locally), embodiments of the present invention may be performed using software on the cloud server to achieve similar effects.
Embodiments of the present invention may be further configured to account for potential privacy issues based on the objects/events detected. For example, the CV event detected may comprise detecting people (e.g., faces of strangers, faces of known people, faces of family members, combinations of strangers and known people, etc.). When there is an interesting CV event (or type of object) det
This application relates to U.S. patent application Ser. No. 17/130,442, filed on Dec. 22, 2020, which relates to China Patent Application No. 202010845108.6, filed on Aug. 20, 2020. U.S. patent application Ser. No. 17/130,442 also relates to U.S. patent application Ser. No. 17/126,108, filed on Dec. 18, 2020. Each of the above applications are hereby incorporated by reference in its entirety.
FIELD OF THE INVENTION
The invention relates to computer vision generally and, more particularly, to a method and/or apparatus for implementing a person-of-interest centric timelapse video with AI input on home security camera to protect privacy.
BACKGROUND
A timelapse video mode for conventional internet-connected/cloud-enabled cameras is usually implemented by software operated on a cloud server. The software relies on using the resources of distributed processing of the cloud server (i.e., scalable computing). Timelapse video clips are displayed at a fixed frame rate for a fast-forward effect. For example, video frames are selected at fixed intervals to create the timelapse video (i.e., every thirtieth frame is selected from a thirty frames per second video to create the timelapse video).
Internet-connected/cloud-enabled cameras encode video data and then communicate the encoded video streams to the cloud servers. In order to transcode the encoded video streams into a timelapse video, the cloud server has to first decode the encoded video stream. The cloud servers spend lots of CPU cycles to decode the conventional video streams (i.e., compressed video using AVC or HEVC encoding), extract the video frames at fixed frame intervals, then transcode these frames into a timelapse video. Meanwhile, even with a timelapse video, users have difficulty finding important details captured by the internet-connected/cloud-enabled cameras. Since a timelapse video always uses a fixed frame rate that does not use all the video data originally captured by the internet-connected/cloud-enabled, video frames at a normal display speed are not available for the entire duration of time that something of interest to the user is in the original video captured. For example, when a security camera observes a potential event of interest, such as a person on the premises, the timelapse will be the same as when the security camera observes nothing of particular interest.
Conventional internet-connected/cloud-enabled cameras upload full video content to the cloud for easy access from a mobile phone. The video can be preserved for security purposes (i.e., used as evidence) or the cloud service provider might not delete uploaded content. Videos uploaded can remain available indefinitely. The uploaded video content can be a privacy concern. Uploaded video content can include video clips of family members. Uploaded video content can indicate behavioral patterns of people. Even if a user desired to upload the video when it was uploaded, the user might later regret having the video available. The privacy concern can make end-users uncomfortable or the video content could be used by hackers.
It would be desirable to implement a person-of-interest centric timelapse video with AI input on home security camera to protect privacy.
SUMMARY
The invention concerns an apparatus including an interface and a processor. The interface may be configured to receive pixel data generated by a capture device. The processor may be configured to generate video frames in response to the pixel data, perform computer vision operations on the video frames to detect objects, perform a classification of the objects detected based on characteristics of the objects, determine whether the classification of the objects corresponds to a user-defined event and a user-defined identity and generate encoded video frames from the video frames. The encoded video frames may comprise a first sample of the video frames selected at a first rate when the user-defined event is not detected and a second sample of the video frames selected at a second rate while the user-defined event is detected. The video frames comprising the user-defined identity without a second person may be excluded from the encoded video frames.
BRIEF DESCRIPTION OF THE FIGURES
Embodiments of the invention will be apparent from the following detailed description and the appended claims and drawings.
FIG. 1 is a diagram illustrating an example context of the present invention.
FIG. 2 is a diagram illustrating example internet-connected cameras implementing an example embodiment of the present invention.
FIG. 3 is a block diagram illustrating components of an apparatus configured to provide an event centric timelapse video with the assistance of a neural network.
FIG. 4 is a diagram illustrating an interconnected camera communicating with a cloud server and a video processing pipeline for generating a timelapse video.
FIG. 5 is a diagram illustrating a smart timelapse mode on an edge AI camera with CV analysis excluding privacy event video frames.
FIG. 6 is a diagram illustrating a smart timelapse mode on an edge AI camera with CV analysis including mixed event video frames.
FIG. 7 is a diagram illustrating event detection in a captured video frame.
FIG. 8 is a diagram illustrating an encoded video frame of a smart timelapse video stream with a privacy effect applied.
FIG. 9 is a diagram illustrating an application operating on a smart phone for controlling preferences for a timelapse video.
FIG. 10 is a diagram illustrating an application operating on a smart phone for controlling privacy settings for a timelapse video.
FIG. 11 is a flow diagram illustrating a method for implementing a person-of-interest centric timelapse video with AI input on home security camera to protect privacy.
FIG. 12 is a flow diagram illustrating a method for adding a distortion effect to protect privacy of a pre-defined identity.
FIG. 13 is a flow diagram illustrating a method for updating feature set information for identifying particular people using a local network connection.
FIG. 14 is a flow diagram illustrating a method for identifying a person to determine whether a privacy event has been detected.
DETAILED DESCRIPTION OF THE EMBODIMENTS
Embodiments of the present invention include providing a person-of-interest centric timelapse video with AI input on home security camera to protect privacy that may (i) implement event detection on an edge device, (ii) implement video encoding on an edge device, (iii) generate smart timelapse videos with varying framerates based on events/objects detected, (iv) detect objects/events using a convolutional neural network implemented on a processor, (v) adjust a timelapse framerate to capture all video frames when an event/object is detected, (vi) perform facial recognition and/or object classification local to the camera device, (vii) upload an encoded smart timelapse video to a cloud storage server, (viii) enable configuration of parameters for event/object detection and privacy concerns, (ix) exclude video frames that may be a privacy concern, (x) blur or otherwise conceal faces in an encoded video frame to preserve privacy and/or (xi) be implemented as one or more integrated circuits.
Embodiments of the present invention may be configured to generate a smart timelapse video. The smart timelapse video may be implemented by adjusting a video display speed automatically based on content detected in the captured video. The video display speed may be adjusted in response to a selection rate of video frames from the originally captured video to be used for the smart timelapse video. Embodiments of the present invention may be configured to generate encoded smart timelapse videos. The smart timelapse video may be generated on a cloud service (e.g., using scalable computing). The smart timelapse video may be generated on an edge device (e.g., an artificial intelligence (AI) camera). Generating the smart timelapse video on the edge device may comprise only using processing capabilities within the edge device (e.g., without outsourcing processing to an external device). In an example, the edge device may be an internet-connected and/or cloud-enabled camera that may implement a home security camera.
Embodiments of the present invention may be configured to protect the privacy of particular people when the smart timelapse video is generated. The video content (e.g., what appears in the smart timelapse video) may be automatically adjusted in response to the objects/events detected. Particular objects/events may be shown as captured in the smart timelapse video and other objects/events may be excluded and/or removed from the smart timelapse video stream. For example, the smart timelapse video stream may be generated to include the faces and behaviors of strangers but exclude the faces and behaviors of family members. Generally, the faces and/or behaviors excluded from the smart timelapse video stream may correspond to privacy concerns (e.g., identifying particular people, a person being uncomfortable being on video, preventing the storage of potentially embarrassing behaviors, etc.). The criteria for including or excluding video content may be varied according to the design criteria of a particular implementation.
The edge AI home security device/camera may be configured to implement artificial intelligence (AI) technology. Using AI technology, the edge AI camera may be a more powerful (e.g., by providing relevant data for the user) and a more power efficient solution than using a cloud server in many aspects. An edge AI camera may be configured to execute computer readable instructions locally (e.g., internally) on the device (e.g., without relying on external processing resources) to analyze the video content frame by frame. Based on the analysis, content in the video frames may be tagged with metadata information. The metadata information may be used to select video frames for a smart timelapse video. For example, the video frames may be classified by being tagged as having no interesting object/event and as having an interesting object/event. Computer vision (CV) operations may determine whether there is an interesting object/event (e.g., based on a pre-defined feature set).
When there is no interesting CV event (or type of object) in the video frames for a duration of N seconds (N=60/120/190/ . . . ), the edge AI camera may select one of the video frames captured during the no-event duration. The selected no-event video frame may be used for video encoding (e.g., a video encoder built in the edge AI camera device that performs video encoding internally). Selecting one video frame from the no-event duration N for encoding may be repeated for each N second duration with no event detected. Selecting one video frame from the no-event duration N for encoding may result in an encoded output that provides a condensed portion of the captured video (e.g., to effectively fast-forward through âmeaningless contentâ portions of video captured with a high display speed).
When there is an interesting CV event (or type of object) detected in the video frames for a duration of M seconds (M=5/15/30/ . . . ), the edge AI camera may adjust how many and/or the rate of selection of the CV event video frames for the event duration M. The event detected may be defined based on a pre-determined feature set. In an example, the object and/or event of interest may be considered to be detected when a person is detected, a car is detected, an animal detected, an amount of motion is detected, a particular face detected, etc. The selection of the video frames for the smart timelapse video may be adjusted to select all of the video frames from the M second duration of the CV event (e.g., for a 2 minute event captured at 60 frames per second, the entire 7200 frames may be selected). The selection of the CV event video frames may be adjusted to select video frames at a higher rate than for the no-event duration (e.g., select more frames, but not all frames) for the M second duration of the event (e.g., for a 2 minute event captured at 60 frames per second, the rate of selection may be changed to 30 frames and every other frame may be selecting resulting in 3600 video frames being selected).
The selected CV event video frames may be encoded (e.g., using the on-device video encoding of the AI edge camera to perform encoding locally). Selecting video frames at a higher rate for the event duration M for encoding may result in an encoded output that provides either a portion of the smart timelapse video with the normal display speed for the âmeaningful contentâ or a portion of the smart timelapse video with a slightly condensed (but not as high speed as the âmeaningless contentâ) display speed for the âmeaningful contentâ.
With the smart timelapse video generation implemented on an edge camera, the user may quickly browse through the captured video content for a long period of time (e.g., days/weeks/months), and the user may be confident that no interesting CV event will be missed. An interesting CV event may comprise detecting a person (e.g., a known person) by face detection and/or face recognition, detecting a car license plate (e.g., using a license plate reader), detecting a pet (e.g., a known animal) using animal/pet recognition, detecting motion (e.g., any motion detected above a pre-defined threshold), etc. The type of event and/or object detected that may be considered an interesting event may be varied according to the design criteria of a particular implementation.
Embodiments of the present invention may enable a user to specify the type of object/event that may be considered to be interesting. In one example, an app operating on a smartphone (e.g., a companion app for the edge AI camera) may be configured to adjust settings for the edge AI camera. In another example, the edge AI camera may be configured to provide a web interface (e.g., over a local area network) to enable a user to remotely select objects/events to be considered interesting events. In yet another example, the edge AI camera may connect to a cloud server, and the user may use a web interface to adjust settings stored on the cloud server that may then be sent to the edge AI camera to control the smart timelapse object type(s) of interest.
The duration of the smart timelapse video may be configured as regular intervals and/or since the last time the user has received a timelapse video. For example, if the user has missed 20 event notifications, the moment the users interacts with the app (e.g., swipes on a smartphone) to view the events, the event-focused timelapse may be presented to the user for viewing. While embodiments of the present invention may generally be performed local to an edge AI camera (e.g., implementing a processor configured to implement a convolutional neural network and/or video encoding locally), embodiments of the present invention may be performed using software on the cloud server to achieve similar effects.
Embodiments of the present invention may be further configured to account for potential privacy issues based on the objects/events detected. For example, the CV event detected may comprise detecting people (e.g., faces of strangers, faces of known people, faces of family members, combinations of strangers and known people, etc.). When there is an interesting CV event (or type of object) detected for the video frames for a duration of M seconds (M=5/15/30/ . . . ), the CV event may be also be a privacy event of P duration. The edge AI camera may be configured to determine whether the CV event is a CV event without a privacy event or a CV event with a privacy event. For example, a privacy event may comprise the detection of a face of a known person (e.g., a face that matches a pre-defined or user-defined face input). In one example, if the face corresponds to a privacy event, the edge AI camera may be configured to draw a privacy mask, apply a blur effect and/or apply another type of distortion effect to the face and record metadata (e.g., apply a tag to the video frames). In another example, if the face corresponds to a privacy event, the edge AI camera may be configured to discard the video frames (e.g., not select the video frames with the privacy event for encoding in the smart timelapse video stream). If the face corresponds to a stranger (e.g., does not match the pre-defined or user-defined face input) the video frame may be selected for encoding (e.g., to be part of the smart timelapse video).
The edge AI camera may be configured to record the full video content (e.g., the input or raw video stream). The full video content may be recorded to a local storage (e.g., eMMC, a microSD card, etc.). The edge AI camera may perform fast face-detection and/or facial recognition on the video captured. If the video frames comprise any user-defined face (e.g., any family member) a privacy mask may be drawn (e.g., a green mask, a black mask, etc. may be drawn over the face). Other distortion effects may be applied (e.g., a blur effect, a mosaic effect, a graphic and/or alternate image, etc.). If the video frames comprise the face of a stranger, then the video frame may be selected (e.g., because an event has been detected), but the face may be unchanged. If a video frame comprises both a user-defined face and a stranger face, the edge AI camera may detect where the user-defined face is in the video frame and only apply the distortion effect to the user-defined face, but not the stranger face. In some embodiments, the edge AI camera may not apply the distortion effect to either face (e.g., based on user settings/preferences). In some embodiments, when the video frames comprise the user-defined face, the video frames may not be selected for the smart time-lapse video stream (e.g., the video frames may be excluded entirely and not selected at the framerate for the non CV event video frames or the CV event video frames).
The user-defined faces may be selected by the end-user. The app may be configured to receive the user-defined faces and provide the user-defined faces to the edge AI camera. In an example, the user-defined faces may be provided over a local network (e.g., rather than the cloud network). Submitting the user-defined faces over a local network may ensure that private information (e.g., the faces of family members, nudity, people under a particular age, particular behaviors, particular logos, etc.) are not exposed to the cloud services (e.g., never uploaded to the cloud). The duration of the timelapse may be defined as regular intervals for non CV events, stranger faces (e.g., regular CV events) and family faces (e.g., privacy events). Referring to FIG. 1 , a diagram illustrating an example context of the present invention is shown. A home 50 and a vehicle 52 are shown. Camera systems 100 a - 100 n are shown. Each of the cameras 100 a - 100 n may be configured to generate pixel data of the environment, generate video frames from the pixel data, encode the video frames and/or generate the smart timelapse videos. For example, each of the cameras 100 a - 100 n may be configured to operate independently of each other. Each of the cameras 100 a - 100 n may capture video and generate smart timelapse videos. In one example, the respective smart timelapse videos may be uploaded to a cloud storage service. In another example, the respective smart timelapse videos may be stored locally (e.g., on a microSD card, to a local network attached storage device, etc.).
Each of the cameras 100 a - 100 n may be configured to detect different or the same events/objects that may be considered interesting. For example, the camera system 100 b may capture an area near an entrance of the home 50 . For an entrance of the home 50 , objects/events of interest may be detecting people. The camera system 100 b may be configured to analyze video frames to detect people and the smart timelapse video may slow down (e.g., select video frames for encoding at a higher frame rate) when a person is detected. In another example, the camera system 100 d may capture an area near the vehicle 52 . For the vehicle 52 , objects/events of interest may be detecting other vehicles and pedestrians. The camera system 100 b may be configured to analyze video frames to detect vehicles (or road signs) and people and the smart timelapse video may slow down when a vehicle or a pedestrian is detected.
Each of the cameras 100 a - 100 n may operate independently from each other. For example, each of the cameras 100 a - 100 n may individually analyze the pixel data captured and perform the event/object detection locally. In some embodiments, the cameras 100 a - 100 n may be configured as a network of cameras (e.g., security cameras that send video data to a central source such as network-attached storage and/or a cloud service). The locations and/or configurations of the cameras 100 a - 100 n may be varied according to the design criteria of a particular implementation.
Referring to FIG. 2 , a diagram illustrating example internet-connected cameras implementing an example embodiment of the present invention is shown. Camera systems 100 a - 100 n are shown. Each camera device 100 a - 100 n may have a different style and/or use case. For example, the camera 100 a may be an action camera, the camera 100 b may be a ceiling mounted security camera, the camera 100 n may be webcam, etc. Other types of cameras may be implemented (e.g., home security cameras, battery powered cameras, doorbell cameras, stereo cameras, etc.). The design/style of the cameras 100 a - 100 n may be varied according to the design criteria of a particular implementation.
Each of the camera systems 100 a - 100 n may comprise a block (or circuit) 102 and/or a block (or circuit) 104 . The circuit 102 may implement a processor. The circuit 104 may implement a capture device. The camera systems 100 a - 100 n may comprise other components (not shown). Details of the components of the cameras 100 a - 100 n may be described in association with FIG. 3 .
The processor 102 may be configured to implement a convolutional neural network (CNN). The processor 102 may be configured to implement a video encoder. The processor 102 may generate the smart timelapse videos. The capture device 104 may be configured to capture pixel data that may be used by the processor 102 to generate video frames.
The cameras 100 a - 100 n may be edge devices. The processor 102 implemented by each of the cameras 100 a - 100 n may enable the cameras 100 a - 100 n to implement various functionality internally (e.g., at a local level). For example, the processor 102 may be configured to perform object/event detection (e.g., computer vision operations), video encoding and/or video transcoding on-device. For example, even advanced processes such as computer vision may be performed by the processor 102 without uploading video data to a cloud service in order to offload computation-heavy functions (e.g., computer vision, video encoding, video transcoding, etc.).
Referring to FIG. 3 , a block diagram illustrating components of an apparatus configured to provide an event centric timelapse video with the assistance of a neural network is shown. A block diagram of the camera system 100 i is shown. The camera system 100 i may be a representative example of the camera system 100 a - 100 n shown in association with FIGS. 1 - 2 . The camera system 100 i generally comprises the processor 102 , the capture devices 104 a - 104 n , blocks (or circuits) 150 a - 150 n , a block (or circuit) 152 , blocks (or circuits) 154 a - 154 n , a block (or circuit) 156 , blocks (or circuits) 158 a - 158 n , a block (or circuit) 160 and/or a block (or circuit) 162 . The blocks 150 a - 150 n may implement lenses. The circuit 152 may implement sensors. The circuits 154 a - 154 n may implement microphones (e.g., audio capture devices). The circuit 156 may implement a communication device. The circuits 158 a - 158 n may implement audio output devices (e.g., speakers). The circuit 160 may implement a memory. The circuit 162 may implement a power supply (e.g., a battery). The camera system 100 i may comprise other components (not shown). In the example shown, some of the components 150 - 158 are shown external to the camera system 100 i . However, the components 150 - 158 may be implemented within and/or attached to the camera system 100 i (e.g., the speakers 158 a - 158 n may provide better functionality if not located inside a housing of the camera system 100 i ). The number, type and/or arrangement of the components of the camera system 100 i may be varied according to the design criteria of a particular implementation.
In an example implementation, the processor 102 may be implemented as a video processor. The processor 102 may comprise inputs 170 a - 170 n and/or other inputs. The processor 102 may comprise an input/ output 172 . The processor 102 may comprise an input 174 and an input 176 . The processor 102 may comprise an output 178 . The processor 102 may comprise an output 180 a and an input 180 b . The number of inputs, outputs and/or bi-directional ports implemented by the processor 102 may be varied according to the design criteria of a particular implementation.
In the embodiment shown, the capture devices 104 a - 104 n may be components of the camera system 100 i . In some embodiments, the capture devices 104 a - 104 n may be separate devices (e.g., remotely connected to the camera system 100 i , such as a drone, a robot and/or a system of security cameras configured capture video data) configured to send data to the camera system 100 i . In one example, the capture devices 104 a - 104 n may be implemented as part of an autonomous robot configured to patrol particular paths such as hallways. Similarly, in the example shown, the sensors 152 , the microphones 154 a - 154 n , the wireless communication device 156 , and/or the speakers 158 a - 158 n are shown external to the camera system 100 i but in some embodiments may be a component of (e.g., within) the camera system 100 i.
The camera system 100 i may receive one or more signals (e.g., IMF_A-IMF_N), a signal (e.g., SEN), a signal (e.g., FEAT_SET) and/or one or more signals (e.g., DIR_AUD). The camera system 100 i may present a signal (e.g., ENC_VIDEO) and/or a signal (e.g., DIR_AOUT). The capture devices 104 a - 104 n may receive the signals IMF_A-IMF_N from the corresponding lenses 150 a - 150 n . The processor 102 may receive the signal SEN from the sensors 152 . The processor 102 may receive the signal DIR_AUD from the microphones 154 a - 154 n . The processor 102 may present the signal ENC_VIDEO to the communication device 156 and receive the signal FEAT_SET from the communication device 156 . For example, the wireless communication device 156 may be a radio-frequency (RF) transmitter. In another example, the communication device 156 may be a Wi-Fi module. In another example, the communication device 156 may be a device capable of implementing RF transmission, Wi-Fi, Bluetooth and/or other wireless communication protocols. In some embodiments, the signal ENC_VIDEO may be presented to a display device connected to the camera 100 i . The processor 102 may present the signal DIR_AOUT to the speakers 158 a - 158 n.
The lenses 150 a - 150 n may capture signals (e.g., IM_A-IM_N). The signals IM_A-IM_N may be an image (e.g., an analog image) of the environment near the camera system 100 i presented by the lenses 150 a - 150 n to the capture devices 104 a - 104 n as the signals IMF_A-IMF_N. The lenses 150 a - 150 n may be implemented as an optical lens. The lenses 150 a - 150 n may provide a zooming feature and/or a focusing feature. The capture devices 104 a - 104 n and/or the lenses 150 a - 150 n may be implemented, in one example, as a single lens assembly. In another example, the lenses 150 a - 150 n may be a separate implementation from the capture devices 104 a - 104 n . The capture devices 104 a - 104 n are shown within the circuit 100 i . In an example implementation, the capture devices 104 a - 104 n may be implemented outside of the circuit 100 i (e.g., along with the lenses 150 a - 150 n as part of a lens/capture device assembly).
In some embodiments, two or more of the lenses 150 a - 150 n may be configured as a stereo pair of lenses. For example, the camera 100 i may implement stereo vision. The lenses 150 a - 150 n implemented as a stereo pair may be implemented at a pre-determined distance apart from each other and at a pre-determined inward angle. The pre-determined distance and/or the pre-determined inward angle may be used by the processor 102 to build disparity maps for stereo vision.
The capture devices 104 a - 104 n may be configured to capture image data for video (e.g., the signals IMF_A-IMF_N from the lenses 150 a - 150 n ). In some embodiments, the capture devices 104 a - 104 n may be video capturing devices such as cameras. The capture devices 104 a - 104 n may capture data received through the lenses 150 a - 150 n to generate raw pixel data. In some embodiments, the capture devices 104 a - 104 n may capture data received through the lenses 150 a - 150 n to generate bitstreams (e.g., generate video frames). For example, the capture devices 104 a - 104 n may receive focused light from the lenses 150 a - 150 n . The lenses 150 a - 150 n may be directed, tilted, panned, zoomed and/or rotated to provide a targeted view from the camera system 100 i (e.g., a view for a video frame, a view for a panoramic video frame captured using multiple capture devices 104 a - 104 n , a target image and reference image view for stereo vision, etc.). The capture devices 104 a - 104 n may generate signals (e.g., PIXELD_A-PIXELD_N). The signals PIXELD_A-PIXELD_N may be pixel data (e.g., a sequence of pixels that may be used to generate video frames). In some embodiments, the signals PIXELD_A-PIXELD_N may be video data (e.g., a sequence of video frames). The signals PIXELD_A-PIXELD_N may be presented to the inputs 170 a - 170 n of the processor 102 .
The capture devices 104 a - 104 n may transform the received focused light signals IMF_A-IMF_N into digital data (e.g., bitstreams). In some embodiments, the capture devices 104 a - 104 n may perform an analog to digital conversion. For example, the capture devices 104 a - 104 n may perform a photoelectric conversion of the focused light received by the lenses 150 a - 150 n . The capture devices 104 a - 104 n may transform the bitstreams into pixel data, images and/or video frames. In some embodiments, the pixel data generated by the capture devices 104 a - 104 n may be uncompressed and/or raw data generated in response to the focused light from the lenses 150 a - 150 n . In some embodiments, the output of the capture devices 104 a - 104 n may be digital video signals.
The sensors 152 may comprise one or more input devices. The sensors 152 may be configured to detect physical input from the environment and convert the physical input into computer readable signals. The signal SEN may comprise the computer readable signals generated by the sensors 152 . In an example, one of the sensors 152 may be configured to detect an amount of light and present a computer readable signal representing the amount of light detected. In another example, one of the sensors 152 may be configured to detect motion and present a computer readable signal representing the amount of motion detected. The sensors 152 may be configured to detect temperature (e.g., a thermometer), orientation (e.g., a gyroscope), a movement speed (e.g., an accelerometer), etc. The types of input detected by the sensors 152 may be varied according to the design criteria of a particular implementation.
The data provided in the signal SEN provided by the sensors 152 may be read and/or interpreted by the processor 102 . The processor 102 may use the data provided by the signal SEN for various operations. In some embodiments, the processor 102 may use a light reading from the sensors 152 to determine whether to activate an infrared light (e.g., to provide night vision). In another example, the processor 102 may use information about movement from an accelerometer and/or a gyroscope to perform motion correction on video frames generated. The types of operations performed by the processor 102 in response to the signal SEN may be varied according to the design criteria of a particular implementation.
The communication device 156 may send and/or receive data to/from the camera system 100 i . In some embodiments, the communication device 156 may be implemented as a wireless communications module. In some embodiments, the communication device 156 may be implemented as a satellite connection to a proprietary system. In one example, the communication device 156 may be a hard-wired data port (e.g., a USB port, a mini-USB port, a USB-C connector, HDMI port, an Ethernet port, a DisplayPort interface, a Lightning port, etc.). In another example, the communication device 156 may be a wireless data interface (e.g., Wi-Fi, Bluetooth, ZigBee, cellular, etc.).
The communication device 156 may be configured to receive the signal FEAT_SET. The signal FEAT_SET may comprise a feature set. The feature set received may be used to detect events and/or objects. For example, the feature set may be used to perform the computer vision operations. The feature set information may comprise instructions for the processor 102 for determining which types of objects correspond to an object and/or event of interest.
The processor 102 may receive the signals PIXELD_A-PIXELD_N from the capture devices 104 a - 104 n at the inputs 170 a - 170 n . The processor 102 may send/receive a signal (e.g., DATA) to/from the memory 160 at the input/ output 172 . The processor 102 may receive the signal SEN from the sensors 152 at the input port 174 . The processor 102 may receive the signal DIR_AUD from the microphones 154 a - 154 n at the port 176 . The <figure-callout id="102" label="processor" filenames="US11869241-20240109-D00002.png,US1
CLAIMS
Claims ( 19 )
The invention claimed is:
1. An apparatus comprising:
an interface configured to receive pixel data; and
a processor configured to (i) receive said pixel data from said interface, (ii) process said pixel data arranged as video frames, (iii) perform computer vision operations on said video frames to detect objects, (iv) perform a classification of said objects detected based on characteristics of said objects, (v) determine whether said classification of said objects corresponds to (a) a user-defined event and (b) a user-defined identity, and (vi) generate encoded video frames from said video frames, wherein
(a) said encoded video frames comprise (i) a first sample of said video frames selected at a first rate when said user-defined event is not detected and (ii) a second sample of said video frames selected at a second rate while said user-defined event is detected,
(b) said second rate is greater than said first rate, and
(c) a distortion effect is applied to said second sample of said video frames in a region of said second sample of said video frames that comprises said user-defined identity when an additional one of said objects is detected in said video frames that does not correspond to said user-defined identity.
2. The apparatus according to claim 1 , wherein said second rate is the same as a frame rate of said video frames.
3. The apparatus according to claim 1 , wherein said second rate is less than a frame rate of said video frames.
4. The apparatus according to claim 1 , wherein said distortion effect comprises at least one of a blur, a mask and an alternate graphic.
5. The apparatus according to claim 1 , wherein (i) said apparatus is implemented on an edge device comprising a capture device and (ii) said edge device communicates with a cloud storage service.
6. The apparatus according to claim 5 , wherein said edge device is an internet connected camera.
7. The apparatus according to claim 5 , wherein applying said distortion effect to said region of said second sample of said video frames that comprises said user-defined identity before generating said encoded video frames prevents said user-defined identity from being visible in said encoded video frames uploaded to said cloud storage service.
8. The apparatus according to claim 5 , wherein uploading said encoded video frames to said cloud storage service instead of uploading all of said video frames generated (a) reduces an amount of bandwidth used for communication between said apparatus and said cloud storage service, (b) reduces an amount of storage capacity used by said cloud storage service and (c) reduces an amount of CPU cycles used by said cloud storage service for transcoding video.
9. The apparatus according to claim 5 , wherein a user accesses said encoded video frames stored on said cloud storage service using a smartphone app.
10. The apparatus according to claim 1 , wherein said video frames selected for said encoded video frames comprise any of an I-frame, a B-frame, or a P-frame.
11. The apparatus according to claim 1 , wherein said encoded video frames provide a smart timelapse video stream comprising a video display speed that is automatically adjusted by said processor in response to content in said video frames.
12. The apparatus according to claim 1 , wherein said user-defined event comprises said classification of said objects using said computer vision operations as at least one a person, and a license plate, an animal.
13. The apparatus according to claim 1 , wherein said classification is performed in response to said processor locally analyzing content in said video frames and tagging said content in said video frames with metadata information.
14. The apparatus according to claim 1 , wherein (i) said user-defined identity comprises a face of a person and (ii) applying said distortion effect to said region of said second sample of said video frames that comprises said user-defined identity provides privacy protection of said person.
15. The apparatus according to claim 1 , wherein said first sample of said video frames and said second sample of said video frames are selected with a response of neural network input implemented in said processor.
16. The apparatus according to claim 1 , wherein said computer vision operations are configured to detect said objects by performing feature extraction based on neural network weight values for each of a plurality of visual features that are associated with said objects extracted from said video frames and (d) said neural network weight values are determined in response to an analysis of training data by said processor prior to said feature extraction.
17. The apparatus according to claim 1 , wherein (i) said user-defined event is detected using said computer vision operations in response to first feature set data received from a cloud service and (ii) said user-defined identity is detected in response to second feature set data received from a local network.
18. A method for generating a timelapse video stream, comprising the steps of:
receiving pixel data generated by a capture device;
processing said pixel data arranged as video frames using a processor;
performing computer vision operations on said video frames to detect objects;
performing a classification of said objects detected based on characteristics of said objects;
determining whether said classification of said objects corresponds to (a) a user-defined event and (b) a user-defined identity; and
generating encoded video frames from said video frames, wherein
(a) said encoded video frames comprise (i) a first sample of said video frames selected at a first rate when said user-defined event is not detected and (ii) a second sample of said video frames selected at a second rate while said user-defined event is detected,
(b) said second rate is greater than said first rate and
(c) said video frames comprising said user-defined identity without a second person are excluded from said encoded video frames.
19. The apparatus according to claim 1 , wherein said processor (i) generates said first sample of said video frames selected at said first rate, (ii) increases a frame rate of said encoded video frames to said second rate to generate said second sample of said video frames while said user-defined event is detected and (iii) decreases said frame rate of said encoded video frames to said first rate to generate said first sample of said video frames after said user-defined event is no longer detected.
US17/976,093
2020-08-20
2022-10-28
Person-of-interest centric timelapse video with AI input on home security camera to protect privacy
Active
US11869241B2
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US17/976,093
US11869241B2
( en )
2020-08-20
2022-10-28
Person-of-interest centric timelapse video with AI input on home security camera to protect privacy
Applications Claiming Priority (4)
Application Number
Priority Date
Filing Date
Title
CN202010845108.6A
CN114079750A
( en )
2020-08-20
2020-08-20
Capturing video at intervals of interest person-centric using AI input on a residential security camera to protect privacy
CN202010845108.6
2020-08-20
US17/130,442
US11551449B2
( en )
2020-08-20
2020-12-22
Person-of-interest centric timelapse video with AI input on home security camera to protect privacy
US17/976,093
US11869241B2
( en )
2020-08-20
2022-10-28
Person-of-interest centric timelapse video with AI input on home security camera to protect privacy
Related Parent Applications (1)
Application Number
Title
Priority Date
Filing Date
US17/130,442
Continuation
US11551449B2
( en )
2020-08-20
2020-12-22
Person-of-interest centric timelapse video with AI input on home security camera to protect privacy
Publications (2)
Publication Number
Publication Date
US20230046676A1
US20230046676A1 ( en )
2023-02-16
US11869241B2
true
US11869241B2 ( en )
2024-01-09
Family
ID=80270836
Family Applications (2)
Application Number
Title
Priority Date
Filing Date
US17/130,442
Active
2041-08-04
US11551449B2
( en )
2020-08-20
2020-12-22
Person-of-interest centric timelapse video with AI input on home security camera to protect privacy
US17/976,093
Active
US11869241B2
( en )
2020-08-20
2022-10-28
Person-of-interest centric timelapse video with AI input on home security camera to protect privacy
Family Applications Before (1)
Application Number
Title
Priority Date
Filing Date
US17/130,442
Active
2041-08-04
US11551449B2
( en )
2020-08-20
2020-12-22
Person-of-interest centric timelapse video with AI input on home security camera to protect privacy
Country Status (2)
Country
Link
US
( 2 )
US11551449B2
( en )
CN
( 1 )
CN114079750A
( en )
Families Citing this family (13)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20220164569A1
( en )
*
2020-11-26
2022-05-26
POSTECH Research and Business Development Foundation
Action recognition method and apparatus based on spatio-temporal self-attention
US11894019B2
( en )
*
2020-12-30
2024-02-06
Linearity Gmbh
Time-lapse
US11468676B2
( en )
*
2021-01-08
2022-10-11
University Of Central Florida Research Foundation, Inc.
Methods of real-time spatio-temporal activity detection and categorization from untrimmed video segments
US20220237316A1
( en )
*
2021-01-28
2022-07-28
Capital One Services, Llc
Methods and systems for image selection and push notification
US20220301403A1
( en )
*
2021-03-16
2022-09-22
Motorola Solutions, Inc.
Clustering and active learning for teach-by-example
GB2621305B
( en )
*
2022-06-09
2024-10-09
Sony Interactive Entertainment Inc
Data processing apparatus and method
CN115278004B
( en )
*
2022-07-06
2023-10-31
æå·æµ·åº·æ±½è½¦è½¯ä»¶æéå ¬å¸
Method, device, equipment and storage medium for transmitting monitoring video data
CN116389811B
( en )
*
2023-03-10
2025-07-15
ä¸èå¸ä¹é¼å®ä¸æéå ¬å¸
Synchronous control method and system for distributed video image stitching
CN116366919B
( en )
*
2023-04-13
2025-12-12
æ å·å¸å¾·èµè¥¿å¨æ±½è½¦çµåè¡ä»½æéå ¬å¸
Video processing methods, apparatus, equipment and storage media
IT202300008532A1
( en )
*
2023-05-02
2024-11-02
Vlab S R L
Method for generating a time-lapse video and associated generating device
WO2025058548A1
( en )
*
2023-09-15
2025-03-20
Telefonaktiebolaget Lm Ericsson (Publ)
Controlling how privacy enhancing technology is applied
US20250111638A1
( en )
*
2023-09-28
2025-04-03
Ati Technologies Ulc
Imaging privacy filter for objects of interest in hardware firmware platform
CN118509542B
( en )
*
2024-07-18
2024-11-29
åå¨çç§æ(常å·)æéå ¬å¸
Video generation method, device, computer equipment and storage medium
Citations (13)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20160004390A1
( en )
*
2014-07-07
2016-01-07
Google Inc.
Method and System for Generating a Smart Time-Lapse Video Clip
US20180068540A1
( en )
*
2015-05-12
2018-03-08
Apical Ltd
Image processing method
US20180268240A1
( en )
*
2017-03-20
2018-09-20
Conduent Business Services, Llc
Video redaction method and system
US20180330591A1
( en )
*
2015-11-18
2018-11-15
Jörg Tilkin
Protection of privacy in video monitoring systems
US20190130188A1
( en )
*
2017-10-26
2019-05-02
Qualcomm Incorporated
Object classification in a video analytics system
US20190266346A1
( en )
*
2018-02-28
2019-08-29
Walmart Apollo, Llc
System and method for privacy protection of sensitive information from autonomous vehicle sensors
US20200082851A1
( en )
*
2018-09-11
2020-03-12
Avigilon Corporation
Bounding box doubling as redaction boundary
US10681313B1
( en )
*
2018-09-28
2020-06-09
Ambarella International Lp
Home monitoring camera featuring intelligent personal audio assistant, smart zoom and face recognition features
US11140292B1
( en )
*
2019-09-30
2021-10-05
Gopro, Inc.
Image capture device for generating time-lapse videos
US20210342479A1
( en )
*
2020-04-29
2021-11-04
Cobalt Robotics Inc.
Privacy protection in mobile robot
US11282367B1
( en )
*
2020-08-16
2022-03-22
Vuetech Health Innovations LLC
System and methods for safety, security, and well-being of individuals
US11427195B1
( en )
*
2020-02-07
2022-08-30
Ambarella International Lp
Automatic collision detection, warning, avoidance and prevention in parked cars
US20220292827A1
( en )
*
2021-03-09
2022-09-15
The Research Foundation For The State University Of New York
Interactive video surveillance as an edge service using unsupervised feature queries
Family Cites Families (3)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US9953187B2
( en )
*
2014-11-25
2018-04-24
Honeywell International Inc.
System and method of contextual adjustment of video fidelity to protect privacy
EP3343561B1
( en )
*
2016-12-29
2020-06-24
Axis AB
Method and system for playing back recorded video
US10083360B1
( en )
*
2017-05-10
2018-09-25
Vivint, Inc.
Variable rate time-lapse with saliency
2020
2020-08-20
CN
CN202010845108.6A
patent/CN114079750A/en
active
Pending
2020-12-22
US
US17/130,442
patent/US11551449B2/en
active
Active
2022
2022-10-28
US
US17/976,093
patent/US11869241B2/en
active
Active
Patent Citations (13)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20160004390A1
( en )
*
2014-07-07
2016-01-07
Google Inc.
Method and System for Generating a Smart Time-Lapse Video Clip
US20180068540A1
( en )
*
2015-05-12
2018-03-08
Apical Ltd
Image processing method
US20180330591A1
( en )
*
2015-11-18
2018-11-15
Jörg Tilkin
Protection of privacy in video monitoring systems
US20180268240A1
( en )
*
2017-03-20
2018-09-20
Conduent Business Services, Llc
Video redaction method and system
US20190130188A1
( en )
*
2017-10-26
2019-05-02
Qualcomm Incorporated
Object classification in a video analytics system
US20190266346A1
( en )
*
2018-02-28
2019-08-29
Walmart Apollo, Llc
System and method for privacy protection of sensitive information from autonomous vehicle sensors
US20200082851A1
( en )
*
2018-09-11
2020-03-12
Avigilon Corporation
Bounding box doubling as redaction boundary
US10681313B1
( en )
*
2018-09-28
2020-06-09
Ambarella International Lp
Home monitoring camera featuring intelligent personal audio assistant, smart zoom and face recognition features
US11140292B1
( en )
*
2019-09-30
2021-10-05
Gopro, Inc.
Image capture device for generating time-lapse videos
US11427195B1
( en )
*
2020-02-07
2022-08-30
Ambarella International Lp
Automatic collision detection, warning, avoidance and prevention in parked cars
US20210342479A1
( en )
*
2020-04-29
2021-11-04
Cobalt Robotics Inc.
Privacy protection in mobile robot
US11282367B1
( en )
*
2020-08-16
2022-03-22
Vuetech Health Innovations LLC
System and methods for safety, security, and well-being of individuals
US20220292827A1
( en )
*
2021-03-09
2022-09-15
The Research Foundation For The State University Of New York
Interactive video surveillance as an edge service using unsupervised feature queries
Also Published As
Publication number
Publication date
CN114079750A
( en )
2022-02-22
US20230046676A1
( en )
2023-02-16
US11551449B2
( en )
2023-01-10
US20220058394A1
( en )
2022-02-24
Similar Documents
Publication
Publication Date
Title
US11594254B2
( en )
2023-02-28
Event/object-of-interest centric timelapse video generation on camera device with the assistance of neural network input
US11551449B2
( en )
2023-01-10
Person-of-interest centric timelapse video with AI input on home security camera to protect privacy
US12069251B2
( en )
2024-08-20
Smart timelapse video to conserve bandwidth by reducing bit rate of video on a camera device with the assistance of neural network input
TWI897954B
( en )
2025-09-21
Maintaining fixed sizes for target objects in frames
US11704908B1
( en )
2023-07-18
Computer vision enabled smart snooze home security cameras
US11798340B1
( en )
2023-10-24
Sensor for access control reader for anti-tailgating applications
CN113920010B
( en )
2025-07-15
Method and device for realizing super-resolution of image frames
US12073581B2
( en )
2024-08-27
Adaptive face depth image generation
US12400337B2
( en )
2025-08-26
Automatic exposure metering for regions of interest that tracks moving subjects using artificial intelligence
US11922697B1
( en )
2024-03-05
Dynamically adjusting activation sensor parameters on security cameras using computer vision
CN116803079A
( en )
2023-09-22
Scalable decoding of video and related features
US11651456B1
( en )
2023-05-16
Rental property monitoring solution using computer vision and audio analytics to detect parties and pets while preserving renter privacy
US12020314B1
( en )
2024-06-25
Generating detection parameters for rental property monitoring solution using computer vision and audio analytics from a rental agreement
US20240312036A1
( en )
2024-09-19
Accelerating speckle image block matching using convolution techniques
US20230052553A1
( en )
2023-02-16
Adding an adaptive offset term using convolution techniques to a local adaptive binarization expression
US11924555B2
( en )
2024-03-05
Intelligent auto-exposure control for RGB-IR sensor
US11743450B1
( en )
2023-08-29
Quick RGB-IR calibration verification for a mass production process
US12452540B1
( en )
2025-10-21
Object-based auto exposure using neural network models
US11812007B2
( en )
2023-11-07
Disparity map building using guide node
US12610141B2
( en )
2026-04-21
Electronic image stabilization for large zoom ratio lens
Legal Events
Date
Code
Title
Description
2022-10-28
FEPP
Fee payment procedure
Free format text : ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY
2022-11-13
STPP
Information on status: patent application and granting procedure in general
Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION
2023-05-08
STPP
Information on status: patent application and granting procedure in general
Free format text : NON FINAL ACTION MAILED
2023-08-09
STPP
Information on status: patent application and granting procedure in general
Free format text : RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER
2023-08-28
STPP
Information on status: patent application and granting procedure in general
Free format text : NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS
2023-12-01
STPP
Information on status: patent application and granting procedure in general
Free format text : PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED
2023-12-20
STCF
Information on status: patent grant
Free format text : PATENTED CASE