Frequency-domain convolutional neural network — Intel Corporation (US11836240B2)

ABSTRACT

Abstract

In one embodiment, an apparatus comprises a memory and a processor. The memory is to store visual data associated with a visual representation captured by one or more sensors. The processor is to: obtain the visual data associated with the visual representation captured by the one or more sensors, wherein the visual data comprises uncompressed visual data or compressed visual data; process the visual data using a convolutional neural network (CNN), wherein the CNN comprises a plurality of layers, wherein the plurality of layers comprises a plurality of filters, and wherein the plurality of filters comprises one or more pixel-domain filters to perform processing associated with uncompressed data and one or more compressed-domain filters to perform processing associated with compressed data; and classify the visual data based on an output of the CNN.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a continuation (and claims the benefit under 35 U.S.C. § 120) of U.S. patent application Ser. No. 16/948,304, filed on Sep. 11, 2020, which is a continuation of U.S. patent application Ser. No. 16/024,397, filed on Jun. 29, 2018, and entitled “MULTI-DOMAIN CONVOLUTIONAL NEURAL NETWORK”, which claims the benefit of the filing date of U.S. Provisional Patent Application Ser. No. 62/611,536, filed on Dec. 28, 2017, and entitled “VISUAL FOG,” and U.S. Provisional Patent Application Ser. No. 62/691,464, filed on Jun. 28, 2018, and entitled “VISUAL FOG,” the contents of which are hereby expressly incorporated by reference.

FIELD OF THE SPECIFICATION

This disclosure relates in general to the field of visual computing, and more particularly, though not exclusively, to a multi-domain convolutional neural network (CNN).

BACKGROUND

In distributed computing systems, images and video captured by devices at the edge of a network are typically compressed and transmitted to the cloud for further processing. The images and video must subsequently be decompressed before any processing can be performed. This approach is inefficient, however, as it often wastes significant bandwidth on transmission of insignificant and/or trivial visual data, it requires full decompression as a prerequisite to performing any meaningful processing and analysis on visual data, and it fails to leverage the processing capabilities of other nodes in the network efficiently.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is best understood from the following detailed description when read with the accompanying figures. It is emphasized that, in accordance with the standard practice in the industry, various features are not necessarily drawn to scale, and are used for illustration purposes only. Where a scale is shown, explicitly or implicitly, it provides only one illustrative example. In other embodiments, the dimensions of the various features may be arbitrarily increased or reduced for clarity of discussion.

FIG. 1 illustrates an example embodiment of a visual fog system in accordance with certain embodiments.

FIGS. 2 , 3 , 4 , and 5 illustrate examples of Internet-of-Things (IoT) networks and architectures that can be used in accordance with certain embodiments.

FIGS. 6 and 7 illustrate example computer architectures that can be used in accordance with certain embodiments.

FIG. 8 illustrates an example embodiment of an architecture for visual fog nodes.

FIGS. 9 , 10 , 11 , and 12 A -B illustrate example embodiments of a visual fog architecture.

FIGS. 13 and 14 illustrate example embodiments associated with a visual question answering (VCA) framework.

FIGS. 15 and 16 illustrate example embodiments of device-centric scheduling for visual fog computing.

FIG. 17 illustrates an example embodiment of a runtime processing pipeline for a visual fog architecture.

FIG. 18 illustrates an example embodiment of a visual data storage architecture.

FIG. 19 illustrates an example of a vision processing pipeline that leverages metadata for searching visual data.

FIGS. 20 and 21 illustrate examples of representing visual metadata using a property graph.

FIG. 22 illustrates an example embodiment of an analytic image format designed to aid in visual data processing.

FIG. 23 illustrates a performance graph for various image formats.

FIGS. 24 A-C illustrate an example embodiment of a multi-domain cascade convolutional neural network (CNN).

FIGS. 25 A-B , 26 , 27 , 28 , 29 , 30 and 31 A-B illustrate the use of butterfly operations for a multi-domain convolutional neural network (CNN).

FIGS. 32 and 33 illustrate an example embodiment of a three-dimensional (3D) CNN for processing compressed visual data.

FIG. 34 illustrates an example of a pixel-domain CNN.

FIG. 35 illustrates an example of a pixel-domain visual analytics pipeline.

FIGS. 36 and 37 illustrate example embodiments of compressed-domain visual analytics pipelines.

FIG. 38 illustrates a performance graph showing the precision of a CNN trained using compressed visual data.

FIG. 39 illustrates a flowchart for an example embodiment of context-aware image compression.

FIGS. 40 A-C illustrate an example embodiment of a privacy-preserving demographic identification system.

FIGS. 41 , 42 , and 43 illustrate an example embodiment of privacy-preserving distributed visual data processing.

FIGS. 44 , 45 , and 46 illustrate example embodiments of self-sovereign device identification for distributed computing networks.

FIG. 47 illustrates an example of device onboarding/commissioning in a visual fog network without conflict resolution.

FIGS. 48 and 49 illustrate example embodiments of algorithm identification for distributed computing using a self-sovereign blockchain.

FIGS. 50 , 51 , and 52 illustrate example embodiments for processing traditional and analytic image formats.

FIG. 53 illustrates a flowchart for an example embodiment of privacy-preserving demographics identification.

FIG. 54 illustrates a flowchart for an example embodiment of privacy-preserving distributed visual processing.

FIG. 55 illustrates an example use case for an automotive ubiquitous witness.

FIG. 56 illustrates an example dataflow for a ubiquitous witness.

FIG. 57 illustrates an example use case for automotive anomaly detection and reconstruction.

FIG. 58 illustrates an example process flow for IP-based anomaly detection and reconstruction.

FIG. 59 illustrates an example process flow for ICN-based anomaly detection and reconstruction.

FIG. 60 illustrates an example process flow of an individual node for ICN-based anomaly detection and reconstruction.

FIG. 61 illustrates a flowchart for an example embodiment of a ubiquitous visual computing witness.

FIG. 62 illustrates an example embodiment of a converged node router.

FIG. 63 illustrates an example network topology using converged node routers.

FIG. 64 illustrates an example embodiment of an image storage system that leverages context-aware image compression.

FIG. 65 illustrates an example of inter-frame encoding for context-aware image compression.

FIGS. 66 , 67 , and 68 illustrate examples embodiments for performing automated semantic inference of visual objects using smart cameras.

FIGS. 69 and 70 illustrate example embodiments associated with visual fog stream multiplexing for improved security.

FIG. 71 illustrates an example embodiment of a privacy-preserving VQA dataflow.

FIGS. 72 and 73 illustrate example sanitization techniques for a privacy-preserving VQA dataflow.

FIGS. 74 and 75 illustrate pseudocode for a recursive quadrant division algorithm associated with an analytic image format.

FIGS. 76 A-E illustrate an example of a recursive quadrant division algorithm on an image with a single region of interest.

FIGS. 77 A-F illustrate an example of a recursive quadrant division algorithm on an image with multiple regions of interest.

FIGS. 78 A-B and 79 A-B illustrate pseudocode for a region-based tiling algorithm associated with an analytic image format.

FIGS. 80 A-C illustrate an example of a region-based tiling algorithm on an image with a single region of interest.

FIGS. 81 A-C illustrate an example of various tiling algorithms on an image with multiple overlapping regions of interest.

FIG. 82 illustrates an example of an integrated customer model based on both online and offline customer activity.

FIG. 83 illustrates an example of linking the in-store visual footprint and online activity of a customer.

FIG. 84 illustrates an example of using online and in-store customer activities to construct a robust customer model.

FIG. 85 illustrates a comparison of the shopping activities of multiple users.

FIG. 86 illustrates an example of using head pose to infer customer data points.

FIG. 87 illustrates an example dataflow for creating an improved customer model using both traditional and visual data.

FIG. 88 illustrates the architecture of an example application that leverages integrated online and in-store customer profiles.

FIG. 89 illustrates a flowchart for an example embodiment of a multi-domain cascade CNN.

FIG. 90 illustrates a flowchart for an example embodiment of a multi-domain CNN.

FIG. 91 illustrates a flowchart for an example embodiment of a blockchain for managing self-sovereign device identities.

FIG. 92 illustrates a flowchart for an example embodiment of a blockchain for managing distributed computing algorithms.

EMBODIMENTS OF THE DISCLOSURE

This patent application claims the benefit of the filing date of U.S. Provisional Patent Application Ser. No. 62/611,536, filed on Dec. 28, 2017, and entitled “VISUAL FOG,” and U.S. Provisional Patent Application Ser. No. 62/691,464, filed on Jun. 28, 2018, and entitled “VISUAL FOG,” the contents of which are hereby expressly incorporated by reference.

The following disclosure provides many different embodiments, or examples, for implementing different features of the present disclosure. Specific examples of components and arrangements are described below to simplify the present disclosure. These are, of course, merely examples and are not intended to be limiting. Further, the present disclosure may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. Different embodiments may have different advantages, and no particular advantage is necessarily required of any embodiment.

Example embodiments that may be used to implement the features and functionality of this disclosure will now be described with more particular reference to the attached FIGURES.

Visual Fog Introduction

FIG. 1 illustrates an example embodiment of a visual fog system 100 in accordance with certain embodiments. Advancements in modern computing have led to an increased use of computer vision technologies and large-scale visual computing for a variety of mainstream computing applications. In particular, rapid deployments of cameras and other types of computer vision technologies have been leveraged for a variety of visual computing applications that rely on large-scale video analytics and visual data processing. For example, large-scale visual computing can be leveraged for security and surveillance, transportation (e.g., traffic monitoring, navigation, park

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a continuation (and claims the benefit under 35 U.S.C. § 120) of U.S. patent application Ser. No. 16/948,304, filed on Sep. 11, 2020, which is a continuation of U.S. patent application Ser. No. 16/024,397, filed on Jun. 29, 2018, and entitled “MULTI-DOMAIN CONVOLUTIONAL NEURAL NETWORK”, which claims the benefit of the filing date of U.S. Provisional Patent Application Ser. No. 62/611,536, filed on Dec. 28, 2017, and entitled “VISUAL FOG,” and U.S. Provisional Patent Application Ser. No. 62/691,464, filed on Jun. 28, 2018, and entitled “VISUAL FOG,” the contents of which are hereby expressly incorporated by reference.

FIELD OF THE SPECIFICATION

This disclosure relates in general to the field of visual computing, and more particularly, though not exclusively, to a multi-domain convolutional neural network (CNN).

BACKGROUND

In distributed computing systems, images and video captured by devices at the edge of a network are typically compressed and transmitted to the cloud for further processing. The images and video must subsequently be decompressed before any processing can be performed. This approach is inefficient, however, as it often wastes significant bandwidth on transmission of insignificant and/or trivial visual data, it requires full decompression as a prerequisite to performing any meaningful processing and analysis on visual data, and it fails to leverage the processing capabilities of other nodes in the network efficiently.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is best understood from the following detailed description when read with the accompanying figures. It is emphasized that, in accordance with the standard practice in the industry, various features are not necessarily drawn to scale, and are used for illustration purposes only. Where a scale is shown, explicitly or implicitly, it provides only one illustrative example. In other embodiments, the dimensions of the various features may be arbitrarily increased or reduced for clarity of discussion.

FIG. 1 illustrates an example embodiment of a visual fog system in accordance with certain embodiments.

FIGS. 2 , 3 , 4 , and 5 illustrate examples of Internet-of-Things (IoT) networks and architectures that can be used in accordance with certain embodiments.

FIGS. 6 and 7 illustrate example computer architectures that can be used in accordance with certain embodiments.

FIG. 8 illustrates an example embodiment of an architecture for visual fog nodes.

FIGS. 9 , 10 , 11 , and 12 A -B illustrate example embodiments of a visual fog architecture.

FIGS. 13 and 14 illustrate example embodiments associated with a visual question answering (VCA) framework.

FIGS. 15 and 16 illustrate example embodiments of device-centric scheduling for visual fog computing.

FIG. 17 illustrates an example embodiment of a runtime processing pipeline for a visual fog architecture.

FIG. 18 illustrates an example embodiment of a visual data storage architecture.

FIG. 19 illustrates an example of a vision processing pipeline that leverages metadata for searching visual data.

FIGS. 20 and 21 illustrate examples of representing visual metadata using a property graph.

FIG. 22 illustrates an example embodiment of an analytic image format designed to aid in visual data processing.

FIG. 23 illustrates a performance graph for various image formats.

FIGS. 24 A-C illustrate an example embodiment of a multi-domain cascade convolutional neural network (CNN).

FIGS. 25 A-B , 26 , 27 , 28 , 29 , 30 and 31 A-B illustrate the use of butterfly operations for a multi-domain convolutional neural network (CNN).

FIGS. 32 and 33 illustrate an example embodiment of a three-dimensional (3D) CNN for processing compressed visual data.

FIG. 34 illustrates an example of a pixel-domain CNN.

FIG. 35 illustrates an example of a pixel-domain visual analytics pipeline.

FIGS. 36 and 37 illustrate example embodiments of compressed-domain visual analytics pipelines.

FIG. 38 illustrates a performance graph showing the precision of a CNN trained using compressed visual data.

FIG. 39 illustrates a flowchart for an example embodiment of context-aware image compression.

FIGS. 40 A-C illustrate an example embodiment of a privacy-preserving demographic identification system.

FIGS. 41 , 42 , and 43 illustrate an example embodiment of privacy-preserving distributed visual data processing.

FIGS. 44 , 45 , and 46 illustrate example embodiments of self-sovereign device identification for distributed computing networks.

FIG. 47 illustrates an example of device onboarding/commissioning in a visual fog network without conflict resolution.

FIGS. 48 and 49 illustrate example embodiments of algorithm identification for distributed computing using a self-sovereign blockchain.

FIGS. 50 , 51 , and 52 illustrate example embodiments for processing traditional and analytic image formats.

FIG. 53 illustrates a flowchart for an example embodiment of privacy-preserving demographics identification.

FIG. 54 illustrates a flowchart for an example embodiment of privacy-preserving distributed visual processing.

FIG. 55 illustrates an example use case for an automotive ubiquitous witness.

FIG. 56 illustrates an example dataflow for a ubiquitous witness.

FIG. 57 illustrates an example use case for automotive anomaly detection and reconstruction.

FIG. 58 illustrates an example process flow for IP-based anomaly detection and reconstruction.

FIG. 59 illustrates an example process flow for ICN-based anomaly detection and reconstruction.

FIG. 60 illustrates an example process flow of an individual node for ICN-based anomaly detection and reconstruction.

FIG. 61 illustrates a flowchart for an example embodiment of a ubiquitous visual computing witness.

FIG. 62 illustrates an example embodiment of a converged node router.

FIG. 63 illustrates an example network topology using converged node routers.

FIG. 64 illustrates an example embodiment of an image storage system that leverages context-aware image compression.

FIG. 65 illustrates an example of inter-frame encoding for context-aware image compression.

FIGS. 66 , 67 , and 68 illustrate examples embodiments for performing automated semantic inference of visual objects using smart cameras.

FIGS. 69 and 70 illustrate example embodiments associated with visual fog stream multiplexing for improved security.

FIG. 71 illustrates an example embodiment of a privacy-preserving VQA dataflow.

FIGS. 72 and 73 illustrate example sanitization techniques for a privacy-preserving VQA dataflow.

FIGS. 74 and 75 illustrate pseudocode for a recursive quadrant division algorithm associated with an analytic image format.

FIGS. 76 A-E illustrate an example of a recursive quadrant division algorithm on an image with a single region of interest.

FIGS. 77 A-F illustrate an example of a recursive quadrant division algorithm on an image with multiple regions of interest.

FIGS. 78 A-B and 79 A-B illustrate pseudocode for a region-based tiling algorithm associated with an analytic image format.

FIGS. 80 A-C illustrate an example of a region-based tiling algorithm on an image with a single region of interest.

FIGS. 81 A-C illustrate an example of various tiling algorithms on an image with multiple overlapping regions of interest.

FIG. 82 illustrates an example of an integrated customer model based on both online and offline customer activity.

FIG. 83 illustrates an example of linking the in-store visual footprint and online activity of a customer.

FIG. 84 illustrates an example of using online and in-store customer activities to construct a robust customer model.

FIG. 85 illustrates a comparison of the shopping activities of multiple users.

FIG. 86 illustrates an example of using head pose to infer customer data points.

FIG. 87 illustrates an example dataflow for creating an improved customer model using both traditional and visual data.

FIG. 88 illustrates the architecture of an example application that leverages integrated online and in-store customer profiles.

FIG. 89 illustrates a flowchart for an example embodiment of a multi-domain cascade CNN.

FIG. 90 illustrates a flowchart for an example embodiment of a multi-domain CNN.

FIG. 91 illustrates a flowchart for an example embodiment of a blockchain for managing self-sovereign device identities.

FIG. 92 illustrates a flowchart for an example embodiment of a blockchain for managing distributed computing algorithms.

EMBODIMENTS OF THE DISCLOSURE

This patent application claims the benefit of the filing date of U.S. Provisional Patent Application Ser. No. 62/611,536, filed on Dec. 28, 2017, and entitled “VISUAL FOG,” and U.S. Provisional Patent Application Ser. No. 62/691,464, filed on Jun. 28, 2018, and entitled “VISUAL FOG,” the contents of which are hereby expressly incorporated by reference.

The following disclosure provides many different embodiments, or examples, for implementing different features of the present disclosure. Specific examples of components and arrangements are described below to simplify the present disclosure. These are, of course, merely examples and are not intended to be limiting. Further, the present disclosure may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. Different embodiments may have different advantages, and no particular advantage is necessarily required of any embodiment.

Example embodiments that may be used to implement the features and functionality of this disclosure will now be described with more particular reference to the attached FIGURES.

Visual Fog Introduction

FIG. 1 illustrates an example embodiment of a visual fog system 100 in accordance with certain embodiments. Advancements in modern computing have led to an increased use of computer vision technologies and large-scale visual computing for a variety of mainstream computing applications. In particular, rapid deployments of cameras and other types of computer vision technologies have been leveraged for a variety of visual computing applications that rely on large-scale video analytics and visual data processing. For example, large-scale visual computing can be leveraged for security and surveillance, transportation (e.g., traffic monitoring, navigation, parking, infrastructure planning, security or amber alerts), retail (e.g., customer analytics), enterprise applications, and so forth.

Existing approaches to large-scale visual computing, however, suffer from numerous limitations. In particular, existing visual computing approaches are implemented using rigid designs that utilize resources inefficiently (e.g., processing, bandwidth, and storage resources) and provide limited functionality. For example, using existing approaches, visual data is typically captured by devices at the edge of a network and simply funneled to the cloud for processing and storage, thus relying heavily on the cloud infrastructure. Due to the large size of visual data, however, this approach typically consumes significant network bandwidth and requires substantial processing and storage resources in the cloud. As a result, existing approaches often suffer from high latency and inefficient resource utilization, and may also be inaccurate, unreliable, inflexible, and incapable of scaling efficiently.

Accordingly, this disclosure describes various embodiments of a visual fog computing system 100 for performing large-scale visual computing in an efficient and reliable manner. For example, rather than relying exclusively or primarily on cloud resources 130 for visual computing tasks, visual fog system 100 leverages both cloud 130 and edge 110 resources, which may be collectively referred to as the “fog.” In this manner, visual fog system 100 can leverage all available “fog” resources to perform visual computing tasks more efficiently, thus improving resource utilization, latency, accuracy, precision, and reliability. Moreover, as described further throughout this disclosure, visual fog system 100 can be implemented using a flexible design that supports ad-hoc queries and is highly scalable, thus rendering it suitable for many visual computing applications and use cases.

In the illustrated embodiment of FIG. 1 , visual fog system 100 includes edge resources 110 and a plurality of associated visual sensors 120 , cloud resources 130 , and communication networks 150 , which are respectively discussed further below. Moreover, in various embodiments, these components of visual fog system 100 may be implemented some or all aspects of the visual computing functionality described throughout this disclosure in connection with the remaining FIGURES.

Edge resources 110 may include any equipment, devices, and/or components deployed or connected near the “edge” of a communication network. In the illustrated embodiment, for example, edge resources 110 include end- user devices 112 a,b (e.g., desktops, laptops, mobile devices), Internet-of-Things (IoT) devices 114 , and gateways or routers 116 , as described further below. Edge resources 110 may communicate with each other and/or with other remote networks and resources (e.g., cloud resources 130 ) through one or more communication networks 150 , such as local area network 150 a and/or wide area network 150 b . Moreover, in the illustrated embodiment, edge resources 110 collectively include a plurality of visual sensors 120 (e.g., cameras) for capturing visual representations and data associated with their surroundings. In some embodiments, for example, certain end-user devices 112 and/or IoT devices 114 may include one or more cameras and/or other types of visual sensors 120 . Visual sensors 120 may include any type of visual or optical sensors, such as cameras, ultraviolet (UV) sensors, laser rangefinders (e.g., light detection and ranging (LIDAR)), infrared (IR) sensors, electro-optical/infrared (EO/IR) sensors, and so forth.

End-user devices 112 may include any device that enables or facilitates interaction with a user in visual fog system 100 , including, for example, desktop computers, laptops, tablets, mobile phones and other mobile devices, and wearable devices (e.g., smart watches, smart glasses, headsets), among other examples.

IoT devices 114 may include any device capable of communicating and/or participating in an Internet-of-Things (IoT) system or network. IoT systems may refer to new or improved ad-hoc systems and networks composed of a variety of different devices (e.g., IoT devices 114 ) interoperating and synergizing for a particular application or use case. Such ad-hoc systems are emerging as more and more products and equipment evolve to become “smart,” meaning they are controlled or monitored by computer processors and are capable of communicating with other devices. For example, an IoT device 114 may include a computer processor and/or communication interface to allow interoperation with other components of visual fog system 100 , such as with cloud resources 130 and/or other edge resources 110 . IoT devices 114 may be “greenfield” devices that are developed with IoT capabilities from the ground-up, or “brownfield” devices that are created by integrating IoT capabilities into existing legacy devices that were initially developed without IoT capabilities. For example, in some cases, IoT devices 114 may be built from sensors and communication modules integrated in or attached to “things,” such as equipment, toys, tools, vehicles, living things (e.g., plants, animals, humans), and so forth. Alternatively, or additionally, certain IoT devices 114 may rely on intermediary components, such as edge gateways or routers 116 , to communicate with the various components of system 100 .

IoT devices 114 may include various types of sensors for monitoring, detecting, measuring, and generating sensor data and signals associated with characteristics of their environment. In some embodiments, for example, certain IoT devices 114 may include visual sensors 120 (e.g., cameras) for capturing visual representations and data associated with their surroundings. IoT devices 114 may also include other types of sensors configured to detect characteristics such as movement, weight, physical contact, temperature, wind, noise, light, position, humidity, radiation, liquid, specific chemical compounds, battery life, wireless signals, computer communications, and bandwidth, among other examples. Sensors can include physical sensors (e.g., physical monitoring components) and virtual sensors (e.g., software-based monitoring components). IoT devices 114 may also include actuators to perform various actions in their respective environments. For example, an actuator may be used to selectively activate certain functionality, such as toggling the power or operation of a security system (e.g., alarm, camera, locks) or household appliance (e.g., audio system, lighting, HVAC appliances, garage doors), among other examples.

Indeed, this disclosure contemplates use of a potentially limitless universe of IoT devices 114 and associated sensors/actuators. IoT devices 114 may include, for example, any type of equipment and/or devices associated with any type of system 100 and/or industry, including transportation (e.g., automobile, airlines), industrial manufacturing, energy (e.g., power plants), telecommunications (e.g., Internet, cellular, and television service providers), retail, medical (e.g., healthcare, pharmaceutical), and/or food and beverage, among others. In the transportation industry, for example, IoT devices 114 may include equipment and devices associated with aircrafts, automobiles, or vessels, such as navigation systems, autonomous flight or driving systems, traffic monitoring and/or planning systems, parking systems, and/or any internal mechanical or electrical components that are monitored by sensors (e.g., engines). IoT devices 114 may also include equipment, devices, and/or infrastructure associated with industrial manufacturing and production, shipping (e.g., cargo tracking), communications networks (e.g., gateways, routers, servers, cellular towers), server farms, electrical power plants, wind farms, oil and gas pipelines, water treatment and distribution, wastewater collection and treatment, and weather monitoring (e.g., temperature, wind, and humidity sensors), among other examples. IoT devices 114 may also include, for example, any type of “smart” device or system, such as smart entertainment systems (e.g., televisions, audio systems, videogame systems), smart household or office appliances (e.g., heat-ventilation-air-conditioning (HVAC) appliances, refrigerators, washers and dryers, coffee brewers), power control systems (e.g., automatic electricity, light, and HVAC controls), security systems (e.g., alarms, locks, cameras, motion detectors, fingerprint scanners, facial recognition systems), and other home automation systems, among other examples. IoT devices 114 can be statically located, such as mounted on a building, wall, floor, ground, lamppost, sign, water tower, or any other fixed or static structure. IoT devices 114 can also be mobile, such as devices in vehicles or aircrafts, drones, packages (e.g., for tracking cargo), mobile devices, and wearable devices, among other examples. Moreover, any type of edge resource 110 may also be considered as an IoT device 114 , including end-user devices 112 and edge gateways 116 , among other examples.

Edge gateways and/or routers 116 may be used to facilitate communication to and from edge resources 110 . For example, gateways 116 may provide communication capabilities to existing legacy devices that were initially developed without any such capabilities (e.g., “brownfield” IoT devices 114 ). Gateways 116 can also be utilized to extend the geographical reach of edge resources 110 with short-range, proprietary, or otherwise limited communication capabilities, such as IoT devices 114 with Bluetooth or ZigBee communication capabilities. For example, gateways 116 can serve as intermediaries between IoT devices 114 and remote networks or services, by providing a front-haul to the IoT devices 114 using their native communication capabilities (e.g., Bluetooth, ZigBee), and providing a back-haul to other networks 150 and/or cloud resources 130 using another wired or wireless communication medium (e.g., Ethernet, Wi-Fi, cellular). In some embodiments, a gateway 116 may be implemented by a dedicated gateway device, or by a general-purpose device, such as another IoT device 114 , end-user device 112 , or other type of edge resource 110 . In some instances, gateways 116 may also implement certain network management and/or application functionality (e.g., visual computing functionality, IoT application and management functionality), either separately or in conjunction with other components, such as cloud resources 130 and/or other edge resources 110 .

Cloud resources 130 may include any resources or services that are hosted remotely over a network, which may otherwise be referred to as in the “cloud.” In some embodiments, for example, cloud resources 130 may be remotely hosted on servers in a datacenter (e.g., application servers, database servers). Cloud resources 130 may include any resources, services, and/or functionality that can be utilized by or for edge resources 110 , including but not limited to, visual computing applications and services, IoT application and management services, data storage, computational services (e.g., data analytics, searching, diagnostics and fault management), security services (e.g., surveillance, alarms, user authentication), mapping and navigation, geolocation services, network or infrastructure management, payment processing, audio and video streaming, messaging, social networking, news, and weather, among other examples.

Communication networks 150 a,b may be used to facilitate communication between components of system 100 . In the illustrated embodiment, for example, edge resources 110 are connected to local area network (LAN) 150 a in order to facilitate communication with each other and/or other remote networks or resources, such as wide area network (WAN) 150 b and/or cloud resources 130 . In various embodiments, visual fog system 100 may be implemented using any number or type of communication network(s) 150 , including local area networks, wide area networks, public networks, the Internet, cellular networks, Wi-Fi networks, short-range networks (e.g., Bluetooth or ZigBee), and/or any other wired or wireless communication networks or mediums.

In general, edge resources 110 (and in particular IoT devices 114 ) may generate an extremely large volume and variety of data. As one example, edge resources 110 with visual sensors 120 may generate large volumes of visual data, such as video and/or images. Edge resources 110 typically offload this data to the cloud 130 for processing and/or storage. Cloud resources 130 , however, may not necessarily be suited to handle the rapidly growing volume, variety, and velocity of data generated by IoT devices 114 and other edge resources 110 . For example, cloud-based processing may not be ideal in certain circumstances, such as processing time-sensitive or highly confidential data, or when faced with network bandwidth constraints, among other examples. Accordingly, in some embodiments, visual fog system 100 may leverage “edge” processing to augment the performance and capabilities of the cloud 130 using edge resources 110 . Edge processing is an approach that involves processing certain data at the network edge (e.g., using edge resources 110 ), near where the data is generated, rather than simply funneling large volumes of data to the cloud for processing and storage. Certain data may still be sent to the cloud, as appropriate, such as for deeper analysis and/or long-term storage. Edge processing may be used to complement the shortcomings of cloud-based processing (e.g., when cloud-based processing is inefficient, ineffective, and/or unsecure), and thus improve the handling of the growing volume, variety, and velocity of data generated by IoT devices 114 and/or other edge resources 110 . For example, in some cases, processing data near its source (e.g., in the network edge) rather than in the cloud may improve performance and/or avoid system failures or disasters. Edge processing may also conserve network bandwidth, which may be particularly beneficial when facing bandwidth constraints and/or limited network connectivity.

In some cases, the collective use of both edge 110 and cloud 130 resources may be referred to as “fog” computing, as functionality of the “cloud” 130 is effectively extended by the edge resources 110 , thus forming a “fog” over the network edge. Moreover, in some embodiments, devices 110 in the “fog” may connect and/or communicate with each other using an interconnection standard or protocol, such as the open interconnect consortium (OIC) standard specification 1.0, released by the Open Connectivity Foundation™ (OCF) on Dec. 23, 2015, which enables devices to discover and connect with each other; Thread, a networking protocol for Internet-of-Things (IoT) devices used in “smart” home automation and similar deployments, developed by an alliance of organizations named the “Thread Group”; the optimized link state routing (OLSR) protocol; and/or the better approach to mobile ad-hoc networking (B.A.T.M.A.N.), among other examples.

Moreover, in some embodiments, fog computing may be leveraged by visual fog system 100 for large-scale visual computing applications. For example, in some embodiments, the components of visual fog system 100 (e.g., edge resources 110 , cloud resources 130 ) may be implemented with some or all aspects of the visual computing functionality described throughout this disclosure in connection with the remaining FIGURES.

Any, all, or some of the computing devices of system 100 may be adapted to execute any operating system, including Linux or other UNIX-based operating systems, Microsoft Windows, Windows Server, MacOS, Apple iOS, Google Android, or any customized and/or proprietary operating system, along with virtual machines adapted to virtualize execution of a particular operating system.

While FIG. 1 is described as containing or being associated with a plurality of elements, not all elements illustrated within system 100 of FIG. 1 may be utilized in each alternative implementation of the present disclosure. Additionally, one or more of the elements described in connection with the examples of FIG. 1 may be located external to system 100 , while in other instances, certain elements may be included within or as a portion of one or more of the other described elements, as well as other elements not described in the illustrated implementation. Further, certain elements illustrated in FIG. 1 may be combined with other components, as well as used for alternative or additional purposes in addition to those purposes described herein.

Additional embodiments associated with the implementation of a visual fog computing system 100 are described further in connection with the remaining FIGURES. Accordingly, it should be appreciated that visual fog system 100 of FIG. 1 may be implemented with any aspects of the embodiments described throughout this disclosure.

Example Internet-of-Things (IoT) Implementations

FIGS. 2 - 5 illustrate examples of Internet-of-Things (IoT) networks and devices that can be used in accordance with embodiments disclosed herein. For example, the operations and functionality described throughout this disclosure may be embodied by an IoT device or machine in the example form of an electronic processing system, within which a set or sequence of instructions may be executed to cause the electronic processing system to perform any one of the methodologies discussed herein, according to an example embodiment. The machine may be an IoT device or an IoT gateway, including a machine embodied by aspects of a personal computer (PC), a tablet PC, a personal digital assistant (PDA), a mobile telephone or smartphone, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine may be depicted and referenced in the example above, such machine shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein. Further, these and like examples to a processor-based system shall be taken to include any set of one or more machines that are controlled by or operated by a processor (e.g., a computer) to individually or jointly execute instructions to perform any one or more of the methodologies discussed herein.

FIG. 2 illustrates an example domain topology for respective internet-of-things (IoT) networks coupled through links to respective gateways. The internet of things (IoT) is a concept in which a large number of computing devices are interconnected to each other and to the Internet to provide functionality and data acquisition at very low levels. Thus, as used herein, an IoT device may include a semiautonomous device performing a function, such as sensing or control, among others, in communication with other IoT devices and a wider network, such as the Internet.

Often, IoT devices are limited in memory, size, or functionality, allowing larger numbers to be deployed for a similar cost to smaller numbers of larger devices. However, an IoT device may be a smart phone, laptop, tablet, or PC, or other larger device. Further, an IoT device may be a virtual device, such as an application on a smart phone or other computing device. IoT devices may include IoT gateways, used to couple IoT devices to other IoT devices and to cloud applications, for data storage, process control, and the like.

Networks of IoT devices may include commercial and home automation devices, such as water distribution systems, electric power distribution systems, pipeline control systems, plant control systems, light switches, thermostats, locks, cameras, alarms, motion sensors, and the like. The IoT devices may be accessible through remote computers, servers, and other systems, for example, to control systems or access data.

The future growth of the Internet and like networks may involve very large numbers of IoT devices. Accordingly, in the context of the techniques discussed herein, a number of innovations for such future networking will address the need for all these layers to grow unhindered, to discover and make accessible connected resources, and to support the ability to hide and compartmentalize connected resources. Any number of network protocols and communications standards may be used, wherein each protocol and standard is designed to address specific objectives. Further, the protocols are part of the fabric supporting human accessible services that operate regardless of location, time or space. The innovations include service delivery and associated infrastructure, such as hardware and software; security enhancements; and the provision of services based on Quality of Service (QoS) terms specified in service level and service delivery agreements. As will be understood, the use of IoT devices and networks, such as those introduced in FIGS. 2 - 5 , present a number of new challenges in a heterogeneous network of connectivity comprising a combination of wired and wireless technologies.

FIG. 2 specifically provides a simplified drawing of a domain topology that may be used for a number of internet-of-things (IoT) networks comprising IoT devices 204 , with the

IoT networks

256 , 258 , 260 , 262 , coupled through backbone links 202 to respective gateways 254 . For example, a number of IoT devices 204 may communicate with a gateway 254 , and with each other through the gateway 254 . To simplify the drawing, not every IoT device 204 , or communications link (e.g., link 216 , 222 , 228 , or 232 ) is labeled. The backbone links 202 may include any number of wired or wireless technologies, including optical networks, and may be part of a local area network (LAN), a wide area network (WAN), or the Internet. Additionally, such communication links facilitate optical signal paths among both IoT devices 204 and gateways 254 , including the use of MUXing/deMUXing components that facilitate interconnection of the various devices.

The network topology may include any number of types of IoT networks, such as a mesh network provided with the network 256 using Bluetooth low energy (BLE) links 222 . Other types of IoT networks that may be present include a wireless local area network (WLAN) network 258 used to communicate with IoT devices 204 through IEEE 802.11 (Wi-Fi®) links 228 , a cellular network 260 used to communicate with IoT devices 204 through an LTE/LTE-A (4G) or 5G cellular network, and a low-power wide area (LPWA) network 262 , for example, a LPWA network compatible with the LoRaWan specification promulgated by the LoRa alliance, or a IPv6 over Low Power Wide-Area Networks (LPWAN) network compatible with a specification promulgated by the Internet Engineering Task Force (IETF). Further, the respective IoT networks may communicate with an outside network provider (e.g., a tier 2 or <figure-callout id="3" label="tier" filenames="

CLAIMS

Claims ( 20 )

What is claimed is:

1. At least one non-transitory computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed on processing circuitry, cause the processing circuitry to:

receive a frequency-domain representation of image data, wherein the frequency-domain representation comprises a plurality of sets of transform coefficients for a plurality of blocks of the image data; and

detect content in the image data using a convolutional neural network (CNN), wherein the CNN is trained to detect the content in the image data based on corresponding transform coefficients among the plurality of blocks.

2. The storage medium of claim 1 , wherein the corresponding transform coefficients comprise groups of transform coefficients for corresponding frequencies from the plurality of sets of transform coefficients.

3. The storage medium of claim 1 , wherein:

the frequency-domain representation of the image data is arranged into a plurality of dimensions, wherein the plurality of dimensions comprise a first dimension, a second dimension, and a third dimension;

the first dimension and the second dimension represent spatial indices of the plurality of blocks within the image data; and

the third dimension represents a plurality of channels, wherein each channel comprises a group of transform coefficients for a corresponding frequency from the plurality of sets of transform coefficients.

4. The storage medium of claim 1 , wherein each set of transform coefficients represents a transform performed on one of the plurality of blocks.

5. The storage medium of claim 4 , wherein:

the transform is a discrete cosine transform (DCT); and

the transform coefficients are DCT coefficients.

6. The storage medium of claim 1 , wherein the CNN comprises one or more butterfly layers, wherein the one or more butterfly layers are to perform one or more partial inverse transforms based on the frequency-domain representation of the image data.

7. The storage medium of claim 1 , wherein the instructions that cause the processing circuitry to receive the frequency-domain representation of the image data further cause the processing circuitry to:

receive, via interface circuitry, compressed image data, wherein the compressed image data comprises the frequency-domain representation of the image data; and

extract the frequency-domain representation of the image data from the compressed image data.

8. The storage medium of claim 1 , wherein the image data comprises an image or a video frame.

9. A device, comprising:

processing circuitry to:

receive a frequency-domain representation of image data, wherein the frequency-domain representation comprises a plurality of sets of transform coefficients for a plurality of blocks of the image data; and

detect content in the image data using a convolutional neural network (CNN), wherein the CNN is trained to detect the content in the image data based on corresponding transform coefficients among the plurality of blocks.

10. The device of claim 9 , wherein the corresponding transform coefficients comprise groups of transform coefficients for corresponding frequencies from the plurality of sets of transform coefficients.

11. The device of claim 9 , wherein:

the frequency-domain representation of the image data is arranged into a plurality of dimensions, wherein the plurality of dimensions comprise a first dimension, a second dimension, and a third dimension;

the first dimension and the second dimension represent spatial indices of the plurality of blocks within the image data; and

the third dimension represents a plurality of channels, wherein each channel comprises a group of transform coefficients for a corresponding frequency from the plurality of sets of transform coefficients.

12. The device of claim 9 , wherein each set of transform coefficients represents a transform performed on one of the plurality of blocks.

13. The device of claim 12 , wherein:

the transform is a discrete cosine transform (DCT); and

the transform coefficients are DCT coefficients.

14. The device of claim 9 , wherein the CNN comprises one or more butterfly layers, wherein the one or more butterfly layers are to perform one or more partial inverse transforms based on the frequency-domain representation of the image data.

15. The device of claim 9 , wherein the processing circuitry to receive the frequency-domain representation of the image data is further to:

receive, via interface circuitry, compressed image data, wherein the compressed image data comprises the frequency-domain representation of the image data; and

extract the frequency-domain representation of the image data from the compressed image data.

16. The device of claim 9 , wherein the device is:

a smart camera; or

an edge server.

17. A method, comprising:

receiving a frequency-domain representation of image data, wherein the frequency-domain representation comprises a plurality of sets of transform coefficients for a plurality of blocks of the image data, wherein each set of transform coefficients represents a transform performed on one of the plurality of blocks; and

detecting content in the image data using a convolutional neural network (CNN), wherein the CNN is trained to detect the content in the image data based on corresponding transform coefficients among the plurality of blocks.

18. The method of claim 17 , wherein the corresponding transform coefficients comprise groups of transform coefficients for corresponding frequencies from the plurality of sets of transform coefficients.

19. The method of claim 17 , wherein:

the frequency-domain representation of the image data is arranged into a plurality of dimensions, wherein the plurality of dimensions comprise a first dimension, a second dimension, and a third dimension;

the first dimension and the second dimension represent spatial indices of the plurality of blocks within the image data; and

the third dimension represents a plurality of channels, wherein each channel comprises a group of transform coefficients for a corresponding frequency from the plurality of sets of transform coefficients.

20. The method of claim 17 , wherein:

the transform is a discrete cosine transform (DCT); and

the transform coefficients are DCT coefficients.

US18/157,154

2017-12-28

2023-01-20

Frequency-domain convolutional neural network

Active

US11836240B2

( en )

Priority Applications (1)

Application Number

Priority Date

Filing Date

Title

US18/157,154

US11836240B2

( en )

2017-12-28

2023-01-20

Frequency-domain convolutional neural network

Applications Claiming Priority (5)

Application Number

Priority Date

Filing Date

Title

US201762611536P

2017-12-28

2017-12-28

US201862691464P

2018-06-28

2018-06-28

US16/024,397

US10778412B2

( en )

2017-12-28

2018-06-29

Multi-domain convolutional neural network

US16/948,304

US11562181B2

( en )

2017-12-28

2020-09-11

Multi-domain convolutional neural network

US18/157,154

US11836240B2

( en )

2017-12-28

2023-01-20

Frequency-domain convolutional neural network

Related Parent Applications (1)

Application Number

Title

Priority Date

Filing Date

US16/948,304

Continuation

US11562181B2

( en )

2017-12-28

2020-09-11

Multi-domain convolutional neural network

Publications (2)

Publication Number

Publication Date

US20230237144A1

US20230237144A1 ( en )

2023-07-27

US11836240B2

true

US11836240B2 ( en )

2023-12-05

Family

ID=65230711

Family Applications (16)

Application Number

Title

Priority Date

Filing Date

US16/024,364

Active

2038-09-22

US10742399B2

( en )

2017-12-28

2018-06-29

Context-aware image compression

US16/652,038

Abandoned

US20200250003A1

( en )

2017-12-28

2018-06-29

Visual fog

US16/024,397

Active

2038-12-05

US10778412B2

( en )

2017-12-28

2018-06-29

Multi-domain convolutional neural network

US16/024,356

Active

2038-12-11

US10797863B2

( en )

2017-12-28

2018-06-29

Multi-domain cascade convolutional neural network

US16/024,366

Active

2038-11-13

US11481583B2

( en )

2017-12-28

2018-06-29

Algorithm management blockchain

US16/141,812

Active

US10719744B2

( en )

2017-12-28

2018-09-25

Automated semantic inference of visual features and scenes

US16/142,468

Active

2039-01-04

US11068757B2

( en )

2017-12-28

2018-09-26

Analytic image format for visual computing

US16/145,047

Active

US11139958B2

( en )

2017-12-28

2018-09-27

Privacy-preserving sanitization for visual computing queries

US16/947,096

Active

US11462036B2

( en )

2017-12-28

2020-07-17

Automated semantic inference of visual features and scenes

US16/947,590

Active

US11531850B2

( en )

2017-12-28

2020-08-07

Context-aware image compression

US16/948,304

Active

2039-01-10

US11562181B2

( en )

2017-12-28

2020-09-11

Multi-domain convolutional neural network

US16/948,861

Active

US11521024B2

( en )

2017-12-28

2020-10-02

Cascade convolutional neural network

US17/374,217

Active

US11450123B2

( en )

2017-12-28

2021-07-13

Analytic image format for visual computing

US17/942,304

Active

US12111907B2

( en )

2017-12-28

2022-09-12

Analytic image format for visual computing

US18/060,207

Active

2039-06-21

US12499204B2

( en )

2017-12-28

2022-11-30

Compressed video recognition

US18/157,154

Active

US11836240B2

( en )

2017-12-28

2023-01-20

Frequency-domain convolutional neural network

Family Applications Before (15)

Application Number

Title

Priority Date

Filing Date

US16/024,364

Active

2038-09-22

US10742399B2

( en )

2017-12-28

2018-06-29

Context-aware image compression

US16/652,038

Abandoned

US20200250003A1

( en )

2017-12-28

2018-06-29

Visual fog

US16/024,397

Active

2038-12-05

US10778412B2

( en )

2017-12-28

2018-06-29

Multi-domain convolutional neural network

US16/024,356

Active

2038-12-11

US10797863B2

( en )

2017-12-28

2018-06-29

Multi-domain cascade convolutional neural network

US16/024,366

Active

2038-11-13

US11481583B2

( en )

2017-12-28

2018-06-29

Algorithm management blockchain

US16/141,812

Active

US10719744B2

( en )

2017-12-28

2018-09-25

Automated semantic inference of visual features and scenes

US16/142,468

Active

2039-01-04

US11068757B2

( en )

2017-12-28

2018-09-26

Analytic image format for visual computing

US16/145,047

Active

US11139958B2

( en )

2017-12-28

2018-09-27

Privacy-preserving sanitization for visual computing queries

US16/947,096

Active

US11462036B2

( en )

2017-12-28

2020-07-17

Automated semantic inference of visual features and scenes

US16/947,590

Active

US11531850B2

( en )

2017-12-28

2020-08-07

Context-aware image compression

US16/948,304

Active

2039-01-10

US11562181B2

( en )

2017-12-28

2020-09-11

Multi-domain convolutional neural network

US16/948,861

Active

US11521024B2

( en )

2017-12-28

2020-10-02

Cascade convolutional neural network

US17/374,217

Active

US11450123B2

( en )

2017-12-28

2021-07-13

Analytic image format for visual computing

US17/942,304

Active

US12111907B2

( en )

2017-12-28

2022-09-12

Analytic image format for visual computing

US18/060,207

Active

2039-06-21

US12499204B2

( en )

2017-12-28

2022-11-30

Compressed video recognition

Country Status (5)

Country

Link

US

( 16 )

US10742399B2

( en )

KR

( 1 )

KR102656864B1

( en )

CN

( 1 )

CN112384947A

( en )

DE

( 1 )

DE112018006630T5

( en )

WO

( 1 )

WO2019133052A1

( en )

Families Citing this family (607)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US11669562B2

( en )

*

2013-10-10

2023-06-06

Aura Home, Inc.

Method of clustering photos for digital picture frames with split screen display

US12417317B1

( en )

*

2014-05-13

2025-09-16

Anonos Ip Llc

Systems and methods for dynamic de-identification and anonymity for user input to artificial intelligence (AI)-based models

US10147984B2

( en )

2015-07-31

2018-12-04

SynCells, Inc.

Portable and modular energy storage for multiple applications

US10303887B2

( en )

*

2015-09-14

2019-05-28

T0.Com, Inc.

Data verification methods and systems using a hash tree, such as a time-centric merkle hash tree

SG11201803263WA

( en )

2015-10-21

2018-05-30

15 Seconds Of Fame Inc

Methods and apparatus for false positive minimization in facial recognition applications

US12276420B2

( en )

2016-02-03

2025-04-15

Strong Force Iot Portfolio 2016, Llc

Industrial internet of things smart heating systems and methods that produce and use hydrogen fuel

CN109074473B

( en )

*

2016-04-11

2020-04-24

北京市商汤科技开发有限公司

Method and system for object tracking

JP6528723B2

( en )

*

2016-05-25

2019-06-12

トヨタ自動車株式会社

Object recognition apparatus, object recognition method and program

US10261652B1

( en )

*

2016-07-07

2019-04-16

Northrop Grumman Systems Corporation

4D interactive mission analytics for visualization of unmanned vehicle performance indicators

EP3340624B1

( en )

2016-12-20

2019-07-03

Axis AB

Encoding a privacy masked image

CN110678877B

( en )

*

2017-03-16

2022-07-26

西门子股份公司

System and method for visual localization in test images

US11481740B1

( en )

*

2017-04-25

2022-10-25

EMC IP Holding Company LLC

Distributed ledger for peer-to-peer cloud data asset valuation

US11125461B2

( en )

2017-06-13

2021-09-21

Gerard O&#39;Hora

Smart vent system with local and central control

US11394573B2

( en )

2017-06-13

2022-07-19

SynCells, Inc.

Energy virtualization layer with a universal smart gateway

US11271766B2

( en )

*

2017-06-13

2022-03-08

SynCells, Inc.

Energy virtualization layer with a universal smart gateway

US10937083B2

( en )

2017-07-03

2021-03-02

Medici Ventures, Inc.

Decentralized trading system for fair ordering and matching of trades received at multiple network nodes and matched by multiple network nodes within decentralized trading system

US10761743B1

( en )

2017-07-17

2020-09-01

EMC IP Holding Company LLC

Establishing data reliability groups within a geographically distributed data storage environment

US10742750B2

( en )

*

2017-07-20

2020-08-11

Cisco Technology, Inc.

Managing a distributed network of function execution environments

US10817388B1

( en )

2017-07-21

2020-10-27

EMC IP Holding Company LLC

Recovery of tree data in a geographically distributed environment

US10880040B1

( en )

2017-10-23

2020-12-29

EMC IP Holding Company LLC

Scale-out distributed erasure coding

US10592767B2

( en )

*

2017-10-27

2020-03-17

Salesforce.Com, Inc.

Interpretable counting in visual question answering

CA3026837A1

( en )

*

2017-12-07

2019-06-07

Fifth Third Bancorp

Geospatial market analytics

US10540574B2

( en )

*

2017-12-07

2020-01-21

Shanghai Cambricon Information Technology Co., Ltd

Image compression method and related device

CN108062562B

( en )

*

2017-12-12

2020-03-10

北京图森未来科技有限公司

Object re-identification method and device

US10630769B2

( en )

*

2017-12-26

2020-04-21

Akamai Technologies, Inc.

Distributed system of record transaction receipt handling in an overlay network

EP3783477B1

( en )

2017-12-27

2021-11-17

Cambricon Technologies Corporation Limited

Integrated circuit chip device

US11349753B2

( en )

*

2017-12-28

2022-05-31

Intel Corporation

Converged routing for distributed computing systems

DE112018006630T5

( en )

*

2017-12-28

2020-09-24

Intel Corporation

VISUAL FOG

US10607484B2

( en )

*

2017-12-28

2020-03-31

Intel Corporation

Privacy-preserving distributed visual data processing

US11409576B2

( en )

*

2017-12-29

2022-08-09

Entefy Inc.

Dynamic distribution of a workload processing pipeline on a computing infrastructure

US10382554B1

( en )

2018-01-04

2019-08-13

Emc Corporation

Handling deletes with distributed erasure coding

US11227382B2

( en )

*

2018-01-11

2022-01-18

Intelinair, Inc.

Change detection system

WO2019143318A1

( en )

*

2018-01-16

2019-07-25

Hall David R

Health monitoring system including privacy-ensuring obfuscated camera images

CN110084068B

( en )

2018-01-26

2023-09-29

阿里巴巴集团控股有限公司

Blockchain system and data processing method for blockchain system

WO2019161229A1

( en )

2018-02-15

2019-08-22

DMAI, Inc.

System and method for reconstructing unoccupied 3d space

CN112204563A

( en )

*

2018-02-15

2021-01-08

得麦股份有限公司

System and method for visual scene construction based on user communication

WO2019161207A1

( en )

2018-02-15

2019-08-22

DMAI, Inc.

System and method for conversational agent via adaptive caching of dialogue tree

US10826902B1

( en )

*

2018-03-01

2020-11-03

The United States Of America As Represented By The Secretary Of The Air Force

Internet of things (IoT) identifying system and associated methods

US11265168B2

( en )

2018-03-07

2022-03-01

Private Identity Llc

Systems and methods for privacy-enabled biometric processing

US10938852B1

( en )

2020-08-14

2021-03-02

Private Identity Llc

Systems and methods for private authentication with helper networks

US11210375B2

( en )

2018-03-07

2021-12-28

Private Identity Llc

Systems and methods for biometric processing with liveness

US11489866B2

( en )

2018-03-07

2022-11-01

Private Identity Llc

Systems and methods for private authentication with helper networks

US11394552B2

( en )

2018-03-07

2022-07-19

Private Identity Llc

Systems and methods for privacy-enabled biometric processing

US11170084B2

( en )

2018-06-28

2021-11-09

Private Identity Llc

Biometric authentication

US11138333B2

( en )

2018-03-07

2021-10-05

Private Identity Llc

Systems and methods for privacy-enabled biometric processing

US10721070B2

( en )

*

2018-03-07

2020-07-21

Private Identity Llc

Systems and methods for privacy-enabled biometric processing

US11392802B2

( en )

2018-03-07

2022-07-19

Private Identity Llc

Systems and methods for privacy-enabled biometric processing

US11789699B2

( en )

2018-03-07

2023-10-17

Private Identity Llc

Systems and methods for private authentication with helper networks

US11514371B2

( en )

*

2018-03-13

2022-11-29

Woven Planet North America, Inc.

Low latency image processing using byproduct decompressed images

US10733448B2

( en )

*

2018-03-15

2020-08-04

Microsoft Technology Licensing, Llc

Machine learning of context data for social and contextual scene inferences

US11004565B2

( en )

*

2018-03-22

2021-05-11

Digital Diagnostics Inc.

Data aggregation, integration and analysis system and related devices and methods

US11308222B2

( en )

*

2018-03-22

2022-04-19

Via Science, Inc.

Neural-network training using secure data processing

US10965516B2

( en )

2018-03-27

2021-03-30

Cisco Technology, Inc.

Deep fusion reasoning engine (DFRE) for prioritizing network monitoring alerts

US10887197B2

( en )

2018-03-27

2021-01-05

Cisco Technology, Inc.

Deep fusion reasoning engine (DFRE) for dynamic and explainable wireless network QoE metrics

US10650616B2

( en )

*

2018-04-06

2020-05-12

University Of Connecticut

Fault diagnosis using distributed PCA architecture

US10771488B2

( en )

*

2018-04-10

2020-09-08

Cisco Technology, Inc.

Spatio-temporal anomaly detection in computer networks using graph convolutional recurrent neural networks (GCRNNs)

EP3553748A1

( en )

*

2018-04-10

2019-10-16

InterDigital VC Holdings, Inc.

Deep learning based image partitioning for video compression

US10817374B2

( en )

2018-04-12

2020-10-27

EMC IP Holding Company LLC

Meta chunks

US10642656B2

( en )

*

2018-04-17

2020-05-05

Cognizant Technology Solutions India Pvt. Ltd.

System and method for efficiently and securely managing a network using fog computing

US11429725B1

( en )

*

2018-04-26

2022-08-30

Citicorp Credit Services, Inc. (Usa)

Automated security risk assessment systems and methods

US10579297B2

( en )

2018-04-27

2020-03-03

EMC IP Holding Company LLC

Scaling-in for geographically diverse storage

US12353203B2

( en )

2018-05-07

2025-07-08

Strong Force Iot Portfolio 2016, Llc

Methods and systems for data collection, learning, and streaming of machine signals for analytics and maintenance using the industrial Internet of Things

CA3099783A1

( en )

2018-05-10

2019-11-14

Axon Enterprise, Inc.

Systems and methods for cross-redaction

US10949087B2

( en )

2018-05-15

2021-03-16

Samsung Electronics Co., Ltd.

Method for rapid reference object storage format for chroma subsampled images

US11449256B2

( en )

2018-05-15

2022-09-20

Samsung Electronics Co., Ltd.

Method for accelerating image storing and retrieving differential latency storage devices based on access rates

US12058214B2

( en )

*

2018-05-18

2024-08-06

Nant Holdings Ip, Llc

Fine grained network management to edge device features

US11157524B2

( en )

*

2018-05-18

2021-10-26

At&amp;T Intellectual Property I, L.P.

Automated learning of anomalies in media streams with external feed labels

US10355865B1

( en )

2018-05-24

2019-07-16

DeepTruth, LLC

Systems and techniques for certification of trusted media data

US10938568B2

( en )

*

2018-06-05

2021-03-02

Eight Plus Ventures, LLC

Image inventory production

US10606888B2

( en )

2018-06-05

2020-03-31

Eight Plus Ventures, LLC

Image inventory production

US11868321B2

( en )

*

2018-06-12

2024-01-09

Salesforce, Inc.

Cryptographically secure multi-tenant data exchange platform

US11023130B2

( en )

2018-06-15

2021-06-01

EMC IP Holding Company LLC

Deleting data in a geographically diverse storage construct

US10936196B2

( en )

2018-06-15

2021-03-02

EMC IP Holding Company LLC

Data convolution for geographically diverse storage

US10936920B2

( en )

*

2018-06-17

2021-03-02

Uber Technologies, Inc.

Determining geographical map features with multi-sensor input

EP3585008A1

( en )

*

2018-06-20

2019-12-25

Siemens Aktiengesellschaft

Cloud gateway device and method for operating a cloud gateway device

CN110633725B

( en )

*

2018-06-25

2023-08-04

富士通株式会社

Method and device for training classification model and classification method and device

US11070763B2

( en )

*

2018-06-27

2021-07-20

Snap-On Incorporated

Method and system for displaying images captured by a computing device including a visible light camera and a thermal camera

GB201811263D0

( en )

*

2018-07-10

2018-08-29

Netmaster Solutions Ltd

A method and system for managing digital using a blockchain

CN109117950B

( en )

*

2018-08-01

2021-03-09

上海天数智芯半导体有限公司

Layered sparse tensor compression method based on artificial intelligence equipment

US10694032B2

( en )

*

2018-08-05

2020-06-23

Michael Francis Byrne

Systems and methods for blockchain wireless services in a controlled environment

US11218907B2

( en )

*

2018-08-08

2022-01-04

Intel Corporation

Publisher control in an information centric network

DE102018213554A1

( en )

*

2018-08-10

2020-02-13

Audi Ag

Method and display device for visualizing an arrangement and mode of operation of environmental sensors of a motor vehicle

CN109359470B

( en )

2018-08-14

2020-09-01

阿里巴巴集团控股有限公司

Multi-party security calculation method and device and electronic equipment

CN109255247B

( en )

2018-08-14

2020-08-14

阿里巴巴集团控股有限公司

Multi-party secure computing method and device, electronic device

CN109241016B

( en )

2018-08-14

2020-07-07

阿里巴巴集团控股有限公司

Multi-party secure computing method and device, and electronic device

US10479356B1

( en )

2018-08-17

2019-11-19

Lyft, Inc.

Road segment similarity determination

US10467391B1

( en )

2018-08-23

2019-11-05

Eight Plus Ventures, LLC

Manufacture of secure printed image inventories

US11055854B2

( en )

*

2018-08-23

2021-07-06

Seoul National University R&amp;Db Foundation

Method and system for real-time target tracking based on deep learning

SG11202101513UA

( en )

*

2018-08-28

2021-03-30

Sohail Alavi

Systems and method for mapping and referencing physical geographic addresses to a dynamically linked identifier

US10936856B2

( en )

*

2018-08-31

2021-03-02

15 Seconds of Fame, Inc.

Methods and apparatus for reducing false positives in facial recognition

<a href

Related documents

• Comparison of Image Processing Models in Quark Gluon Jet Classification#1014601
• Applications of Neural Cellular Automata: State of the Art, Challenges and Opportunities#1051005
• Do Vision Model See Like the Brain? A Comparison Across EEG Encoding Model#1062402
• Do Vision Model See Like the Brain? A Comparison Across EEG Encoding Model#1043139
• Penerapan Algoritma Convolutional Neural Network untuk Meningkatkan Akurasi Pengenalan Bahasa Isyarat Indonesia#272515