ABSTRACT
Abstract
Techniques that facilitate feedback loop learning between artificial intelligence systems are provided. In one example, a system includes a monitoring component and a machine learning component. The monitoring component identifies a data pattern associated with data for an artificial intelligence system. The machine learning component compares the data pattern to historical data patterns for the artificial intelligence system to facilitate modification of at least a component of the artificial intelligence system and/or one or more dependent systems of the artificial intelligence system.
Description
BACKGROUND
The subject disclosure relates to data analytics systems, and more specifically, to learning associated with artificial intelligence for data analytics systems.
SUMMARY
The following presents a summary to provide a basic understanding of one or more embodiments of the invention. This summary is not intended to identify key or critical elements, or delineate any scope of the particular embodiments or any scope of the claims. Its sole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. In one or more embodiments described herein, devices, systems, computer-implemented methods, apparatus and/or computer program products that facilitate feedback loop learning between artificial intelligence systems are described.
According to an embodiment, a system can comprise a monitoring component and a machine learning component. The monitoring component can identify a data pattern associated with data for an artificial intelligence system. The machine learning component can compare the data pattern to historical data patterns for the artificial intelligence system to facilitate modification of a component of the artificial intelligence system. In certain embodiments, the machine learning component can compare the data pattern to historical data patterns for the artificial intelligence system to facilitate modification of at least a component of the artificial intelligence system and/or a dependent system associated with the artificial intelligence system.
According to another embodiment, a computer-implemented method is provided. The computer-implemented method can comprise monitoring, by a system operatively coupled to a processor, an artificial intelligence system to identify a data pattern associated with data for the artificial intelligence system. The computer-implemented method can also comprise comparing, by the system, the data pattern to historical data patterns for the artificial intelligence system.
According to yet another embodiment, a computer program product for facilitating feedback loop learning between artificial intelligence systems can comprise a computer readable storage medium having program instructions embodied therewith. The program instructions can be executable by a processor and cause the processor to monitor, by the processor, an artificial intelligence system to identify a data pattern associated with data for the artificial intelligence system. The program instructions can also cause the processor to compare, by the processor, the data pattern to historical data patterns for the artificial intelligence system. Furthermore, the program instructions can also cause the processor to modify, by the processor, one or more portions of the artificial intelligence system based on the data pattern and the historical data patterns.
DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates a block diagram of an example, non-limiting system that includes a data analytics component in accordance with one or more embodiments described herein.
FIG. 2 illustrates a block diagram of another example, non-limiting system that includes a data analytics component in accordance with one or more embodiments described herein.
FIG. 3 illustrates an example, non-limiting system that includes an electronic device and an artificial intelligence system in accordance with one or more embodiments described herein.
FIG. 4 illustrates an example, non-limiting system associated with feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein.
FIG. 5 illustrates an example, non-limiting system associated with a data analytics process in accordance with one or more embodiments described herein.
FIG. 6 illustrates another example, non-limiting system associated with feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein.
FIG. 7 illustrates yet another example, non-limiting system associated with feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein.
FIG. 8 illustrates an example, non-limiting system associated with a dependency mapping graph in accordance with one or more embodiments described herein.
FIG. 9 illustrates a flow diagram of an example, non-limiting computer-implemented method for facilitating feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein.
FIG. 10 illustrates a flow diagram of another example, non-limiting computer-implemented method for facilitating feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein.
FIG. 11 illustrates a block diagram of an example, non-limiting operating environment in which one or more embodiments described herein can be facilitated.
FIG. 12 illustrates a block diagram of an example, non-limiting cloud computing environment in accordance with one or more embodiments of the present invention.
FIG. 13 illustrates a block diagram of example, non-limiting abstraction model layers in accordance with one or more embodiments of the present invention.
DETAILED DESCRIPTION
The following detailed description is merely illustrative and is not intended to limit embodiments and/or application or uses of embodiments. Furthermore, there is no intention to be bound by any expressed or implied information presented in the preceding Background or Summary sections, or in the Detailed Description section.
One or more embodiments are now described with reference to the drawings, wherein like referenced numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a more thorough understanding of the one or more embodiments. It is evident, however, in various cases, that the one or more embodiments can be practiced without these specific details.
It is to be understood that although this disclosure includes a detailed description on cloud computing, implementation of the teachings recited herein are not limited to a cloud computing environment. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of computing environment now known or later developed.
Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.
Characteristics are as follows:
On-demand self-service: a cloud consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with the service's provider.
Broad network access: capabilities are available over a network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).
Resource pooling: the provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to demand There is a sense of location independence in that the consumer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter).
Rapid elasticity: capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time.
Measured service: cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service.
Service Models are as follows:
Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.
Platform as a Service (PaaS): the capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.
Infrastructure as a Service (IaaS): the capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).
Deployment Models are as follows:
Private cloud: the cloud infrastructure is operated solely for an organization. It may be managed by the organization or a third party and may exist on-premises or off-premises.
Community cloud: the cloud infrastructure is shared by several organizations and supports a specific community that has shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be managed by the organizations or a third party and may exist on-premises or off-premises.
Public cloud: the cloud infrastructure is made available to the general public or a large industry group and is owned by an organization selling cloud services.
Hybrid cloud: the cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load-balancing between clouds).
A cloud computing environment is service oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.
In this regard, artificial intelligence (AI) service systems often have a separation between offline learning/training and online prediction. Due to the separation between offline learning/training and online prediction in an AI service system, online machine learning and/or incremental learning is generally difficult. For instance, an AI service system is generally updated based on generated models and/or new use cases. However, due to the separation between offline learning/training and online prediction, it is generally difficult to cohesively update offline learning/training and online prediction. In an example, a development system for an AI service system and an operation system for an AI service system are generally separated and execute AI processes with minimal communication. The development system generally is not informed when new data should be ingested to train a new model. Furthermore, timing for offline learning/training is often determined when there are new use cases generated by the operations system. A new use case in an AI service systems generally requires a new implementation to facilitate execution in by the operations system. In certain implementations, the operations system can directly monitor a model. For example, the operations system can monitor how an accuracy rate changes over time for the model. However, the operations system generally is not able to adequately update the model based on data generated by the development system develop due to lack of process and/or communication between the operations system and the development system. As such, learning and/or training by an AI system (e.g., an AI service system) can be improved.
Embodiments described herein include systems, computer-implemented methods, and computer program products that provide feedback loop learning between artificial intelligence systems. For example, feedback loop learning between a development system for an artificial intelligence system and an operations system for the artificial intelligence systems can be provided. In an embodiment, data associated with an artificial intelligence system can be collected and/or analyzed. In an example, one or more events associated with the artificial intelligence system, one or more communications (e.g., one or more application programming interface (API) communications) associated with the artificial intelligence system, one or more execution results associated with the artificial intelligence system, and/or other data associated with the artificial intelligence system can be collected and/or analyzed. Based on the data associated with the artificial intelligence system, deviation from an existing model of the artificial intelligence system can be inferred. For example, deviation from one or more patterns (e.g., one or more communication patterns) for an existing model of the artificial intelligence system can be inferred based on the data associated with the artificial intelligence system. Furthermore, the one or more patterns can be provided to a development system of the artificial intelligence system via a communication channel. For instance, a feedback loop associated with communication channel for a distributed version control system can provide the one or more patterns to the development system of the artificial intelligence system. Additionally, online machine learning can be employed to determine whether or not there are new potential classes (e.g. new use cases) for the artificial intelligence system. The data associated with the new potential classes (e.g., the new use cases) can additionally or alternatively be provided to the development system of the artificial intelligence system via the communication channel. In an embodiment, active learning can be employed to enable determining need for component changes in the artificial intelligence system based on data patterns associated with operations of the artificial intelligence system. Data associated with the artificial
BACKGROUND
The subject disclosure relates to data analytics systems, and more specifically, to learning associated with artificial intelligence for data analytics systems.
SUMMARY
The following presents a summary to provide a basic understanding of one or more embodiments of the invention. This summary is not intended to identify key or critical elements, or delineate any scope of the particular embodiments or any scope of the claims. Its sole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. In one or more embodiments described herein, devices, systems, computer-implemented methods, apparatus and/or computer program products that facilitate feedback loop learning between artificial intelligence systems are described.
According to an embodiment, a system can comprise a monitoring component and a machine learning component. The monitoring component can identify a data pattern associated with data for an artificial intelligence system. The machine learning component can compare the data pattern to historical data patterns for the artificial intelligence system to facilitate modification of a component of the artificial intelligence system. In certain embodiments, the machine learning component can compare the data pattern to historical data patterns for the artificial intelligence system to facilitate modification of at least a component of the artificial intelligence system and/or a dependent system associated with the artificial intelligence system.
According to another embodiment, a computer-implemented method is provided. The computer-implemented method can comprise monitoring, by a system operatively coupled to a processor, an artificial intelligence system to identify a data pattern associated with data for the artificial intelligence system. The computer-implemented method can also comprise comparing, by the system, the data pattern to historical data patterns for the artificial intelligence system.
According to yet another embodiment, a computer program product for facilitating feedback loop learning between artificial intelligence systems can comprise a computer readable storage medium having program instructions embodied therewith. The program instructions can be executable by a processor and cause the processor to monitor, by the processor, an artificial intelligence system to identify a data pattern associated with data for the artificial intelligence system. The program instructions can also cause the processor to compare, by the processor, the data pattern to historical data patterns for the artificial intelligence system. Furthermore, the program instructions can also cause the processor to modify, by the processor, one or more portions of the artificial intelligence system based on the data pattern and the historical data patterns.
DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates a block diagram of an example, non-limiting system that includes a data analytics component in accordance with one or more embodiments described herein.
FIG. 2 illustrates a block diagram of another example, non-limiting system that includes a data analytics component in accordance with one or more embodiments described herein.
FIG. 3 illustrates an example, non-limiting system that includes an electronic device and an artificial intelligence system in accordance with one or more embodiments described herein.
FIG. 4 illustrates an example, non-limiting system associated with feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein.
FIG. 5 illustrates an example, non-limiting system associated with a data analytics process in accordance with one or more embodiments described herein.
FIG. 6 illustrates another example, non-limiting system associated with feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein.
FIG. 7 illustrates yet another example, non-limiting system associated with feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein.
FIG. 8 illustrates an example, non-limiting system associated with a dependency mapping graph in accordance with one or more embodiments described herein.
FIG. 9 illustrates a flow diagram of an example, non-limiting computer-implemented method for facilitating feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein.
FIG. 10 illustrates a flow diagram of another example, non-limiting computer-implemented method for facilitating feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein.
FIG. 11 illustrates a block diagram of an example, non-limiting operating environment in which one or more embodiments described herein can be facilitated.
FIG. 12 illustrates a block diagram of an example, non-limiting cloud computing environment in accordance with one or more embodiments of the present invention.
FIG. 13 illustrates a block diagram of example, non-limiting abstraction model layers in accordance with one or more embodiments of the present invention.
DETAILED DESCRIPTION
The following detailed description is merely illustrative and is not intended to limit embodiments and/or application or uses of embodiments. Furthermore, there is no intention to be bound by any expressed or implied information presented in the preceding Background or Summary sections, or in the Detailed Description section.
One or more embodiments are now described with reference to the drawings, wherein like referenced numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a more thorough understanding of the one or more embodiments. It is evident, however, in various cases, that the one or more embodiments can be practiced without these specific details.
It is to be understood that although this disclosure includes a detailed description on cloud computing, implementation of the teachings recited herein are not limited to a cloud computing environment. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of computing environment now known or later developed.
Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.
Characteristics are as follows:
On-demand self-service: a cloud consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with the service's provider.
Broad network access: capabilities are available over a network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).
Resource pooling: the provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to demand There is a sense of location independence in that the consumer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter).
Rapid elasticity: capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time.
Measured service: cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service.
Service Models are as follows:
Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.
Platform as a Service (PaaS): the capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.
Infrastructure as a Service (IaaS): the capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).
Deployment Models are as follows:
Private cloud: the cloud infrastructure is operated solely for an organization. It may be managed by the organization or a third party and may exist on-premises or off-premises.
Community cloud: the cloud infrastructure is shared by several organizations and supports a specific community that has shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be managed by the organizations or a third party and may exist on-premises or off-premises.
Public cloud: the cloud infrastructure is made available to the general public or a large industry group and is owned by an organization selling cloud services.
Hybrid cloud: the cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load-balancing between clouds).
A cloud computing environment is service oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.
In this regard, artificial intelligence (AI) service systems often have a separation between offline learning/training and online prediction. Due to the separation between offline learning/training and online prediction in an AI service system, online machine learning and/or incremental learning is generally difficult. For instance, an AI service system is generally updated based on generated models and/or new use cases. However, due to the separation between offline learning/training and online prediction, it is generally difficult to cohesively update offline learning/training and online prediction. In an example, a development system for an AI service system and an operation system for an AI service system are generally separated and execute AI processes with minimal communication. The development system generally is not informed when new data should be ingested to train a new model. Furthermore, timing for offline learning/training is often determined when there are new use cases generated by the operations system. A new use case in an AI service systems generally requires a new implementation to facilitate execution in by the operations system. In certain implementations, the operations system can directly monitor a model. For example, the operations system can monitor how an accuracy rate changes over time for the model. However, the operations system generally is not able to adequately update the model based on data generated by the development system develop due to lack of process and/or communication between the operations system and the development system. As such, learning and/or training by an AI system (e.g., an AI service system) can be improved.
Embodiments described herein include systems, computer-implemented methods, and computer program products that provide feedback loop learning between artificial intelligence systems. For example, feedback loop learning between a development system for an artificial intelligence system and an operations system for the artificial intelligence systems can be provided. In an embodiment, data associated with an artificial intelligence system can be collected and/or analyzed. In an example, one or more events associated with the artificial intelligence system, one or more communications (e.g., one or more application programming interface (API) communications) associated with the artificial intelligence system, one or more execution results associated with the artificial intelligence system, and/or other data associated with the artificial intelligence system can be collected and/or analyzed. Based on the data associated with the artificial intelligence system, deviation from an existing model of the artificial intelligence system can be inferred. For example, deviation from one or more patterns (e.g., one or more communication patterns) for an existing model of the artificial intelligence system can be inferred based on the data associated with the artificial intelligence system. Furthermore, the one or more patterns can be provided to a development system of the artificial intelligence system via a communication channel. For instance, a feedback loop associated with communication channel for a distributed version control system can provide the one or more patterns to the development system of the artificial intelligence system. Additionally, online machine learning can be employed to determine whether or not there are new potential classes (e.g. new use cases) for the artificial intelligence system. The data associated with the new potential classes (e.g., the new use cases) can additionally or alternatively be provided to the development system of the artificial intelligence system via the communication channel. In an embodiment, active learning can be employed to enable determining need for component changes in the artificial intelligence system based on data patterns associated with operations of the artificial intelligence system. Data associated with the artificial intelligence system and/or the operations of the artificial intelligence system can be obtained in approximately real-time. In certain embodiments, user feedback data can additionally or alternatively be employed via a learning engine of the artificial intelligence system. As such, an operation of the artificial intelligence system can obtain knowledge regarding for updating and/or developing a new feature for the artificial intelligence system. Furthermore, an inline microservice of the artificial intelligence system can be provided that monitors an environment of the artificial intelligence system. A learning model can also be provided to identify and/or learn when to update and/or develop a new feature for the artificial intelligence system. The new feature can also be provided to the development system of the artificial intelligence system via a feedback loop of the artificial intelligence system. Additionally or alternatively, additional new information associated with the artificial intelligence system can be provided to the development system of the artificial intelligence system via the feedback loop of the artificial intelligence system.
As such, a technique to detect a need for artificial intelligence component changes can be provided. Communication driven monitoring capability for artificial intelligence systems can also be provided to detect and/or infer communication pattern changes in the artificial intelligence system. Additionally, an ability to detect and/or infer data pattern changes via communication traffics can be provided. Relational service component changes can also be detected to facilitate feedback of dependency and/or conformation of communications between the artificial intelligence system and one or more dependent systems of the artificial intelligence system, where dependent systems should be updated together with a new model and/or a new use case. New classes (e.g., untrained classes) can also be dynamically inferred based on observed data for the artificial intelligence system that is collected over time. In addition, an indication for a need to update artificial intelligence components can be provided to a development system to update and/or train respective artificial intelligence components for the changes. Accordingly, performance and/or accuracy of an artificial intelligence system can be improved. Changes to different portions of an artificial intelligence system can also be provided with conformity. Moreover, accuracy of data generated by a machine learning process can be improved, quality of data generated by a machine learning process can be improved, speed of data generated by a machine learning process can be improved, and/or a cost for analyzing data using a machine learning process can be reduced. Accuracy and/or quality of a machine learning model and/or training data associated with an artificial intelligence system can also be provided.
FIG. 1 illustrates a block diagram of an example, non-limiting system 100 that facilitates feedback loop learning between artificial intelligence systems in accordance with one or more embodiments described herein. In various embodiments, the system 100 can be a system associated with technologies such as, but not limited to, artificial intelligence technologies, machine learning technologies, data analytics technologies, cognitive computing technologies, cloud computing technologies, computer technologies, server technologies, server/client technologies, internet technology (IT) technologies, information technologies, digital technologies, data processing technologies, and/or other computer technologies. The system 100 can employ hardware and/or software to solve problems that are highly technical in nature, that are not abstract and that cannot be performed as a set of mental acts by a human. Further, some of the processes performed may be performed by one or more specialized computers (e.g., one or more specialized processing units, a specialized computer with a data analytics component, etc.) for carrying out defined tasks related to machine learning and/or data analytics. The system 100 and/or components of the system can be employed to solve new problems that arise through advancements in technologies mentioned above, and/or computer architecture, and the like. One or more embodiments of the system 100 can provide technical improvements to artificial intelligence systems, machine learning systems, data analytics systems, cognitive computing systems, cloud computing systems, computer systems, server systems, server/client systems, IT systems, information systems, digital systems, data processing systems, and/or other systems. One or more embodiments of the system 100 can also provide technical improvements to a processing unit (e.g., a processor) associated with a data analytics process by improving processing performance of the processing unit, improving processing efficiency of the processing unit, and/or reducing an amount of time for the processing unit to perform a data analytics process. One or more embodiments of the system 100 can also provide technical improvements to a server/client computing environment (e.g., a server/client computing platform) by improving processing performance of the server/client computing environment and/or improving processing efficiency of the server/client computing environment. In one example, the system 100 can be associated with a machine learning process and/or a data analytics process.
In the embodiment shown in FIG. 1 , the system 100 can include a data analytics component 102 . As shown in FIG. 1 , the data analytics component 102 can include a monitoring component 104 and a machine learning component 106 . Aspects of the data analytics component 102 can constitute machine-executable component(s) embodied within machine(s), e.g., embodied in one or more computer readable mediums (or media) associated with one or more machines. Such component(s), when executed by the one or more machines, e.g., computer(s), computing device(s), virtual machine(s), etc. can cause the machine(s) to perform the operations described. In an aspect, the data analytics component 102 can also include memory 108 that stores computer executable components and instructions. Furthermore, the data analytics component 102 can include a processor 110 to facilitate execution of the instructions (e.g., computer executable components and corresponding instructions) by the data analytics component 102 . As shown, the monitoring component 104 , the machine learning component 106 , the memory 108 and/or the processor 110 can be electrically and/or communicatively coupled to one another in one or more embodiments.
The data analytics component 102 (e.g., the monitoring component 104 of the data analytics component 102 ) can receive data 112 . For example, the data analytics component 102 (e.g., the monitoring component 104 of the data analytics component 102 ) can receive the data 112 in response to monitoring by the monitoring component 104 . The monitoring component 104 can, for example, monitor one or more portions of an artificial intelligence system. As such, the data 112 can be related to the artificial intelligence system. The data 112 can include, for example, data related to one or more events associated with the artificial intelligence system, data related to one or more communications (e.g., one or more API communications) associated with the artificial intelligence system, data related to one or more execution results associated with the artificial intelligence system, and/or other data associated with the artificial intelligence system. The artificial intelligence system can employ one or more artificial intelligence techniques and/or one or more machine learning techniques in a distributed computing environment. For instance, a first portion of the artificial intelligence system can be associated with learning/training (e.g., offline learning/training) and a second portion of the artificial intelligence system can be associated with prediction (e.g., online prediction). In an embodiment, the artificial intelligence system can be an artificial intelligence service system. For instance, the artificial intelligence system can manage a set of artificial intelligence components for a microservice. An artificial intelligence component can be, for example, a system that provides supervised learning to map input to output via an artificial intelligence model. In certain embodiments, the artificial intelligence system can include a development system (e.g., a first artificial intelligence system) and an operations system (e.g., a second artificial intelligence system) to facilitate one or more development processes and/or one or more operations processes associated with machine learning. In certain embodiments, the data 112 can be generated by one or more electronic devices. Additionally or alternatively, the data 112 can be stored in one or more databases that receives and/or stores the data 112 associated with the one or more electronic devices. The one or more electronic devices can include, for example, one or more computing devices, one or more computers, one or more desktop computers, one or more laptop computers, one or more monitor devices, one or more smart devices, one or more smart phones, one or more mobile devices, one or more handheld devices, one or more tablets, one or more wearable devices, one or more portable computing devices, one or more medical devices, one or more sensor devices, one or more controller devices, and/or or one or more other computing devices. In an aspect, the data 112 can be digital data. Furthermore, the data 112 can include one or more types of data, such as but not limited to, electronic device data, sensor data, network data, metadata, user data, geolocation data, wearable device data, health-related data, medical imaging data, audio data, image data, video data, textual data and/or other data. In an embodiment, the data 112 can be raw data. In another embodiment, at least a portion of the data 112 can be encoded data and/or processed data.
In an embodiment, the monitoring component 104 can identify a data pattern associated with the data 112 . For example, the monitoring component 104 can monitor the data 112 to identity one or more data patterns in the data 112 . In certain embodiments, the monitoring component 104 can monitor one or more communication channels associated with the artificial intelligence system. For example, the monitoring component 104 can monitor one or more communications (e.g., one or more API communications) associated with the artificial intelligence system. In certain embodiments, the monitoring component 104 can monitor API requests, API responses and/or logs associated with the artificial intelligence system. In another example, the monitoring component 104 can monitor input data provided to one or more artificial intelligence components of the artificial intelligence system. Additionally or alternatively, the monitoring component 104 can monitor output data provided by one or more artificial intelligence components of the artificial intelligence system. In certain embodiments, the monitoring component 104 can monitor one or more events associated with the artificial intelligence system. For example, the monitoring component 104 can monitor one or more events associated with one or more artificial intelligence components of the artificial intelligence system. Additionally or alternatively, the monitoring component 104 can monitor one or more events associated with a server, middleware, an application and/or another component of the artificial intelligence system. In another example, the monitoring component 104 can monitor one or more events associated with different artificial intelligence subsystems of the artificial intelligence system. In yet another example, the monitoring component 104 can monitor one or more events associated with an operations system associated with the artificial intelligence system and/or a development system associated with the artificial intelligence system. In certain embodiments, the monitoring component 104 can monitor accuracy of output data generated by one or more components of the artificial intelligence system. For example, the monitoring component 104 can monitor accuracy of output data generated by one or more artificial intelligence components of the artificial intelligence system. In certain embodiments, the monitoring component 104 can employ one or more correctness inference techniques to augment data monitored by the monitoring component 104 . For example the monitoring component 104 can employ similarity metrics for input data provided to an artificial intelligence component to infer whether output data provided by the artificial intelligence component is acceptable or not acceptable. In certain embodiments, the monitoring component 104 can employ one or more similarity measure techniques such as Euclidean distance, Manhattan distance, Cosine similarity and/or another technique to facilitate monitoring of the data 112 . In certain embodiments, the monitoring component 104 can employ temporal information and/or spatial information associated with the data 112 to facilitate monitoring of the data 112 . In certain embodiments, the monitoring component 104 can employ unsupervised learning to learn over time how the data 112 changes over time and/or to group the data 112 into a set of clusters to facilitate identifying data patterns in the data 112 . In an aspect, the monitoring component 104 can compare one or more portions of the data 112 to a set of previously defined data patterns. For instance, the monitoring component 104 can compare one or more portions of the data 112 to a set of previously defined data patterns stored in the memory 108 or another data store associated with the data analytics component 102 . A data pattern can be, for example, a digital data pattern associated with a classification. For example, a data pattern can be a defined arrangement of one or more characteristics of data. In certain embodiments, the monitoring component 104 can identify a data pattern associated with the data 112 based on a dependency mapping graph for the artificial intelligence system. The dependency mapping graph can be, for example, a graph that represents dependencies of two or more components (e.g., two or more artificial intelligence components). For example, the dependency mapping graph can provide a mapping of components (e.g., artificial intelligence components) represented by vertices connected by edges associated with relationship among the components (e.g., artificial intelligence components). Additionally or alternatively, the dependency mapping graph can be, for example, a graph that represents dependencies of two or more data elements. For example, the dependency mapping graph can provide a mapping of data elements represented by vertices connected by edges associated with relationship among the data elements.
The machine learning component 106 can compare, using one or more machine learning techniques, the data pattern to historical data patterns for the artificial intelligence system. For example, the machine learning component 106 can determine a classification for the data pattern based on the historical data patterns for the artificial intelligence system. In an embodiment, the machine learning component 106 can infer a new classification for one or more components of the artificial intelligence system. For instance, the machine learning component 106 can dynamically infer one or more new classes for one or more components of the artificial intelligence system based on, for example, observed data for the artificial intelligence system collected over a period of time. In an aspect, the machine learning component 106 can compare the data pattern to the historical data patterns to facilitate modification of one or more components of the artificial intelligence system. In an example, the machine learning component 106 can compare the data pattern to the historical data patterns to facilitate updating one or more portions of a model associated with the artificial intelligence system. In an example, the machine learning component 106 can compare the data pattern to the historical data patterns to facilitate updating one or more portions of source code associated with the artificial intelligence system. The machine learning component 106 can employ machine learning and/or principles of artificial intelligence (e.g., one or more machine learning processes) to compare the data pattern to the historical data patterns. The machine learning component 106 can perform learning explicitly or implicitly with respect to learning one or more patterns, one or more inferences, one or more correlations, one or more features and/or information related to the data 112 and/or a data pattern associated with the data 112 . In an aspect, the machine learning component 106 can learn one or more patterns, one or more inferences, one or more correlations, one or more features and/or information related to the data 112 and/or a data pattern associated with the data 112 based on classifications, correlations, inferences and/or expressions associated with principles of artificial intelligence. For instance, the machine learning component 106 can employ an automatic classification system and/or an automatic classification process to learn one or more patterns, one or more inferences, one or more correlations, one or more features and/or information related to the data 112 and/or a data pattern associated with the data 112 . In one example, the machine learning component 106 can employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to learn and/or generate inferences with respect to the data 112 and/or a data pattern associated with the data 112 . In an aspect, the machine learning component 106 can include an inference component (not shown) that can further enhance automated aspects of the machine learning component 106 utilizing in part inference-based schemes to learn one or more patterns, one or more inferences, one or more correlations, one or more features and/or information related to the data 112 and/or a data pattern associated with the data 112 .
The machine learning component 106 can employ any suitable machine-learning based techniques, statistical-based techniques and/or probabilistic-based techniques. For example, the machine learning component 106 can employ deep learning, expert systems, fuzzy logic, SVMs, Hidden Markov Models (HMMs), greedy search algorithms, rule-based systems, Bayesian models (e.g., Bayesian networks), neural networks, other non-linear training techniques, data fusion, utility-based analytical systems, systems employing Bayesian models, etc. In another aspect, the machine learning component 106 can perform a set of machine learning computations associated with learning one or more patterns, one or more inferences, one or more correlations, one or more features and/or information related to the data 112 and/or a data pattern associated with the data 112 . For example, the machine learning component 106 can perform a set of clustering machine learning computations, a set of logistic regression machine learning computations, a set of decision tree machine learning computations, a set of random forest machine learning computations, a set of regression tree machine learning computations, a set of least square machine learning computations, a set of instance-based machine learning computations, a set of regression machine learning computations, a set of support vector regression machine learning computations, a set of k-means machine learning computations, a set of spectral clustering machine learning computations, a set of rule learning machine learning computations, a set of Bayesian machine learning computations, a set of deep Boltzmann machine computations, a set of deep belief network computations, and/or a set of different machine learning computations to learn one or more patterns, one or more inferences, one or more correlations, one or more features and/or information related to the data 112 and/or a data pattern associated with the data 112 . In an embodiment, the machine learning component 106 can generate feedback data 114 . The feedback data 114 can be data to be provided to one or more components of the artificial intelligence system. Furthermore, the feedback data 114 can include, for example, a classification for the data pattern associated with the data 112 . Additionally or alternatively, the feedback data 114 can include data associated with one or more system updates for the artificial intelligence system. For example, the feedback data 114 can include data to modify one or more components of the artificial intelligence system.
It is to be appreciated that the data analytics component 102 (e.g., the monitoring component 104 and/or the machine learning component 106 ) performs a machine learning process and/or a data analytics process that cannot be performed by a human (e.g., is greater than the capability of a single human mind). For example, an amount of data processed, a speed of processing of data and/or data types processed by the data analytics component 102 (e.g., the monitoring component 104 and/or the machine learning component 1
CLAIMS
Claims ( 20 )
What is claimed is:
1. A system, comprising:
a memory that stores computer executable components; and
a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise:
a data analytics component that, based on an analysis of a microservices mesh, constructs a dependency mapping graph for artificial intelligence components of the microservices mesh, wherein the nodes of dependency mapping graph represent the artificial intelligence components, and edges of the dependency mapping graph represent respective dependencies between the artificial intelligence components;
a monitoring component that identifies, in real time, a data pattern associated with data generated during execution of the artificial intelligence components of the microservices mesh in a runtime environment of an artificial intelligence system;
a machine learning component that employs machine learning to:
determine, in real time, a deviation of the data pattern from one or more historical data patterns associated with one or more prior executions of the artificial intelligence components in the runtime environment of the artificial intelligence system, wherein the deviation indicates a need for a modification of at least one corresponding artificial intelligence component, in a development environment of the artificial intelligence system, that corresponds to at least one artificial intelligence component of the artificial intelligence components, and wherein the deviation comprises a new class of data representative of a new use case for the artificial intelligence components, and
determine, based on the dependency mapping graph, a need for another modification of at least one other artificial intelligence component that depends on the at least one corresponding artificial intelligence component based on the modification of at least one corresponding artificial intelligence component; and
a development component that implements the modification of the at least one corresponding artificial intelligence component and the other modification of the at least one other artificial intelligence component in the development environment of the artificial intelligence system.
2. The system of claim 1 , wherein the monitoring component identifies the data pattern associated with the data based on the dependency mapping graph for the artificial intelligence components.
3. The system of claim 1 , wherein the monitoring component monitors one or more application programming interface communications in the data associated with the artificial intelligence components.
4. The system of claim 1 , wherein the monitoring component monitors one or more events in the data associated with the artificial intelligence components.
5. The system of claim 1 , wherein the monitoring component monitors accuracy of output data in the data generated by the at least one artificial intelligence component of the artificial intelligence components.
6. The system of claim 1 , wherein the machine learning component infers a new classification for the at least one corresponding artificial intelligence component.
7. The system of claim 1 , wherein the development component that trains the at least one corresponding artificial intelligence component based on the data pattern.
8. The system of claim 1 , wherein the development component updates respective artificial intelligence models of the at least one corresponding artificial intelligence component based on the data pattern.
9. The system of claim 1 , wherein the machine learning component compares the data pattern to the one or more historical data patterns to improve performance of the artificial intelligence system.
10. A computer-implemented method, comprising:
based on an analysis of a microservices mesh, constructing, by a system operatively coupled to a processor, a dependency mapping graph for artificial intelligence components of the microservices mesh, wherein the nodes of dependency mapping graph represent the artificial intelligence components, and edges of the dependency mapping graph represent respective dependencies between the artificial intelligence components;
identifying, by the system, in real time, a data pattern associated with data generated during execution of artificial intelligence components of the microservices mesh in a runtime environment of a the artificial intelligence system; and
determining, by the system, using machine learning, in real time, a deviation of the data pattern from one or more historical data patterns associated with one or more prior executions of the artificial intelligence components in the runtime environment of the artificial intelligence system, wherein the deviation indicates a need for a modification of at least one corresponding artificial intelligence component, in a development environment of the artificial intelligence system, that corresponds to at least one artificial intelligence component of the artificial intelligence components, and wherein the deviation comprises a new class of data representative of a new use case for the artificial intelligence components;
determining, by the system, based on the dependency mapping graph, a need for another modification of at least one other artificial intelligence component that depends on the at least one corresponding artificial intelligence component based on the modification of at least one corresponding artificial intelligence component; and
implementing, by the system, the modification of the at least one corresponding artificial intelligence component and the other modification of the at least one other artificial intelligence component in the development environment of the artificial intelligence system.
11. The computer-implemented method of claim 10 , further comprising:
inferring, by the system, a new classification for the at least one corresponding artificial intelligence component based on the comparing.
12. The computer-implemented method of claim 10 , wherein the modification comprises:
modifying, by the system, one or more weights of the at least one corresponding artificial intelligence component.
13. The computer-implemented method of claim 10 , wherein the modification comprises:
modifying, by the system, one or more respective models associated with the at least one corresponding artificial intelligence component.
14. The computer-implemented method of claim 10 , wherein the modification comprises:
modifying, by the system, respective training data associated with the at least one corresponding artificial intelligence component.
15. The computer-implemented method of claim 10 , wherein the identifying the data pattern is based on the dependency mapping graph for the artificial intelligence components.
16. A computer program product facilitating feedback loop learning for an artificial intelligence system, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
based on an analysis of a microservices mesh, constructing, by the processor, a dependency mapping graph for artificial intelligence components of the microservices mesh, wherein the nodes of dependency mapping graph represent the artificial intelligence components, and edges of the dependency mapping graph represent respective dependencies between the artificial intelligence components;
identify, by the processor, in real time, a data pattern associated with data generated during execution of artificial intelligence components of the microservices mesh in a runtime environment of the artificial intelligence system;
determine, by the processor, using machine learning, in real time, a deviation of the data pattern from one or more historical data patterns associated with one or more prior executions of the artificial intelligence components in the runtime environment of the artificial intelligence system, wherein the deviation indicates a need for a modification of at least one corresponding artificial intelligence component, in a development environment of the artificial intelligence system, that corresponds to at least one artificial intelligence component of the artificial intelligence components, and wherein the deviation comprises a new class of data representative of a new use case for the artificial intelligence components;
determining, by the processor, based on the dependency mapping graph, a need for another modification of at least one other artificial intelligence component that depends on the at least one corresponding artificial intelligence component based on the modification of at least one corresponding artificial intelligence component; and
implement, by the processor, the modification of the at least one corresponding artificial intelligence component and the other modification of the at least one other artificial intelligence component in the development environment of the artificial intelligence system.
17. The computer program product of claim 16 , wherein the modification comprises:
modify, by the processor, one or more weights of the at least one corresponding artificial intelligence component.
18. The computer program product of claim 16 , wherein the modification comprises:
modify, by the processor, one or more respective artificial intelligence models of the at least one corresponding artificial intelligence component.
19. The computer program product of claim 16 , wherein the modifying comprises:
modification, by the processor, respective training data for the at least one corresponding artificial intelligence component.
20. The computer program product of claim 16 , wherein the identifying the data pattern is based on the dependency mapping graph for the artificial intelligence components.
US16/521,185
2019-07-24
2019-07-24
Feedback loop learning between artificial intelligence systems
Active
2041-08-31
US11720826B2
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US16/521,185
US11720826B2
( en )
2019-07-24
2019-07-24
Feedback loop learning between artificial intelligence systems
Applications Claiming Priority (1)
Application Number
Priority Date
Filing Date
Title
US16/521,185
US11720826B2
( en )
2019-07-24
2019-07-24
Feedback loop learning between artificial intelligence systems
Publications (2)
Publication Number
Publication Date
US20210027136A1
US20210027136A1 ( en )
2021-01-28
US11720826B2
true
US11720826B2 ( en )
2023-08-08
Family
ID=74189503
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US16/521,185
Active
2041-08-31
US11720826B2
( en )
2019-07-24
2019-07-24
Feedback loop learning between artificial intelligence systems
Country Status (1)
Country
Link
US
( 1 )
US11720826B2
( en )
Families Citing this family (21)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US11178065B2
( en )
*
2019-08-07
2021-11-16
Oracle International Corporation
System and methods for optimal allocation of multi-tenant platform infrastructure resources
US12619414B2
( en )
*
2019-09-09
2026-05-05
Red Hat, Inc.
Artificial intelligence-powered diagram-to-deployment converter
US11769587B2
( en )
*
2019-10-08
2023-09-26
GE Precision Healthcare LLC
Systems and methods to configure, program, and personalize a medical device using a digital assistant
WO2021167659A1
( en )
*
2019-11-14
2021-08-26
Trideum Corporation
Systems and methods of monitoring and controlling remote assets
US11550652B1
( en )
*
2019-11-14
2023-01-10
Amazon Technologies, Inc.
Integrated remediation system for network-based services
US12230406B2
( en )
2020-07-13
2025-02-18
Vignet Incorporated
Increasing diversity and engagement in clinical trails through digital tools for health data collection
US12277507B2
( en )
*
2021-01-22
2025-04-15
International Business Machines Corporation
Factchecking artificial intelligence models using blockchain
US11281553B1
( en )
2021-04-16
2022-03-22
Vignet Incorporated
Digital systems for enrolling participants in health research and decentralized clinical trials
US11586524B1
( en )
*
2021-04-16
2023-02-21
Vignet Incorporated
Assisting researchers to identify opportunities for new sub-studies in digital health research and decentralized clinical trials
US11789837B1
( en )
*
2021-02-03
2023-10-17
Vignet Incorporated
Adaptive data collection in clinical trials to increase the likelihood of on-time completion of a trial
US12211594B1
( en )
2021-02-25
2025-01-28
Vignet Incorporated
Machine learning to predict patient engagement and retention in clinical trials and increase compliance with study aims
US12248383B1
( en )
2021-02-25
2025-03-11
Vignet Incorporated
Digital systems for managing health data collection in decentralized clinical trials
US12248384B1
( en )
2021-02-25
2025-03-11
Vignet Incorporated
Accelerated clinical trials using patient-centered, adaptive digital health tools
WO2023021553A1
( en )
*
2021-08-16
2023-02-23
æ¥æ¬é»ä¿¡é»è©±æ ªå¼ä¼ç¤¾
Pattern extraction visualization device, pattern extraction visualization method, and pattern extraction visualization program
US12346780B2
( en )
*
2021-10-04
2025-07-01
Kyndryl, Inc.
Artificial intelligence module communication
US20230153725A1
( en )
*
2021-11-18
2023-05-18
Cisco Technology, Inc.
Techniques for determining service risks and causes
US11729082B2
( en )
2021-11-18
2023-08-15
Cisco Technology, Inc.
Techniques for providing inter-cluster dependencies
US12619479B2
( en )
*
2022-01-04
2026-05-05
International Business Machines Corporation
Artificial intelligence model management
US12474926B2
( en )
*
2022-12-14
2025-11-18
Advanced Micro Devices, Inc.
Fused data generation and associated communication
WO2025160415A1
( en )
*
2024-01-26
2025-07-31
Strong Force Iot Portfolio 2016, Llc
Systems, methods, devices, and platforms for industrial internet of things
US12493802B2
( en )
*
2024-03-05
2025-12-09
Squirrel Applications Llc
System and method for cross-platform personalized information management
Citations (21)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
WO2005033934A2
( en )
2003-10-02
2005-04-14
Siemens Aktiengesellschaft
Flexible software update for automation systems via internet
US20070074149A1
( en )
2005-08-26
2007-03-29
Microsoft Corporation
Automated product defects analysis and reporting
US20070094168A1
( en )
*
2005-07-29
2007-04-26
The Florida International University Board Of Trustees
Artificial neural network design and evaluation tool
US20090210368A1
( en )
2008-02-14
2009-08-20
Ebay Inc.
System and method for real time pattern identification
US7636698B2
( en )
*
2006-03-16
2009-12-22
Microsoft Corporation
Analyzing mining pattern evolutions by comparing labels, algorithms, or data patterns chosen by a reasoning component
US20100063948A1
( en )
*
2008-09-10
2010-03-11
Digital Infuzion, Inc.
Machine learning methods and systems for identifying patterns in data
US8806644B1
( en )
*
2012-05-25
2014-08-12
Symantec Corporation
Using expectation measures to identify relevant application analysis results
US20180068271A1
( en )
2016-09-08
2018-03-08
International Business Machines Corporation
Agile team structure and processes recommendation
US20180089591A1
( en )
*
2016-09-27
2018-03-29
Clairfai, Inc.
Artificial intelligence model and data collection/development platform
US20190019095A1
( en )
*
2017-07-11
2019-01-17
Sap Se
Pattern recognition to detect erroneous data
US20190042290A1
( en )
2017-08-07
2019-02-07
Open Data Group Inc.
Dynamically configurable microservice model for data analysis using sensors
US20190114417A1
( en )
*
2017-10-13
2019-04-18
Ping Identity Corporation
Methods and apparatus for analyzing sequences of application programming interface traffic to identify potential malicious actions
US10296848B1
( en )
*
2018-03-05
2019-05-21
Clinc, Inc.
Systems and method for automatically configuring machine learning models
US20190318245A1
( en )
*
2016-12-26
2019-10-17
Huawei Technologies Co., Ltd.
Method, terminal-side device, and cloud-side device for data processing and terminal-cloud collaboration system
US20190356555A1
( en )
*
2018-05-17
2019-11-21
Microsoft Technology Licensing, Llc
Machine learning microservice architecture design tools and methods
US20200160170A1
( en )
*
2018-11-20
2020-05-21
Bank Of America Corporation
System and method for incremental learning through state-based real-time adaptations in neural networks
US20200160227A1
( en )
*
2018-11-15
2020-05-21
General Electric Company
Model update based on change in edge data
US10783801B1
( en )
*
2016-12-21
2020-09-22
Aptima, Inc.
Simulation based training system for measurement of team cognitive load to automatically customize simulation content
US20200366574A1
( en )
*
2019-05-17
2020-11-19
Citrix Systems, Inc.
Systems and methods for identifying a context of an endpoint accessing a plurality of microservices
US20200380381A1
( en )
*
2019-05-31
2020-12-03
At&T Intellectual Property I, L.P.
Public policy rule enhancement of machine learning/artificial intelligence solutions
US11151468B1
( en )
*
2015-07-02
2021-10-19
Experian Information Solutions, Inc.
Behavior analysis using distributed representations of event data
2019
2019-07-24
US
US16/521,185
patent/US11720826B2/en
active
Active
Patent Citations (21)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
WO2005033934A2
( en )
2003-10-02
2005-04-14
Siemens Aktiengesellschaft
Flexible software update for automation systems via internet
US20070094168A1
( en )
*
2005-07-29
2007-04-26
The Florida International University Board Of Trustees
Artificial neural network design and evaluation tool
US20070074149A1
( en )
2005-08-26
2007-03-29
Microsoft Corporation
Automated product defects analysis and reporting
US7636698B2
( en )
*
2006-03-16
2009-12-22
Microsoft Corporation
Analyzing mining pattern evolutions by comparing labels, algorithms, or data patterns chosen by a reasoning component
US20090210368A1
( en )
2008-02-14
2009-08-20
Ebay Inc.
System and method for real time pattern identification
US20100063948A1
( en )
*
2008-09-10
2010-03-11
Digital Infuzion, Inc.
Machine learning methods and systems for identifying patterns in data
US8806644B1
( en )
*
2012-05-25
2014-08-12
Symantec Corporation
Using expectation measures to identify relevant application analysis results
US11151468B1
( en )
*
2015-07-02
2021-10-19
Experian Information Solutions, Inc.
Behavior analysis using distributed representations of event data
US20180068271A1
( en )
2016-09-08
2018-03-08
International Business Machines Corporation
Agile team structure and processes recommendation
US20180089591A1
( en )
*
2016-09-27
2018-03-29
Clairfai, Inc.
Artificial intelligence model and data collection/development platform
US10783801B1
( en )
*
2016-12-21
2020-09-22
Aptima, Inc.
Simulation based training system for measurement of team cognitive load to automatically customize simulation content
US20190318245A1
( en )
*
2016-12-26
2019-10-17
Huawei Technologies Co., Ltd.
Method, terminal-side device, and cloud-side device for data processing and terminal-cloud collaboration system
US20190019095A1
( en )
*
2017-07-11
2019-01-17
Sap Se
Pattern recognition to detect erroneous data
US20190042290A1
( en )
2017-08-07
2019-02-07
Open Data Group Inc.
Dynamically configurable microservice model for data analysis using sensors
US20190114417A1
( en )
*
2017-10-13
2019-04-18
Ping Identity Corporation
Methods and apparatus for analyzing sequences of application programming interface traffic to identify potential malicious actions
US10296848B1
( en )
*
2018-03-05
2019-05-21
Clinc, Inc.
Systems and method for automatically configuring machine learning models
US20190356555A1
( en )
*
2018-05-17
2019-11-21
Microsoft Technology Licensing, Llc
Machine learning microservice architecture design tools and methods
US20200160227A1
( en )
*
2018-11-15
2020-05-21
General Electric Company
Model update based on change in edge data
US20200160170A1
( en )
*
2018-11-20
2020-05-21
Bank Of America Corporation
System and method for incremental learning through state-based real-time adaptations in neural networks
US20200366574A1
( en )
*
2019-05-17
2020-11-19
Citrix Systems, Inc.
Systems and methods for identifying a context of an endpoint accessing a plurality of microservices
US20200380381A1
( en )
*
2019-05-31
2020-12-03
At&T Intellectual Property I, L.P.
Public policy rule enhancement of machine learning/artificial intelligence solutions
Non-Patent Citations (8)
* Cited by examiner, â Cited by third party
Title
Bahadori et al., " DevOps Meets Dynamic Orchestration, " International Workshop on Software Engineering Aspects of Continuous Development and New Paradigms of Software Production and Deployment, pp. 142-154, Springer, Cham, (2018).
https://sudo.hailoapp.com/services/2015/03/09/journey-into-a-microservice-world-part-3/.
Hummer et al., " ModelOps: Cloud-based Lifecycle Management for Reliable and Trusted AI, " (2019).
Mell et al., " The NIST Definition of Cloud Computing, " Recommendations of the National Institute of Standards and Technology, NIST Special Publication 800-145, Sep. 2011, 7 pages.
Miguel et al., " Marvin-Open Source Artificial Intelligence Platform, " International Conference on Predictive Applications and APIs, pp. 33-44, (2018).
Pahl et al., " Machine Learning as a Reusable Microservice, " NOMS 2018-2018 IEEE/IFIP Network Operations and Management Symposium, pp. 1-7, IEEE, (2018).
slideshare.net, " The Case for Chaos, " https://www.slideshare.net/BRUCEWONG3/THE-CASE-FOR-CHAOS, Retrieved: May 14, 2019, 28 pages.
twitter.com, " Twitter microservices map looks just like the Netflix one. We called this the "Death Star" diagram, " https://twitter.com/adrianco/status/441883572618948608, Retrieved: May 14, 2019, 1 page.
Also Published As
Publication number
Publication date
US20210027136A1
( en )
2021-01-28
Similar Documents
Publication
Publication Date
Title
US20210027136A1
( en )
2021-01-28
Feedback loop learning between artificial intelligence systems
US11748648B2
( en )
2023-09-05
Quantum pulse optimization using machine learning
US11681914B2
( en )
2023-06-20
Determining multivariate time series data dependencies
US11601468B2
( en )
2023-03-07
Detection of an adversarial backdoor attack on a trained model at inference time
US12079730B2
( en )
2024-09-03
Transfer learning for molecular structure generation
US20200082290A1
( en )
2020-03-12
Adaptive anonymization of data using statistical inference
US11221855B2
( en )
2022-01-11
Transformation of an enterprise application into a cloud native application
US10592368B2
( en )
2020-03-17
Missing values imputation of sequential data
US11843565B2
( en )
2023-12-12
Dialogue system based on contextual information
US20190317728A1
( en )
2019-10-17
Graph similarity analytics
US11425000B2
( en )
2022-08-23
On-the-fly reorganization of directed acyclic graph nodes of a computing service for high integration flexibility
US12026613B2
( en )
2024-07-02
Transfer learning across automated machine learning systems
US20190156243A1
( en )
2019-05-23
Efficient Large-Scale Kernel Learning Using a Distributed Processing Architecture
US11221846B2
( en )
2022-01-11
Automated transformation of applications to a target computing environment
US11853017B2
( en )
2023-12-26
Machine learning optimization framework
US11947940B2
( en )
2024-04-02
Training data augmentation via program simplification
US10833962B2
( en )
2020-11-10
Orchestration engine blueprint aspects for hybrid cloud composition
US20230115723A1
( en )
2023-04-13
Complexity based artificial intelligence model training
US12333379B2
( en )
2025-06-17
Optimized selection of data for quantum circuits
US20230130781A1
( en )
2023-04-27
Artificial intelligence model learning introspection
US12165019B2
( en )
2024-12-10
Symbolic model training with active learning
US20230035522A1
( en )
2023-02-02
Certification of computer pipeline results
WO2023109134A1
( en )
2023-06-22
Quantum circuit buffering
US20240037439A1
( en )
2024-02-01
Quantum system selection via coupling map comparison
US11928519B2
( en )
2024-03-12
Modernization of an application for related image generation
Legal Events
Date
Code
Title
Description
2019-07-24
AS
Assignment
Owner name : INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW YORK
Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HWANG, JINHO;SHWARTZ, LARISA;VOELZER, HAGEN;AND OTHERS;SIGNING DATES FROM 20190615 TO 20190724;REEL/FRAME:049851/0482
2019-07-24
FEPP
Fee payment procedure
Free format text : ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY
2022-05-06
STPP
Information on status: patent application and granting procedure in general
Free format text : NON FINAL ACTION MAILED
2022-08-11
STPP
Information on status: patent application and granting procedure in general
Free format text : RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER
2022-08-26
STPP
Information on status: patent application and granting procedure in general
Free format text : FINAL REJECTION MAILED
2022-11-01
STPP
Information on status: patent application and granting procedure in general
Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION
2022-12-06
STPP
Information on status: patent application and granting procedure in general
Free format text : NON FINAL ACTION MAILED
2023-06-16
STPP
Information on status: patent application and granting procedure in general
Free format text : PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED
2023-07-19
STCF
Information on status: patent grant
Free format text : PATENTED CASE