ABSTRACT
Abstract
Artificial intelligence systems are created for end users based on raw data received from the end users or obtained from any source. Training, validation and testing data is maintained securely and subject to authentication prior to use. A machine learning model is selected for providing solutions of any type or form and trained, verified and tested by an artificial intelligence engine using such data. A trained model is distributed to end users, and feedback regarding the performance of the trained model is returned to the artificial intelligence engine, which updates the model on account of such feedback before redistributing the model to the end users. When an end user provides data to an artificial intelligence engine and requests a trained model, the end user monitors progress of the training of the model, along with the performance of the model in providing quality artificial intelligence solutions, via one or more dashboards.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims priority to U.S. Patent Application No. 62/852,245, filed May 23, 2019, the contents of which are incorporated by reference herein in their entirety.
BACKGROUND
Advancements in processing capacity and network connectivity have enabled computers to permeate into nearly every aspect of the human experience. For example, what we now know as the computer processor was born from the development of a number of components in the mid-20th Century, such as vacuum tube processors, transistors, integrated circuits and printed circuit boards. Since then, computer processors have been integrated into machines ranging from wrist watches to rockets, and become ever smaller over time, while growing faster and increasingly powerful according to an observed trend commonly referred to as Moore's Law.
Similarly, the Internet traces its roots to ARPANET, a packet-switching network of computers that was established by the United States Department of Defense during the Cold War. Eventually, civilian computer networks such as BITNET, which began with a single network link between Yale University and the City University of New York in 1981, joined universities and government institutions over networked connections according to the Transmission Control Protocol and Internet Protocol, or TCP/IP. Internet connectivity ultimately reached into businesses and homes via dial-up systems, subscriber lines and television networks, and eventually through wireless components such as transceivers, modems or routers. Today, a theory known as the âInternet of thingsâ predicts a world of the future in which a variety of systems, machines, objects or people are interconnected via wired or wireless computer networks, and seamlessly transfer information or data between one another.
Artificial intelligence is a term that was first coined in the 1950s, and generally refers to the simulation of intelligent human behavior in computers, or the capability of a computer-based machine to learn and imitate intelligent human behavior, particularly in the making of intelligent decisions. Computer devices may effectively âlearnâ human behavior by providing a set of training data that includes training inputs and training outputs (or âtargetsâ) to a machine learning model or algorithm, in an effort to train the model or algorithm to learn patterns within the training inputs and to associate such training inputs with their corresponding training outputs or targets. During a training process, other data, e.g., a set of validation data including validation inputs and validation outputs, is used to validate the training of the model or algorithm with respect to data that was not used for training the model or algorithm. Finally, after the training process is complete, still other data, e.g., a set of test data that includes both test inputs and test outputs, is used to obtain an unbiased evaluation of the model or algorithm.
Unlike processors or network connectivity, however, artificial intelligence has yet to be adopted into mainstream computer systems or networks on a wide scale. Some hurdles to making artificial intelligence available to the masses include, but are not limited to, the fact that generating machine learning models typically requires highly skilled programmers and engineers, and top-heavy investments in computer infrastructure. For these reasons, and others, the many benefits of artificial intelligence are presently limited to substantially large and sophisticated entities having both the will and the means to obtain timely and relevant data associated with a task, to generate machine learning models that are best suited for performing the task, and to train such models using the timely and relevant data.
BRIEF DESCRIPTION OF THE DRAWINGS
FIGS. 1A through 1G are views of aspects of one system in accordance with embodiments of the present disclosure.
FIGS. 2A and 2B are block diagrams of one system in accordance with embodiments of the present disclosure.
FIGS. 3A and 3B are a flow chart of one process in accordance with embodiments of the present disclosure.
FIGS. 4A through 4C are views of aspects of one system in accordance with embodiments of the present disclosure.
FIG. 5 is a flow chart of one process in accordance with embodiments of the present disclosure.
FIGS. 6A through 6D are views of aspects of one system in accordance with embodiments of the present disclosure.
FIG. 7 is a flow chart of one process in accordance with embodiments of the present disclosure.
FIG. 8 is a view of aspects of one system in accordance with embodiments of the present disclosure.
FIG. 9 is a flow chart of one process in accordance with embodiments of the present disclosure.
FIGS. 10A through 10D are views of aspects of one system in accordance with embodiments of the present disclosure.
FIGS. 11A and 11B are a flow chart of one process in accordance with embodiments of the present disclosure.
FIGS. 12A through 12E are views of aspects of one system in accordance with embodiments of the present disclosure.
FIG. 13 is a flow chart of one process in accordance with embodiments of the present disclosure.
FIG. 14 is a view of aspects of one system in accordance with embodiments of the present disclosure.
DETAILED DESCRIPTION
As is set forth in greater detail below, the present disclosure is directed to generating and updating trained machine learning models for providing artificial intelligence solutions to end users using relevant data received from the end users or from other sources. More specifically, the systems and methods of the present disclosure make artificial intelligence solutions available to end users at scales and levels of quality that are presently beyond the core capacities of their personnel or the availability of their technical infrastructure. Such end users may include companies, customers or other clients that are engaged in task-based operations of any sizes, durations or complexity, and in any fields or industries. The systems and methods of the present disclosure thereby lower barriers to entry to artificial intelligence for end users of any levels of sophistication, and provide them with well-trained models even where the end users have limited amounts of data, while protecting their data through the process, and presenting them with easy, intuitive systems for monitoring the process from cradle to grave.
In accordance with the present disclosure, an artificial intelligence engine or other computer device or system may generate and train machine learning models of any type or form are generated behalf of end users, or in concert with such end users, e.g., according to one or more federated learning techniques. The artificial intelligence engine or other device or system may select or train a machine learning model for performing a task for an end user, or for multiple end users, based on any attributes or characteristics of the task or of data ordinarily required to perform the task. Sets of data for training the machine learning model may be obtained from such end users or from any other sources, e.g., open sources, and augmented or supplemented by simulated data that is generated based on the data received by the artificial intelligence engine from the end users or the open sources, as necessary. Trained machine learning models may be furnished to the end users, and updated based on feedback received from the end users or newly available data. Feedback may be returned to the artificial intelligence engine in any manner and in any format. For example, the feedback may represent the effectiveness or level of performance of a model in performing a task on behalf of one or more end users, and may be encrypted according to one or more algorithms or protocols (e.g., homomorphic encryption algorithms), or unencrypted. Furthermore, an artificial intelligence solution in accordance with the present disclosure may utilize one or more sets of rules, as well as trained machine learning models. Where a result associated with a given data point may be more accurately or easily obtained by evaluating the data point with respect to one or more rules, the data point need not be provided to a trained machine learning model, which may then be freed to determine results for data points that may not be accurately or easily processed according to any of the rules.
Referring to FIGS. 1A through 1G , views of aspects of one system in accordance with the present disclosure are shown. As is shown in FIG. 1A , a system 100 includes an artificial intelligence engine (or other data processing system) 110 and a plurality of end users 150 - 1 , 150 - 2 . . . 150 - n . The artificial intelligence engine 110 operates a server 112 or other computer device or machine having one or more processors, memory components or other data storage components. Each of the end users 150 - 1 , 150 - 2 . . . 150 - n includes a computer device 152 - 1 , 152 - 2 . . . 152 - n in communication with a sensor 160 - 1 , 160 - 2 . . . 160 - n associated with the performance of a task. For example, as is shown in FIG. 1A , each of the sensors 160 - 1 , 160 - 2 . . . 160 - n comprises an imaging device configured to capture data for use in the performance of one or more computer vision, anomaly detection or other image-based tasks. Additionally, each of the computer devices 152 - 1 , 152 - 2 . . . 152 - n is in communication with the server 112 over a network 190 , which may include the Internet in whole or in part.
As is shown in FIG. 1B , each of the end users 150 - 1 , 150 - 2 . . . 150 - n may provide sets of training data 175 - 1 , 175 - 2 . . . 175 - n to the server 112 over the network 190 . The training data 175 - 1 , 175 - 2 . . . 175 - n may include any relevant information, data or metadata regarding one or more tasks of interest to the end users 150 - 1 , 150 - 2 . . . 150 - n . For example, where the end users 150 - 1 , 150 - 2 . . . 150 - n are engaged in the performance of computer vision-related tasks, the training data 175 - 1 , 175 - 2 . . . 175 - n may include sets of images along with one or more annotations of relevant aspects of such images, as applicable. Such annotations may be applied to the respective images themselves, or stored in one or more data files or records, which may include one or more coordinate pairs identifying the relevant aspects depicted within the respective images. Where the end users 150 - 1 , 150 - 2 . . . 150 - n are engaged in the performance of anomaly detection-related tasks, the training data 175 - 1 , 175 - 2 . . . 175 - n may include data (e.g., images) representing one or more anomalous and anomaly-free conditions, as well as metadata or other records identifying the presence of such anomalies within such data. Where the end users 150 - 1 , 150 - 2 . . . 150 - n are engaged in the performance of voice recognition or natural language processing tasks, the training data 175 - 1 , 175 - 2 . . . 175 - n may include sets of acoustic data corresponding to spoken words, or representations of the acoustic data, e.g., plots or waveforms of sound amplitudes and/or frequencies, as well as labels or identifiers of words, parts of speech, participants or sentiments represented within the acoustic data.
Additionally, as is also shown in FIG. 1B , the server 112 may also receive or otherwise obtain open source data 185 from an open source 180 having one or more servers or other computer devices or machines that are connected to the network 190 . The open source data 185 may be publicly or commonly available, free of charge or for a fee, and may also correspond to one or more tasks performed by the end users 150 - 1 , 150 - 2 . . . 150 - n . Whereas the training data 175 - 1 , 175 - 2 . . . 175 - n may have been generated or obtained by the respective end users 150 - 1 , 150 - 2 . . . 150 - n , e.g., using one or more of the sensors 160 - 1 , 160 - 2 . . . 160 - n shown in FIG. 1A , during an ordinary course of business or operation, the open source data 185 may have been generated or received by the open source 180 at any time and from any source, and made available to one or more computer devices or machines over the network 190 .
The training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 may be identified and furnished to the server 112 on any basis, and in any manner, in accordance with the present disclosure. For example, in some embodiments, the
CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims priority to U.S. Patent Application No. 62/852,245, filed May 23, 2019, the contents of which are incorporated by reference herein in their entirety.
BACKGROUND
Advancements in processing capacity and network connectivity have enabled computers to permeate into nearly every aspect of the human experience. For example, what we now know as the computer processor was born from the development of a number of components in the mid-20th Century, such as vacuum tube processors, transistors, integrated circuits and printed circuit boards. Since then, computer processors have been integrated into machines ranging from wrist watches to rockets, and become ever smaller over time, while growing faster and increasingly powerful according to an observed trend commonly referred to as Moore's Law.
Similarly, the Internet traces its roots to ARPANET, a packet-switching network of computers that was established by the United States Department of Defense during the Cold War. Eventually, civilian computer networks such as BITNET, which began with a single network link between Yale University and the City University of New York in 1981, joined universities and government institutions over networked connections according to the Transmission Control Protocol and Internet Protocol, or TCP/IP. Internet connectivity ultimately reached into businesses and homes via dial-up systems, subscriber lines and television networks, and eventually through wireless components such as transceivers, modems or routers. Today, a theory known as the âInternet of thingsâ predicts a world of the future in which a variety of systems, machines, objects or people are interconnected via wired or wireless computer networks, and seamlessly transfer information or data between one another.
Artificial intelligence is a term that was first coined in the 1950s, and generally refers to the simulation of intelligent human behavior in computers, or the capability of a computer-based machine to learn and imitate intelligent human behavior, particularly in the making of intelligent decisions. Computer devices may effectively âlearnâ human behavior by providing a set of training data that includes training inputs and training outputs (or âtargetsâ) to a machine learning model or algorithm, in an effort to train the model or algorithm to learn patterns within the training inputs and to associate such training inputs with their corresponding training outputs or targets. During a training process, other data, e.g., a set of validation data including validation inputs and validation outputs, is used to validate the training of the model or algorithm with respect to data that was not used for training the model or algorithm. Finally, after the training process is complete, still other data, e.g., a set of test data that includes both test inputs and test outputs, is used to obtain an unbiased evaluation of the model or algorithm.
Unlike processors or network connectivity, however, artificial intelligence has yet to be adopted into mainstream computer systems or networks on a wide scale. Some hurdles to making artificial intelligence available to the masses include, but are not limited to, the fact that generating machine learning models typically requires highly skilled programmers and engineers, and top-heavy investments in computer infrastructure. For these reasons, and others, the many benefits of artificial intelligence are presently limited to substantially large and sophisticated entities having both the will and the means to obtain timely and relevant data associated with a task, to generate machine learning models that are best suited for performing the task, and to train such models using the timely and relevant data.
BRIEF DESCRIPTION OF THE DRAWINGS
FIGS. 1A through 1G are views of aspects of one system in accordance with embodiments of the present disclosure.
FIGS. 2A and 2B are block diagrams of one system in accordance with embodiments of the present disclosure.
FIGS. 3A and 3B are a flow chart of one process in accordance with embodiments of the present disclosure.
FIGS. 4A through 4C are views of aspects of one system in accordance with embodiments of the present disclosure.
FIG. 5 is a flow chart of one process in accordance with embodiments of the present disclosure.
FIGS. 6A through 6D are views of aspects of one system in accordance with embodiments of the present disclosure.
FIG. 7 is a flow chart of one process in accordance with embodiments of the present disclosure.
FIG. 8 is a view of aspects of one system in accordance with embodiments of the present disclosure.
FIG. 9 is a flow chart of one process in accordance with embodiments of the present disclosure.
FIGS. 10A through 10D are views of aspects of one system in accordance with embodiments of the present disclosure.
FIGS. 11A and 11B are a flow chart of one process in accordance with embodiments of the present disclosure.
FIGS. 12A through 12E are views of aspects of one system in accordance with embodiments of the present disclosure.
FIG. 13 is a flow chart of one process in accordance with embodiments of the present disclosure.
FIG. 14 is a view of aspects of one system in accordance with embodiments of the present disclosure.
DETAILED DESCRIPTION
As is set forth in greater detail below, the present disclosure is directed to generating and updating trained machine learning models for providing artificial intelligence solutions to end users using relevant data received from the end users or from other sources. More specifically, the systems and methods of the present disclosure make artificial intelligence solutions available to end users at scales and levels of quality that are presently beyond the core capacities of their personnel or the availability of their technical infrastructure. Such end users may include companies, customers or other clients that are engaged in task-based operations of any sizes, durations or complexity, and in any fields or industries. The systems and methods of the present disclosure thereby lower barriers to entry to artificial intelligence for end users of any levels of sophistication, and provide them with well-trained models even where the end users have limited amounts of data, while protecting their data through the process, and presenting them with easy, intuitive systems for monitoring the process from cradle to grave.
In accordance with the present disclosure, an artificial intelligence engine or other computer device or system may generate and train machine learning models of any type or form are generated behalf of end users, or in concert with such end users, e.g., according to one or more federated learning techniques. The artificial intelligence engine or other device or system may select or train a machine learning model for performing a task for an end user, or for multiple end users, based on any attributes or characteristics of the task or of data ordinarily required to perform the task. Sets of data for training the machine learning model may be obtained from such end users or from any other sources, e.g., open sources, and augmented or supplemented by simulated data that is generated based on the data received by the artificial intelligence engine from the end users or the open sources, as necessary. Trained machine learning models may be furnished to the end users, and updated based on feedback received from the end users or newly available data. Feedback may be returned to the artificial intelligence engine in any manner and in any format. For example, the feedback may represent the effectiveness or level of performance of a model in performing a task on behalf of one or more end users, and may be encrypted according to one or more algorithms or protocols (e.g., homomorphic encryption algorithms), or unencrypted. Furthermore, an artificial intelligence solution in accordance with the present disclosure may utilize one or more sets of rules, as well as trained machine learning models. Where a result associated with a given data point may be more accurately or easily obtained by evaluating the data point with respect to one or more rules, the data point need not be provided to a trained machine learning model, which may then be freed to determine results for data points that may not be accurately or easily processed according to any of the rules.
Referring to FIGS. 1A through 1G , views of aspects of one system in accordance with the present disclosure are shown. As is shown in FIG. 1A , a system 100 includes an artificial intelligence engine (or other data processing system) 110 and a plurality of end users 150 - 1 , 150 - 2 . . . 150 - n . The artificial intelligence engine 110 operates a server 112 or other computer device or machine having one or more processors, memory components or other data storage components. Each of the end users 150 - 1 , 150 - 2 . . . 150 - n includes a computer device 152 - 1 , 152 - 2 . . . 152 - n in communication with a sensor 160 - 1 , 160 - 2 . . . 160 - n associated with the performance of a task. For example, as is shown in FIG. 1A , each of the sensors 160 - 1 , 160 - 2 . . . 160 - n comprises an imaging device configured to capture data for use in the performance of one or more computer vision, anomaly detection or other image-based tasks. Additionally, each of the computer devices 152 - 1 , 152 - 2 . . . 152 - n is in communication with the server 112 over a network 190 , which may include the Internet in whole or in part.
As is shown in FIG. 1B , each of the end users 150 - 1 , 150 - 2 . . . 150 - n may provide sets of training data 175 - 1 , 175 - 2 . . . 175 - n to the server 112 over the network 190 . The training data 175 - 1 , 175 - 2 . . . 175 - n may include any relevant information, data or metadata regarding one or more tasks of interest to the end users 150 - 1 , 150 - 2 . . . 150 - n . For example, where the end users 150 - 1 , 150 - 2 . . . 150 - n are engaged in the performance of computer vision-related tasks, the training data 175 - 1 , 175 - 2 . . . 175 - n may include sets of images along with one or more annotations of relevant aspects of such images, as applicable. Such annotations may be applied to the respective images themselves, or stored in one or more data files or records, which may include one or more coordinate pairs identifying the relevant aspects depicted within the respective images. Where the end users 150 - 1 , 150 - 2 . . . 150 - n are engaged in the performance of anomaly detection-related tasks, the training data 175 - 1 , 175 - 2 . . . 175 - n may include data (e.g., images) representing one or more anomalous and anomaly-free conditions, as well as metadata or other records identifying the presence of such anomalies within such data. Where the end users 150 - 1 , 150 - 2 . . . 150 - n are engaged in the performance of voice recognition or natural language processing tasks, the training data 175 - 1 , 175 - 2 . . . 175 - n may include sets of acoustic data corresponding to spoken words, or representations of the acoustic data, e.g., plots or waveforms of sound amplitudes and/or frequencies, as well as labels or identifiers of words, parts of speech, participants or sentiments represented within the acoustic data.
Additionally, as is also shown in FIG. 1B , the server 112 may also receive or otherwise obtain open source data 185 from an open source 180 having one or more servers or other computer devices or machines that are connected to the network 190 . The open source data 185 may be publicly or commonly available, free of charge or for a fee, and may also correspond to one or more tasks performed by the end users 150 - 1 , 150 - 2 . . . 150 - n . Whereas the training data 175 - 1 , 175 - 2 . . . 175 - n may have been generated or obtained by the respective end users 150 - 1 , 150 - 2 . . . 150 - n , e.g., using one or more of the sensors 160 - 1 , 160 - 2 . . . 160 - n shown in FIG. 1A , during an ordinary course of business or operation, the open source data 185 may have been generated or received by the open source 180 at any time and from any source, and made available to one or more computer devices or machines over the network 190 .
The training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 may be identified and furnished to the server 112 on any basis, and in any manner, in accordance with the present disclosure. For example, in some embodiments, the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 may be provided to the server 112 over the network 190 via one or more wired or wireless connections. Alternatively, the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 may be physically delivered to a location associated with the server 112 via a data transport system, and uploaded to the server 112 upon its arrival. Moreover, in some embodiments, data may be received from fewer than all of the end users 150 - 1 , 150 - 2 . . . 150 - n , e.g., from just one of the end users 150 - 1 , 150 - 2 . . . 150 - n , or from the open source 180 . In some other embodiments, the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 may be augmented with simulated data, which may be modified versions of one or more data points of the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 received from the end users 150 - 1 , 150 - 2 . . . 150 - n , or from the open source 180 . For example, where the training data 175 - 1 , 175 - 2 . . . 175 - n or the open source data 185 includes imaging data, one or more images may be rotated, zoomed, cropped or otherwise altered, and may augment the training data 175 - 1 , 175 - 2 . . . 175 - n or the open source data 185 . Where the training data 175 - 1 , 175 - 2 . . . 175 - n or the open source data 185 includes or describes acoustic signals, the acoustic signals or corresponding data may be filtered or otherwise altered, and the filtered or altered data may augment the training data 175 - 1 , 175 - 2 . . . 175 - n or the open source data 185 .
Upon receiving the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 from the end users 150 - 1 , 150 - 2 . . . 150 - n and the open source 180 , the server 112 may securely store the data in one or more data stores. For example, in some embodiments, any annotations or other information or data associated with the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 may be identified, evaluated and modified, as necessary, by one or more human operators other devices or machines, prior to storing the data on the server 112 , or on one or more other computer devices or machines (not shown). Alternatively, or additionally, the server 112 may execute one or more authentication functions (e.g., hash functions) on the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 prior to storing the data on the server 112 or elsewhere. The authentication functions may return values (e.g., hashes) representative of the contents of the respective data points within the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 , as well as values (e.g., hashes) representative of their respective annotations or other identifiers or metadata associated with such data points.
As is shown in FIG. 1C , the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 may be used to generate and/or train a baseline machine learning model 170 - 1 . For example, in some embodiments, the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 may be split or parsed into a set of training data, a set of validation data, and a set of test data, each including corresponding inputs and outputs. The baseline model 170 - 1 may be trained to map inputs to desired outputs, e.g., by adjusting connections between one or more neurons in layers, in order to provide an output that most closely approximates or associates with an input to a maximum practicable extent. In accordance with embodiments of the present disclosure, any type or form of machine learning model may be generated or trained, including but not limited to artificial neural networks, deep learning systems, support vector machines, or others. Furthermore, in some embodiments, information or data regarding the transfer of the training data 175 - 1 , 175 - 2 . . . 175 - n and/or the open source data 185 to the server 112 , or the training of the baseline model 170 - 1 based on such data, may be returned to the respective end users 150 - 1 , 150 - 2 . . . 150 - n and displayed in one or more user interfaces (e.g., dashboards) by the respective computer devices 152 - 1 , 152 - 2 . . . 152 - n.
As is shown in FIG. 1D , once the baseline model 170 - 1 has been generated and sufficiently trained, the server 112 may distribute the baseline model 170 - 1 to the one or more end users 150 - 1 , 150 - 2 . . . 150 - n . For example, in some embodiments, code for operating the baseline model 170 - 1 may be transmitted to the one or more end users 150 - 1 , 150 - 2 . . . 150 - n , e.g., over the network 190 . The code may identify or represent numbers of layers or of neurons within such layers, synaptic weights between neurons, or any factors describing the operation of the baseline model 170 - 1 . Alternatively, the baseline model 170 - 1 may be provided to the respective end users 150 - 1 , 150 - 1 . . . 150 - n in any other manner. As is shown in FIG. 1E , after receiving the baseline model 170 - 1 is received, the computer devices 152 - 1 , 152 - 2 . . . 152 - n may operate the baseline model 170 - 1 to process data captured using the sensors 160 - 1 , 160 - 2 . . . 160 - n or obtained from any other sources, and to make decisions based on such data.
As is shown in FIG. 1F , one or more of the end users 150 - 1 , 150 - 1 . . . 150 - n may provide feedback regarding the performance of the baseline model 170 - 1 with respect to the task for which the model was trained. For example, as is shown in FIG. 1F , each of the respective computer devices 152 - 1 , 152 - 2 . . . 152 - n of the end users 150 - 1 , 150 - 1 . . . 150 - n may further train the baseline model 170 - 1 based on data encountered during the performance of the task, and may calculate or determine differences between the baseline model 170 - 1 , or BM 1 , as generated and trained by the server 112 , and the subsequently trained models M i , or (ÎM i ) 1 =(BM 1 -M i ) 1 . The differences (ÎM 1 ) 1 , (ÎM 2 ) 1 , (ÎM 3 ) 1 calculated by the respective computer devices 152 - 1 , 152 - 2 . . . 152 - n may be returned to the server 112 as feedback over the network 190 , or in any other manner. Alternatively, any other type or form of feedback may be returned to the server 112 . For example, where the baseline model 170 - 1 is used to perform a task requiring the identification of an output based on one or more data points provided to the baseline model 170 - 1 as an input, the outputs generated by the baseline model 170 - 1 may be manually evaluated with respect to the respective inputs, e.g., by one or more human operators, to determine a measure of efficiency or performance of the baseline model 170 - 1 .
As is shown in FIG. 1G , upon receiving the feedback from one or more of the end users 150 - 1 , 150 - 1 . . . 150 - n , the server 112 may generate a modified baseline model 170 - 2 based on the feedback received and the original baseline model 170 - 1 , and may distribute the baseline model 170 - 2 to the end users 150 - 1 , 150 - 1 . . . 150 - n . Subsequently, one or more of the end users 150 - 1 , 150 - 1 . . . 150 - n , or any additional end users (not shown), may utilize the baseline model 170 - 2 in the performance of the task, and may generate and return feedback regarding the performance of the baseline model 170 - 1 to the server 112 .
Accordingly, the systems and methods of the present disclosure may generate artificial intelligence solutions for end users by training a machine learning model to perform a task based on data received from such end users or obtained from an open source, as well as data that has been simulated or modified from such data.
Artificial intelligence solutions may be generated, trained and utilized for the performance of any task or function in accordance with the present disclosure. For example, a machine learning model may be trained to execute any number of computer vision applications in accordance with the present disclosure. In some embodiments, an artificial intelligence solution generated according to the present disclosure may be used in medical applications, such as where images of samples of tissue or blood, or radiographic images, must be interpreted in order to properly diagnose a patient. Alternatively, an artificial intelligence solution generated according to the present disclosure may be used in autonomous vehicles, such as to enable an autonomous vehicle to detect and recognize one or more obstacles, features or other vehicles based on imaging data, and making one or more decisions regarding the safe operation of an autonomous vehicle accordingly. Likewise, a machine learning model may also be trained to execute any number of anomaly detection (or outlier detection) tasks for use in any application. In some embodiments, an artificial intelligence solution generated according to the present disclosure may be used to determine that objects such as manufactured goods, food products (e.g., fruits or meats) or faces or other identifying features of humans comply with or deviate from one or more established standards or requirements. In some embodiments, an artificial intelligence may also be trained to perform voice interpretation, or natural language processing functions, for use in any application in which computers must understand and interpret spoken languages. The number of applications, tasks or functions that may be enhanced through the use of artificial intelligence solutions generated according to the present disclosure is not limited.
Any type or form of machine learning model may be generated, trained and utilized using one or more of the embodiments disclosed herein. For example, machine learning models, such as artificial neural networks, have been utilized to identify relations between respective elements of apparently unrelated sets of data. An artificial neural network is a parallel distributed computing processor system comprised of individual units that may collectively learn and store experimental knowledge, and make such knowledge available for use in one or more applications. Such a network may simulate the non-linear mental performance of the many neurons of the human brain in multiple layers by acquiring knowledge from an environment through one or more flexible learning processes, determining the strengths of the respective connections between such neurons, and utilizing such strengths when storing acquired knowledge. Like the human brain, an artificial neural network may use any number of neurons in any number of layers. In view of their versatility, and their inherent mimicking of the human brain, machine learning models including not only artificial neural networks but also deep learning systems, support vector machines, nearest neighbor methods or analyses, factorization methods or techniques, K-means clustering analyses or techniques, similarity measures such as log likelihood similarities or cosine similarities, latent Dirichlet allocations or other topic models, decision trees, or latent semantic analyses have been utilized in many applications, including but not limited to computer vision applications, anomaly detection applications, and voice recognition or natural language processing.
Artificial neural networks may be trained to map inputted data to desired outputs by adjusting strengths of connections between one or more neurons, which are sometimes called synaptic weights. An artificial neural network may have any number of layers, including an input layer, an output layer, and any number of intervening hidden layers. Each of the neurons in a layer within a neural network may receive an input and generate an output in accordance with an activation or energy function, with parameters corresponding to the various strengths or synaptic weights. For example, in a heterogeneous neural network, each of the neurons within the network may be understood to have different activation or energy functions. In some neural networks, at least one of the activation or energy functions may take the form of a sigmoid function, wherein an output thereof may have a range of zero to one or 0 to 1. In other neural networks, at least one of the activation or energy functions may take the form of a hyperbolic tangent function, wherein an output thereof may have a range of negative one to positive one, or â1 to +1. Thus, the training of a neural network according to an identity function results in the redefinition or adjustment of the strengths or weights of such connections between neurons in the various layers of the neural network, in order to provide an output that most closely approximates or associates with the input to the maximum practicable extent.
Artificial neural networks may typically be characterized as either feedforward neural networks or recurrent neural networks, and may be fully or partially connected. In a feedforward neural network, e.g., a convolutional neural network, information may specifically flow in one direction from an input layer to an output layer, while in a recurrent neural network, at least one feedback loop returns information regarding the difference between the actual output and the targeted output for training purposes. Additionally, in a fully connected neural network architecture, each of the neurons in one of the layers is connected to all of the neurons in a subsequent layer. By contrast, in a sparsely connected neural network architecture, the number of activations of each of the neurons is limited, such as by a sparsity parameter.
Moreover, the training of a neural network is typically characterized as supervised or unsupervised. In supervised learning, a training set comprises at least one input and at least one target output for the input. Thus, the neural network is trained to identify the target output, to within an acceptable level of error. In unsupervised learning of an identity function, such as that which is typically performed by a sparse autoencoder, target output of the training set is the input, and the neural network is trained to recognize the input as such. Sparse autoencoders employ backpropagation in order to train the autoencoders to recognize an approximation of an identity function for an input, or to otherwise approximate the input. Such backpropagation algorithms may operate according to methods of steepest descent, conjugate gradient methods, or other like methods or techniques, in accordance with the systems and methods of the present disclosure. Those of ordinary skill in the pertinent art would recognize that any algorithm or method may be used to train one or more layers of a neural network. Likewise, any algorithm or method may be used to determine and minimize errors in an output of such a network. Additionally, those of ordinary skill in the pertinent art would further recognize that the various layers of a neural network may be trained collectively, such as in a sparse autoencoder, or individually, such that each output from one hidden layer of the neural network acts as an input to a subsequent hidden layer.
Once a neural network has been trained to recognize dominant characteristics of an input of a training set, e.g., to associate a point or a set of data such as an image with a label to within an acceptable tolerance, an input in the form of a data point may be provided to the trained network, and a label may be identified based on the output thereof.
In some embodiments, a machine learning model may be selected for use in any application, on any basis. For example, the machine learning model may be selected based on aspects of a particular task to be performed on behalf of one or more end users, or based on a level of similarity between data that is received from end users, and data that was previously received or obtained from other end users to train a machine learning model to perform similar tasks. Furthermore, in some embodiments, a machine learning model may be generated organically based at least in part on data received from the end users and trained accordingly. In some embodiments, the machine learning system may be obtained from another source, e.g., an open source, and trained based at least in part on data received from the end users.
In some embodiments, the systems and methods of the present disclosure may receive raw data (or physical data) from end users, or obtain open source data from one or more sources. The raw data or the open source data may be properly annotated or otherwise prepared for use by the end users or sources from which the data is received, or upon its receipt, and securely stored in one or more data stores. In some embodiments, raw data received from end users or open source data may be modified or augmented to increase the data that is made available for generating a trained machine learning model. For example, in some embodiments, where imaging data is received from a number of end users or from an open source, e.g., for use in computer vision applications or anomaly detection purposes, some of the images may be synthetically altered (e.g., rotating, zooming, cropping, or the like) to emphasize or enhance flaws or defects depicted therein, e.g., by altering portions of such subjects within the images. Moreover, once images of undamaged subjects have been synthetically altered, such images may be refined to enhance the appearance of such flaws or defects within such images, and make such images appear to be more realistic, e.g., by providing the images to a generative adversarial network, or GAN, to enhance the realism of the synthetic images. In some embodiments, a generative adversarial network may refine such images using a refiner network that is configured to make a synthetically altered image appear more realistic, and a discriminator network that is configured to distinguish between synthetic images and real images. Synthesizing data may be substantially less expensive than obtaining additional data, e.g., other images, by capturing data.
Data received from end users or obtained from open source may be subject to one or more annotation processes in which regions of such images, or objects depicted therein, are designated accordingly. In computer vision applications, annotation is commonly known as marking or labeling of images or video files captured from a scene, such as to denote the presence and location of one or more objects or other features within the scene in the images or video files. Annotating a video file typically involves placing a virtual marking such as a box or other shape on an image frame of a video file, thereby denoting that the image frame depicts an item, or includes pixels of significance, within the box or shape. Alternatively, in some embodiments, a video file may be annotated by applying markings or layers including alphanumeric characters, hyperlinks or other markings on specific frames of the video file, thereby enhancing the functionality or interactivity of the video file in general, or of the video frames in particular. In some other embodiments, annotation may involve generating a table or record identifying positions of objects depicted within image frames, e.g., by one or more pairs of coordinates.
In some embodiments, data received from end users, or obtained from open sources, may be split or parsed into training sets, validation sets or test sets, each having any size or containing any proportion of the total data received or obtained. The data and any annotations may be authenticated or validated according to one or more functions (e.g., an authentication function, or a hash function) prior to training a machine learning model to perform one or more tasks, prior to validating the training of the model, and prior to testing the trained model. Each time the data is accessed for training, or for validation or testing, the data and annotations may be further subjected to the authentication function in order to confirm that the data is unchanged and has neither been altered nor compromised. Once a machine learning model has been sufficiently trained, validated and tested by an artificial intelligence engine, the model may be distributed to one or more end users, e.g., over a network, to the artificial intelligence engine including but not limited to one or more of the end users that requested the model, or provided data by which the model was trained, validated and tested.
Subsequently, in some embodiments, end users that receive a trained machine learning model for performing a task from an artificial intelligence engine may return feedback regarding the performance or the efficacy of the model to the artificial intelligence engine, including the accuracy or efficiency of the model in performing the task for which the model was generated. The feedback may take any form, including but not limited to one or more measures of the effectiveness of the trained machine learning model in performing a given task, including an identification of one or more sets of data regarding inaccuracies of the model in interpreting inputs and generating outputs for performing the task.
Moreover, in some embodiments, after receiving a baseline trained machine learning model for performing a task, an end user may be configured to continue training the baseline model using raw data (or physical data) that is obtained by the end users, or from an open source. In such embodiments, feedback may be returned to an artificial intelligence engine that generated the baseline model, including differences between results generated by a model, as further trained by the end user in performing the task, and results generated by the baseline model in performing the task. Regardless of the form or format of the feedback received from the end users, an artificial intelligence engine may utilize such feedback in further updating the baseline model. Thereafter, an updated baseline model may be returned to one or more of the end users that received the baseline model originally, or to any other end users that may subsequently request a trained machine learning model for performing the same task.
In some embodiments, differences between performance of a task using a baseline model and performance of the task using the baseline model as further trained by an end user may be returned directly to an artificial intelligence engine from each of the end users that received the baseline model from the artificial intelligence engine. In some other embodiments, however, differences between the performance of the baseline model and the performance of the further trained model may be accumulated by one or more of the end users and returned to the artificial intelligence engine. For example, differences between the performance of the baseline model and the performance of the further trained model may be encrypted by each of the end users, e.g., according to an encryption algorithm or function, and exchanged among the end users, such that the artificial intelligence engine receives sequentially averaged differences from all of the end users.
In some embodiments, an artificial intelligence solution generated in accordance with the present disclosure may be configured to pre-process data, prior to providing the data to a trained machine learning model as inputs, to determine whether decisions may be made more appropriately by rule rather than based on outputs received from the trained machine learning model. For example, where a rule is more appropriately aimed to a selected data point, such as where the data point equals, falls below or exceeds a threshold associated with a rule having a designated or assigned outcome, the rule may be used to assign the outcome to the data point, rather than having the data point be provided as an input to a machine learning model, and predicting an outcome based on an output from the machine learning model.
In some embodiments, artificial intelligence solutions may be provided to clients in a âtoolbox,â or a computing environment that enables end users to apply the artificial intelligence solutions with little to no background in machine learning or other computer systems. For example, the artificial intelligence solutions may be applied in a scalable, âcloudâ-based architecture that may be accessed by end users via one or more user interfaces, and need not maintain or store code or other aspects of the baseline models on their respective computer systems or devices. Such interfaces, for example, may enable end users to upload relevant data to an artificial intelligence engine or other data processing system for evaluation or processing, and for secure storage. The end users may maintain ownership over their own respective data, while being restricted from accessing the data of others, yet may exploit the benefits of a machine learning model that has been trained using all of such data and others. Alternatively, in some embodiments, an artificial intelligence solution may be provided directly to an end user, and stored directly on a computer system or device of the end user, before being operated to process data captured or obtained by the end user and to make decisions based on such data. Any of the functions or actions described herein with regard to the generation, training, validation or testing of machine learning models, or the transfer of information or data regarding results obtained by the machine learning models, or decisions made based on such results, may be performed by one or more computer devices or systems associated with an end user or with an artificial intelligence engine in accordance with the present disclosure.
Furthermore, in some embodiments, information regarding the generation of a machine learning model, and the training of the machine learning model, may be provided to end users in one or more dashboards or other user interfaces. For example, an end user that provides raw data to an artificial intelligence engine for the purpose of generating a trained machine learning model for performing a task may monitor a status of the transfer of the raw data to the artificial intelligence engine on one or more dashboards rendered on a mobile device or other computer system, which may display characters, icons, graphs, meters, charts or other information regarding the transfer to one or more personnel associated with the end user. Likewise, as the training of the machine learning model is in progress, the end user may be further updated regarding the training with information displayed on one or more dashboards. Furthermore, when the machine learning model has been adequately trained and returned to the end user, information regarding the performance of the machine learning model may also be displayed on one or more dashboards.
The systems and methods of the present disclosure are not limited to use in any of the embodiments disclosed herein, including but not limited to computer vision, anomaly detection or natural language processing applications. For example, one or more of the artificial intelligence solutions generated in accordance with the present disclosure may be utilized to process data and make decisions in connection with banking, education, manufacturing or retail applications, or any other applications, in accordance with the present disclosure. Moreover, those of ordinary skill in the pertinent arts will recognize that any of the aspects of embodiments disclosed herein may be utilized with or applicable to any other aspects of any of the other embodiments disclosed herein.
Referring to FIGS. 2A and 2B , block diagrams of one system 200 in accordance with embodiments of the present disclosure is shown. As is shown in FIG. 2A , the system 200 includes a data processing system 210 , a plurality of end users 250 - 1 , 250 - 2 . . . 250 - n and a third- party data source 280 that are connected to one another over a network 290 . Except where otherwise noted, reference numerals preceded by the number â2â shown in the block diagram of FIG. 2A or 2B indicate components or features that are similar to components or features having reference numerals preceded by the number â1â shown in FIGS. 1A through 1G .
The data processing system 210 may be an artificial intelligence engine or any other system that includes one or more physical or virtual computer servers 212 or other computer devices or machines having any number of processors that may be provided for any specific or general purpose, and one or more data stores (e.g., data bases) 214 and transceivers 216 associated therewith. For example, the data processing system 210 of FIGS. 2A and 2B may be independently provided for the exclusive purpose of receiving, analyzing, processing or storing data received from the end users 250 - 1 , 250 - 2 . . . 250 - n or the third- party data source 280 or, alternatively, provided in connection with one or more physical or virtual services that are configured to receive, analyze or store such data, or perform any other functions. The data stores 274 may store any type of information or data, including but not limited to imaging data, acoustic signals, or any other information or data, for any purpose. The servers 212 and/or the data stores 214 may also connect to or otherwise communicate with the network 290 , as indicated by line 218 , through the sending and receiving of digital data.
The data processing system 210 may further include any facility, structure, or station for receiving, analyzing, processing or storing data using the servers 212 , the data stores <b
CLAIMS
Claims ( 20 )
What is claimed is:
1. A system comprising:
an artificial intelligence engine; and
a plurality of servers, wherein each of the plurality of servers is associated with one of a plurality of end users,
wherein the artificial intelligence engine is in communication with each of the plurality of servers over at least one network, and
wherein the artificial intelligence engine includes one or more computer processors configured to at least:
receive at least a first plurality of images from at least one of the plurality of end users;
identify a first plurality of annotations, wherein each of the first plurality of annotations identifies at least a portion of one of the first plurality of images that depicts at least a portion of an object of a type;
modify at least a second plurality of images, wherein each of the second plurality of images is one of the first plurality of images;
define a set of training inputs, wherein the set of training inputs comprises:
at least some of the second plurality of images; and
a third plurality of images, wherein each of the third plurality of images is one of the first plurality of images and not one of the second plurality of images; define a set of training outputs, wherein the set of training outputs comprises:
a second plurality of annotations, wherein each of the second plurality of annotations identifies at least a portion of one of the second plurality of images that depicts at least the portion of the object of the type; and
a third plurality of annotations, wherein each of the third plurality of annotations identifies at least a portion of one of the third plurality of images that depicts at least the portion of the object of the type;
train a machine learning tool to detect at least a portion of the object of the type within an image using the training inputs and the training outputs; and
distribute code for operating the machine learning tool to at least some of the plurality of servers.
2. The system of claim 1 , wherein the one or more computer processors are configured to modify at least the second plurality of images by at least one of:
rotating one of the second plurality of images by a predetermined angle;
zooming the one of the second plurality of images by a predetermined ratio;
cropping at least a portion of the one of the second plurality of images, wherein the cropped portion depicts at least the portion of the object of the type; or
varying a contrast of the portion of the one of the second plurality of images by a predetermined amount.
3. The system of claim 1 , wherein each of the plurality of end users is associated with one of a hospital, a university, a research laboratory, a military facility, a financial institution, a manufacturing facility or a retail establishment.
4. A computer-implemented method comprising:
identifying a first plurality of data points, wherein each of the first plurality of data points is associated with performance of a computer-based task by at least one computer device;
identifying a first plurality of annotations, wherein each of the first plurality of annotations identifies at least a portion of one of the first plurality of data points that is relevant to the performance of the computer-based task;
modifying at least some of the first plurality of data points, wherein each of the modified first plurality of data points is relevant to the performance of the computer-based task;
defining a training set, wherein the training set comprises:
a second plurality of data points, wherein each of the second plurality of data points is a modified one of the first plurality of data points; and
a second plurality of annotations, wherein each of the second plurality of annotations is one of the first plurality of annotations that identifies at least the portion of one of the second plurality of data points that is relevant to the performance of the computer-based task;
training a first machine learning model to perform the computer-based task based at least in part on the training set; and
distributing code for operating the first machine learning model to a plurality of end users over at least one computer network.
5. The computer-implemented method of claim 4 , wherein modifying the at least some of the first plurality of data points comprises:
rotating, zooming, cropping or filtering each of the at least some of the first plurality of data points.
6. The computer-implemented method of claim 4 , wherein the computer-based task comprises detecting objects in imaging data,
wherein each of the first plurality of data points comprises an image,
wherein each of the images depicts at least a portion of at least one object, and
wherein each of the first plurality of annotations identifies a portion of one of the images that depicts the portion of the at least one object.
7. The computer-implemented method of claim 4 , wherein the computer-based task comprises natural language processing,
wherein each of the first plurality of data points comprises an acoustic signal,
wherein each of the acoustic signals includes at least one spoken word, and
wherein each of the first plurality of annotations is a label of one of the at least one spoken word or a part of speech of the at least one spoken word in one of the acoustic signals.
8. The computer-implemented method of claim 4 , wherein the computer-based task comprises detecting anomalies in data,
wherein at least some of the first plurality of data points are anomalous, and
wherein each of the first plurality of annotations indicates whether one of the first plurality of data points is anomalous or not anomalous.
9. The computer-implemented method of claim 4 , wherein the first machine learning model is an artificial neural network comprising an input layer having a first plurality of neurons, at least one hidden layer having at least a second plurality of neurons, and an output layer having a third plurality of neurons,
wherein a first connection between at least one of the first plurality of neurons and at least one of the second plurality of neurons has a first synaptic weight in the first machine learning model,
wherein a second connection between at least one of the second plurality of neurons and at least one of the third plurality of neurons has a second synaptic weight in the first machine learning model, and
wherein training the first machine learning model to perform the computer-based task based at least in part on the training set comprises:
selecting at least one of the first synaptic weight for the first connection or the second synaptic weight for the second connection based at least in part on at least some of the second plurality of data points and at least some of the second plurality of annotations.
10. The computer-implemented method of claim 9 , further comprising:
receiving feedback regarding effectiveness of the first machine learning model in performing the computer-based task from at least one of the plurality of end users;
selecting at least one of a third synaptic weight for the first connection or a fourth synaptic weight for the second connection based at least in part on the feedback;
generating code for operating a second machine learning model comprising the input layer, the at least one hidden layer and the output layer, wherein the first connection has the third synaptic weight in the second machine learning model and wherein the second connection has the fourth synaptic weight in the second machine learning model; and
distributing the code for operating the second machine learning model to at least some of the plurality of end users over the at least one computer network.
11. The computer-implemented method of claim 10 , wherein the feedback comprises information regarding a difference between the first machine learning model and a third machine learning model, and
wherein the third machine model is trained by the first end user to perform the computer-based task based at least in part on second data captured by at least the first end user.
12. The computer-implemented method of claim 10 , wherein the feedback comprises information regarding a difference between at least a first result generated by the first machine learning model in response to at least one of a third plurality of data points and a second result identified by a human operator for the at least one of the third plurality of data points.
13. The computer-implemented method of claim 10 , wherein receiving the feedback regarding the effectiveness of the first machine learning model in performing the computer-based task comprises:
receiving a plurality of differences from a plurality of computer devices over a computer network, wherein each of the plurality of computer devices is associated with one of the plurality of end users, wherein each of the plurality of differences is a difference between the first machine learning model and a machine learning model trained by one of the computer devices based at least in part on the first trained machine learning model and data obtained by the one of the computer devices that is homomorphically encrypted by the one of the computer devices, and wherein the plurality of differences are sequentially averaged.
14. The computer-implemented method of claim 4 , further comprising:
splitting the first data into the training set and a test set, wherein the test set comprises a third plurality of data points and a third plurality of annotations, and
testing the training of the first machine learning model based at least in part on the test set.
15. The computer-implemented method of claim 4 , wherein the first machine learning model is one of:
an artificial neural network, a deep learning system, a support vector machine, a nearest neighbor analysis, a factorization method, a K-means clustering technique, a similarity measure, a latent Dirichlet allocation, a decision tree or a latent semantic analysis.
16. The computer-implemented method of claim 4 , wherein at least some of the first plurality of data points are received from at least one of:
a computer device associated with an end user of the plurality of end users; or
a computer device associated with an open source.
17. The computer-implemented method of claim 4 , wherein receiving the first data associated with the computer-based task comprises:
providing at least the first plurality of data points as a first input to at least one authentication function;
determining a first value corresponding to at least the first plurality of data points based at least in part on a first output received from the at least one authentication function in response to the first input; and
storing the first value in association with the first plurality of data points in at least one data store,
wherein modifying the at least some of the first plurality of data points comprises:
retrieving the at least some of the first plurality of data points from the at least one data store;
providing the at least some of the first plurality of data points as a second input to the at least one authentication function;
determining a second value corresponding to the at least some of the first plurality of data points based at least in part on a second output received from the at least one authentication function in response to the second input; and
determining that the second value is consistent with the first value, and
wherein the at least some of the first plurality of data points are modified in response to determining that the second value is consistent with the first value.
18. The computer-implemented method of claim 4 , wherein modifying the at least some of the first plurality of data points comprises:
causing a display of a first user interface on a computer display associated with a first computer device associated with a first end user, wherein the first user interface indicates that the at least some of the first plurality of data points are being modified by an artificial intelligence engine,
wherein defining the training set comprises:
causing a display of a second user interface on the computer display associated with the first computer device, wherein the second user interface indicates that the training set has been defined to include the second plurality of data points and the second plurality of annotations, and
wherein the method further comprises:
causing a display of a third user interface on the computer display associated with the first computer device, wherein the third user interface includes information regarding effectiveness of the first machine learning model in performing the computer-based task on the first computer device.
19. A computer-implemented method comprising:
receiving, by an artificial intelligence engine over a computer network, a plurality of data points, wherein each of the data points is received from one of a plurality of end users engaged in performance of a computer-based task, and wherein each of the data points relates to the computer-based task;
receiving, by the artificial intelligence engine, a plurality of annotations, wherein each of the annotations identifies a portion of a corresponding one of the data points that is relevant to the computer-based task;
executing, by the artificial intelligence engine, a modification to at least some of the plurality of data points;
defining a training set of training inputs and training outputs, wherein the training inputs comprises each of the plurality of data points and each of the modified at least some of the plurality of data points, and wherein the training outputs comprises the plurality of annotations;
training a machine learning model to perform the computer-based task based at least in part on the training inputs and the training outputs;
generating code for operating the machine learning model, wherein the code defines a matrix of weights associated with the machine learning model, and wherein at least one of the matrix of weights is defined based at least in part on the modified at least some of the plurality of data points; and
distributing the code for operating the machine learning model to each of the plurality of end users over a computer network.
20. The computer-implemented method of claim 19 , wherein each of the first plurality of data points is one of imaging data or acoustic data.
US16/880,824
2019-05-23
2020-05-21
Generating artificial intelligence solutions using raw data and simulated data
Expired - Fee Related
US11227187B1
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US16/880,824
US11227187B1
( en )
2019-05-23
2020-05-21
Generating artificial intelligence solutions using raw data and simulated data
Applications Claiming Priority (2)
Application Number
Priority Date
Filing Date
Title
US201962852245P
2019-05-23
2019-05-23
US16/880,824
US11227187B1
( en )
2019-05-23
2020-05-21
Generating artificial intelligence solutions using raw data and simulated data
Publications (1)
Publication Number
Publication Date
US11227187B1
true
US11227187B1 ( en )
2022-01-18
Family
ID=79293852
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US16/880,824
Expired - Fee Related
US11227187B1
( en )
2019-05-23
2020-05-21
Generating artificial intelligence solutions using raw data and simulated data
Country Status (1)
Country
Link
US
( 1 )
US11227187B1
( en )
Cited By (38)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20200034701A1
( en )
*
2018-07-26
2020-01-30
Sap Se
Dynamic, automated fulfillment of computer-based resource request provisioning using deep reinforcement learning
US20210144207A1
( en )
*
2019-11-12
2021-05-13
Marvell Asia Pte, Ltd.
Automotive network with centralized storage
US20210150270A1
( en )
*
2019-11-19
2021-05-20
International Business Machines Corporation
Mathematical function defined natural language annotation
US20210216077A1
( en )
*
2020-06-30
2021-07-15
Beijing Baidu Netcom Science And Technology Co., Ltd.
Method, apparatus and computer storage medium for training trajectory planning model
US20210248009A1
( en )
*
2020-02-12
2021-08-12
Beijing Xiaomi Mobile Software Co., Ltd.
Resource Processing Method and Apparatus for Mobile Terminal, Computer Device and Storage Medium
US20210374689A1
( en )
*
2020-05-30
2021-12-02
Mastercard International Incorporated
Methods and systems for performing secure transactions associated with instructions received in natural language form
US20210390352A1
( en )
*
2020-06-10
2021-12-16
Toyota Research Institute, Inc.
Confidence-estimated domain adaptation
US20220036129A1
( en )
*
2020-07-31
2022-02-03
EMC IP Holding Company LLC
Method, device, and computer program product for model updating
US20220051133A1
( en )
*
2020-08-12
2022-02-17
NEC Laboratories Europe GmbH
Decentralized multi-task learning
US20220067181A1
( en )
*
2020-09-01
2022-03-03
Argo AI, LLC
Methods and systems for secure data analysis and machine learning
US20220122308A1
( en )
*
2020-10-16
2022-04-21
Adobe Inc.
Techniques for smooth region merging in image editing
US20220172100A1
( en )
*
2020-11-27
2022-06-02
Amazon Technologies, Inc.
Feedback-based training for anomaly detection
US20230004807A1
( en )
*
2020-03-13
2023-01-05
International Business Machines Corporation
Determining optimal augmentations for a training data set
US20230096243A1
( en )
*
2019-04-11
2023-03-30
Sunghee Woo
Electronic device comprising user interface for providing user-participating-type ai training service, and server and method for providing user-participating-type ai training service using the electronic device
CN115919273A
( en )
*
2022-12-07
2023-04-07
å京ä¸çµæ®åä¿¡æ¯ææ¯æéå ¬å¸
Sub-health early warning system based on deep learning and related equipment
US20230196073A1
( en )
*
2020-05-18
2023-06-22
Idemia Identity & Security France
Method for secure use of a first neural network on an input datum and method for learning parameters of a second neural network
US11741944B2
( en )
*
2020-11-24
2023-08-29
Google Llc
Speech personalization and federated training using real world noise
US20230353637A1
( en )
*
2020-10-30
2023-11-02
Johnson Controls Tyco IP Holdings LLP
Systems and methods of configuring a building management system
US20230419113A1
( en )
*
2019-09-30
2023-12-28
Amazon Technologies, Inc.
Attention-based deep reinforcement learning for autonomous agents
US11868855B2
( en )
*
2019-11-04
2024-01-09
Hewlett Packard Enterprise Development Lp
Resiliency for machine learning workloads
US20240062098A1
( en )
*
2022-08-16
2024-02-22
Snowflake Inc.
Automated machine learning for network-based database systems
US20250053664A1
( en )
*
2023-01-19
2025-02-13
Citibank, N.A.
Detection and mitigation of machine learning model adversarial attacks
US20250117646A1
( en )
*
2023-10-04
2025-04-10
Cisco Technology, Inc.
Sustainable training of artificial intelligence and machine learning models
WO2025077876A1
( en )
*
2023-10-13
2025-04-17
å为ææ¯æéå ¬å¸
Communication method and communication apparatus
US12282565B2
( en )
2023-01-19
2025-04-22
Citibank, N.A.
Generative cybersecurity exploit synthesis and mitigation
US12316737B2
( en )
*
2022-08-30
2025-05-27
Inventec (Pudong) Technology Corporation
Method for verifying model update
US12314406B1
( en )
2023-01-19
2025-05-27
Citibank, N.A.
Generative cybersecurity exploit discovery and evaluation
US12322264B2
( en )
*
2022-07-14
2025-06-03
Paramount Bed Co., Ltd.
Data processor, care support system, and data processing method
US12367292B2
( en )
2023-01-19
2025-07-22
Citibank, N.A.
Providing user-induced variable identification of end-to-end computing system security impact information systems and methods
US12475235B2
( en )
2023-01-19
2025-11-18
Citibank, N.A.
Generative cybersecurity exploit discovery and evaluation
US12505352B2
( en )
2024-04-11
2025-12-23
Citibank, N.A
Identifying and remediating gaps in artificial intelligence use cases using a generative artificial intelligence model
CN121212259A
( en )
*
2025-11-26
2025-12-26
å±±ä¸é«éç§æåå±æéå ¬å¸
A cloud data model information processing method and system based on artificial intelligence
US20260044541A1
( en )
*
2023-05-04
2026-02-12
Vijay Madisetti
Method and System for Processing Artificial Intelligence User Requests
US12596813B2
( en )
2023-01-19
2026-04-07
Citibank, N.A
Autonomous agent observation and control
US12596738B2
( en )
2024-04-11
2026-04-07
Citibank, N.A.
Explainable large language model routing with immutable audit trails
US12602624B2
( en )
2023-12-11
2026-04-14
Citibank, N.A.
Anomaly detection method for model outputs
US12602418B2
( en )
2024-04-11
2026-04-14
Citibank, N.A.
Intelligent query decomposition, specialized model routing, and hierarchical aggregation with conflict resolution
US12608486B2
( en )
2023-01-19
2026-04-21
Citibank, N.A.
Generating predicted end-to-end cyber-security attack characteristics via bifurcated machine learning-based processing of multi-modal data systems and methods
Citations (6)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20170221072A1
( en )
*
2013-05-23
2017-08-03
GiriSrinivasaRao AthuluruTlrumala
End-to-end situation aware operations solution for customer experience centric businesses
US20170364539A1
( en )
2016-06-19
2017-12-21
Data.World, Inc.
Platform management of integrated access of public and privately-accessible datasets utilizing federated query generation and query schema rewriting optimization
WO2019112667A1
( en )
2017-12-06
2019-06-13
Google Llc
Systems and methods for distributed on-device learning with data-correlated availability
US10438094B1
( en )
*
2016-11-15
2019-10-08
Google Llc
Automatic suggestion to share images
US20200065966A1
( en )
*
2018-08-22
2020-02-27
EIO Diagnostics, Inc.
System for high performance, ai-based dairy herd management and disease detection
US10657461B2
( en )
2016-09-26
2020-05-19
Google Llc
Communication efficient federated learning
2020
2020-05-21
US
US16/880,824
patent/US11227187B1/en
not_active
Expired - Fee Related
Patent Citations (6)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20170221072A1
( en )
*
2013-05-23
2017-08-03
GiriSrinivasaRao AthuluruTlrumala
End-to-end situation aware operations solution for customer experience centric businesses
US20170364539A1
( en )
2016-06-19
2017-12-21
Data.World, Inc.
Platform management of integrated access of public and privately-accessible datasets utilizing federated query generation and query schema rewriting optimization
US10657461B2
( en )
2016-09-26
2020-05-19
Google Llc
Communication efficient federated learning
US10438094B1
( en )
*
2016-11-15
2019-10-08
Google Llc
Automatic suggestion to share images
WO2019112667A1
( en )
2017-12-06
2019-06-13
Google Llc
Systems and methods for distributed on-device learning with data-correlated availability
US20200065966A1
( en )
*
2018-08-22
2020-02-27
EIO Diagnostics, Inc.
System for high performance, ai-based dairy herd management and disease detection
Non-Patent Citations (2)
* Cited by examiner, â Cited by third party
Title
Lauter, K., et al., " Can Homomorphic Encryption be Practical? ", Proceedings of the 3rd ACM Cloud Computing Security Workshop, CCSW 2011, Chicago, IL, USA, Oct. 21, 2011, 18 pages.
Yang, T., et al., " Applied Federated Learning: Improving Google Keyboard Query Suggestions. " arXiv preprint arXiv:1812.02903, https://arxiv.org/pdf/1812.02903.pdf (2018), 9 pages.
Cited By (70)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20200034701A1
( en )
*
2018-07-26
2020-01-30
Sap Se
Dynamic, automated fulfillment of computer-based resource request provisioning using deep reinforcement learning
US11423295B2
( en )
*
2018-07-26
2022-08-23
Sap Se
Dynamic, automated fulfillment of computer-based resource request provisioning using deep reinforcement learning
US20230096243A1
( en )
*
2019-04-11
2023-03-30
Sunghee Woo
Electronic device comprising user interface for providing user-participating-type ai training service, and server and method for providing user-participating-type ai training service using the electronic device
US12468963B2
( en )
*
2019-04-11
2025-11-11
Sunghee Woo
Server and method for providing user-participating-type AI training service using the electronic device
US20230419113A1
( en )
*
2019-09-30
2023-12-28
Amazon Technologies, Inc.
Attention-based deep reinforcement learning for autonomous agents
US11868855B2
( en )
*
2019-11-04
2024-01-09
Hewlett Packard Enterprise Development Lp
Resiliency for machine learning workloads
US20210144207A1
( en )
*
2019-11-12
2021-05-13
Marvell Asia Pte, Ltd.
Automotive network with centralized storage
US11683371B2
( en )
*
2019-11-12
2023-06-20
Marvell Asia Pte Ltd
Automotive network with centralized storage
US12197846B2
( en )
*
2019-11-19
2025-01-14
International Business Machines Corporation
Mathematical function defined natural language annotation
US20210150270A1
( en )
*
2019-11-19
2021-05-20
International Business Machines Corporation
Mathematical function defined natural language annotation
US20210248009A1
( en )
*
2020-02-12
2021-08-12
Beijing Xiaomi Mobile Software Co., Ltd.
Resource Processing Method and Apparatus for Mobile Terminal, Computer Device and Storage Medium
US11561837B2
( en )
*
2020-02-12
2023-01-24
Beijing Xiaomi Mobile Software Co., Ltd.
Resource processing method and apparatus for mobile terminal, computer device and storage medium
US11790231B2
( en )
*
2020-03-13
2023-10-17
International Business Machines Corporation
Determining optimal augmentations for a training data set
US20230004807A1
( en )
*
2020-03-13
2023-01-05
International Business Machines Corporation
Determining optimal augmentations for a training data set
US11551088B2
( en )
*
2020-03-13
2023-01-10
International Business Machines Corporation
Determining optimal augmentations for a training data set
US20230196073A1
( en )
*
2020-05-18
2023-06-22
Idemia Identity & Security France
Method for secure use of a first neural network on an input datum and method for learning parameters of a second neural network
US20210374689A1
( en )
*
2020-05-30
2021-12-02
Mastercard International Incorporated
Methods and systems for performing secure transactions associated with instructions received in natural language form
US12450577B2
( en )
*
2020-05-30
2025-10-21
Mastercard International Incorporated
Methods and systems for performing secure transactions associated with instructions received in natural language form
US11887066B2
( en )
*
2020-05-30
2024-01-30
Mastercard International Incorporated
Methods and systems for performing secure transactions associated with instructions received in natural language form
US20240020658A1
( en )
*
2020-05-30
2024-01-18
Mastercard International Incorporated
Methods and systems for performing secure transactions associated with instructions received in natural language form
US11741698B2
( en )
*
2020-06-10
2023-08-29
Toyota Research Institute, Inc.
Confidence-estimated domain adaptation for training machine learning models
US20210390352A1
( en )
*
2020-06-10
2021-12-16
Toyota Research Institute, Inc.
Confidence-estimated domain adaptation
US11940803B2
( en )
*
2020-06-30
2024-03-26
Beijing Baidu Netcom Science And Technology Co., Ltd.
Method, apparatus and computer storage medium for training trajectory planning model
US20210216077A1
( en )
*
2020-06-30
2021-07-15
Beijing Baidu Netcom Science And Technology Co., Ltd.
Method, apparatus and computer storage medium for training trajectory planning model
US11562173B2
( en )
*
2020-07-31
2023-01-24
EMC IP Holding Company LLC
Method, device, and computer program product for model updating
US20220036129A1
( en )
*
2020-07-31
2022-02-03
EMC IP Holding Company LLC
Method, device, and computer program product for model updating
US20220051133A1
( en )
*
2020-08-12
2022-02-17
NEC Laboratories Europe GmbH
Decentralized multi-task learning
US12400010B2
( en )
*
2020-09-01
2025-08-26
Volkswagen Group of America Investments, LLC
Methods and systems for secure data analysis and machine learning
US20220067181A1
( en )
*
2020-09-01
2022-03-03
Argo AI, LLC
Methods and systems for secure data analysis and machine learning
US11915133B2
( en )
*
2020-10-16
2024-02-27
Adobe Inc.
Techniques for smooth region merging in image editing
US20220122308A1
( en )
*
2020-10-16
2022-04-21
Adobe Inc.
Techniques for smooth region merging in image editing
US11875221B2
( en )
2020-10-16
2024-01-16
Adobe Inc.
Attribute decorrelation techniques for image editing
US11907839B2
( en )
2020-10-16
2024-02-20
Adobe Inc.
Detail-preserving image editing techniques
US12333427B2
( en )
2020-10-16
2025-06-17
Adobe Inc.
Multi-scale output techniques for generative adversarial networks
US12327188B2
( en )
2020-10-16
2025-06-10
Adobe Inc.
Direct regression encoder architecture and training
US12412089B2
( en )
2020-10-16
2025-09-09
Adobe Inc.
Supervised learning techniques for encoder training
US11983628B2
( en )
2020-10-16
2024-05-14
Adobe Inc.
Attribute control techniques for image editing
US12530592B2
( en )
2020-10-16
2026-01-20
Adobe Inc.
Non-linear latent filter techniques for image editing
US11880766B2
( en )
2020-10-16
2024-01-23
Adobe Inc.
Techniques for domain to domain projection using a generative model
US12231496B2
( en )
2020-10-30
2025-02-18
Tyco Fire & Security Gmbh
Building management system with dynamic building model enhanced by digital twins
US12432277B2
( en )
*
2020-10-30
2025-09-30
Tyco Fire & Security Gmbh
Systems and methods of configuring a building management system
US12542830B2
( en )
2020-10-30
2026-02-03
Tyco Fire & Security Gmbh
Building management system with configuration by building model augmentation
US20230353637A1
( en )
*
2020-10-30
2023-11-02
Johnson Controls Tyco IP Holdings LLP
Systems and methods of configuring a building management system
US11741944B2
( en )
*
2020-11-24
2023-08-29
Google Llc
Speech personalization and federated training using real world noise
US12165630B2
( en )
*
2020-11-24
2024-12-10
Google Llc
Speech personalization and federated training using real world noise
US20230360639A1
( en )
*
2020-11-24
2023-11-09
Google Llc
Speech Personalization and Federated Training Using Real World Noise
US12147878B2
( en )
*
2020-11-27
2024-11-19
Amazon Technologies, Inc.
Feedback-based training for anomaly detection
US20220172100A1
( en )
*
2020-11-27
2022-06-02
Amazon Technologies, Inc.
Feedback-based training for anomaly detection
US12322264B2
( en )
*
2022-07-14
2025-06-03
Paramount Bed Co., Ltd.
Data processor, care support system, and data processing method
US20240062098A1
( en )
*
2022-08-16
2024-02-22
Snowflake Inc.
Automated machine learning for network-based database systems
US12626186B2
( en )
*
2022-08-16
2026-05-12
Snowflake Inc.
Automated machine learning for network-based database systems
US12316737B2
( en )
*
2022-08-30
2025-05-27
Inventec (Pudong) Technology Corporation
Method for verifying model update
CN115919273A
( en )
*
2022-12-07
2023-04-07
å京ä¸çµæ®åä¿¡æ¯ææ¯æéå ¬å¸
Sub-health early warning system based on deep learning and related equipment
US20250053664A1
( en )
*
2023-01-19
2025-02-13
Citibank, N.A.
Detection and mitigation of machine learning model adversarial attacks
US12367292B2
( en )
2023-01-19
2025-07-22
Citibank, N.A.
Providing user-induced variable identification of end-to-end computing system security impact information systems and methods
US12314406B1
( en )
2023-01-19
2025-05-27
Citibank, N.A.
Generative cybersecurity exploit discovery and evaluation
US12282565B2
( en )
2023-01-19
2025-04-22
Citibank, N.A.
Generative cybersecurity exploit synthesis and mitigation
US12475235B2
( en )
2023-01-19
2025-11-18
Citibank, N.A.
Generative cybersecurity exploit discovery and evaluation
US12271491B2
( en )
*
2023-01-19
<td itemprop="publicat