ABSTRACT
Abstract
Methods, apparatus, systems, and articles of manufacture are disclosed for automatically updating artificial intelligence models operating on data of a first factory production line, the apparatus comprising, an intelligent trigger circuitry to trigger an automated model update process, an automated model search circuitry to, in response to a model update, generate a plurality of candidate artificial intelligence models, and an intelligent model deployment circuitry to output a prediction of an artificial intelligence model combination to improve prediction performance over time.
Description
RELATED APPLICATION
This patent claims the benefit of U.S. Provisional Patent Application No. 63/182,585, filed Apr. 30, 2021, which is hereby incorporated herein by reference in its entirety. Priority to U.S. Patent Application No. 63/182,585 is hereby claimed.
FIELD OF THE DISCLOSURE
This disclosure relates generally to machine learning, and, more particularly, to methods and apparatus to automatically update artificial intelligence models for autonomous factories.
BACKGROUND
Data collection and technology for data analysis continues to advance at a rapid pace. For example, factories that manufacture products through the use of assembly lines may gather data throughout the manufacturing process. In some examples, a first part is produced on a first assembly (e.g., production) line, and a second part that is identical to the first part may be produced on a second assembly line that is identical to the first assembly line. In recent years, machine learning algorithms have been used to model such data.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates an overview of an edge cloud configuration for edge computing.
FIG. 2 illustrates operational layers among endpoints, an edge cloud, and cloud computing environments.
FIG. 3 illustrates an example approach for networking and services in an edge computing system.
FIG. 4 is a block diagram of an example environment in which a model update controller circuitry operates to automatically update artificial intelligence models for autonomous factories.
FIG. 5 is a block diagram of an example implementation of the model update controller circuitry of FIG. 4 .
FIG. 6 is a flowchart representative of example machine readable instructions that may be executed by example processor circuitry to implement the model update controller circuitry of FIG. 5 .
FIG. 7 is a flowchart representative of example machine readable instructions that may be executed by example processor circuitry to implement the model update controller circuitry of FIG. 5 .
FIG. 8 is a flowchart of the process executed by the environment of where the model update controller circuitry of FIG. 4 operates.
FIG. 9 is an illustration of a data table generated by the deployment circuitry of the model update controller circuitry of FIG. 5 .
FIG. 10A provides an overview of example components for compute deployed at a compute node in an edge computing system.
FIG. 10B provides a further overview of example components within a computing device in an edge computing system.
FIG. 11 is a block diagram of an example processing platform including processor circuitry structured to execute the example machine readable instructions of FIG. 6 and FIG. 7 to implement the model update controller circuitry of FIG. 5 .
FIG. 12 is a block diagram of an example implementation of the processor circuitry of FIG. 11 .
FIG. 13 is a block diagram of another example implementation of the processor circuitry of FIG. 11 .
FIG. 14 is a block diagram of an example software distribution platform (e.g., one or more servers) to distribute software (e.g., software corresponding to the example machine readable instructions of FIGS. 6-7 ) to client devices associated with end users and/or consumers (e.g., for license, sale, and/or use), retailers (e.g., for sale, re-sale, license, and/or sub-license), and/or original equipment manufacturers (OEMs) (e.g., for inclusion in products to be distributed to, for example, retailers and/or to other end users such as direct buy customers).
The figures are not to scale. Instead, the thickness of the layers or regions may be enlarged in the drawings. As used herein, connection references (e.g., attached, coupled, connected, and joined) may include intermediate members between the elements referenced by the connection reference and/or relative movement between those elements unless otherwise indicated. As such, connection references do not necessarily infer that two elements are directly connected and/or in fixed relation to each other. As used herein, stating that any part is in âcontactâ with another part is defined to mean that there is no intermediate part between the two parts.
Unless specifically stated otherwise, descriptors such as âfirst,â âsecond,â âthird,â etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. In some examples, the descriptor âfirstâ may be used to refer to an element in the detailed description, while the same element may be referred to in a claim with a different descriptor such as âsecondâ or âthird.â In such instances, it should be understood that such descriptors are used merely for identifying those elements distinctly that might, for example, otherwise share a same name. As used herein, âapproximatelyâ and âaboutâ refer to dimensions that may not be exact due to manufacturing tolerances and/or other real world imperfections. As used herein âsubstantially real timeâ refers to occurrence in a near instantaneous manner recognizing there may be real world delays for computing time, transmission, etc. Thus, unless otherwise specified, âsubstantially real timeâ refers to real time+/â1 second. As used herein, the phrase âin communication,â including variations thereof, encompasses direct communication and/or indirect communication through one or more intermediary components, and does not require direct physical (e.g., wired) communication and/or constant communication, but rather additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and/or one-time events. As used herein, âprocessor circuitryâ is defined to include (i) one or more special purpose electrical circuits structured to perform specific operation(s) and including one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors), and/or (ii) one or more general purpose semiconductor-based electrical circuits programmed with instructions to perform specific operations and including one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors). Examples of processor circuitry include programmed microprocessors, Field Programmable Gate Arrays (FPGAs) that may instantiate instructions, Central Processor Units (CPUs), Graphics Processor Units (GPUs), Digital Signal Processors (DSPs), XPUs, or microcontrollers and integrated circuits such as Application Specific Integrated Circuits (ASICs). For example, an XPU may be implemented by a heterogeneous computing system including multiple types of processor circuitry (e.g., one or more FPGAs, one or more CPUs, one or more GPUs, one or more DSPs, etc., and/or a combination thereof) and application programming interface(s) (API(s)) that may assign computing task(s) to whichever one(s) of the multiple types of the processing circuitry is/are best suited to execute the computing task(s).
As used herein, data is information in any form that may be ingested, processed, interpreted and/or otherwise manipulated by processor circuitry to produce a result. The produced result may itself be data.
As used herein âthresholdâ is expressed as data such as a numerical value represented in any form, that may be used by processor circuitry as a reference for a comparison operation.
As used herein, a model is a set of instructions and/or data that may be ingested, processed, interpreted and/or otherwise manipulated by processor circuitry to produce a result. Often, a model is operated using input data to produce output data in accordance with one or more relationships reflected in the model. The model may be based on training data. In some examples, a model is a structure of numbers and relationships to be used in artificial intelligence and/or decision-making logic.
As used herein, a configuration is an arrangement of data to identify and define how a machine is set up.
As used herein, a score may be a numerical value or dimensionless number such as a percentage.
DETAILED DESCRIPTION
Artificial intelligence (AI), including machine learning (ML), deep learning (DL), and/or other artificial machine-driven logic, enables machines (e.g., computers, logic circuits, etc.) to use a model to process input data to generate an output based on patterns and/or associations previously learned by the model via a training process. For instance, the model may be trained with data to recognize patterns and/or associations and follow such patterns and/or associations when processing input data such that other input(s) result in output(s) consistent with the recognized patterns and/or associations.
Many different types of AI models and/or AI architectures exist. In general, AI models/architectures that are suitable to use in the example approaches disclosed herein will be artificial neural network models (e.g., convolutional neural networks, recurrent neural networks, etc.) and machine learning models (e.g., random forest classifiers, support vector machines, etc.). However, other types of machine learning models could additionally or alternatively be used such as reinforcement learning models etc.
In general, implementing a ML/AI system involves two phases, a learning/training phase and an inference phase. In the learning/training phase, a training algorithm is used to train a model to operate in accordance with patterns and/or associations based on, for example, training data. In general, the model includes internal parameters that guide how input data is transformed into output data, such as through a series of nodes and connections within the model to transform input data into output data. Additionally, hyperparameters are used as part of the training process to control how the learning is performed (e.g., a learning rate, a number of layers to be used in the machine learning model, etc.). Hyperparameters are defined to be training parameters that are determined prior to initiating the training process.
Different types of training may be performed based on the type of ML/AI model and/or the expected output. For example, supervised training uses inputs and corresponding expected (e.g., labeled) outputs to select parameters (e.g., by iterating over combinations of select parameters) for the ML/AI model that reduce model error. As used herein, labelling refers to an expected output of the machine learning model (e.g., a classification, an expected output value, etc.) Alternatively, unsupervised training (e.g., used in deep learning, a subset of machine learning, etc.) involves inferring patterns from inputs to select parameters for the ML/AI model (e.g., without the benefit of expected (e.g., labeled) outputs).
In examples disclosed herein, ML/AI models are trained based on the model type and architecture, for example, neural networks can be trained with stochastic gradient descent. However, any other training algorithm may additionally or alternatively be used. In examples disclosed herein, training is performed until the model output is converged. In examples disclosed herein, training is performed at remotely (e.g., at a central facility). In other examples, training is performed locally (e.g., at an edge device at the factory). Training is performed using hyperparameters that control how the learning is performed (e.g., a learning rate, a number of layers to be used in the machine learning model, etc.). In examples disclosed herein, hyperparameters that control the output of the model include the number of nodes, the number of layers etc. Such hyperparameters are selected by, for example, by an update of the previous model, randomly generated, or searched by an optimization algorithm. In some examples re-training may be performed. Such re-training may be performed in response to intelligent trigger circuitry as described in FIG. 5 .
Training is performed using training data. In some examples supervised training is used, and the training data is labeled.
Once training is complete, the model is deployed for use as an executable construct that processes an input and provides an output based on the network of nodes and connections defined in the model. In some examples, the model is a logistic regression model, a random forest model, or a gradient boosted tree model, etc. The model is stored at the model repository as described in FIG. 5 . The model may then be executed by the intelligent deployment circuitry as described in FIG. 5 . In some examples, multiple models are deployed and evaluated by the intelligent deployment circuitry as described in FIG. 5 .
Once trained, the deployed model may be operated in an inference phase to process data. In the inference phase, data to be analyzed (e.g., live data) is input to the model, and the model executes to create an output. This inference phase can be thought of as the AI âthinkingâ to generate the output based on what it learned from the training (e.g., by executing the model to apply the learned patterns and/or associations to the live data). In some examples, input data undergoes pre-processing before being used as an input to the machine learning model. Moreover, in some examples, the output data may undergo post-processing after it is generated by the AI model to transform the output into a useful result (e.g., a display of data, an instruction to be executed by a machine, etc.).
In some examples, output of the deployed model may be captured and provided as feedback. By analyzing the feedback, an accuracy of the deployed model can be determined. If the feedback indicates that the accuracy of the deployed model is less than a threshold or othe
RELATED APPLICATION
This patent claims the benefit of U.S. Provisional Patent Application No. 63/182,585, filed Apr. 30, 2021, which is hereby incorporated herein by reference in its entirety. Priority to U.S. Patent Application No. 63/182,585 is hereby claimed.
FIELD OF THE DISCLOSURE
This disclosure relates generally to machine learning, and, more particularly, to methods and apparatus to automatically update artificial intelligence models for autonomous factories.
BACKGROUND
Data collection and technology for data analysis continues to advance at a rapid pace. For example, factories that manufacture products through the use of assembly lines may gather data throughout the manufacturing process. In some examples, a first part is produced on a first assembly (e.g., production) line, and a second part that is identical to the first part may be produced on a second assembly line that is identical to the first assembly line. In recent years, machine learning algorithms have been used to model such data.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates an overview of an edge cloud configuration for edge computing.
FIG. 2 illustrates operational layers among endpoints, an edge cloud, and cloud computing environments.
FIG. 3 illustrates an example approach for networking and services in an edge computing system.
FIG. 4 is a block diagram of an example environment in which a model update controller circuitry operates to automatically update artificial intelligence models for autonomous factories.
FIG. 5 is a block diagram of an example implementation of the model update controller circuitry of FIG. 4 .
FIG. 6 is a flowchart representative of example machine readable instructions that may be executed by example processor circuitry to implement the model update controller circuitry of FIG. 5 .
FIG. 7 is a flowchart representative of example machine readable instructions that may be executed by example processor circuitry to implement the model update controller circuitry of FIG. 5 .
FIG. 8 is a flowchart of the process executed by the environment of where the model update controller circuitry of FIG. 4 operates.
FIG. 9 is an illustration of a data table generated by the deployment circuitry of the model update controller circuitry of FIG. 5 .
FIG. 10A provides an overview of example components for compute deployed at a compute node in an edge computing system.
FIG. 10B provides a further overview of example components within a computing device in an edge computing system.
FIG. 11 is a block diagram of an example processing platform including processor circuitry structured to execute the example machine readable instructions of FIG. 6 and FIG. 7 to implement the model update controller circuitry of FIG. 5 .
FIG. 12 is a block diagram of an example implementation of the processor circuitry of FIG. 11 .
FIG. 13 is a block diagram of another example implementation of the processor circuitry of FIG. 11 .
FIG. 14 is a block diagram of an example software distribution platform (e.g., one or more servers) to distribute software (e.g., software corresponding to the example machine readable instructions of FIGS. 6-7 ) to client devices associated with end users and/or consumers (e.g., for license, sale, and/or use), retailers (e.g., for sale, re-sale, license, and/or sub-license), and/or original equipment manufacturers (OEMs) (e.g., for inclusion in products to be distributed to, for example, retailers and/or to other end users such as direct buy customers).
The figures are not to scale. Instead, the thickness of the layers or regions may be enlarged in the drawings. As used herein, connection references (e.g., attached, coupled, connected, and joined) may include intermediate members between the elements referenced by the connection reference and/or relative movement between those elements unless otherwise indicated. As such, connection references do not necessarily infer that two elements are directly connected and/or in fixed relation to each other. As used herein, stating that any part is in âcontactâ with another part is defined to mean that there is no intermediate part between the two parts.
Unless specifically stated otherwise, descriptors such as âfirst,â âsecond,â âthird,â etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. In some examples, the descriptor âfirstâ may be used to refer to an element in the detailed description, while the same element may be referred to in a claim with a different descriptor such as âsecondâ or âthird.â In such instances, it should be understood that such descriptors are used merely for identifying those elements distinctly that might, for example, otherwise share a same name. As used herein, âapproximatelyâ and âaboutâ refer to dimensions that may not be exact due to manufacturing tolerances and/or other real world imperfections. As used herein âsubstantially real timeâ refers to occurrence in a near instantaneous manner recognizing there may be real world delays for computing time, transmission, etc. Thus, unless otherwise specified, âsubstantially real timeâ refers to real time+/â1 second. As used herein, the phrase âin communication,â including variations thereof, encompasses direct communication and/or indirect communication through one or more intermediary components, and does not require direct physical (e.g., wired) communication and/or constant communication, but rather additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and/or one-time events. As used herein, âprocessor circuitryâ is defined to include (i) one or more special purpose electrical circuits structured to perform specific operation(s) and including one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors), and/or (ii) one or more general purpose semiconductor-based electrical circuits programmed with instructions to perform specific operations and including one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors). Examples of processor circuitry include programmed microprocessors, Field Programmable Gate Arrays (FPGAs) that may instantiate instructions, Central Processor Units (CPUs), Graphics Processor Units (GPUs), Digital Signal Processors (DSPs), XPUs, or microcontrollers and integrated circuits such as Application Specific Integrated Circuits (ASICs). For example, an XPU may be implemented by a heterogeneous computing system including multiple types of processor circuitry (e.g., one or more FPGAs, one or more CPUs, one or more GPUs, one or more DSPs, etc., and/or a combination thereof) and application programming interface(s) (API(s)) that may assign computing task(s) to whichever one(s) of the multiple types of the processing circuitry is/are best suited to execute the computing task(s).
As used herein, data is information in any form that may be ingested, processed, interpreted and/or otherwise manipulated by processor circuitry to produce a result. The produced result may itself be data.
As used herein âthresholdâ is expressed as data such as a numerical value represented in any form, that may be used by processor circuitry as a reference for a comparison operation.
As used herein, a model is a set of instructions and/or data that may be ingested, processed, interpreted and/or otherwise manipulated by processor circuitry to produce a result. Often, a model is operated using input data to produce output data in accordance with one or more relationships reflected in the model. The model may be based on training data. In some examples, a model is a structure of numbers and relationships to be used in artificial intelligence and/or decision-making logic.
As used herein, a configuration is an arrangement of data to identify and define how a machine is set up.
As used herein, a score may be a numerical value or dimensionless number such as a percentage.
DETAILED DESCRIPTION
Artificial intelligence (AI), including machine learning (ML), deep learning (DL), and/or other artificial machine-driven logic, enables machines (e.g., computers, logic circuits, etc.) to use a model to process input data to generate an output based on patterns and/or associations previously learned by the model via a training process. For instance, the model may be trained with data to recognize patterns and/or associations and follow such patterns and/or associations when processing input data such that other input(s) result in output(s) consistent with the recognized patterns and/or associations.
Many different types of AI models and/or AI architectures exist. In general, AI models/architectures that are suitable to use in the example approaches disclosed herein will be artificial neural network models (e.g., convolutional neural networks, recurrent neural networks, etc.) and machine learning models (e.g., random forest classifiers, support vector machines, etc.). However, other types of machine learning models could additionally or alternatively be used such as reinforcement learning models etc.
In general, implementing a ML/AI system involves two phases, a learning/training phase and an inference phase. In the learning/training phase, a training algorithm is used to train a model to operate in accordance with patterns and/or associations based on, for example, training data. In general, the model includes internal parameters that guide how input data is transformed into output data, such as through a series of nodes and connections within the model to transform input data into output data. Additionally, hyperparameters are used as part of the training process to control how the learning is performed (e.g., a learning rate, a number of layers to be used in the machine learning model, etc.). Hyperparameters are defined to be training parameters that are determined prior to initiating the training process.
Different types of training may be performed based on the type of ML/AI model and/or the expected output. For example, supervised training uses inputs and corresponding expected (e.g., labeled) outputs to select parameters (e.g., by iterating over combinations of select parameters) for the ML/AI model that reduce model error. As used herein, labelling refers to an expected output of the machine learning model (e.g., a classification, an expected output value, etc.) Alternatively, unsupervised training (e.g., used in deep learning, a subset of machine learning, etc.) involves inferring patterns from inputs to select parameters for the ML/AI model (e.g., without the benefit of expected (e.g., labeled) outputs).
In examples disclosed herein, ML/AI models are trained based on the model type and architecture, for example, neural networks can be trained with stochastic gradient descent. However, any other training algorithm may additionally or alternatively be used. In examples disclosed herein, training is performed until the model output is converged. In examples disclosed herein, training is performed at remotely (e.g., at a central facility). In other examples, training is performed locally (e.g., at an edge device at the factory). Training is performed using hyperparameters that control how the learning is performed (e.g., a learning rate, a number of layers to be used in the machine learning model, etc.). In examples disclosed herein, hyperparameters that control the output of the model include the number of nodes, the number of layers etc. Such hyperparameters are selected by, for example, by an update of the previous model, randomly generated, or searched by an optimization algorithm. In some examples re-training may be performed. Such re-training may be performed in response to intelligent trigger circuitry as described in FIG. 5 .
Training is performed using training data. In some examples supervised training is used, and the training data is labeled.
Once training is complete, the model is deployed for use as an executable construct that processes an input and provides an output based on the network of nodes and connections defined in the model. In some examples, the model is a logistic regression model, a random forest model, or a gradient boosted tree model, etc. The model is stored at the model repository as described in FIG. 5 . The model may then be executed by the intelligent deployment circuitry as described in FIG. 5 . In some examples, multiple models are deployed and evaluated by the intelligent deployment circuitry as described in FIG. 5 .
Once trained, the deployed model may be operated in an inference phase to process data. In the inference phase, data to be analyzed (e.g., live data) is input to the model, and the model executes to create an output. This inference phase can be thought of as the AI âthinkingâ to generate the output based on what it learned from the training (e.g., by executing the model to apply the learned patterns and/or associations to the live data). In some examples, input data undergoes pre-processing before being used as an input to the machine learning model. Moreover, in some examples, the output data may undergo post-processing after it is generated by the AI model to transform the output into a useful result (e.g., a display of data, an instruction to be executed by a machine, etc.).
In some examples, output of the deployed model may be captured and provided as feedback. By analyzing the feedback, an accuracy of the deployed model can be determined. If the feedback indicates that the accuracy of the deployed model is less than a threshold or other criterion, training of an updated model can be triggered using the feedback and an updated training data set, hyperparameters, etc., to generate an updated, deployed model.
FIG. 1 is a block diagram 100 showing an overview of a configuration for edge computing, which includes a layer of processing referred to in many of the following examples as an âedge cloudâ. As shown, the edge cloud 110 is co-located at an edge location, such as an access point or base station 140 , a local processing hub 150 , or a central office 120 , and thus may include multiple entities, devices, and equipment instances. The edge cloud 110 is located much closer to the endpoint (consumer and producer) data sources 160 (e.g., autonomous vehicles 161 , user equipment 162 , business and industrial equipment 163 , video capture devices 164 , drones 165 , smart cities and building devices 166 , sensors and IoT devices 167 , etc.) than the cloud data center 130 . Compute, memory, and storage resources which are offered at the edges in the edge cloud 110 are critical to providing ultra-low latency response times for services and functions used by the endpoint data sources 160 as well as reduce network backhaul traffic from the edge cloud 110 toward cloud data center 130 thus improving energy consumption and overall network usages among other benefits.
Compute, memory, and storage are scarce resources, and generally decrease depending on the edge location (e.g., fewer processing resources being available at consumer endpoint devices, than at a base station, than at a central office). However, the closer that the edge location is to the endpoint (e.g., user equipment (UE)), the more that space and power is often constrained. Thus, edge computing attempts to reduce the amount of resources needed for network services, through the distribution of more resources which are located closer both geographically and in network access time. In this manner, edge computing attempts to bring the compute resources to the workload data where appropriate, or, bring the workload data to the compute resources.
The following describes aspects of an edge cloud architecture that covers multiple potential deployments and addresses restrictions that some network operators or service providers may have in their own infrastructures. These include, variation of configurations based on the edge location (because edges at a base station level, for instance, may have more constrained performance and capabilities in a multi-tenant scenario); configurations based on the type of compute, memory, storage, fabric, acceleration, or like resources available to edge locations, tiers of locations, or groups of locations; the service, security, and management and orchestration capabilities; and related objectives to achieve usability and performance of end services. These deployments may accomplish processing in network layers that may be considered as ânear edgeâ, âclose edgeâ, âlocal edgeâ, âmiddle edgeâ, or âfar edgeâ layers, depending on latency, distance, and timing characteristics.
Edge computing is a developing paradigm where computing is performed at or closer to the âedgeâ of a network, typically through the use of a compute platform (e.g., x86 or ARM compute hardware architecture) implemented at base stations, gateways, network routers, or other devices which are much closer to endpoint devices producing and consuming the data. For example, edge gateway servers may be equipped with pools of memory and storage resources to perform computation in real-time for low latency use-cases (e.g., autonomous driving or video surveillance) for connected client devices. Or as an example, base stations may be augmented with compute and acceleration resources to directly process service workloads for connected user equipment, without further communicating data via backhaul networks. Or as another example, central office network management hardware may be replaced with standardized compute hardware that performs virtualized network functions and offers compute resources for the execution of services and consumer functions for connected devices. Within edge computing networks, there may be scenarios in services which the compute resource will be âmovedâ to the data, as well as scenarios in which the data will be âmovedâ to the compute resource. Or as an example, base station compute, acceleration and network resources can provide services in order to scale to workload demands on an as needed basis by activating dormant capacity (subscription, capacity on demand) in order to manage corner cases, emergencies or to provide longevity for deployed resources over a significantly longer implemented lifecycle.
FIG. 2 illustrates operational layers among endpoints, an edge cloud, and cloud computing environments. Specifically, FIG. 2 depicts examples of computational use cases 205 , utilizing the edge cloud 110 among multiple illustrative layers of network computing. The layers begin at an endpoint (devices and things) layer 200 , which accesses the edge cloud 110 to conduct data creation, analysis, and data consumption activities. The edge cloud 110 may span multiple network layers, such as an edge devices layer 210 having gateways, on-premise servers, or network equipment (nodes 215 ) located in physically proximate edge systems; a network access layer 220 , encompassing base stations, radio processing units, network hubs, regional data centers (DC), or local network equipment (equipment 225 ); and any equipment, devices, or nodes located therebetween (in layer 212 , not illustrated in detail). The network communications within the edge cloud 110 and among the various layers may occur via any number of wired or wireless mediums, including via connectivity architectures and technologies not depicted.
Examples of latency, resulting from network communication distance and processing time constraints, may range from less than a millisecond (ms) when among the endpoint layer 200 , under 5 ms at the edge devices layer 210 , to even between 10 to 40 ms when communicating with nodes at the network access layer 220 . Beyond the edge cloud 110 are core network 230 and cloud data center 240 layers, each with increasing latency (e.g., between 50-60 ms at the core network layer 230 , to 100 or more ms at the cloud data center layer). As a result, operations at a core network data center 235 or a cloud data center 245 , with latencies of at least 50 to 100 ms or more, will not be able to accomplish many time-critical functions of the use cases 205 . Each of these latency values are provided for purposes of illustration and contrast; it will be understood that the use of other access network mediums and technologies may further reduce the latencies. In some examples, respective portions of the network may be categorized as âclose edgeâ, âlocal edgeâ, ânear edgeâ, âmiddle edgeâ, or âfar edgeâ layers, relative to a network source and destination. For instance, from the perspective of the core network data center 235 or a cloud data center 245 , a central office or content data network may be considered as being located within a ânear edgeâ layer (ânearâ to the cloud, having high latency values when communicating with the devices and endpoints of the use cases 205 ), whereas an access point, base station, on-premise server, or network gateway may be considered as located within a âfar edgeâ layer (âfarâ from the cloud, having low latency values when communicating with the devices and endpoints of the use cases 205 ). It will be understood that other categorizations of a particular network layer as constituting a âcloseâ, âlocalâ, ânearâ, âmiddleâ, or âfarâ edge may be based on latency, distance, number of network hops, or other measurable characteristics, as measured from a source in any of the network layers 200 - 240 .
The various use cases 205 may access resources under usage pressure from incoming streams, due to multiple services utilizing the edge cloud. To achieve results with low latency, the services executed within the edge cloud 110 balance varying requirements in terms of: (a) Priority (throughput or latency) and Quality of Service (QoS) (e.g., traffic for an autonomous car may have higher priority than a temperature sensor in terms of response time requirement; or, a performance sensitivity/bottleneck may exist at a compute/accelerator, memory, storage, or network resource, depending on the application); (b) Reliability and Resiliency (e.g., some input streams need to be acted upon and the traffic routed with mission-critical reliability, where as some other input streams may be tolerate an occasional failure, depending on the application); and (c) Physical constraints (e.g., power, cooling and form-factor).
The end-to-end service view for these use cases involves the concept of a service-flow and is associated with a transaction. The transaction details the overall service requirement for the entity consuming the service, as well as the associated services for the resources, workloads, workflows, and business functional and business level requirements. The services executed with the âtermsâ described may be managed at each layer in a way to assure real time, and runtime contractual compliance for the transaction during the lifecycle of the service. When a component in the transaction is missing its agreed to SLA, the system as a whole (components in the transaction) may provide the ability to (1) understand the impact of the SLA violation, and (2) augment other components in the system to resume overall transaction SLA, and (3) implement steps to remediate.
Thus, with these variations and service features in mind, edge computing within the edge cloud 110 may provide the ability to serve and respond to multiple applications of the use cases 205 (e.g., object tracking, video surveillance, connected cars, etc.) in real-time or near real-time, and meet ultra-low latency requirements for these multiple applications. These advantages enable a whole new class of applications (Virtual Network Functions (VNFs), Function as a Service (FaaS), Edge as a Service (EaaS), standard processes, etc.), which cannot leverage conventional cloud computing due to latency or other limitations.
However, with the advantages of edge computing comes the following caveats. The devices located at the edge are often resource constrained and therefore there is pressure on usage of edge resources. Typically, this is addressed through the pooling of memory and storage resources for use by multiple users (tenants) and devices. The edge may be power and cooling constrained and therefore the power usage needs to be accounted for by the applications that are consuming the most power. There may be inherent power-performance tradeoffs in these pooled memory resources, as many of them are likely to use emerging memory technologies, where more power requires greater memory bandwidth. Likewise, improved security of hardware and root of trust trusted functions are also required, because edge locations may be unmanned and may even need permissioned access (e.g., when housed in a third-party location). Such issues are magnified in the edge cloud 110 in a multi-tenant, multi-owner, or multi-access setting, where services and applications are requested by many users, especially as network usage dynamically fluctuates and the composition of the multiple stakeholders, use cases, and services changes.
At a more generic level, an edge computing system may be described to encompass any number of deployments at the previously discussed layers operating in the edge cloud 110 (network layers 200 - 240 ), which provide coordination from client and distributed computing devices. One or more edge gateway nodes, one or more edge aggregation nodes, and one or more core data centers may be distributed across layers of the network to provide an implementation of the edge computing system by or on behalf of a telecommunication service provider (âtelcoâ, or âTSPâ), internet-of-things service provider, cloud service provider (CSP), enterprise entity, or any other number of entities. Various implementations and configurations of the edge computing system may be provided dynamically, such as when orchestrated to meet service objectives.
Consistent with the examples provided herein, a client compute node may be embodied as any type of endpoint component, device, appliance, or other thing capable of communicating as a producer or consumer of data. Further, the label ânodeâ or âdeviceâ as used in the edge computing system does not necessarily mean that such node or device operates in a client or agent/minion/follower role; rather, any of the nodes or devices in the edge computing system refer to individual entities, nodes, or subsystems which include discrete or connected hardware or software configurations to facilitate or use the edge cloud 110 .
As such, the edge cloud 110 is formed from network components and functional features operated by and within edge gateway nodes, edge aggregation nodes, or other edge compute nodes among network layers 210 - 230 . The edge cloud 110 thus may be embodied as any type of network that provides edge computing and/or storage resources which are proximately located to radio access network (RAN) capable endpoint devices (e.g., mobile computing devices, IoT devices, smart devices, etc.), which are discussed herein. In other words, the edge cloud 110 may be envisioned as an âedgeâ which connects the endpoint devices and traditional network access points that serve as an ingress point into service provider core networks, including mobile carrier networks (e.g., Global System for Mobile Communications (GSM) networks, Long-Term Evolution (LTE) networks, 5G/6G networks, etc.), while also providing storage and/or compute capabilities. Other types and forms of network access (e.g., Wi-Fi, long-range wireless, wired networks including optical networks) may also be utilized in place of or in combination with such 3GPP carrier networks.
The network components of the edge cloud 110 may be servers, multi-tenant servers, appliance computing devices, and/or any other type of computing devices. For example, the edge cloud 110 may include an appliance computing device that is a self-contained electronic device including a housing, a chassis, a case or a shell. In some circumstances, the housing may be dimensioned for portability such that it can be carried by a human and/or shipped. Example housings may include materials that form one or more exterior surfaces that partially or fully protect contents of the appliance, in which protection may include weather protection, hazardous environment protection (e.g., EMI, vibration, extreme temperatures), and/or enable submergibility. Example housings may include power circuitry to provide power for stationary and/or portable implementations, such as AC power inputs, DC power inputs, AC/DC or DC/AC converter(s), power regulators, transformers, charging circuitry, batteries, wired inputs and/or wireless power inputs. Example housings and/or surfaces thereof may include or connect to mounting hardware to enable attachment to structures such as buildings, telecommunication structures (e.g., poles, antenna structures, etc.) and/or racks (e.g., server racks, blade mounts, etc.). Example housings and/or surfaces thereof may support one or more sensors (e.g., temperature sensors, vibration sensors, light sensors, acoustic sensors, capacitive sensors, proximity sensors, etc.). One or more such sensors may be contained in, carried by, or otherwise embedded in the surface and/or mounted to the surface of the appliance. Example housings and/or surfaces thereof may support mechanical connectivity, such as propulsion hardware (e.g., wheels, propellers, etc.) and/or articulating hardware (e.g., robot arms, pivotable appendages, etc.). In some circumstances, the sensors may include any type of input devices such as user interface hardware (e.g., buttons, switches, dials, sliders, etc.). In some circumstances, example housings include output devices contained in, carried by, embedded therein and/or attached thereto. Output devices may include displays, touchscreens, lights, LEDs, speakers, I/O ports (e.g., USB), etc. In some circumstances, edge devices are devices presented in the network for a specific purpose (e.g., a traffic light), but may have processing and/or other capacities that may be utilized for other purposes. Such edge devices may be independent from other networked devices and may be provided with a housing having a form factor suitable for its primary purpose; yet be available for other compute tasks that do not interfere with its primary task. Edge devices include Internet of Things devices. The appliance computing device may include hardware and software components to manage local issues such as device temperature, vibration, resource utilization, updates, power issues, physical and network security, etc. Example hardware for implementing an appliance computing device is described in conjunction with FIG. 9B . The edge cloud 110 may also include one or more servers and/or one or more multi-tenant servers. Such a server may include an operating system and implement a virtual computing environment. A virtual computing environment may include a hypervisor managing (e.g., spawning, deploying, destroying, etc.) one or more virtual machines, one or more containers, etc. Such virtual computing environments provide an execution environment in which one or more applications and/or other software, code or scripts may execute while being isolated from one or more other applications, software, code or scripts.
In FIG. 3 , various client endpoints 310 (in the form of mobile devices, computers, autonomous vehicles, business computing equipment, industrial processing equipment) exchange requests and responses that are specific to the type of endpoint network aggregation. For instance, client endpoints 310 may obtain network access via a wired broadband network, by exchanging requests and responses 322 through an on- premise network system 332 . Some client endpoints 310 , such as mobile computing devices, may obtain network access via a wireless broadband network, by exchanging requests and responses 324 through an access point (e.g., cellular network tower) 334 . Some client endpoints 310 , such as autonomous vehicles may obtain network access for requests and responses 326 via a wireless vehicular network through a street-located network system 336 . However, regardless of the type of network access, the TSP may deploy aggregation points
342 , 344 within the edge cloud 110 to aggregate traffic and requests. Thus, within the edge cloud 110 , the TSP may deploy various compute and storage resources, such as at edge aggregation nodes 340 , to provide requested content. The edge aggregation nodes 340 and other systems of the edge cloud 110 are connected to a cloud or data center 360 , which uses a backhaul network 350 to fulfill higher-latency requests from a cloud/data center for websites, applications, database servers, etc. Additional or consolidated instances of the edge aggregation nodes 340 and the aggregation points 342 , 344 , including those deployed on a single server framework, may also be present within the edge cloud 110 or other areas of the TSP infrastructure.
Examples disclosed herein relate to automating the development and/or the deployment of machine learning models in, for example, a factory environment. For example, machine learning models that may be deployed in edge infrastructure such as the edge infrastructure described in conjunction with FIGS. 1-3 . The artificial intelligence models (e.g., machine learning models) use data generated by factories or other data sources (e.g., industrial data sources, commercial data sources, etc.). For example, manufacturing processes in factories are subject to environmental variations. For example, a temperature fluctuation in the morning using a first set of settings for manufacturing a gear, may produce unacceptable results in the afternoon, if the first set of settings are still applied. Unacceptable results may be determined as deviation from a standard or threshold as applied by the factory or the supervisor. The artificial intelligence models (e.g., machine learning models) are updated, but if a human is to update the models, there is a period of time when a model that is not as efficient as the updated model is running. This period of time wastes factory resources.
Prior techniques to automate the development and/or deployment of machine learning models include AutoML techniques as implemented by Google's Cloud AutoML, Microsoft's Azure AutoML, or DataRobot. However, these three prior techniques require humans to monitor the model performance and trigger the process to retrain the model. The model update process is more subjective as the update process is dependent on the human supervisor. Humans use time to react (e.g., substantially more time than a machine), so the model update process is discontinuous and sub-optimal models would run for longer if humans controlled the update process.
Additionally, AutoML has a limited scope and may not be applicable to industrial use cases (e.g., such as a factory production line). The prior AutoML techniques include neural network architecture search and transfer learning, and to perform such neural network architecture search a large amount of labeled training data is required, which is typically not generated by factories. Transfer learning is primarily limited to computer vision applications and natural language processing applications where mature neural network models have been trained with large datasets from sources other than the industrial use cases. In some examples, transfer learning is applicable to non-computer vision applications, but the application is limited by the need for a large dataset.
FIG. 4 is a block diagram of an example environment 400 in which model update controller circuitry operates to automatically update artificial intelligence models for autonomous factories. The example environment 400 includes an example model update central facility 402 , an example network 406 , an example model update controller circuitry 410 , and other example model update controller circuitry 418 .
The example model update central facility 402 includes an example artificial intelligence model repository 404 and is configured to receive deployed artificial intelligence models from the example model update controller circuitry 410 and the other example model update controller circuitry 418 . The example model update central facility 402 is connected through a network 406 to the example model update controller circuitry 410 and the other example model update controller circuitry 418 .
The example network 406 shown is the internet. The example model update controller circuitry 410 accesses the internet. Alternatively, the network 406 may be any other type of devices. In some examples, the example artificial intelligence model repository 404 and/or the example model update central facility 402 exist in the example network 406 .
The example model update controller circuitry 410 is similar to the other example model update controller circuitry 418 . In some examples the example model update controller circuitry 410 is identical to the other example model update controller circuitry 418 . The example model update controller circuitry 410 includes access to a database 412 of sensor data or environmental metadata. The other example model update controller circuitry 418 includes access to a database 416 of sensor data or environmental metadata. The other example model update controller circuitry 418 produces an ensemble (e.g., at least one) of artificial intelligence models in the database 420 which is communicated to the <figure-callout id="406" label="example network" filenames="US20210325861A1-20211021-D00
CLAIMS
Claims ( 55 )
1 . An apparatus for automatically updating artificial intelligence models operating on data of a first factory production line, the apparatus comprising:
an intelligent trigger circuitry to trigger an automated model update process; an automated model search circuitry to, in response to a model update, generate a plurality of candidate artificial intelligence models; and an intelligent model deployment circuitry to output a prediction of an artificial intelligence model combination to improve prediction performance over time.
2 . The apparatus of claim 1 , wherein the intelligent trigger circuitry is to trigger the model update based on at least one of: a metric baseline, an output from a first artificial intelligence model operating on the data of the first factory production line, metadata, and an output from a second artificial intelligence model, wherein the second artificial intelligence model is operating on data of a second factory production line.
3 . The apparatus of claim 2 , wherein the metadata includes at least one of environmental sensor readings, equipment configurations, and process configurations.
4 . The apparatus of claim 1 , further including a first artificial intelligence model operating on the data of the first factory production line, wherein the automated search circuitry, in response to the intelligent trigger circuitry triggering the automated model update process, generates a plurality of candidate artificial intelligence models or selects a plurality of candidate artificial intelligence models from a repository of trained artificial intelligence models.
5 . The apparatus of claim 4 , wherein a first candidate artificial intelligence model of the plurality of candidate artificial intelligence models implements a model architecture of the first artificial intelligence model operating on the data of the first factory production line, further including updated model parameters based on newly collected data.
6 . The apparatus of claim 4 , wherein a second candidate artificial intelligence model of the plurality of candidate artificial intelligence models implements a similar model architecture of the first artificial intelligence model operating on the data of the first factory production line, further including hyperparameter updates.
7 . The apparatus of claim 4 , wherein a third candidate artificial intelligence model of the plurality of candidate artificial intelligence models implements a model architecture not based on a model architecture of the first artificial intelligence model operating on the data of the first factory production line.
8 . The apparatus of claim 4 , wherein the automated model search circuitry selects a first candidate artificial intelligence model from a repository of trained artificial intelligence models based on a similarity score and a performance score.
9 . The apparatus of claim 4 , wherein the intelligent model deployment circuitry runs the plurality of candidate artificial intelligence models in parallel and the intelligent model deployment circuitry removes outlier candidate artificial intelligence models from the plurality of candidate artificial intelligence models based on an output of the plurality of candidate artificial intelligence models.
10 . An apparatus comprising:
a non-transitory computer readable medium; instructions at the apparatus; a logic circuit to execute the instructions to at least:
trigger an automated model update process;
in response to a model update, generate a plurality of candidate artificial intelligence models; and
output a prediction of an artificial intelligence model combination to improve prediction performance over time.
11 . The non-transitory computer readable medium of claim 10 , wherein the instructions, when executed, further cause the logic circuit to trigger the model update based on at least one of: a metric baseline, an output from a first artificial intelligence model operating on data of a first factory production line, metadata, and an output from a second artificial intelligence model, wherein the second artificial intelligence model is operating on data of a second factory production line.
12 . The non-transitory computer readable medium of claim 11 , wherein the metadata includes at least one of environmental sensor readings, equipment configurations, and process configurations.
13 . The non-transitory computer readable medium of claim 10 , further including a first artificial intelligence model operating on data of a first factory production line, wherein the instructions, when executed, further cause the logic circuit to, in response to a triggered automated model update process, generate a plurality of candidate artificial intelligence models or selects a plurality of artificial intelligence models from a repository of trained artificial intelligence models.
14 . The non-transitory computer readable medium of claim 13 , wherein a first candidate artificial intelligence model of the plurality of candidate artificial intelligence models implements a model architecture of the first artificial intelligence model operating on the data of a first factory production line, further including updated model parameters based on newly collected data.
15 . The non-transitory computer readable medium of claim 13 , wherein a second candidate artificial intelligence model of the plurality of candidate artificial intelligence models implements a similar model architecture of the first artificial intelligence model operating on the data of a first factory production line, further including hyperparameter updates.
16 . The non-transitory computer readable medium of claim 13 , wherein a third candidate artificial intelligence model of the plurality of candidate artificial intelligence models implements a model architecture not based on a model architecture of a first artificial intelligence model operating on the data of the first factory production line.
17 . The non-transitory computer readable medium of claim 13 , wherein the instructions, when executed, further cause the logic circuit to select a first candidate artificial intelligence model from a repository of trained artificial intelligence models based on a similarity score and a performance score.
18 . The non-transitory computer readable medium of claim 13 , wherein the instructions, when executed, further cause the logic circuit to run the plurality of candidate artificial intelligence models in parallel and, remove outlier candidate artificial intelligence models from the plurality of candidate artificial intelligence models based on an output of the plurality of candidate artificial intelligence models.
19 . An apparatus comprising:
at least one memory; instructions in the apparatus; and processor circuitry including control circuitry to control data movement within the processor circuitry, arithmetic and logic circuitry to perform one or more operations on the data, and one or more registers to store a result of one or more operations, the processor circuitry to execute the instructions to: trigger an automated model update process; in response to a model update, generate a plurality of candidate artificial intelligence models; and output a prediction of an artificial intelligence model combination to improve prediction performance over time.
20 . The apparatus of claim 19 , wherein the processor circuitry further executes the instructions to trigger the model update based on at least one of: a metric baseline, an output from a first artificial intelligence model operating on data of a first factory production line, metadata, and an output from a second artificial intelligence model, wherein the second artificial intelligence model is operating on data of a second factory production line.
21 . (canceled)
22 . (canceled)
23 . (canceled)
24 . (canceled)
25 . (canceled)
26 . (canceled)
27 . (canceled)
28 . A method for automatically updating artificial intelligence models operating on data of a first factory production line, the method comprising:
triggering an automated model update process; in response to a model update, generating a plurality of candidate artificial intelligence models; and outputting a prediction of an artificial intelligence model combination to improve prediction performance over time.
29 . The method of claim 28 , wherein the method further includes triggering the model update based on at least one of: a metric baseline, an output from a first artificial intelligence model operating on the data of the first factory production line, metadata, and an output from a second artificial intelligence model, wherein the second artificial intelligence model is operating on data of a second factory production line.
30 . (canceled)
31 . (canceled)
32 . (canceled)
33 . (canceled)
34 . (canceled)
35 . (canceled)
36 . (canceled)
37 . An apparatus for automatically updating artificial intelligence models operating on data of a first factory production line, the apparatus comprising:
means for triggering an automated model update process; means for, in response to a model update, generating a plurality of candidate artificial intelligence models; and means for outputting a prediction of an artificial intelligence model combination to improve prediction performance over time.
38 . The apparatus of claim 37 , the apparatus further including means for triggering the model update based on at least one of: a metric baseline, an output from a first artificial intelligence model operating on the data of the first factory production line, metadata, and an output from a second artificial intelligence model, wherein the second artificial intelligence model is operating on data of a second factory production line.
39 . (canceled)
40 . (canceled)
41 . (canceled)
42 . (canceled)
43 . (canceled)
44 . (canceled)
45 . (canceled)
46 . (canceled)
47 . (canceled)
48 . (canceled)
49 . (canceled)
50 . (canceled)
51 . (canceled)
52 . (canceled)
53 . (canceled)
54 . (canceled)
55 . (canceled)
US17/359,206
2021-04-30
2021-06-25
Methods and apparatus to automatically update artificial intelligence models for autonomous factories
Pending
US20210325861A1
( en )
Priority Applications (4)
Application Number
Priority Date
Filing Date
Title
US17/359,206
US20210325861A1
( en )
2021-04-30
2021-06-25
Methods and apparatus to automatically update artificial intelligence models for autonomous factories
KR1020220036598A
KR20220149418A
( en )
2021-04-30
2022-03-24
Methods and apparatus to automatically update artificial intelligence models for autonomous factories
CN202210302181.8A
CN115270697A
( en )
2021-04-30
2022-03-25
Method and apparatus for automatically updating an artificial intelligence model of an autonomous plant
DE102022203266.7A
DE102022203266A1
( en )
2021-04-30
2022-04-01
METHODS AND DEVICES FOR AUTOMATICALLY UPDATING ARTIFICIAL INTELLIGENCE MODELS FOR AUTONOMOUS FACTORIES
Applications Claiming Priority (2)
Application Number
Priority Date
Filing Date
Title
US202163182585P
2021-04-30
2021-04-30
US17/359,206
US20210325861A1
( en )
2021-04-30
2021-06-25
Methods and apparatus to automatically update artificial intelligence models for autonomous factories
Publications (1)
Publication Number
Publication Date
US20210325861A1
true
US20210325861A1 ( en )
2021-10-21
Family
ID=78082447
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US17/359,206
Pending
US20210325861A1
( en )
2021-04-30
2021-06-25
Methods and apparatus to automatically update artificial intelligence models for autonomous factories
Country Status (4)
Country
Link
US
( 1 )
US20210325861A1
( en )
KR
( 1 )
KR20220149418A
( en )
CN
( 1 )
CN115270697A
( en )
DE
( 1 )
DE102022203266A1
( en )
Cited By (15)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20220377844A1
( en )
*
2021-05-18
2022-11-24
Qualcomm Incorporated
Ml model training procedure
US20230107309A1
( en )
*
2021-10-01
2023-04-06
International Business Machines Corporation
Machine learning model selection
US20230133373A1
( en )
*
2021-11-04
2023-05-04
UiPath, Inc.
Building and managing artificial intelligence flows using long-running workflows for robotic process automation
WO2023091624A1
( en )
*
2021-11-21
2023-05-25
Schlumberger Technology Corporation
Machine learning model deployment, management and monitoring at scale
WO2023102714A1
( en )
*
2021-12-07
2023-06-15
Intel Corporation
Decentralized active-learning model update and broadcast mechanism in internet-of-things environment
US20230385692A1
( en )
*
2022-05-24
2023-11-30
Palantir Technologies Inc.
Systems and methods for artificial intelligence inference platform and model controller
EP4343459A1
( en )
*
2022-09-22
2024-03-27
Rockwell Automation Technologies, Inc.
Data driven digital twins for industrial automation device operation enhancement
US20240111263A1
( en )
*
2022-09-29
2024-04-04
Johnson Controls Tyco IP Holdings LLP
Systems and methods for optimization and autoconfiguration of edge processing devices
WO2024188939A1
( en )
*
2023-03-16
2024-09-19
Signify Holding B.V.
A controller for selecting machine learning models from an ensemble of models and a method thereof
US20240323074A1
( en )
*
2021-06-30
2024-09-26
Siemens Aktiengesellschaft
Computer-Implemented Method for Operating a Terminal with High Availability in a Network
US20240386348A1
( en )
*
2023-05-15
2024-11-21
Tyco Fire & Security Gmbh
Building management system with building lifecycle workflow applcation
EP4586148A1
( en )
*
2024-01-11
2025-07-16
StatSketch Inc.
Performance optimization predictions related to an entity dataset based on a modified version of a predefined feature set for a candidate machine learning model
US20250252729A1
( en )
*
2024-02-01
2025-08-07
Assert Secure Tech Pvt. Limited
System and method for detecting and tracking objects using a computer vision model
EP4617806A1
( en )
*
2024-03-13
2025-09-17
ABB Schweiz AG
A system and a method for mitigating data drift in an industrial plant
US12632697B1
( en )
*
2023-05-30
2026-05-19
Marvell Asia Pte Ltd
Method and system to configure a neural processing unit
Families Citing this family (6)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
WO2024098398A1
( en )
*
2022-11-11
2024-05-16
Nec Corporation
Methods, devices and medium for communication
KR20250089281A
( en )
2023-12-11
2025-06-18
ê³ ë ¤ëíêµ ì°ííë ¥ë¨
In-memory sharing method for deep neural network models in container environment
KR102691558B1
( en )
*
2024-02-16
2024-08-05
주ìíì¬ ì ëí ì¤í¸
Vibration and industrial environment integrated monitoring system using artificial intelligence
KR20260045190A
( en )
*
2024-09-27
2026-04-03
ì¼ì±ì ì주ìíì¬
Method and apparatus for artificial intelligence related memory and storage management in wireless communication system
CN119291347B
( en )
*
2024-10-29
2025-11-28
æ·±å³å¸é¼æ³°ä½³åç§ææéå ¬å¸
Aging room product accompany measurement method and system
KR102849212B1
( en )
*
2024-12-11
2025-08-22
주ìíì¬ ì¸í°ìì¤
System and method for manufacturing process autonomous operation based on multi-agent
Citations (52)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20090106178A1
( en )
*
2007-10-23
2009-04-23
Sas Institute Inc.
Computer-Implemented Systems And Methods For Updating Predictive Models
US20120284213A1
( en )
*
2011-05-04
2012-11-08
Google Inc.
Predictive Analytical Modeling Data Selection
US20120284212A1
( en )
*
2011-05-04
2012-11-08
Google Inc.
Predictive Analytical Modeling Accuracy Assessment
US20180060330A1
( en )
*
2016-08-26
2018-03-01
International Business Machines Corporation
Parallel scoring of an ensemble model
US20180150036A1
( en )
*
2016-11-28
2018-05-31
General Electric Company
Systems and methods for concept drift learning with alternating learners
US10089383B1
( en )
*
2017-09-25
2018-10-02
Maana, Inc.
Machine-assisted exemplar based similarity discovery
US10209974B1
( en )
*
2017-12-04
2019-02-19
Banjo, Inc
Automated model management methods
US20190094843A1
( en )
*
2017-09-27
2019-03-28
International Business Machines Corporation
Orchestration of learning and execution of model predictive control tool for manufacturing processes
US20190102361A1
( en )
*
2017-09-29
2019-04-04
Linkedin Corporation
Automatically detecting and managing anomalies in statistical models
US20190102700A1
( en )
*
2017-09-29
2019-04-04
Oracle International Corporation
Machine learning platform
US20190155717A1
( en )
*
2017-11-21
2019-05-23
International Business Machines Corporation
Method and apparatus for test modeling
US20200125900A1
( en )
*
2018-10-19
2020-04-23
Oracle International Corporation
Selecting an algorithm for analyzing a data set based on the distribution of the data set
US20200151619A1
( en )
*
2018-11-09
2020-05-14
Hewlett Packard Enterprise Development Lp
Systems and methods for determining machine learning training approaches based on identified impacts of one or more types of concept drift
US10783469B2
( en )
*
2016-11-01
2020-09-22
Hitachi, Ltd.
Production support system using multiple plan candidates and different types of learning devices
US20200387836A1
( en )
*
2019-06-04
2020-12-10
Accenture Global Solutions Limited
Machine learning model surety
US20210117869A1
( en )
*
2018-03-29
2021-04-22
Benevolentai Technology Limited
Ensemble model creation and selection
US20210125066A1
( en )
*
2019-10-28
2021-04-29
Lightmatter, Inc.
Quantized architecture search for machine learning models
US20210201209A1
( en )
*
2019-12-31
2021-07-01
Bull Sas
Method and system for selecting a learning model from among a plurality of learning models
US11055639B1
( en )
*
2020-04-28
2021-07-06
Sas Institute Inc.
Optimizing manufacturing processes using one or more machine learning models
US11164093B1
( en )
*
2018-08-03
2021-11-02
Amazon Technologies, Inc.
Artificial intelligence system incorporating automatic model switching based on model parameter confidence sets
US20210357785A1
( en )
*
2020-05-12
2021-11-18
Unisys Corporation
Predictive analytics system and method for implementing machine learning models into prediction systems
US20210358065A1
( en )
*
2020-05-13
2021-11-18
Accenture Global Solutions Limited
Utilizing similarity, knowledge graphs, and machine learning models to refine/redefine a target skill profile based on skills data from employees of an entity and activate a recommendation system
US20210390455A1
( en )
*
2020-06-11
2021-12-16
DataRobot, Inc.
Systems and methods for managing machine learning models
US20220012641A1
( en )
*
2020-07-09
2022-01-13
International Business Machines Corporation
Determining and selecting prediction models over multiple points in time
US11227188B2
( en )
*
2017-08-04
2022-01-18
Fair Ip, Llc
Computer system for building, training and productionizing machine learning models
US20220051140A1
( en )
*
2019-03-13
2022-02-17
Nec Corporation
Model creation method, model creation apparatus, and program
US20220067622A1
( en )
*
2020-08-25
2022-03-03
Noodle Analytics, Inc.
Systems and methods for automating production intelligence across value streams using interconnected machine-learning models
US20220067573A1
( en )
*
2020-08-31
2022-03-03
Accenture Global Solutions Limited
In-production model optimization
US20220076164A1
( en )
*
2020-09-09
2022-03-10
DataRobot, Inc.
Automated feature engineering for machine learning models
US20220122000A1
( en )
*
2020-10-19
2022-04-21
International Business Machines Corporation
Ensemble machine learning model
US11315039B1
( en )
*
2018-08-03
2022-04-26
Domino Data Lab, Inc.
Systems and methods for model management
US20220138574A1
( en )
*
2020-11-04
2022-05-05
Hon Hai Precision Industry Co., Ltd.
Method of training models in ai and electronic device
US20220147672A1
( en )
*
2019-05-17
2022-05-12
Tata Consultancy Services Limited
Method and system for adaptive learning of models for manufacturing systems
US20220199266A1
( en )
*
2020-12-10
2022-06-23
DataRobot, Inc.
Systems and methods for using machine learning with epidemiological modeling
US20220222927A1
( en )
*
2022-03-31
2022-07-14
Intel Corporation
Apparatus, system, and method of generating a multi-model machine learning (ml) architecture
US20220230024A1
( en )
*
2021-01-20
2022-07-21
Hewlett Packard Enterprise Development Lp
Artificial intelligence optimization platform
US20220269248A1
( en )
*
2021-02-19
2022-08-25
Factorypal Gmbh
Method and device for automatically determining an optimized process configuration of a process for manufacturing or processing products
US20220277231A1
( en )
*
2021-02-26
2022-09-01
Hitachi, Ltd.
System to invoke update of machine learning models on edge computers
US20220318689A1
( en )
*
2021-04-06
2022-10-06
Unlearn.AI, Inc.
Systems and Methods for Model Selection
US20220391760A1
( en )
*
2019-11-08
2022-12-08
Koninklijke Philips N.V.
Combining model outputs into a combined model output
US11544494B2
( en )
*
2017-09-28
2023-01-03
Oracle International Corporation
Algorithm-specific neural network architectures for automatic machine learning model selection
US20230080357A1
( en )
*
2021-09-16
2023-03-16
Saudi Arabian Oil Company
Method and system for managing model updates for process models
US20230097885A1
( en )
*
2021-09-24
2023-03-30
Rockwell Automation Technologies, Inc.
Machine learning models for asset optimization within industrial automation environments
US11640447B1
( en )
*
2018-04-18
2023-05-02
Meta Platforms, Inc.
Meta-model classifier
US20230148321A1
( en )
*
2020-04-03
2023-05-11
Presagen Pty Ltd
Method for artificial intelligence (ai) model selection
US20230185652A1
( en )
*
2021-12-11
2023-06-15
Adapdix Corporation
Real-time self-adaptive tuning and control of a device using machine learning
US20230186175A1
( en )
*
2021-12-10
2023-06-15
DataRobot, Inc.
Retraining based on challenger model comparison
US20240061410A1
( en )
*
2021-01-25
2024-02-22
Omron Corporation
Quality prediction system, model-generating device, quality prediction method, and recording medium
US20240104394A1
( en )
*
2022-03-11
2024-03-28
Google Llc
Platform for Automatic Production of Machine Learning Models and Deployment Pipelines
US20240176732A1
( en )
*
2021-03-30
2024-05-30
Schlumberger Technology Corporation
Advanced application of model operations in energy
US20240185117A1
( en )
*
2022-12-05
2024-06-06
S&P Global Inc.
Knowledge Graph Based Modeling System for a Production Environment
US20240185576A1
( en )
*
2021-04-05
2024-06-06
Panasonic Intellectual Property Management Co., Ltd.
Image determination device, image determination method, and recording medium
2021
2021-06-25
US
US17/359,206
patent/US20210325861A1/en
active
Pending
2022
2022-03-24
KR
KR1020220036598A
patent/KR20220149418A/en
active
Pending
2022-03-25
CN
CN202210302181.8A
patent/CN115270697A/en
active
Pending
2022-04-01
DE
DE102022203266.7A
patent/DE102022203266A1/en
active
Pending
Patent Citations (52)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20090106178A1
( en )
*
2007-10-23
2009-04-23
Sas Institute Inc.
Computer-Implemented Systems And Methods For Updating Predictive Models
US20120284213A1
( en )
*
2011-05-04
2012-11-08
Google Inc.
Predictive Analytical Modeling Data Selection
US20120284212A1
( en )
*
2011-05-04
2012-11-08
Google Inc.
Predictive Analytical Modeling Accuracy Assessment
US20180060330A1
( en )
*
2016-08-26
2018-03-01
International Business Machines Corporation
Parallel scoring of an ensemble model
US10783469B2
( en )
*
2016-11-01
2020-09-22
Hitachi, Ltd.
Production support system using multiple plan candidates and different types of learning devices
US20180150036A1
( en )
*
2016-11-28
2018-05-31
General Electric Company
Systems and methods for concept drift learning with alternating learners
US11227188B2
( en )
*
2017-08-04
2022-01-18
Fair Ip, Llc
Computer system for building, training and productionizing machine learning models
US10089383B1
( en )
*
2017-09-25
2018-10-02
Maana, Inc.
Machine-assisted exemplar based similarity discovery
US20190094843A1
( en )
*
2017-09-27
2019-03-28
International Business Machines Corporation
Orchestration of learning and execution of model predictive control tool for manufacturing processes
US11544494B2
( en )
*
2017-09-28
2023-01-03
Oracle International Corporation
Algorithm-specific neural network architectures for automatic machine learning model selection
US20190102361A1
( en )
*
2017-09-29
2019-04-04
Linkedin Corporation
Automatically detecting and managing anomalies in statistical models
US20190102700A1
( en )
*
2017-09-29
2019-04-04
Oracle International Corporation
Machine learning platform
US20190155717A1
( en )
*
2017-11-21
2019-05-23
International Business Machines Corporation
Method and apparatus for test modeling
US10209974B1
( en )
*
2017-12-04
2019-02-19
Banjo, Inc
Automated model management methods
US20210117869A1
( en )
*
2018-03-29
2021-04-22
Benevolentai Technology Limited
Ensemble model creation and selection
US11640447B1
( en )
*
2018-04-18
2023-05-02
Meta Platforms, Inc.
Meta-model classifier
US11164093B1
( en )
*
2018-08-03
2021-11-02
Amazon Technologies, Inc.
Artificial intelligence system incorporating automatic model switching based on model parameter confidence sets
US11315039B1
( en )
*
2018-08-03
2022-04-26
Domino Data Lab, Inc.
Systems and methods for model management
US20200125900A1
( en )
*
2018-10-19
2020-04-23
Oracle International Corporation
Selecting an algorithm for analyzing a data set based on the distribution of the data set
US20200151619A1
( en )
*
2018-11-09
2020-05-14
Hewlett Packard Enterprise Development Lp
Systems and methods for determining machine learning training approaches based on identified impacts of one or more types of concept drift
US20220051140A1
( en )
*
2019-03-13
2022-02-17
Nec Corporation
Model creation method, model creation apparatus, and program
US20220147672A1
( en )
*
2019-05-17
2022-05-12
Tata Consultancy Services Limited
Method and system for adaptive learning of models for manufacturing systems
US20200387836A1
( en )
*
2019-06-04
2020-12-10
Accenture Global Solutions Limited
Machine learning model surety
US20210125066A1
( en )
*
2019-10-28
2021-04-29
Lightmatter, Inc.
Quantized architecture search for machine learning models
US20220391760A1
( en )
*
2019-11-08
2022-12-08
Koninklijke Philips N.V.
Combining model outputs into a combined model output
US20210201209A1
( en )
*
2019-12-31
2021-07-01
<sp