ABSTRACT
Abstract
An apparatus having components implemented on one or more solid-state chips. The apparatus includes an input device constructed to generate an input data value (input value), and a neural network implemented on solid-state chips trained to generate an output to control the apparatus by processing the input value. The apparatus also includes another neural network implemented on solid-state chips and configured to receive the output from the neural network. The another neural network is trained to determine whether the output of the neural network corresponds to a predetermined condition and generate a control output from the output of the neural network. The apparatus includes a processor configured receive the control output from the aforementioned another neural network, and in response to the control output indicating the output of the first neural network corresponds to a predetermined condition, and control an operation of the neural network. Corresponding methods are also disclosed.
Description
RELATED APPLICATIONS
This application is a continuation-in-part of U.S. application Ser. No. 16/377,964, filed on Apr. 8, 2019. U.S. application Ser. No. 16/377,964 is a continuation of U.S. application Ser. No. 15/997,192, filed Jun. 4, 2018, now U.S. Pat. No. 10,254,760, which claims the benefit of U.S. Provisional Application No. 62/612,008, filed Dec. 29, 2017, U.S. Provisional Application No. 62/630,596, filed Feb. 14, 2018, and U.S. Provisional Application No. 62/659,359, filed Apr. 18, 2018. U.S. application Ser. No. 16/377,964 is also a continuation-in-part of U.S. patent application Ser. No. 15/991,769, now U.S. Pat. No. 10,324,467. U.S. application Ser. No. 16/377,964 is also a continuation-in-part of U.S. application Ser. No. 16/363,183. U.S. application Ser. No. 16/363,183 is a continuation-in-part of U.S. application Ser. No. 15/991,769. U.S. application Ser. No. 16/363,183 is also a continuation-in-part of U.S. application Ser. No. 15/997,192. U.S. application Ser. No. 16/363,183 is also a continuation of U.S. application Ser. No. 15/997,031, now U.S. Pat. No. 10,242,665. U.S. patent application Ser. No. 15/997,031 claims the benefit of U.S. Provisional Application No. 62/612,008, filed Dec. 29, 2017, U.S. Provisional Application No. 62/630,596, filed Feb. 14, 2018, and U.S. Provisional Application No. 62/659,359, filed Apr. 18, 2018. U.S. patent application Ser. No. 15/991,769, filed May 29, 2018, which claims the benefit of U.S. Provisional Application No. 62/612,008, filed Dec. 29, 2017, U.S. Provisional Application No. 62/630,596, filed Feb. 14, 2018, and U.S. Provisional Application No. 62/659,359, filed Apr. 18, 2018. Each of the above-listed applications is incorporated by reference herein in its entirety.
TECHNICAL FIELD
This invention relates to automatically self-correcting or correcting in real-time one or more neural networks after detecting a triggering event, or breaching a boundary condition, that may indicate an incorrect output signal or data being generated by the one or more neural networks.
BACKGROUND
As illustrated in FIG. 1 , a conventional Neural Network (NN) 101 receives an input (a single vector, in this example) at an input layer 102 , and transforms it through a series of hidden layers 103 . Each hidden layer 103 is made up of a set of âneuronsâ or ânodes,â where each neuron is fully connected to all neurons in the previous layer, and where neurons in a single layer function completely independently and do not share any connections. The last fully-connected layer is called the âoutput layerâ 105 , and in classification the output layer setting(s) represents the class scores.
As illustrated in FIG. 2 , a âconvolutionalâ Neural Network 201 can take 3D images as input, for instance. In particular, unlike the neural network described in connection with FIG. 1 , the layers of a convolutional neural network have neurons arranged in 3 dimensions: width, height, depth. Note that the word depth here refers to the third dimension of an activation volume, not to the depth of a full neural network, which can refer to the total number of layers in a network. The neurons in a layer can be configured to be connected to a small region of the layer before it, instead of all of the neurons in a fully-connected manner. The convolutional neural network reduces the full image into a single vector of class scores 205 , arranged along the depth dimension.
Although advances have been made in prior art neural networks, once a prior art neural network is trained using training data, a conventional neural network system is incapable of effectively recognizing or determining when incorrect output is generated. This shortcoming of a conventional system is compounded by the lack of the conventional systems having effective mechanisms to take corrective measures when incorrect output is generated. These shortcomings prevent conventional systems from adaptively reducing mistakes or incorrect output with new information (e.g., training with new data).
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a schematic diagram illustrating a prior art neural network with hidden layers;
FIG. 2 is a schematic diagram illustrating a prior art convolutional neural network;
FIG. 3 a is a diagram illustrating a two-dimensional decision space with two potential classification groupings;
FIG. 3 b is a diagram illustration a two-dimensional decision space with multiple boundary condition regions;
FIG. 4 is a diagram illustrating a one-dimensional decision space with two potential classification groupings;
FIG. 5 is a flow chart illustrating a set of steps in using boundary conditions in a control system;
FIG. 6 is a flow chart illustrating a set of steps in using boundary conditions in a control system and determining follow-up options;
FIG. 7 is a block diagram illustrating an embodiment of a run-time engine that includes an apex controller and one implementation neural network (âImNNâ);
FIG. 8 is a block diagram illustrating an embodiment of a run-time engine that includes parallel connected ImNNs;
FIG. 9 is a block diagram illustrating an embodiment of the present invention controller connected to ImNNs in series;
FIG. 10 is a block diagram illustrating an embodiment of the present invention controller connected to ImNNs in a combination of parallel and serial connections;
FIG. 11 is a block diagram illustrating an embodiment of components within the apex controller;
FIG. 12 is a flow chart illustrating a set of steps executed in an embodiment in re-training ImNN;
FIG. 13 is a block diagram illustrating an embodiment of a run-time engine that includes a back-up ImNN for updating an ImNN with incremental training data set;
FIG. 14 is a flow chart illustrating steps performed in an incremental update of an ImNN;
FIG. 15 is a flow chart illustrating a set of steps executed in an embodiment in replacing the coefficients of the ImNN with a previous version;
FIG. 16 is a block diagram illustrating an embodiment of a run-time engine that includes a back-up ImNN running simultaneously with an ImNN;
FIG. 17 is a timing diagram illustrating sequence of events in exchanging an operational ImNN with a backup ImNN;
FIG. 18 is a block diagram illustrating an exemplary preferred exemplary embodiment for ImNNs implemented using dynamic loading of neural network(s);
FIG. 19 is a flow chart illustrating implementing and running ImNNs using dynamic loading of neural network(s);
FIG. 20 is a top view of an exemplary embodiment of an autonomous vehicle of the present invention;
FIG. 21 is a diagram illustrating multiple regions of mapping for automatic navigation;
FIG. 22 is a diagram illustrating a display screen showing characteristics of nearby vehicles;
FIG. 23 is a block diagram of an embodiment of a run-time engine for controlling an autonomous automobile with one or more ImNNs receiving input from multiple cameras and other sources; and
FIG. 24 is a flow chart of preferred example steps to execute the run-time engine for controlling an autonomous automobile of the present invention.
SUMMARY OF CERTAIN INVENTIVE ASPECTS
Various aspects of the present invention includes inventive features and embodiments to allow machine controllers to limit the operations of neural networks to be within a set of outputs or a condition. For example, the condition may be a boundary condition. Such features and embodiments allow autonomous machines be self-corrected after a breach of a boundary condition is detected. In various examples of embodiments, a âself-correctionâ may be to make autonomous land vehicles be capable of determining the timing of automatic transition to the manual control from automated driving mode, to configure controllers to filter and save input-output data sets that fall within boundary conditions for later training of neural networks, and/or to provide security architectures to prevent damages from virus attacks or system malfunctions.
Certain embodiments of the present invention include a controller for an autonomous machine having a plurality of sensors. The controller includes a first neural network deployed on the autonomous machine, trained to generate predictable output (i.e., output from inferencing) for a given set of input with a first training data set that includes training data generated by other autonomous machines. The controller may also include a first controller coupled to the first neural network. The first controller may include a detector adapted to process the input and output data of the first neural network and to detect a first event; and a neural network manager coupled to the first neural network and adapted to re-train the first neural network incrementally using a second training data set generated by the sensors on the autonomous machine. The neural network manager can be adapted to re-train the first neural network incrementally using the second training data set.
In some embodiments, a second neural network is instantiated on a different virtual machine from a virtual machine on which the first neural network is instantiated. In some other embodiments, the first and second neural network run at the same time using the same input data set. In such embodiments, if different output data are generated by the two neural nets generated, the first neural net is placed off-line while running only the second neural network.
In certain embodiments, the first controller further includes a data filter and DBMS. The data filter selects input-output data pairs to be stored at the DBMS to be used as training data set. The DBMS can also store and retrieve the initial nodal values of the first neural network and subsequent nodal values after the re-training. In some embodiments, the data filter may cause the DBMS to store only those input-output combinations when the outputs do not cause a triggering event (e.g., outputs being within boundary conditions). In these embodiments, the subsequent re-training can be more efficiently conducted because only the âtrainingâ data set is within the boundary conditions.
Some embodiments include method steps for controlling an autonomous machine having a plurality of sensors, the steps comprise initiating a first neural network deployed on the autonomous machine, the first neural network trained to generate predictable output (i.e., output from inferencing) for a given set of input with a first training data set that includes training data generated by other autonomous machines, and executing instructions for a first controller coupled to the first neural network. The executing instruction step for a first controller may further include executing instructions for a detector adapted to process the input and output data of the first neural network and to detect a first event, and executing instructions for a neural network manager coupled to the first neural network and adapted to re-train the first neural network incrementally using a second training data set generated by the sensors on the autonomous machine. The first event can be a virus attack.
Certain embodiments include an apparatus to control an autonomous land vehicle moving in traffic with other land vehicles. The apparatus may include a first camera mounted on the autonomous land vehicle and located to capture image with a wide angle view that includes a front view and at least one side view, a second camera mounted on a front side of the autonomous land vehicle and located to capture images with a view from the front side of the autonomous land vehicle, and a third camera mounted on the at least one side of the autonomous land vehicle to capture images from the at least one side of the autonomous land vehicle. The apparatus may further include an image registering processor coupled to the first, second and third cameras to receive the images captured thereby and adapted to register the images captured by the second and third cameras on to the images captured by the first camera, synchronously, using a first neural network, and a detector coupled to the first, second and third cameras to receive synchronously the images captured thereby and adapted to identify one or more of the other land vehicles captured on the images captured by the first, second, and third cameras using a second neural network, wherein output from the second neural network include a confidence level for each of the identified other land vehicles and classification information for classifying a subset of the identified other land vehicles into a first class. The apparatus may further include an exception generator coupled to the detector to receive the classification information and the confidence level and adapted to generate an exception signal when at least one of a) the confidence level is below a first determined level and b) a number of the identified other land vehicles in the first class exceeds a second predetermined number. In some embodiments of the present invention, the second predetermined number is one. The detector is further adapted to classify the identified other vehicle is classified as the second class if the identified other vehicle is being driven manually. In some embodiments of the present invention, the first camera is a LIDAR and the second and third cameras are optical digital cameras, and the first neural network is a convolutional neural network and the second neural network is a recursive neural network. The apparatus may further include an alarm generator coupled to the exception generator and adapted to produce a human perceptive notice when the exception signal is received. In some embodiments of the present invention, the detector is further adapted to produce the confidence level to be below the first determined level when a vehicle is identified in the registered one of the second and third cameras and no vehicle is identified in the corresponding location in the registered image from the first camera.
Various embodiments of the present invention apparatus may further include an image rendering processor coupled to the image registering processor, coupled to the first, second, third cameras to receive the images captured thereby and adapted to generate a combined image, wherein the combin
RELATED APPLICATIONS
This application is a continuation-in-part of U.S. application Ser. No. 16/377,964, filed on Apr. 8, 2019. U.S. application Ser. No. 16/377,964 is a continuation of U.S. application Ser. No. 15/997,192, filed Jun. 4, 2018, now U.S. Pat. No. 10,254,760, which claims the benefit of U.S. Provisional Application No. 62/612,008, filed Dec. 29, 2017, U.S. Provisional Application No. 62/630,596, filed Feb. 14, 2018, and U.S. Provisional Application No. 62/659,359, filed Apr. 18, 2018. U.S. application Ser. No. 16/377,964 is also a continuation-in-part of U.S. patent application Ser. No. 15/991,769, now U.S. Pat. No. 10,324,467. U.S. application Ser. No. 16/377,964 is also a continuation-in-part of U.S. application Ser. No. 16/363,183. U.S. application Ser. No. 16/363,183 is a continuation-in-part of U.S. application Ser. No. 15/991,769. U.S. application Ser. No. 16/363,183 is also a continuation-in-part of U.S. application Ser. No. 15/997,192. U.S. application Ser. No. 16/363,183 is also a continuation of U.S. application Ser. No. 15/997,031, now U.S. Pat. No. 10,242,665. U.S. patent application Ser. No. 15/997,031 claims the benefit of U.S. Provisional Application No. 62/612,008, filed Dec. 29, 2017, U.S. Provisional Application No. 62/630,596, filed Feb. 14, 2018, and U.S. Provisional Application No. 62/659,359, filed Apr. 18, 2018. U.S. patent application Ser. No. 15/991,769, filed May 29, 2018, which claims the benefit of U.S. Provisional Application No. 62/612,008, filed Dec. 29, 2017, U.S. Provisional Application No. 62/630,596, filed Feb. 14, 2018, and U.S. Provisional Application No. 62/659,359, filed Apr. 18, 2018. Each of the above-listed applications is incorporated by reference herein in its entirety.
TECHNICAL FIELD
This invention relates to automatically self-correcting or correcting in real-time one or more neural networks after detecting a triggering event, or breaching a boundary condition, that may indicate an incorrect output signal or data being generated by the one or more neural networks.
BACKGROUND
As illustrated in FIG. 1 , a conventional Neural Network (NN) 101 receives an input (a single vector, in this example) at an input layer 102 , and transforms it through a series of hidden layers 103 . Each hidden layer 103 is made up of a set of âneuronsâ or ânodes,â where each neuron is fully connected to all neurons in the previous layer, and where neurons in a single layer function completely independently and do not share any connections. The last fully-connected layer is called the âoutput layerâ 105 , and in classification the output layer setting(s) represents the class scores.
As illustrated in FIG. 2 , a âconvolutionalâ Neural Network 201 can take 3D images as input, for instance. In particular, unlike the neural network described in connection with FIG. 1 , the layers of a convolutional neural network have neurons arranged in 3 dimensions: width, height, depth. Note that the word depth here refers to the third dimension of an activation volume, not to the depth of a full neural network, which can refer to the total number of layers in a network. The neurons in a layer can be configured to be connected to a small region of the layer before it, instead of all of the neurons in a fully-connected manner. The convolutional neural network reduces the full image into a single vector of class scores 205 , arranged along the depth dimension.
Although advances have been made in prior art neural networks, once a prior art neural network is trained using training data, a conventional neural network system is incapable of effectively recognizing or determining when incorrect output is generated. This shortcoming of a conventional system is compounded by the lack of the conventional systems having effective mechanisms to take corrective measures when incorrect output is generated. These shortcomings prevent conventional systems from adaptively reducing mistakes or incorrect output with new information (e.g., training with new data).
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a schematic diagram illustrating a prior art neural network with hidden layers;
FIG. 2 is a schematic diagram illustrating a prior art convolutional neural network;
FIG. 3 a is a diagram illustrating a two-dimensional decision space with two potential classification groupings;
FIG. 3 b is a diagram illustration a two-dimensional decision space with multiple boundary condition regions;
FIG. 4 is a diagram illustrating a one-dimensional decision space with two potential classification groupings;
FIG. 5 is a flow chart illustrating a set of steps in using boundary conditions in a control system;
FIG. 6 is a flow chart illustrating a set of steps in using boundary conditions in a control system and determining follow-up options;
FIG. 7 is a block diagram illustrating an embodiment of a run-time engine that includes an apex controller and one implementation neural network (âImNNâ);
FIG. 8 is a block diagram illustrating an embodiment of a run-time engine that includes parallel connected ImNNs;
FIG. 9 is a block diagram illustrating an embodiment of the present invention controller connected to ImNNs in series;
FIG. 10 is a block diagram illustrating an embodiment of the present invention controller connected to ImNNs in a combination of parallel and serial connections;
FIG. 11 is a block diagram illustrating an embodiment of components within the apex controller;
FIG. 12 is a flow chart illustrating a set of steps executed in an embodiment in re-training ImNN;
FIG. 13 is a block diagram illustrating an embodiment of a run-time engine that includes a back-up ImNN for updating an ImNN with incremental training data set;
FIG. 14 is a flow chart illustrating steps performed in an incremental update of an ImNN;
FIG. 15 is a flow chart illustrating a set of steps executed in an embodiment in replacing the coefficients of the ImNN with a previous version;
FIG. 16 is a block diagram illustrating an embodiment of a run-time engine that includes a back-up ImNN running simultaneously with an ImNN;
FIG. 17 is a timing diagram illustrating sequence of events in exchanging an operational ImNN with a backup ImNN;
FIG. 18 is a block diagram illustrating an exemplary preferred exemplary embodiment for ImNNs implemented using dynamic loading of neural network(s);
FIG. 19 is a flow chart illustrating implementing and running ImNNs using dynamic loading of neural network(s);
FIG. 20 is a top view of an exemplary embodiment of an autonomous vehicle of the present invention;
FIG. 21 is a diagram illustrating multiple regions of mapping for automatic navigation;
FIG. 22 is a diagram illustrating a display screen showing characteristics of nearby vehicles;
FIG. 23 is a block diagram of an embodiment of a run-time engine for controlling an autonomous automobile with one or more ImNNs receiving input from multiple cameras and other sources; and
FIG. 24 is a flow chart of preferred example steps to execute the run-time engine for controlling an autonomous automobile of the present invention.
SUMMARY OF CERTAIN INVENTIVE ASPECTS
Various aspects of the present invention includes inventive features and embodiments to allow machine controllers to limit the operations of neural networks to be within a set of outputs or a condition. For example, the condition may be a boundary condition. Such features and embodiments allow autonomous machines be self-corrected after a breach of a boundary condition is detected. In various examples of embodiments, a âself-correctionâ may be to make autonomous land vehicles be capable of determining the timing of automatic transition to the manual control from automated driving mode, to configure controllers to filter and save input-output data sets that fall within boundary conditions for later training of neural networks, and/or to provide security architectures to prevent damages from virus attacks or system malfunctions.
Certain embodiments of the present invention include a controller for an autonomous machine having a plurality of sensors. The controller includes a first neural network deployed on the autonomous machine, trained to generate predictable output (i.e., output from inferencing) for a given set of input with a first training data set that includes training data generated by other autonomous machines. The controller may also include a first controller coupled to the first neural network. The first controller may include a detector adapted to process the input and output data of the first neural network and to detect a first event; and a neural network manager coupled to the first neural network and adapted to re-train the first neural network incrementally using a second training data set generated by the sensors on the autonomous machine. The neural network manager can be adapted to re-train the first neural network incrementally using the second training data set.
In some embodiments, a second neural network is instantiated on a different virtual machine from a virtual machine on which the first neural network is instantiated. In some other embodiments, the first and second neural network run at the same time using the same input data set. In such embodiments, if different output data are generated by the two neural nets generated, the first neural net is placed off-line while running only the second neural network.
In certain embodiments, the first controller further includes a data filter and DBMS. The data filter selects input-output data pairs to be stored at the DBMS to be used as training data set. The DBMS can also store and retrieve the initial nodal values of the first neural network and subsequent nodal values after the re-training. In some embodiments, the data filter may cause the DBMS to store only those input-output combinations when the outputs do not cause a triggering event (e.g., outputs being within boundary conditions). In these embodiments, the subsequent re-training can be more efficiently conducted because only the âtrainingâ data set is within the boundary conditions.
Some embodiments include method steps for controlling an autonomous machine having a plurality of sensors, the steps comprise initiating a first neural network deployed on the autonomous machine, the first neural network trained to generate predictable output (i.e., output from inferencing) for a given set of input with a first training data set that includes training data generated by other autonomous machines, and executing instructions for a first controller coupled to the first neural network. The executing instruction step for a first controller may further include executing instructions for a detector adapted to process the input and output data of the first neural network and to detect a first event, and executing instructions for a neural network manager coupled to the first neural network and adapted to re-train the first neural network incrementally using a second training data set generated by the sensors on the autonomous machine. The first event can be a virus attack.
Certain embodiments include an apparatus to control an autonomous land vehicle moving in traffic with other land vehicles. The apparatus may include a first camera mounted on the autonomous land vehicle and located to capture image with a wide angle view that includes a front view and at least one side view, a second camera mounted on a front side of the autonomous land vehicle and located to capture images with a view from the front side of the autonomous land vehicle, and a third camera mounted on the at least one side of the autonomous land vehicle to capture images from the at least one side of the autonomous land vehicle. The apparatus may further include an image registering processor coupled to the first, second and third cameras to receive the images captured thereby and adapted to register the images captured by the second and third cameras on to the images captured by the first camera, synchronously, using a first neural network, and a detector coupled to the first, second and third cameras to receive synchronously the images captured thereby and adapted to identify one or more of the other land vehicles captured on the images captured by the first, second, and third cameras using a second neural network, wherein output from the second neural network include a confidence level for each of the identified other land vehicles and classification information for classifying a subset of the identified other land vehicles into a first class. The apparatus may further include an exception generator coupled to the detector to receive the classification information and the confidence level and adapted to generate an exception signal when at least one of a) the confidence level is below a first determined level and b) a number of the identified other land vehicles in the first class exceeds a second predetermined number. In some embodiments of the present invention, the second predetermined number is one. The detector is further adapted to classify the identified other vehicle is classified as the second class if the identified other vehicle is being driven manually. In some embodiments of the present invention, the first camera is a LIDAR and the second and third cameras are optical digital cameras, and the first neural network is a convolutional neural network and the second neural network is a recursive neural network. The apparatus may further include an alarm generator coupled to the exception generator and adapted to produce a human perceptive notice when the exception signal is received. In some embodiments of the present invention, the detector is further adapted to produce the confidence level to be below the first determined level when a vehicle is identified in the registered one of the second and third cameras and no vehicle is identified in the corresponding location in the registered image from the first camera.
Various embodiments of the present invention apparatus may further include an image rendering processor coupled to the image registering processor, coupled to the first, second, third cameras to receive the images captured thereby and adapted to generate a combined image, wherein the combined image has the image captured by the first camera as a background image and the images from the second and third cameras are inserted into corresponding registered locations in the background image, and a display screen coupled to the image rendering processor and the exception generator and adapted to display the combined image when the exception signal is received. In some embodiments, the display screen is remotely located from the autonomous land vehicle or a three-dimensional screen having one graphical representation for the first class vehicles and a different graphical representation for the second class vehicles. In certain embodiments the apparatus may also include a third neural network adapted to receive identical, substantially identical, inputs and generate identical, or substantially identical, outputs as the first neural network; and a fourth neural network adapted to receive identical, or substantially identical, inputs and generate identical, or substantially identical, outputs of the second neural network, wherein the first and second neural networks are executed on a first virtual machine and the third and fourth neural networks are executed on a second virtual machine, and a security processor coupled to the first and second neural networks and adapted to detect an attempt to alter the first and second neural networks by an unauthorized source, wherein a security alarm signal is generated when an attempt to alter is detected, and wherein the exception generator is further coupled to the third and fourth neural networks and to receive the classification information and the confidence level therefrom upon the generation of the security alarm.
Certain embodiments include a method of controlling an autonomous land vehicle moving in traffic with other land vehicles. The method may include the step of registering images captured by a second camera and a third camera on to the images captured by a first camera, synchronously, using a first neural network, wherein the first camera is mounted on the autonomous land vehicle and located to capture image with a wide angle view that includes a front view and at least one side view, a second camera is mounted on a front side of the autonomous land vehicle and located to capture images with a view from the front side of the autonomous land vehicle, and a third camera is mounted on the at least one side of the autonomous land vehicle to capture images from the at least one side of the autonomous land vehicle. Some embodiments may also include the steps of identifying one or more of the other land vehicles captured on the images captured by the first, second, and third cameras using a second neural network, wherein output from the second neural network include a confidence level for each of the identified other land vehicles and classification information for classifying a subset of the identified other land vehicles into a first class; and generating an exception signal when at least one of a) the confidence level is below a first determined level and b) a number of the identified other land vehicles in the first class exceeds a second predetermined number. The method may include the steps of determining if other vehicle is being driven manually or autonomously, generating an alarm when the exception signal is received, and producing the confidence level to be below the first determined level when a vehicle is identified in the registered one of the second and third cameras and no vehicle is identified in the corresponding location in the registered image from the first camera. Some embodiments may also include the steps of generating a combined image, wherein the combined image has the image captured by the first camera as a background image and the images from the second and third cameras are inserted into corresponding registered locations in the background image; and displaying the combined image when the exception signal is received; and instantiating on a first virtual machine a third neural network adapted to receive identical, or substantially identical, inputs as the first neural network and generate identical, or substantially identical, outputs as the first neural network and a fourth neural network adapted to receive identical, or substantially identical, inputs as the second neural network and generate identical, or substantially identical, outputs of the second neural network, and instantiating the first and second neural networks on a second virtual machine.
Another innovation includes a method of operating an apparatus using a control system that includes at least one neural network. In one embodiment, the method can include receiving an input vector. The input vector can be captured by the apparatus. In some embodiments, the input vector can be captured by a sensor (or sensor system) in communication with the apparatus. The method can further include processing the input vector using the at least one neural network of the control system, obtaining an output from the at least one neural network resulting from processing the input vector, and using the obtained output from the at least one neural network to control the apparatus unless the obtained output from the at least one neural network is determined to breach a predetermined condition that is unchangeable after an initial installation onto the control system. The method can be performed by one or more computer hardware processors configured to execute computer-executable instructions on a non-transitory computer storage medium. The term âinput vectorâ as used herein, is a broad term which refers to an input value (or input data value). In various embodiments, the input value/vector can be information representative of one data point, multiple data points, or an array of data points (e.g., pixel values of images). In other words, the input value/vector may be (or represent) one data point or multiple data points.
Embodiments of such methods may include one or more other features or aspects. In some embodiments, using the obtained output from the at least one neural network to control the apparatus includes processing the output from the at least one neural network with a second neural network to determine whether the output breaches the predetermined condition. In some embodiments, the second neural network is prevented from being retrained. For example, a processor may be configured such that it does not allow re-training of the second neural network by preventing the neural network data to not be changed, or not allowing access to certain data of the neural network, etc. In some embodiments, the method includes re-training the at least one neural network when the output is determined to breach the predetermined condition. In some embodiments, the method further includes defining the predetermined condition to prevent a damage to the apparatus. In some embodiments, the method further includes defining the predetermined condition with a machine recognizable human speech part. In some embodiments, the apparatus is a human speech generator with a loudspeaker and the step of using the obtained output further includes the step of generating human speech parts to be played on the loudspeaker. In some embodiments, the apparatus is an autonomous land vehicle and the step of using the obtained output further includes the step of generating a signal to control the autonomous land vehicle. In some embodiments, the method further includes replacing nodal values of the at least one neural network to a previously stored nodal values when the obtained output from the at least one neural network is determined to breach a predetermined condition.
Another innovation includes an apparatus being operated in part by a controller, comprising an input device constructed to generate an input vector, at least one neural network coupled to the controller and constructed to receive the input vector and to generate an output; and a comparator constructed to compare the output from the at least one neural network with a predetermined condition that is unchangeable after an initial installation onto the control system, where the controller is further constructed to operate the apparatus using the output unless the obtained output from the at least one neural network is determined to breach the predetermined condition. In some embodiments, the comparator is a second neural network constructed to processing the output from the at least one neural network to determine whether the output breaches the predetermined condition. In some embodiments, the apparatus is configures such that the second neural network is prevented from being retrained. In some embodiments, the controller is further constructed to re-train the at least one neural network when the output is determined to breach the predetermined condition. In some embodiments, controller is further configured to generate a human recognizable notification when the output is determined to breach the predetermined condition. In some embodiments, the predetermined condition is defined to prevent damage to the apparatus. In some embodiments, the predetermined condition is defined with a machine recognizable human speech part. In some embodiments, the apparatus is an autonomous land vehicle coupled to the at least one neural network and constructed to generate a signal to control the autonomous land vehicle. In some embodiments, nodal values of the at least one neural network are replaced by previously stored nodal values when the obtained output from the at least one neural network is determined to breach a predetermined condition.
Another innovation includes an apparatus being operated in part by a controller, the apparatus comprising an input means coupled to the apparatus for generating an input vector, at least one neural network coupled to the controller and constructed to receive the input vector and to generate an output, and a comparator means for comparing the output from the at least one neural network with a predetermined condition that is unchangeable after an initial installation onto the control system, where the controller is further constructed to operate the apparatus unless the obtained output from the at least one neural network is determined to breach the predetermined condition. In some embodiments, the apparatus is an autonomous land vehicle coupled to the at least one neural network and constructed to generate a signal to control the autonomous land vehicle.
Various embodiments of the invention may relate to an autonomous machine or system. The autonomous machine may include a first subordinate neural network having a structure that includes an input layer, an output layer, and at least two hidden layers. The first subordinate neural network may be configured to receive input data and to generate output data. An aspect of the autonomous machine is operated by using one or more of the output data. For instance, output data may be an output signal controlling the temperature of a refrigerator (or other appliance) or a vehicle (manned or unmanned). The autonomous machine also includes a machine controller coupled to the first subordinate neural network and includes (i) a first processor configured to detect a first triggering event, and (ii) a neural network controller coupled to the first processor configured to re-train the first subordinate neural network when the first processor detects the first triggering event. The machine controller may further include a second processor configured to receive and select said input data and the output data, and a memory unit configured to store and retrieve the selected input data and the selected output data, wherein the neural network controller is further configured to use said selected input data and said selected output data in re-training said subordinate neural network.
Some exemplary embodiments of the autonomous machine also include a second subordinate neural network having a structure substantially similar to said structure of said first subordinate neural network. A neural network having a substantially similar structure to another neural network indicates, for example, that the neural networks have the same structure of nodes and layers. The weights associated with the nodes may also be the same. However, because the weights of the nodes can be affected at least slightly with each data used to train a neural network, one of ordinary skill in the art will appreciate that the exact weights associated with each node of each neural network may not be identical, for example, if different training data was used to train the neural network. The machine controller can be coupled to said second subordinate neural network. The neural network controller can be further configured to replace said first neural network with said second neural network when said first processor detects said first triggering event during the operation of the autonomous machine.
In another exemplary embodiment of the autonomous machine, said first processor is further configured to detect a second triggering event and said neural network controller is further configured to take an action different from the action taken when the first triggering event took place.
Moreover, in some exemplary embodiments of the autonomous machine, the first processor is unmodifiable after an initial setup or installation on to the autonomous machine. In some exemplary aspects, the first processor can be a neural network being trained on to recognize said triggering event.
In some embodiments, the subordinate neural networks are continually trained periodically in time and/or can be trained on stored input/output data set that have been sampled from the input data and output data. In some embodiments, the sampling can be based on statistical analysis and/or based on normal operation without detecting a triggering event or based on an affirmed successful operation of a task.
In certain embodiments, a method of operating an apparatus using a control system that includes at least one neural network is provided. The method includes the steps of: receiving organized input data (referred to as an input vector) captured by the operating apparatus, processing the input vector using the at least one neural network of the control system, obtaining an output from the at least one neural network resulting from processing the input vector, comparing the output from the at least one neural network with a predetermined range, and using the obtained output from the at least one neural network in controlling the operating apparatus if the output from the at least one neural network is determined to be within the predetermined range (e.g., the output does not breach the predetermined range).
The method may further include the step(s) of defining the predetermined range with a set of machine recognizable human speech portions and/or processing the output from the at least one neural network with another neural network to determine whether the output is within the predetermined range or not. The method can also include the step of determining the predetermined range to be a safe operating range using another neural network, and/or determining the predetermined range to prevent damage to the operating apparatus.
Another innovation includes an apparatus having a plurality of components implemented on one or more solid-state chips (or sets of solid-state chips). The apparatus can include an input device configured to generate an input value, a first neural network implemented on a first one or more solid-state chips, the first neural network trained to generate an output to control the apparatus by processing the input data value, a second neural network implemented on a second one or more solid-state chips, the second neural network in communication with the first neural network to receive the output from the first neural network, the second neural network configured to determine whether the output of the first neural network corresponds to a predetermined condition and to generate a control output using the output of the first neural network. The apparatus can further include a processor configured to, receive the control output from the second neural network, and in response to the control output indicating the output of the first neural network corresponds to a predetermined condition, control an operation of the first neural network.
Various embodiments of such an apparatus may include more or fewer features, including one or more of the features described herein. In some embodiments, the first one or more solid-state chips are physically located remotely from the second one or more solid-state chips. In some embodiments, the control of an operation of the first neural network comprises terminating the operation of the first neural network when the output of the first neural network corresponds to a predetermined condition. In some embodiments, the control of an operation of the first neural network comprises re-training the first neural network when the output of the first neural network corresponds to a predetermined condition.âIn some embodiments, the processor is further configured to re-train the first neural network using the input data value generated by the input device. In some embodiments, the apparatus further includes a third neural network implemented on third one or more solid-state chips and configured to provide a second output to the second neural network, the third neural network structured and trained substantially identical to the first neural network to generate the second output by processing the input data value, where the second neural network is further configured and trained to compare the output from the first neural network and the second output from the third neural network, and to detect a difference between the output from the first neural network and the second output from the third neural network, and where the processor is further configured to perform a control action if the detected difference corresponds to a predetermined level. In some embodiments, the processor is further configured to terminate the operation of the first neural network if the detected difference is above the predetermined level. In some embodiments, the processor is further configured to re-train the first neural network if the detected difference is above the predetermined level. In some embodiments, the apparatus is an autonomous land vehicle, and wherein the step of using the obtained output further includes the step of generating a message to control the autonomous land vehicle.
Another innovation includes a computer-implemented system, implemented on one or more computer hardware processors, for controlling an apparatus. The system can include means for processing an input data value using a first neural network trained to generate an output by inferencing on the input data value, means for controlling an operation of an aspect of the apparatus using the output from the first neural network, means for generating a control output using a second neural network trained to generate the control output by inferencing on the output generated by the first neural network, the control output indicating the output generated by the first neural network corresponds to a predetermined condition, and in response to the first neural network output corresponding to the predetermined condition, controlling an operation of the first neural network using the control output from the second neural network and not controlling an operation of the apparatus using the output from the first neural network. In some embodiments, the means for controlling an operation of the first neural network comprises means for terminating the first neural network when the output is determined to correspond to the predetermined condition. In some embodiments, the means of controlling an operation of the first neural network comprises means for re-training the first neural network when the output is determined to correspond to the predetermined condition. In some embodiments, the means for controlling an operation of the first neural network comprises means for generating a human recognizable notification in response to determining the output corresponds to the predetermined condition. In some embodiments, the predetermined condition is a condition that prevents damage to the apparatus. In some embodiments, the system comprises means for receiving, from a machine recognizable human speech part, information identifying the predetermined condition; and means for defining the predetermined condition based at least in part on the information received from the machine recognizable human speech part. In some embodiments, the system can further comprise the apparatus, and the apparatus can be a human speech generator comprising a loudspeaker, and means for controlling the operation using the output includes generating human speech parts to be played on the loudspeaker. In some embodiments, the apparatus is an autonomous land vehicle, and the means for controlling the operation using the output includes means for generating a signal to control an autonomous land vehicle based at least in part on the output. In some embodiments, the apparatus comprises an image generator.
Another innovation includes an apparatus having a plurality of components, the apparatus including an input device constructed to generate an input data value, a first neural network implemented on a first one or more solid-state chips, the first neural network trained to generate an output to control the apparatus by processing the input data value, a second neural network implemented on a second one or more solid-state chips, the second neural network configured to receive the output from the first neural network, the second neural network structured and trained to determine whether the output corresponds to a predetermined condition and generate a control output, and a processor configured to receive the control output from the second neural network and to control an operation of the first neural network using the control output from the second neural network and not control the apparatus using the output from the first neural network in response to determining the output of the first neural network corresponds to the predetermined condition. In some embodiments, the processor is further configured to terminate the first neural network when the second neural network determines the output from the first neural network corresponds to the predetermined condition. In some embodiments, the control of the operation of the first neural network comprises not providing the output of the first neural network to the processor when the output of the first neural network corresponds to a predetermined condition.
Some embodiments may include an apparatus being operated in part by a controller. Such an apparatus may include an input device coupled to the apparatus and constructed to generate an input vector, at least one neural network coupled to the controller and constructed to receive the input vector and to generate an output, and a comparator constructed to compare the output from the at least the at least one neural network with a predetermined range, wherein the controller is further constructed to operate the apparatus using the output of the comparator determines the output from the al least one neural network is within the predetermined range (e.g., the output does not breach the predetermined range). The input device may include at least one of a digital camera, a microphone, or a sensor (e.g., a thermometer, optical sensor, accelerometer, etc.).
Some embodiments may provide an apparatus having a plurality of components that includes an input device coupled to the apparatus and generating an input data vector, a first neural network, coupled to the input device to receive the input data vector, configured and trained to generate an output by processing the input data vector; a first processor, coupled to the first neural network to receive the output therefrom and to one of the components of the apparatus, configured to control an operation of the one of the components of the apparatus using the output from the first neural network; a second neural network, coupled to the first neural network to receive the output therefrom, configured to and trained to generate a control output; and a second processor, coupled to the second neural network to receive the control output therefrom and to the first neural network, configured to control an operation of the first neural network using the control output from the second neural network.
In certain embodiments, a method of operating an apparatus using a control system that includes at least one neural network is provided. The method may include steps of controlling the operation of the at least one neural network by using another neural network with defined boundary conditions.
In certain embodiments, the prior art neural network's inability to generalize results to properly predict on closely aligned classes is the basis for some exemplary embodiments the present invention. Various embodiments may include, for example: (1) providing a controller to monitor one or more implementation Neural Network (ImNN), this is another way of referring to the subordinate neural network; the controller may be configured for starting and stopping one or multiple ImNNs; (2) providing a re-parameterization and subsequent re-training of a particular ImNN when one or more triggering events (e.g., incorrect results) generated by one or more of the ImNN; and (3) re-parameterization and subsequent re-training may include the following: re-training on stored and/or updated ImNN reference data and labels, re-parameterization based on stored ImNN reference configurations, and/or shutting down the autonomous machine. Various embodiments relate to autonomous machine capable of self-correction, and autonomous land vehicle capable of determining the timing of transition to manual control from automated driving.
DETAILED DESCRIPTION OF CERTAIN INVENTIVE ASPECTS
The detailed description of various exemplary embodiments below, in relation to the drawings, is intended as a description of various aspects of the various exemplary embodiments of the present invention and is not intended to represent the only aspects in which the various exemplary embodiments described herein may be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of the various exemplary embodiments of the present invention. However, it will be apparent to those skilled in the art that some aspects of the various exemplary embodiments of the present invention may be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form in order to avoid obscuring various examples of various embodiments.
Although particular aspects various exemplary embodiments are described herein, numerous variations, combinations and permutations of these aspects fall within the scope of the disclosure. Although some benefits and advantages of certain aspects are mentioned, the scope of the disclosure is not intended to be limited to particular benefits, uses or objectives.
1. Neural Networks
Some aspects of various exemplary embodiments are described by referring to and/or using neural network(s). Various structural elements of neural network includes layers (input, output, and hidden layers), nodes (or cells) for each, and connections among the nodes. Each node is connected to other nodes and has a nodal value (or a weight) and each connection can also have a weight. The initial nodal values and connections can be random or uniform. A nodal value/weight can be negative, positive, small, large, or zero after a training session with training data set. The value of each of the connection is multiplied (or other mathematical operation) by its respective connection weight. The resulting values are all added together (or other mathematical operation). A bias (e.g., nodal value) can also be added (or other mathematical operation). A bias is sometimes constant (often â1 or 1) and sometimes variable. This resulting value is the value of the node when activated. Another type of nodes is convolutional nodes, which are similar to aforementioned nodal characteristics, are typically connected to only a few nodes from a previous layer, particularly adapted to decode spatial information in images/speech data. Deconvolutional nodes are opposite to convolutional nodes. That is, deconvolutional nodes tend to decode spatial information by being locally connected to a next layer. Other types of nodes include pooling and interpolating nodes, mean and standard deviation nodes to represent probability distributions, recurrent nodes (each with connections other nodes and a memory to store the previous value of itself), long short term memory (LSTM) nodes that may address rapid information loss occurring in recurrent nodes, and gated recurrent units nodes that are a variation of LSTM node by using two gates: update and reset.
A neural network can be a feedforward network that includes multi-level hidden layers with each layer having one or more nodes. In some exemplary embodiments of the present invention, a neural network can be a recurrent neural network either forward moving only in time or bi-directional as including forward moving components and backward moving components. Some exemplary aspects of the present invention contemplate using a recursive neural network that can configure itself adoptively with different number of layers with different number of nodes for each layer depending on given training data. In some embodiments of the present invention, the recursive neural network is a configuration of a neural network created by applying the same set of weights recursively over a structured input (producing a structured prediction over variable-size input structures) or a scalar prediction on it by traversing a given structure in topological order.
In some aspects, various exemplary embodiments contemplate taking advantage of the nonlinearity of a neural network, which may cause loss functions to become nonconvex. In other words, neural networks are typically trained by using training data set on iterative, gradient-based optimizers that would drive the cost function to a very low value. In some exemplary aspects of the present invention, when training data set can be preprocessed to develop characteristic by large linear regression, support vector machines with gradient descent can be used to train a neural network.
For computing the gradient (e.g., in feed-forward neural networks), in some exemplary embodiments contemplate using back-propagation, while another method such as stochastic gradient descent can be used to perform learning using this gradient. In some aspects of the present invention, the back-propagation can also be applicable to other machine learning tasks that involve computing other derivatives, e.g., part of the learning process, or to analyze the learned model.
In some exemplary embodiments, neural networks may undergo regularization (and, optionally, optimization for neural network training) during a training session using training data set. In some aspects of the present invention, regularization contemplates to be modification to the neural network to reduce its generalization error. The optimization, in some exemplary embodiments, can use continuation methods. This option can make optimization more efficient by selecting initial points causing the local optimization efforts in well-behaved regions of training data set space. In another exemplary embodiment, the optimization can use a stochastic curriculum, e.g., gradually increasing the average proportion of the more difficult examples is gradually increased, whereas in a conventional training a random mix of easy and difficult examples is presented to neural nets to be trained.
In some exemplary embodiments, supervised training or unsupervised training (or combination thereof) can be employed to train a given neural network. The unsupervised training allows a neural network to discern the input distribution/pattern on its own. In some exemplary embodiments of the unsupervised training, each layer of a neural network can be trained individually unsupervised, and then the entire network is trained to fine tune.
In some exemplary aspects of present invention, the input data are sampled so that the neural network can be more efficiently trained. In this example embodiment, sampling can be performed by using statistical methods to approximate the input distribution/pattern such as Gibbs sampling. The Gibbs sampling is an example approach in building a Markov chain, which is an example method to perform Monte Carlo estimates.
The above described various types of nodes are used in a number of different neural network structures, such as the feedforward neural network described in connection with FIG. 1 . Other neural network structures includes: a Hopfield network, a network where every neuron is connected to every other neuron; a Boltzmann machines, which is similar to the Hopfield network but with some nodes used as input/output nodes and others remain hidden nodes; and a Restricted Boltzmann machine. These three neural network structures can include Markov chains used as preprocessors.
Another set of neural network structures includes: deep convolutional neural networks and deconvolutional networks, which use the convolutional and deconvolutional nodes described above. The convolutional/deconvolutional networks can be combined with feedforward neural networks. For instance, generative adversarial networks can be formed by two different neural networks such as a combination of a feedforward neural network and convolutional neural network, with one trained to generate content related information (e.g., feature extraction) from input data and the other trained to use the content related information to determine the content (e.g., identifying objects in images).
Another group of neural network structures includes: recurrent neural networks that use the recurrent nodes described above, LSTM use the LSTM the aforementioned LSTM nodes, gated recurrent units having an update gate instead of other gate of LSTM, neural Turing machines that have memories separated from nodes, bidirectional recurrent neural networks, and echo state networks having random connections between recurrent nodes.
Yet another group of neural network structures includes: deep residual networks which is a deep feedforward neural networks with extra connections passing input from one layer to a later layer (often 2 to 5 layers) as well as the next layer, extreme learning machines that is a feedforward neural network with random connections but not recurrent or spiking. Regarding a spiking neural network, liquid state machines are similar to extreme learning machines with spiking nodes, such as replacing sigmoid activations with threshold functions and each node has a memory capable of accumulating.
Other structures include: support vector machines that finds optimal solutions for classification problems, self-organizing neural networks such as Kohonen neural networks. Another set of neural network structures includes: autoencoders configured to automatically encode information, sparse autoencoders that encodes information in more space, variational autoencoders with are pre-injected with an approximated probability distribution of the input training samples, denoising autoencoders that trains with the input data with noise, and deep belief networks are stacked structures of autoencoders. The deep belief networks have been shown to be effectively trainable stack by stack.
In some embodiments, the neural network may include a neural network that has a class of deep, feed-forward artificial neural networks that use a variation of multilayer perceptrons designed to require minimal preprocessing and may also use hidden layers that are convolutional layers (or CNN), pooling layers, fully/partially connected layers and normalization layers. Some embodiments can be referred to as shift invariant or space invariant artificial neural networks (SIANN), based on their shared-weights architecture and translation invariance characteristics. A neural network may self-train (e.g., Alphago Zero) such as by using re-enforcement learning. Variations on this embodiment include the deep Q-network (DQN) which is a type of deep learning model that combines a deep CNN with Q-learning, a form of reinforcement learning. Unlike earlier reinforcement learning agents, DQNs can learn directly from high-dimensional sensory inputs. Variation on this embodiment include convolutional deep belief networks (CDBN) which have structure very similar to the CNN and are trained similarly to deep belief networks. These extensions exploit the 2D structure of images, like CNNs do, and make use of pre-training like deep belief networks. Further variations on this embodiment include time delay neural networks (TDNN) which allow timed signals (e.g. speech) to be processed time-invariantly, analogous to the translation invariance offered by CNNs. The tiling of neuron outputs can cover timed stages. It should be noted that the above-mentioned neural networks can be trained using training data sets using the unsupervised learning, the supervised learning, or the reinforcement learning steps.
2. Boundary Conditions
FIG. 3 a graphically illustrates a simplified decision-making space 301 that shows both the input data space and output results from neural networks. In particular, outer polygonal boundary 303 may depict the entire input sample space (e.g., the decision-making space) in two dimensions, and two smaller circles, 305 and 307 , located therein may depict validated output classes. A neural network can be structured and trained using sample input data, either supervised or unsupervised, to classify input data into output categories. It should be noted that in some embodiments output can be generated from a node(s) of an output layer, and in some embodiments output can be generated from a node(s) from a layer between an input layer and an output layer. In other words, output can be gene
CLAIMS
Claims ( 17 )
What is claimed is:
1. An apparatus having a plurality of components implemented on one or more solid-state chips, comprising:
an input device configured to generate an input value;
a first neural network implemented on a first one or more solid-state chips, the first neural network trained to generate an output to control the apparatus by processing the input data value;
a second neural network implemented on a second one or more solid-state chips, the second neural network in communication with the first neural network to receive the output from the first neural network, the second neural network configured to determine whether the output of the first neural network corresponds to a predetermined condition and to generate a control output using the output of the first neural network; and
a processor configured to:
receive the control output from the second neural network, and
in response to the control output indicating the output of the first neural network corresponds to a predetermined condition, control an operation of the first neural network, wherein the control of an operation of the first neural network comprises re-training the first neural network, using a training data set different than a training data set used to initially train the first neural network, when the output of the first neural network corresponds to a predetermined condition.
2. The apparatus of claim 1 , wherein the processor is further configured to re-train the first neural network using the input data value generated by the input device.
3. An apparatus having a plurality of components implemented on one or more solid-state chips, comprising:
an input device configured to generate an input value;
a first neural network implemented on a first one or more solid-state chips, the first neural network trained to generate an output to control the apparatus by processing the input data value;
a second neural network implemented on a second one or more solid-state chips, the second neural network in communication with the first neural network to receive the output from the first neural network, the second neural network configured to determine whether the output of the first neural network corresponds to a predetermined condition and to generate a control output using the output of the first neural network;
a third neural network implemented on third one or more solid-state chips and configured to provide a second output to the second neural network, the third neural network having the same structure of nodes and layers, and trained identically, as the first neural network to generate the second output by processing the input data value, wherein the second neural network is further configured and trained to compare the output from the first neural network and the second output from the third neural network, and to detect a difference between the output from the first neural network and the second output from the third neural network; and
a processor configured to:
receive the control output from the second neural network, and
in response to the control output indicating the output of the first neural network corresponds to a predetermined condition, control an operation of the first neural network, and wherein the processor is further configured to perform a control action if the detected difference corresponds to a predetermined level.
4. The apparatus of claim 3 , wherein the processor is further configured to terminate the operation of the first neural network if the detected difference is above the predetermined level.
5. The apparatus of claim 3 , wherein the processor is further configured to re-train the first neural network if the detected difference is above the predetermined level.
6. A computer-implemented system, implemented on one or more computer hardware processors, for controlling an apparatus, comprising:
means for processing an input data value using a first neural network trained to generate an output by inferencing on the input data value;
means for controlling an operation of an aspect of the apparatus using the output from the first neural network;
means for generating a control output using a second neural network trained to generate the control output by inferencing on the output generated by the first neural network, the control output indicating the output generated by the first neural network corresponds to a predetermined condition; and
means for controlling an operation of the first neural network using the control output from the second neural network and not controlling an operation of the apparatus using the output from the first neural network in response to the first neural network output corresponding to the predetermined condition, wherein the controlled operation of the first neural network comprises retraining the first neural network using a training data set different than a training data set used to initially train the first neural network.
7. The system of claim 6 , wherein the means for controlling an operation of the first neural network comprises means for terminating the first neural network when the output is determined to correspond to the predetermined condition.
8. The system of claim 6 , wherein the means for controlling an operation of the first neural network comprises means for generating a human recognizable notification in response to determining the output corresponds to the predetermined condition.
9. The system of claim 6 , wherein the predetermined condition is a condition that prevents damage to the apparatus.
10. The system of claim 6 , further comprising:
means for receiving, from a machine recognizable human speech part, information identifying the predetermined condition; and
means for defining the predetermined condition based at least in part on the information received from the machine recognizable human speech part.
11. The system of claim 6 , wherein the apparatus is a human speech generator comprising a loudspeaker, and means for controlling the operation using the output includes generating human speech parts to be played on the loudspeaker.
12. The system of claim 6 , wherein the apparatus is an autonomous land vehicle, and the means for controlling the operation using the output includes means for generating a signal to control an autonomous land vehicle based at least in part on the output.
13. The system of claim 6 , wherein the apparatus comprises an image generator.
14. An apparatus having a plurality of components, the apparatus comprising:
an input device constructed to generate an input data value;
a first neural network implemented on a first one or more solid-state chips, the first neural network trained to generate an output to control the apparatus by processing the input data value;
a second neural network implemented on a second one or more solid-state chips, the second neural network configured to receive the output from the first neural network, the second neural network structured and trained to determine whether the output corresponds to a predetermined boundary condition and generate a control output; and
a processor configured to receive the control output from the second neural network via a wireless communication network and to control the apparatus using the control output from the second neural network and not control the apparatus using the output from the first neural network in response to determining the output of the first neural network corresponds to the predetermined boundary condition.
15. The apparatus of claim 14 , wherein the processor is further configured to terminate the first neural network when the second neural network determines the output from the first neural network corresponds to the predetermined boundary condition.
16. The apparatus of claim 14 , wherein the control of the operation of the first neural network comprises not providing the output of the first neural network to the processor when the output of the first neural network corresponds to a predetermined boundary condition.
17. The apparatus of claim 14 , wherein the second neural network receives the output from the first neural network via a wireless communication network.
US16/540,806
2017-12-29
2019-08-14
Apparatus and method for monitoring and controlling of a neural network using another neural network implemented on one or more solid-state chips
Active
US10802488B1
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US16/540,806
US10802488B1
( en )
2017-12-29
2019-08-14
Apparatus and method for monitoring and controlling of a neural network using another neural network implemented on one or more solid-state chips
Applications Claiming Priority (9)
Application Number
Priority Date
Filing Date
Title
US201762612008P
2017-12-29
2017-12-29
US201862630596P
2018-02-14
2018-02-14
US201862659359P
2018-04-18
2018-04-18
US15/991,769
US10324467B1
( en )
2017-12-29
2018-05-29
Controller systems and methods of limiting the operation of neural networks to be within one or more conditions
US15/997,031
US10242665B1
( en )
2017-12-29
2018-06-04
Controller systems and methods of limiting the operation of neural networks to be within one or more conditions
US15/997,192
US10254760B1
( en )
2017-12-29
2018-06-04
Self-correcting controller systems and methods of limiting the operation of neural networks to be within one or more conditions
US16/363,183
US10672389B1
( en )
2017-12-29
2019-03-25
Controller systems and methods of limiting the operation of neural networks to be within one or more conditions
US16/377,694
US11214176B2
( en )
2018-04-09
2019-04-08
Rail covering for a sliding seat rail between two passenger seats in a cabin of a vehicle
US16/540,806
US10802488B1
( en )
2017-12-29
2019-08-14
Apparatus and method for monitoring and controlling of a neural network using another neural network implemented on one or more solid-state chips
Related Parent Applications (1)
Application Number
Title
Priority Date
Filing Date
US16/377,694
Continuation-In-Part
US11214176B2
( en )
2017-12-29
2019-04-08
Rail covering for a sliding seat rail between two passenger seats in a cabin of a vehicle
Publications (1)
Publication Number
Publication Date
US10802488B1
true
US10802488B1 ( en )
2020-10-13
Family
ID=72749887
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US16/540,806
Active
US10802488B1
( en )
2017-12-29
2019-08-14
Apparatus and method for monitoring and controlling of a neural network using another neural network implemented on one or more solid-state chips
Country Status (1)
Country
Link
US
( 1 )
US10802488B1
( en )
Cited By (23)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20200125107A1
( en )
*
2018-10-23
2020-04-23
Volkswagen Aktiengesellschaft
Method And Device For Abstracting A Data Record
CN112698911A
( en )
*
2020-12-28
2021-04-23
广ä¸ç³æ²¹åå·¥å¦é¢
Cloud job scheduling method based on deep reinforcement learning
US20210209452A1
( en )
*
2020-01-06
2021-07-08
Kabushiki Kaisha Toshiba
Learning device, learning method, and computer program product
US20210228131A1
( en )
*
2019-08-21
2021-07-29
Micron Technology, Inc.
Drowsiness detection for vehicle control
US11112804B2
( en )
*
2018-05-31
2021-09-07
Denso Corporation
Autonomous driving control apparatus and program product
US20220188972A1
( en )
*
2020-12-14
2022-06-16
Defense Agency For Technology And Quality
Method for synthesizing image
US20220318602A1
( en )
*
2021-03-31
2022-10-06
Fujitsu Limited
Provision of semantic feedback on deep neural network (dnn) prediction for decision making
TWI784718B
( en )
*
2021-09-17
2022-11-21
å碩è¯åç§æè¡ä»½æéå ¬å¸
Method and system for processing alarm event in factory
US20220374529A1
( en )
*
2021-04-28
2022-11-24
MatterFi
Secure cryptographic server card
US20220397425A1
( en )
*
2019-11-07
2022-12-15
Sony Group Corporation
Denoising apparatus, denoising method, and unmanned aerial vehicle
US20230275906A1
( en )
*
2022-02-28
2023-08-31
Mellanox Technologies Ltd.
System and method of determining existence of a network attack
US11754408B2
( en )
*
2019-10-09
2023-09-12
Argo AI, LLC
Methods and systems for topological planning in autonomous driving
US12061971B2
( en )
2019-08-12
2024-08-13
Micron Technology, Inc.
Predictive maintenance of automotive engines
US12210401B2
( en )
2019-09-05
2025-01-28
Micron Technology, Inc.
Temperature based optimization of data storage operations
US12249189B2
( en )
2019-08-12
2025-03-11
Micron Technology, Inc.
Predictive maintenance of automotive lighting
US20250232220A1
( en )
*
2024-01-12
2025-07-17
THIA ST Co.
Customization of machine learning tools with occupation training
US12367425B1
( en )
2024-01-12
2025-07-22
THIA ST Co.
Copilot customization with data producer(s)
US12443620B2
( en )
2024-01-12
2025-10-14
THIA ST Co.
Copilot architecture: network of microservices including specialized machine learning tools
US12443387B2
( en )
2019-08-21
2025-10-14
Micron Technology, Inc.
Intelligent audio control in vehicles
US12497055B2
( en )
2019-08-21
2025-12-16
Micron Technology, Inc.
Monitoring controller area network bus for vehicle control
US12518570B2
( en )
2019-12-18
2026-01-06
Lodestar Licensing Group Llc
Predictive maintenance of automotive transmission
US12530564B1
( en )
*
2021-11-29
2026-01-20
Kayak Software Corporation
Combined neural network
US12536045B2
( en )
2024-01-12
2026-01-27
THIA ST Co.
Distribution of tasks among microservices in a copilot
Citations (28)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US5048100A
( en )
1988-12-15
1991-09-10
Michael Kuperstein
Self organizing neural network method and system for general classification of patterns
US5179631A
( en )
1990-09-27
1993-01-12
Board Of Supervisors Of Louisiana State University And Agricultural And Mechanical College
Neural network logic system
US5668926A
( en )
1994-04-28
1997-09-16
Motorola, Inc.
Method and apparatus for converting text into audible signals using a neural network
US5845271A
( en )
1996-01-26
1998-12-01
Thaler; Stephen L.
Non-algorithmically implemented artificial neural networks and components thereof
US5870729A
( en )
1991-02-12
1999-02-09
Mitsubishi Denki Kabushiki Kaisha
Self-organizing neural network for pattern classification
US6574754B1
( en )
*
2000-02-14
2003-06-03
International Business Machines Corporation
Self-monitoring storage device using neural networks
US20030217021A1
( en )
*
2002-05-15
2003-11-20
Caterpillar, Inc.
Engine control system using a cascaded neural network
US7063476B1
( en )
2004-01-16
2006-06-20
Verna M. Pinnix
Liquid dispensing applicator
US20090276385A1
( en )
2008-04-30
2009-11-05
Stanley Hill
Artificial-Neural-Networks Training Artificial-Neural-Networks
CN101763033A
( en )
2009-12-17
2010-06-30
åå°æ»¨å·¥ç¨å¤§å¦
Device for self-correcting control for multi-model RBF neural network of deep submersible rescue vehicle and method thereof
US20100194593A1
( en )
2009-02-05
2010-08-05
Paccar Inc
Neural network for intelligent transportation systems
US7800490B2
( en )
2008-01-09
2010-09-21
Sensormatic Electronics, LLC
Electronic article surveillance system neural network minimizing false alarms and failures to deactivate
US7979173B2
( en )
1997-10-22
2011-07-12
Intelligent Technologies International, Inc.
Autonomous vehicle travel control systems and methods
CN102522945A
( en )
2012-01-10
2012-06-27
æ±è大å¦
Polyphase motor fault-tolerant control method and system based on multi-neural-network inverse model
US20120259804A1
( en )
2011-04-08
2012-10-11
International Business Machines Corporation
Reconfigurable and customizable general-purpose circuits for neural networks
US20130073497A1
( en )
2011-09-16
2013-03-21
Cornell University
Neuromorphic event-driven neural computing architecture in a scalable neural network
US20130141233A1
( en )
2011-02-23
2013-06-06
Embedrf Llc
Position tracking and mobility assessment system
US20130231824A1
( en )
2012-03-05
2013-09-05
Florida A&M University
Artificial Intelligence Valet Systems and Methods
US20130325776A1
( en )
2011-09-21
2013-12-05
Filip Ponulak
Apparatus and methods for reinforcement learning in artificial neural networks
US8677071B2
( en )
2010-03-26
2014-03-18
Virtualmetrix, Inc.
Control of processor cache memory occupancy
US20140156568A1
( en )
2012-12-05
2014-06-05
Microsoft Corporation
Self learning adaptive modeling system
US8775341B1
( en )
2010-10-26
2014-07-08
Michael Lamport Commons
Intelligent control with hierarchical stacked neural networks
US8782653B2
( en )
2010-03-26
2014-07-15
Virtualmetrix, Inc.
Fine grain performance resource management of computer systems
US8892485B2
( en )
2010-07-08
2014-11-18
Qualcomm Incorporated
Methods and systems for neural processor training by encouragement of correct output
US20150042491A1
( en )
*
2010-11-24
2015-02-12
Bcs Business Consulting Services Pte Ltd
Hazard warning system for vehicles
US9460382B2
( en )
2013-12-23
2016-10-04
Qualcomm Incorporated
Neural watchdog
CN107272705A
( en )
2017-07-31
2017-10-20
ä¸å大å¦
A kind of multiple neural network controlling planning method of robot path under intelligent environment
US9875440B1
( en )
*
2010-10-26
2018-01-23
Michael Lamport Commons
Intelligent control with hierarchical stacked neural networks
2019
2019-08-14
US
US16/540,806
patent/US10802488B1/en
active
Active
Patent Citations (28)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US5048100A
( en )
1988-12-15
1991-09-10
Michael Kuperstein
Self organizing neural network method and system for general classification of patterns
US5179631A
( en )
1990-09-27
1993-01-12
Board Of Supervisors Of Louisiana State University And Agricultural And Mechanical College
Neural network logic system
US5870729A
( en )
1991-02-12
1999-02-09
Mitsubishi Denki Kabushiki Kaisha
Self-organizing neural network for pattern classification
US5668926A
( en )
1994-04-28
1997-09-16
Motorola, Inc.
Method and apparatus for converting text into audible signals using a neural network
US5845271A
( en )
1996-01-26
1998-12-01
Thaler; Stephen L.
Non-algorithmically implemented artificial neural networks and components thereof
US7979173B2
( en )
1997-10-22
2011-07-12
Intelligent Technologies International, Inc.
Autonomous vehicle travel control systems and methods
US6574754B1
( en )
*
2000-02-14
2003-06-03
International Business Machines Corporation
Self-monitoring storage device using neural networks
US20030217021A1
( en )
*
2002-05-15
2003-11-20
Caterpillar, Inc.
Engine control system using a cascaded neural network
US7063476B1
( en )
2004-01-16
2006-06-20
Verna M. Pinnix
Liquid dispensing applicator
US7800490B2
( en )
2008-01-09
2010-09-21
Sensormatic Electronics, LLC
Electronic article surveillance system neural network minimizing false alarms and failures to deactivate
US20090276385A1
( en )
2008-04-30
2009-11-05
Stanley Hill
Artificial-Neural-Networks Training Artificial-Neural-Networks
US20100194593A1
( en )
2009-02-05
2010-08-05
Paccar Inc
Neural network for intelligent transportation systems
CN101763033A
( en )
2009-12-17
2010-06-30
åå°æ»¨å·¥ç¨å¤§å¦
Device for self-correcting control for multi-model RBF neural network of deep submersible rescue vehicle and method thereof
US8677071B2
( en )
2010-03-26
2014-03-18
Virtualmetrix, Inc.
Control of processor cache memory occupancy
US8782653B2
( en )
2010-03-26
2014-07-15
Virtualmetrix, Inc.
Fine grain performance resource management of computer systems
US8892485B2
( en )
2010-07-08
2014-11-18
Qualcomm Incorporated
Methods and systems for neural processor training by encouragement of correct output
US8775341B1
( en )
2010-10-26
2014-07-08
Michael Lamport Commons
Intelligent control with hierarchical stacked neural networks
US9875440B1
( en )
*
2010-10-26
2018-01-23
Michael Lamport Commons
Intelligent control with hierarchical stacked neural networks
US20150042491A1
( en )
*
2010-11-24
2015-02-12
Bcs Business Consulting Services Pte Ltd
Hazard warning system for vehicles
US20130141233A1
( en )
2011-02-23
2013-06-06
Embedrf Llc
Position tracking and mobility assessment system
US20120259804A1
( en )
2011-04-08
2012-10-11
International Business Machines Corporation
Reconfigurable and customizable general-purpose circuits for neural networks
US20130073497A1
( en )
2011-09-16
2013-03-21
Cornell University
Neuromorphic event-driven neural computing architecture in a scalable neural network
US20130325776A1
( en )
2011-09-21
2013-12-05
Filip Ponulak
Apparatus and methods for reinforcement learning in artificial neural networks
CN102522945A
( en )
2012-01-10
2012-06-27
æ±è大å¦
Polyphase motor fault-tolerant control method and system based on multi-neural-network inverse model
US20130231824A1
( en )
2012-03-05
2013-09-05
Florida A&M University
Artificial Intelligence Valet Systems and Methods
US20140156568A1
( en )
2012-12-05
2014-06-05
Microsoft Corporation
Self learning adaptive modeling system
US9460382B2
( en )
2013-12-23
2016-10-04
Qualcomm Incorporated
Neural watchdog
CN107272705A
( en )
2017-07-31
2017-10-20
ä¸å大å¦
A kind of multiple neural network controlling planning method of robot path under intelligent environment
Non-Patent Citations (58)
* Cited by examiner, â Cited by third party
Title
" The Internet of Things: Opportunities and Applications across Industries-Discussion Summary " International Institute for Analytics, 8 pages (Dec. 2015).
" The Internet of Things: Opportunities and Applications across IndustriesâDiscussion Summary " International Institute for Analytics, 8 pages (Dec. 2015).
Bengio et al., " Greedy Layer-Wise Training of Deep Networks, " University of Montreal, 8 pages.
Broomhead et al., " Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks, " Royal Signals & Radar Establishment, RSRE Memorandum No. 4148, 39 pages (Mar. 28, 1988).
Chang et al., " CLKN: Cascaded Lucas-Kanade Networks for Image Alignment, " 9 pages (2017).
Chen et al., " Big Data Deep Learning: Challenges and Perspectives, " IEEE Access, vol. 2, pp. 514-525 (2014).
Chen et al., " DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving, " Princeton University paper, 9 pages.
Chung et al., " Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling, " University of Montreal, arXiv:1412.3555v1, 9 pages (Dec. 11, 2014).
Cortes et al., " Support-Vector Networks, " Machine Learning, 20, pp. 273-297 (1995).
Dai et al., " Dynamic Learning from Adaptive Neural Network Control of a Class of Nonaffine Nonlinear Systems, " IEEE Transactions on Neural Networks and Learning Systems, vol. 25, No. 1 (Jan. 2014).
Desai et al., " A Survey of Image Registration Techniques Using Neural Networks ", Int'l Journal of Computer Sciences & Engineering, vol. 3(12), pp. 57-60 (Dec. 2015).
Dixit et al., " Autonomous Vehicles: Disengagements, Accidents and Reaction Times, " Plos One, 15 pages (Dec. 20, 2016).
Durrant-Whyte et al., " Simultaneous Localization and Mapping: Part I, " IEEE Robotics & Automation Magazine, 10 pages (Jun. 2006).
Durrant-Whyte et al., " Simultaneous Localization and Mapping: Part II, " IEEE Robotics & Automation Magazine, 10 pages (Sep. 2006).
Elman, J.L., " Finding Structure in Time, " Cognitive Science 14, pp. 179-211 (1990).
Eykholt et al., " Robust Physical-World Attacks on Deep Learning Visual Classification, " arXiv:1707.08945v5, 11 pages (Apr. 10, 2018).
Fischer et al. " Descriptor Matching with Convolution Neural Networks: a Comparison to SIFT, " arXiv:1405.5769v1, 10 pages (May 22, 2014).
Florian, R.V., " Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity, " Center for Cognitive and Neural Studies, Romania, 36 pages.
Freuder E.C., " Explaining Ourselves: Human-Aware Constraint Reasoning, " Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI-17), pp. 4858-4862 (2017).
Gao et al., " Unsupervised learning to detect loops using deep neural networks for visual SLAM system, " Auton Robot 41, pp. 1-18 (2017).
Goodfellow et al., " Explaining and Harnessing Adversarial Examples, " published as a conference paper at ICLR 2015, ArXiv:1412.6572v3, 11 pages (Mar. 20, 2015).
Goodfellow et al., " Generative Adversarial Nets, " University of Montreal, arXiv:1406.2661v1, 9 pages (Jun. 10, 2014).
Graves et al., " Neural Turing Machines, " Google DeepMind, London, United Kingdom, arXiv:1410.5401v2, 26 pages (Dec. 10, 2014).
Han et al., " A structure optimisation algorithm for feedforward neural network construction, " Neurocomputing, 99, pp. 347-357 (2013).
He et al., " Deep Residual Learning for Image Recognition, " arXiv:1512.03385v1, 12 pages (Dec. 10, 2015).
Hinton et al., " Learning and Relearning in Boltzmann Machines, " Chapter 7, Basic Mechanics, pp. 282-317.
Hopfield, J.J., " Neural Networks and Physical Systems with Emergent Collective Computational Abilities, " PNAS vol. 79, pp. 2554-2558 (Apr. 1982).
Huval et al., " An Empirical Evaluation of Deep Learning on Highway Driving, " arXiv:154:01716v3, 7 pages (Apr. 17, 2015).
Jaeger et al., " Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication, " Science 304, 78, 4 pages (2004).
Kim, G.K., " Deep Learning, " book review,Healthcare Informatics Research, 2016 The Korean Society of Medical Informatics, vol. 22, No. 4. pp. 351-354 (2016).
Kingma et al., " Auto-Encoding Variational Bayes, " arXiv:1312.6114v10, 14 pages (May 1, 2014).
Kohonen, T., " Essentials of the self-organizing map, " Neural Networks 37, pp. 52-65 (2013).
Kohonen, T., " Self-Organized Formation of Topologically Correct Feature Maps, " Biol. Cybern. 43, pp. 59-69 (1982).
Kulkarni et al., " Deep Convolutional Inverse Graphics Network, " arXiv:1503.03167v4, 10 pages (Jun. 22, 2015).
LeCun et al., " Deep Learning-Review " Nature, vol. 521, pp. 436-444 (May 28, 2015).
LeCun et al., " Gradient-Based Learning Applied to Document Recognition, " Proc. Of the IEEE, pp. 1-46 (Nov. 1998).
LeCun et al., " Deep LearningâReview " Nature, vol. 521, pp. 436-444 (May 28, 2015).
Libelium Smart World, 1 page, downloaded Jun. 2, 2018, www.libelium.com.
Ma et al., " Multimodal Convolutional Neural Networks for Matching Image and Sentence, " ICCV paper, pp. 2623-2631 (2015).
Maass et al., " Real-Tim Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations, " Neural Computation 14, pp. 2531-2560 (2002).
Mehri et al., " SampleRNN: An Unconditional End-to-End Neural Audio Generation Model, " published as a conference paper at ICLR 2017, arXiv:1612.07837v2, 11 pages (Feb. 11, 2017).
Nasr et al., " Neural network control of nonlinear dynamic systems using hybrid algorithm-Review article ", Applied Soft Computing 24, pp. 423-431 (2014).
Nasr et al., " Neural network control of nonlinear dynamic systems using hybrid algorithmâReview article ", Applied Soft Computing 24, pp. 423-431 (2014).
Oh et al., " A design of granular-oriented self-organizing hybrid fuzzy polynomial neural networks, " Neurocomputing, 119, pp. 292-307 (2013).
Pecevski et al., " Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons, " PloS Computational Biology, vol. 7, Issue 12, 1-25 (Dec. 2011).
Rahman et al., " Emerging Technologies with Disruptive Effects: A Review, " Perintis eJournal, vol. 7, No. 2, pp. 111-128 (2017).
Ranzato et al., " Efficient Learning of Sparse Representations with an Energy-Based Model, " Courant Institute of Mathematical Sciences, New York University, 8 pages.
Rosenblatt, F., " The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain, " Psychological Review, vol. 65, No. 6, pp. 386-408 (1958).
Schuster et al., " Bidirectional Recurrent Neural Networks, " IEEE Transactions on Signal Processing, vol. 45, No. 11, pp. 2673-2681 (Nov. 1997).
Seff et al., " Learning from Maps: Visual Common Sense for Autonomous Driving, " arXiv:1611.08583v1, 7 pages (Nov. 25, 2016).
Smolensky, P., " Information Processing in Dynamical Systems: Foundations of Harmony Theory, " University of Colorado at Boulder, Department of Computer Science, 56 pages (Feb. 1986).
Svetlov et al., " A New Implementation of the Algorithm of Adaptive Construction of Hierarchical Neural Network Classifiers, " Optical Memory and Neural Networks (Information Optics), vol. 24, No. 4, pp. 288-294 (2015).
Svetlov et al., " Development of the Algorithm of Adaptive Construction of Hierarchical Neural Network Classifiers, " Optical Memory and Neural Networks (Information Optics), vol. 26, No. 1, pp. 40-46 (2017).
Szegedy et al., " Intriguing properties of neural networks, " arXiv:1312.6199v4, 10 pages (Feb. 19, 2014).
Wang, Y., " Deep Learning in Real Time-Inference Acceleration and Continuous Training, " 17 pages, downloaded Dec. 28, 2017, https://medium.com/@Synced/deep-learning-in-real-time-inference-acceleration-and-cont . . . .
Wang, Y., " Deep Learning in Real TimeâInference Acceleration and Continuous Training, " 17 pages, downloaded Dec. 28, 2017, https://medium.com/@Synced/deep-learning-in-real-time-inference