ABSTRACT
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an artificial neural network architecture based on a synaptic connectivity graph. According to one aspect, there is provided a method comprising: obtaining a synaptic resolution image of at least a portion of a brain of a biological organism; processing the image to identify: (i) a plurality of neurons in the brain, and (ii) a plurality of synaptic connections between pairs of neurons in the brain; generating data defining a graph representing synaptic connectivity between the neurons in the brain; determining an artificial neural network architecture corresponding to the graph representing the synaptic connectivity between the neurons in the brain; and processing a network input using an artificial neural network having the artificial neural network architecture to generate a network output.
Description
CROSS REFERENCE TO RELATED APPLICATION
This patent application is a continuation (and claims the benefit of priority under 35 USC 120) of U.S. Pat. Application Serial No. 16/731,396, filed Dec. 31, 2019. The disclosure of the prior application is considered part of (and is incorporated by reference in) the disclosure of this application.
BACKGROUND
This specification relates to processing data using machine learning models.
Machine learning models receive an input and generate an output, e.g., a predicted output, based on the received input. Some machine learning models are parametric models and generate the output based on the received input and on values of the parameters of the model.
Some machine learning models are deep models that employ multiple layers of computational units to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that each apply a non-linear transformation to a received input to generate an output.
SUMMARY
This specification describes systems implemented as computer programs on one or more computers in one or more locations for processing a synaptic resolution image of the brain of a biological organism to generate a synaptic connectivity graph, and implementing an artificial neural network having an architecture specified by the synaptic connectivity graph. A synaptic connectivity graph refers to a graph representing the structure of synaptic connections between neurons in the brain of a biological organism, e.g., a fly.
For convenience, throughout this specification, a neural network having an architecture specified by a synaptic connectivity graph may be referred to as a âbrain emulationâ neural network. Identifying an artificial neural network as a âbrain emulationâ neural network is intended only to conveniently distinguish such neural networks from other neural networks (e.g., with hand-engineered architectures), and should not be interpreted as limiting the nature of the operations that may be performed by the neural network or otherwise implicitly characterizing the neural network.
According to a first aspect there is provided a method performed by one or more data processing apparatus for training a student neural network having a set of student neural network parameters. The method includes repeatedly performing operations comprising processing a training input using the student neural network to generate an output for the training input. The student neural network output is processed using a discriminative neural network to generate a discriminative score for the student neural network output. The discriminative neural network is trained to process a network input to generate a discriminative score that characterizes a prediction for whether the network input was generated using: (i) the student neural network, or (ii) a brain emulation neural network having a set of brain emulation neural network parameters. The brain emulation neural network has a neural network architecture that is specified by a graph representing synaptic connectivity between neurons in a brain of a biological organism. The graph includes a set of nodes and edges, where each edge connects a pair of nodes, each node corresponds to a respective neuron in the brain of the biological organism, and each edge connecting a pair of nodes in the graph corresponds to a synaptic connection between a pair of neurons in the brain of the biological organism. The current values of the student neural network parameters are adjusted using gradients of an objective function that depends on the discriminative score for the student neural network output.
In some implementations, the student neural network is configured to process an input that includes image data, video data, audio data, odor data, point cloud data, magnetic field data, or a combination thereof, to generate an output that includes an embedding of the input.
In some implementations, the neural network architecture of the student neural network is less complex than the neural network architecture of the brain emulation neural network.
In some implementations, adjusting the current values of the student neural network parameters using gradients of an objective function that depends on the discriminative score for the student neural network output encourages the student neural network to generate outputs that are more likely to be misclassified by the discriminative neural network as having been generated by the brain emulation neural network.
In some implementations, specifying the neural network architecture of the brain emulation neural network by the graph representing synaptic connectivity between neurons in the brain of the biological organism includes mapping each node in the graph to a corresponding artificial neuron in the neural network architecture of the brain emulation neural network. Each edge in the graph is mapped to a connection between a pair of artificial neurons in the neural network architecture of the brain emulation neural network that correspond to the pair of nodes in the graph that are connected by the edge.
In some implementations, the graph representing synaptic connectivity between neurons in the brain of the biological organism is generated by processing a synaptic resolution image of at least a portion of the brain of the biological organism to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between pairs of neurons in the brain.
In some implementations, the synaptic resolution image of the brain of the biological organism is generated using electron microscopy techniques.
In some implementations, the graph represents synaptic connectivity between neurons in the brain of the biological organism that are predicted to have a particular function in the brain of the biological organism.
In some implementations, the particular function is a visual data processing function, an audio data processing function, or an odor data processing function.
In some implementations, values of the set of brain emulation neural network parameters are determined randomly prior to training of the student neural network and are not adjusted during the training of the student neural network.
In some implementations, the biological organism is an animal, e.g., a fly.
According to a second aspect there is provided a method performed by one or more data processing apparatus for training a student neural network having a set of student neural network parameters. The method comprises repeatedly performing operations including processing a training input using the student neural network to generate a student neural network output including a respective score for each of multiple classes. The training input is processed using a brain emulation neural network having a set of brain emulation neural network parameters to generate a brain emulation neural network output including a respective score for each of the classes. The brain emulation neural network has a neural network architecture that is specified by a graph representing synaptic connectivity between neurons in a brain of a biological organism. The graph includes a set of nodes and edges, where each edge connects a pair of nodes, each node corresponds to a respective neuron in the brain of the biological organism, and each edge connecting a pair of nodes in the graph corresponds to a synaptic connection between a pair of neurons in the brain of the biological organism. The current values of the student neural network parameters are adjusted using gradients of an objective function that characterizes a similarity between: (i) the student neural network output for the training input, and (ii) the brain emulation neural network output for the training input.
In some implementations, adjusting the current values of the student neural network parameters using gradients of the objective function encourages the student neural network to generate student neural network outputs that match brain emulation neural network outputs generated by the brain emulation neural network.
In some implementations, the student neural network is configured to process an input including image data, video data, audio data, odor data, point cloud data, magnetic field data, or a combination thereof.
In some implementations, the neural network architecture of the student neural network is less complex than the neural network architecture of the brain emulation neural network.
In some implementations, specifying the neural network architecture of the brain emulation neural network by the graph representing synaptic connectivity between neurons in the brain of the biological organism includes mapping each node in the graph to a corresponding artificial neuron in the neural network architecture of the brain emulation neural network. Each edge in the graph is mapped to a connection between a pair of artificial neurons in the neural network architecture of the brain emulation neural network that correspond to the pair of nodes in the graph that are connected by the edge.
In some implementations, the graph representing synaptic connectivity between neurons in the brain of the biological organism is generated by processing a synaptic resolution image of at least a portion of the brain of the biological organism to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between pairs of neurons in the brain.
In some implementations, the synaptic resolution image of the brain of the biological organism is generated using electron microscopy techniques.
In some implementations, the graph represents synaptic connectivity between neurons in the brain of the biological organism that are predicted to have a particular function in the brain of the biological organism.
In some implementations, the particular function is a visual data processing function, an audio data processing function, or an odor data processing function.
In some implementations, the values of the set of brain emulation neural network parameters are trained on a set of training data using machine learning training techniques prior to training of the student neural network.
According to a third aspect, there is provided a method performed by one or more data processing apparatus, the method including obtaining a synaptic resolution image of at least a portion of a brain of a biological organism. The image is processed to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between pairs of neurons in the brain. Data defining a graph representing synaptic connectivity between the neurons in the brain is generated, where the graph includes a set of nodes and edges, where each edge connects a pair of nodes. Each neuron in the brain is identified as a respective node in the graph. Each synaptic connection between a pair of neurons in the brain is identified as an edge between a corresponding pair of nodes in the graph. An artificial neural network architecture is determined that corresponds to the graph representing the synaptic connectivity between the neurons in the brain. A network input is processed using an artificial neural network having the artificial neural network architecture to generate a network output.
In some implementations, determining an artificial neural network architecture corresponding to the graph representing the synaptic connectivity between the neurons in the brain includes mapping each node in the graph to a corresponding artificial neuron in the artificial neural network architecture. Each edge in the graph is mapped to a connection between a pair of artificial neurons in the artificial neural network architecture that correspond to the pair of nodes in the graph that are connected by the edge.
In some implementations, the method further includes processing the image to identify a respective direction of each of the synaptic connections between pairs of neurons in the brain. Generating data defining the graph further includes determining a direction of each edge in the graph based on the direction of the synaptic connection corresponding to the edge. Each connection between a pair of artificial neurons in the artificial neural network architecture has a direction specified by the direction of the corresponding edge in the graph.
In some implementations, the method further includes processing the image to determine a respective weight value for each of the synaptic connections between pairs of neurons in the brain. Generating data defining the graph further includes determining a weight value for each edge in the graph based on the weight value for the synaptic connection corresponding to the edge. Each connection between a pair of artificial neurons in the artificial neural network architecture has a weight value specified by the weight value of the corresponding edge in the graph.
In some implementations, processing a network input using an artificial neural network having the artificial neural network architecture to generate a network output includes, for each of multiple given artificial neurons of the artificial neural network, receiving artificial neuron inputs from other artificial neurons in the artificial neural network that are connected to the given artificial neuron by connections directed towards the given artificial neuron. An artificial neuron output is generated based on the artificial neuron inputs. The artificial neuron output is provided to other artificial neurons in the artificial neural network that are connected to the given artificial neuron by connections directed away from the given artificial neuron.
In some implementations, the method further includes training the artificial neural network having the artificial neural network architecture using machine learning training techniques on a set of training data.
In some implementations, the network input includes image data.
In some implementations, the network output includes classification data that specifies a respective score for each of multiple classes.
In some implementations, the synaptic resolution image of the brain of the biological organism is generated using electron microscopy techniques.
In some implementations, processing the image to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between
CROSS REFERENCE TO RELATED APPLICATION
This patent application is a continuation (and claims the benefit of priority under 35 USC 120) of U.S. Pat. Application Serial No. 16/731,396, filed Dec. 31, 2019. The disclosure of the prior application is considered part of (and is incorporated by reference in) the disclosure of this application.
BACKGROUND
This specification relates to processing data using machine learning models.
Machine learning models receive an input and generate an output, e.g., a predicted output, based on the received input. Some machine learning models are parametric models and generate the output based on the received input and on values of the parameters of the model.
Some machine learning models are deep models that employ multiple layers of computational units to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that each apply a non-linear transformation to a received input to generate an output.
SUMMARY
This specification describes systems implemented as computer programs on one or more computers in one or more locations for processing a synaptic resolution image of the brain of a biological organism to generate a synaptic connectivity graph, and implementing an artificial neural network having an architecture specified by the synaptic connectivity graph. A synaptic connectivity graph refers to a graph representing the structure of synaptic connections between neurons in the brain of a biological organism, e.g., a fly.
For convenience, throughout this specification, a neural network having an architecture specified by a synaptic connectivity graph may be referred to as a âbrain emulationâ neural network. Identifying an artificial neural network as a âbrain emulationâ neural network is intended only to conveniently distinguish such neural networks from other neural networks (e.g., with hand-engineered architectures), and should not be interpreted as limiting the nature of the operations that may be performed by the neural network or otherwise implicitly characterizing the neural network.
According to a first aspect there is provided a method performed by one or more data processing apparatus for training a student neural network having a set of student neural network parameters. The method includes repeatedly performing operations comprising processing a training input using the student neural network to generate an output for the training input. The student neural network output is processed using a discriminative neural network to generate a discriminative score for the student neural network output. The discriminative neural network is trained to process a network input to generate a discriminative score that characterizes a prediction for whether the network input was generated using: (i) the student neural network, or (ii) a brain emulation neural network having a set of brain emulation neural network parameters. The brain emulation neural network has a neural network architecture that is specified by a graph representing synaptic connectivity between neurons in a brain of a biological organism. The graph includes a set of nodes and edges, where each edge connects a pair of nodes, each node corresponds to a respective neuron in the brain of the biological organism, and each edge connecting a pair of nodes in the graph corresponds to a synaptic connection between a pair of neurons in the brain of the biological organism. The current values of the student neural network parameters are adjusted using gradients of an objective function that depends on the discriminative score for the student neural network output.
In some implementations, the student neural network is configured to process an input that includes image data, video data, audio data, odor data, point cloud data, magnetic field data, or a combination thereof, to generate an output that includes an embedding of the input.
In some implementations, the neural network architecture of the student neural network is less complex than the neural network architecture of the brain emulation neural network.
In some implementations, adjusting the current values of the student neural network parameters using gradients of an objective function that depends on the discriminative score for the student neural network output encourages the student neural network to generate outputs that are more likely to be misclassified by the discriminative neural network as having been generated by the brain emulation neural network.
In some implementations, specifying the neural network architecture of the brain emulation neural network by the graph representing synaptic connectivity between neurons in the brain of the biological organism includes mapping each node in the graph to a corresponding artificial neuron in the neural network architecture of the brain emulation neural network. Each edge in the graph is mapped to a connection between a pair of artificial neurons in the neural network architecture of the brain emulation neural network that correspond to the pair of nodes in the graph that are connected by the edge.
In some implementations, the graph representing synaptic connectivity between neurons in the brain of the biological organism is generated by processing a synaptic resolution image of at least a portion of the brain of the biological organism to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between pairs of neurons in the brain.
In some implementations, the synaptic resolution image of the brain of the biological organism is generated using electron microscopy techniques.
In some implementations, the graph represents synaptic connectivity between neurons in the brain of the biological organism that are predicted to have a particular function in the brain of the biological organism.
In some implementations, the particular function is a visual data processing function, an audio data processing function, or an odor data processing function.
In some implementations, values of the set of brain emulation neural network parameters are determined randomly prior to training of the student neural network and are not adjusted during the training of the student neural network.
In some implementations, the biological organism is an animal, e.g., a fly.
According to a second aspect there is provided a method performed by one or more data processing apparatus for training a student neural network having a set of student neural network parameters. The method comprises repeatedly performing operations including processing a training input using the student neural network to generate a student neural network output including a respective score for each of multiple classes. The training input is processed using a brain emulation neural network having a set of brain emulation neural network parameters to generate a brain emulation neural network output including a respective score for each of the classes. The brain emulation neural network has a neural network architecture that is specified by a graph representing synaptic connectivity between neurons in a brain of a biological organism. The graph includes a set of nodes and edges, where each edge connects a pair of nodes, each node corresponds to a respective neuron in the brain of the biological organism, and each edge connecting a pair of nodes in the graph corresponds to a synaptic connection between a pair of neurons in the brain of the biological organism. The current values of the student neural network parameters are adjusted using gradients of an objective function that characterizes a similarity between: (i) the student neural network output for the training input, and (ii) the brain emulation neural network output for the training input.
In some implementations, adjusting the current values of the student neural network parameters using gradients of the objective function encourages the student neural network to generate student neural network outputs that match brain emulation neural network outputs generated by the brain emulation neural network.
In some implementations, the student neural network is configured to process an input including image data, video data, audio data, odor data, point cloud data, magnetic field data, or a combination thereof.
In some implementations, the neural network architecture of the student neural network is less complex than the neural network architecture of the brain emulation neural network.
In some implementations, specifying the neural network architecture of the brain emulation neural network by the graph representing synaptic connectivity between neurons in the brain of the biological organism includes mapping each node in the graph to a corresponding artificial neuron in the neural network architecture of the brain emulation neural network. Each edge in the graph is mapped to a connection between a pair of artificial neurons in the neural network architecture of the brain emulation neural network that correspond to the pair of nodes in the graph that are connected by the edge.
In some implementations, the graph representing synaptic connectivity between neurons in the brain of the biological organism is generated by processing a synaptic resolution image of at least a portion of the brain of the biological organism to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between pairs of neurons in the brain.
In some implementations, the synaptic resolution image of the brain of the biological organism is generated using electron microscopy techniques.
In some implementations, the graph represents synaptic connectivity between neurons in the brain of the biological organism that are predicted to have a particular function in the brain of the biological organism.
In some implementations, the particular function is a visual data processing function, an audio data processing function, or an odor data processing function.
In some implementations, the values of the set of brain emulation neural network parameters are trained on a set of training data using machine learning training techniques prior to training of the student neural network.
According to a third aspect, there is provided a method performed by one or more data processing apparatus, the method including obtaining a synaptic resolution image of at least a portion of a brain of a biological organism. The image is processed to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between pairs of neurons in the brain. Data defining a graph representing synaptic connectivity between the neurons in the brain is generated, where the graph includes a set of nodes and edges, where each edge connects a pair of nodes. Each neuron in the brain is identified as a respective node in the graph. Each synaptic connection between a pair of neurons in the brain is identified as an edge between a corresponding pair of nodes in the graph. An artificial neural network architecture is determined that corresponds to the graph representing the synaptic connectivity between the neurons in the brain. A network input is processed using an artificial neural network having the artificial neural network architecture to generate a network output.
In some implementations, determining an artificial neural network architecture corresponding to the graph representing the synaptic connectivity between the neurons in the brain includes mapping each node in the graph to a corresponding artificial neuron in the artificial neural network architecture. Each edge in the graph is mapped to a connection between a pair of artificial neurons in the artificial neural network architecture that correspond to the pair of nodes in the graph that are connected by the edge.
In some implementations, the method further includes processing the image to identify a respective direction of each of the synaptic connections between pairs of neurons in the brain. Generating data defining the graph further includes determining a direction of each edge in the graph based on the direction of the synaptic connection corresponding to the edge. Each connection between a pair of artificial neurons in the artificial neural network architecture has a direction specified by the direction of the corresponding edge in the graph.
In some implementations, the method further includes processing the image to determine a respective weight value for each of the synaptic connections between pairs of neurons in the brain. Generating data defining the graph further includes determining a weight value for each edge in the graph based on the weight value for the synaptic connection corresponding to the edge. Each connection between a pair of artificial neurons in the artificial neural network architecture has a weight value specified by the weight value of the corresponding edge in the graph.
In some implementations, processing a network input using an artificial neural network having the artificial neural network architecture to generate a network output includes, for each of multiple given artificial neurons of the artificial neural network, receiving artificial neuron inputs from other artificial neurons in the artificial neural network that are connected to the given artificial neuron by connections directed towards the given artificial neuron. An artificial neuron output is generated based on the artificial neuron inputs. The artificial neuron output is provided to other artificial neurons in the artificial neural network that are connected to the given artificial neuron by connections directed away from the given artificial neuron.
In some implementations, the method further includes training the artificial neural network having the artificial neural network architecture using machine learning training techniques on a set of training data.
In some implementations, the network input includes image data.
In some implementations, the network output includes classification data that specifies a respective score for each of multiple classes.
In some implementations, the synaptic resolution image of the brain of the biological organism is generated using electron microscopy techniques.
In some implementations, processing the image to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between pairs of neurons in the brain, includes: identifying positions of the neurons in the image; and identifying the synaptic connections between pairs of neurons based on proximity of the positions of the neurons in the image.
In some implementations, identifying positions of neurons in the image includes processing the image, features derived from the image, or both, using a machine learning model that is trained using supervised learning techniques to identify positions of neurons in images.
In some implementations, identifying the synaptic connections between pairs of neurons based on proximity of the positions of the neurons in the image includes, for one or more pairs of neurons including a first neuron and a second neuron: determining: (i) a first tolerance region in the image around the first neuron, and (ii) a second tolerance region in the image around the second neuron; and determining that the first neuron is connected by a synapse to the second neuron based on an overlap between the first tolerance region and the second tolerance region.
In some implementations, processing the image to determine a respective weight value for each of the synaptic connections between pairs of neurons in the brain includes, for a synaptic connection between a first neuron and a second neuron in the brain, determining the weight value for the synaptic connection between the first neuron and the second neuron based on a proximity of the first neuron and the second neuron in the image.
According to a fourth aspect, there is provided a method performed by one or more data processing apparatus, the method including obtaining data defining a graph representing synaptic connectivity between neurons in a brain of a biological organism. The graph includes a set of nodes and edges, where each edge connects a pair of nodes, each node corresponds to a respective neuron in the brain of the biological organism, and each edge connecting a pair of nodes in the graph corresponds to a synaptic connection between a pair of neurons in the brain of the biological organism. The method includes determining, for each node in the graph, a respective set of one or more node features characterizing a structure of the graph relative to the node. The method includes identifying a sub-graph of the graph, including selecting a proper subset of the nodes in the graph for inclusion in the sub-graph based on the node features of the nodes in the graph. The method includes determining an artificial neural network architecture corresponding to the sub-graph of the graph.
In some implementations, for each node in the graph, the set of node features characterizing the structure of the graph relative to the node includes one or more of: a node degree feature specifying a number of other nodes that are connected to the node by an edge; a path length feature specifying a length of a longest path in the graph starting from the node; or a neighborhood size feature specifying a number of other nodes that are connected to the node by a path in the graph having a length that is less than or equal to a threshold value.
In some implementations, obtaining data defining the graph representing synaptic connectivity between neurons in the brain of the biological organism includes obtaining data defining a weight value for each edge in the graph, where the weight value for each edge in the graph characterizes the corresponding synaptic connection in the brain of the biological organism.
In some implementations, for each node in the graph, the set of node features characterizing the structure of the graph relative to the node are determined based at least in part on the weight values of edges connecting the node to other nodes in the graph.
In some implementations, the proper subset of the nodes in the graph selected for inclusion in the sub-graph are predicted to correspond to neurons having a particular function in the brain of the biological organism.
In some implementations, the particular function is a visual data processing function, and the method further includes providing an artificial neural network having the artificial neural network architecture corresponding to the sub-graph of the graph for performing an image processing task.
In some implementations, the particular function is an audio data processing function, and the method further includes providing an artificial neural network having the artificial neural network architecture corresponding to the sub-graph of the graph for performing an audio data processing task.
In some implementations, the particular function is an odor data processing function, and the method further includes providing an artificial neural network having the artificial neural network architecture corresponding to the sub-graph of the graph for performing an odor data processing function.
In some implementations, the method further includes obtaining a representation of the sub-graph as a two-dimensional array of numerical values, where a value of a component of the array at position (i,j) indicates if the sub-graph includes an edge from node i to node j. The array is processed to identify a set of clusters in the array, where each cluster specifies a contiguous region of the array. Edges from the sub-graph that are not included in the identified clusters are removed prior to determining the artificial neural network architecture corresponding to the sub-graph.
In some implementations, processing the array to identify the set of clusters in the array includes processing the array using a blob detection algorithm.
In some implementations, determining an artificial neural network architecture corresponding to the sub-graph of the graph includes mapping each node in the sub-graph to a corresponding artificial neuron in the artificial neural network architecture. Each edge in the sub-graph is mapped to a connection between a pair of artificial neurons in the artificial neural network architecture that correspond to the pair of nodes in the sub-graph that are connected by the edge.
In some implementations, obtaining data defining a graph representing synaptic connectivity between neurons in a brain of a biological organism includes: obtaining a synaptic resolution image of at least a portion of the brain of the biological organism; and processing the image to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between pairs of neurons in the brain.
According to a fifth aspect, there is provided a system including one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to implement a reservoir computing neural network. The reservoir computing neural network is configured to receive a network input and to generate a network output from the network output, and comprises: (i) a brain emulation sub-network, and (ii) a prediction sub-network. The brain emulation sub-network is configured to process the network input in accordance with values of a set of brain emulation sub-network parameters to generate an alternative representation of the network input. The prediction sub-network is configured to process the alternative representation of the network input in accordance with values of a set of prediction sub-network parameters to generate the network output. The values of the brain emulation sub-network parameters are determined before the reservoir computing neural network is trained and are not adjusting during training of the reservoir computing neural network. The values of the prediction sub-network parameters are adjusted during training of the reservoir computing neural network. The brain emulation sub-network has a neural network architecture that is specified by a graph representing synaptic connectivity between neurons in a brain of a biological organism. The graph includes a set of nodes and edges, where each edge connects a pair of nodes, each node corresponds to a respective neuron in the brain of the biological organism, and each edge connecting a pair of nodes in the graph corresponds to a synaptic connection between a pair of neurons in the brain of the biological organism.
In some implementations, specifying the neural network architecture of the brain emulation sub-network by the graph representing synaptic connectivity between neurons in the brain of the biological organism includes mapping each node in the graph to a corresponding artificial neuron in the neural network architecture of the brain emulation sub-network. Each edge in the graph is mapped to a connection between a pair of artificial neurons in the neural network architecture of the brain emulation sub-network that correspond to the pair of nodes in the graph that are connected by the edge.
In some implementations, the graph representing synaptic connectivity between neurons in the brain of the biological organism is generated by processing a synaptic resolution image of at least a portion of the brain of the biological organism to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between pairs of neurons in the brain.
In some implementations, the synaptic resolution image of the brain of the biological organism is generated using electron microscopy techniques.
In some implementations, the graph represents synaptic connectivity between neurons in the brain of the biological organism that are predicted to have a particular function in the brain of the biological organism.
In some implementations, the particular function is a visual data processing function, an audio data processing function, or an odor data processing function.
In some implementations, the values of the prediction sub-network parameters are adjusted during training of the reservoir computing neural network to optimize an objective function.
In some implementations, the objective function includes a term characterizing a prediction accuracy of the reservoir computing neural network.
In some implementations, the term characterizing the prediction accuracy of the reservoir computing neural network includes a cross-entropy loss term.
In some implementations, the objective function includes a term characterizing a magnitude of the values of the prediction sub-network parameters.
In some implementations, dropout regularization is applied to the brain emulation sub-network parameters during training of the reservoir computing neural network.
In some implementations, the reservoir computing neural network is configured to process a network input including image data, video data, audio data, odor data, point cloud data, magnetic field data, or a combination thereof.
In some implementations, the reservoir computing neural network is configured to generate a classification output that includes a respective score for each of a plurality of classes.
In some implementations, the neural network architecture of the prediction sub-network is less complex than the neural network architecture of the brain emulation sub-network.
In some implementations, the prediction sub-network includes only a single neural network layer.
In some implementations, the values of the brain emulation sub-network parameters are determined based on weight values associated with synaptic connections between neurons in the brain of the biological organism.
According to a sixth aspect there is provided a method performed by one or more data processing apparatus, the method including obtaining data defining a synaptic connectivity graph representing synaptic connectivity between neurons in a brain of a biological organism. The synaptic connectivity graph includes a set of nodes and edges, where each edge connects a pair of nodes, each node corresponds to a respective neuron in the brain of the biological organism, and each edge connecting a pair of nodes in the synaptic connectivity graph corresponds to a synaptic connection between a pair of neurons in the brain of the biological organism. Data defining a set of candidate graphs is generated based on the synaptic connectivity graph. For each candidate graph, a performance measure on a machine learning task of a neural network having a neural network architecture that is specified by the candidate graph is determined. A final neural network architecture is selected for performing the machine learning task based on the performance measures.
In some implementations, obtaining data defining the synaptic connectivity graph representing synaptic connectivity between neurons in the brain of the biological organism includes: obtaining a synaptic resolution image of at least a portion of the brain of the biological organism; and processing the image to identify: (i) a set of neurons in the brain, and (ii) a set of synaptic connections between pairs of neurons in the brain.
In some implementations, the synaptic resolution image of the brain of the biological organism is generated using electron microscopy techniques.
In some implementations, generating data defining the set of candidate graphs based on the synaptic connectivity graph includes, for each of multiple graph features: determining a value of the graph feature for the synaptic connectivity graph; and determining a constraint corresponding to the graph feature based on the value of the graph feature for the synaptic connectivity graph. The constraint corresponding to the graph feature specifies a target value or a range of target values of the graph feature for the candidate graphs. The set of candidate graphs are generated based on the constraints corresponding to the graph features.
In some implementations, the set of graph features includes one or more of: (i) a graph feature that specifies a number of nodes in a graph, (ii) a graph feature that specifies a number of edges in a largest cluster in a two-dimensional array representing a graph, (iii) a graph feature that specifies a number of clusters in a two-dimensional array representing a graph that include a number of edges that is within a predefined range of values, (iv) an average path length between nodes in a graph, or (v) a maximum path length between nodes in a graph.
In some implementations, generating a candidate graph based on the constraints corresponding to the graph features includes: initializing the candidate graph; and at each of one or more iterations, updating the candidate graph to cause the candidate graph to satisfy a corresponding constraint.
In some implementations, initializing the candidate graph includes randomly initializing the candidate graph.
In some implementations, generating data defining the set of candidate graphs based on the synaptic connectivity graph includes generating a set of current graphs based on the synaptic connectivity graph. Each current graph is generated by applying one or more random modifications to the synaptic connectivity graph. The set of current graphs is updated at each of multiple iterations, including, at each iteration: randomly sampling a plurality of current graphs from the set of current graphs; determining, for each sampled graph, a performance measure on the machine learning task of a neural network having a neural network architecture that is specified by the sampled graph; and updating the set of current graphs based on the performance measures of the sampled graphs. After a final iteration of the plurality of iterations, each current graph in the set of current graphs is identified as a candidate graph.
In some implementations, updating the set of current graphs based on the performance measures of the sampled graphs includes removing any sampled graph having a performance measure that does not satisfy a threshold from the set of current graphs.
In some implementations, updating the set of current graphs based on the performance measures of the sampled graphs includes: identifying one or more of the sampled graphs having the highest performance measures; generating one or more new graphs based on the randomly sampled graphs, where each new graph is generated by applying one or more random modifications to a sampled graph; and adding the new graphs to the set of current graphs.
In some implementations, determining a performance measure on a machine learning task of a neural network having a neural network architecture that is specified by a candidate graph includes determining the neural network architecture that is specified by the candidate graph, including mapping each node in the candidate graph to a corresponding artificial neuron in the neural network architecture. Each edge in the candidate graph is mapped to a connection between a pair of artificial neurons in the neural network architecture that correspond to the pair of nodes in the candidate graph that are connected by the edge.
In some implementations, selecting a final neural network architecture for performing the machine learning task based on the performance measures includes selecting the neural network architecture specified by the candidate graph associated with the highest performance measure.
In some implementations, the machine learning task includes processing image data to generate a classification of the image data.
In some implementations, the machine learning task includes processing audio data to generate a classification of the audio data.
In some implementations, generating data defining the set of candidate graphs based on the synaptic connectivity graph includes, at each of multiple iterations, generating a candidate graph by applying one or more transformation operations to the synaptic connectivity graph, where the transformation operations are specified by current values of transformation operation parameters. A performance measure on the machine learning task of a neural network having a neural network architecture that is specified by the candidate graph is determined. The current values of the transformation operation parameters are updated based at least in part on the performance measure.
According to a seventh aspect, there is provided a system including: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, where the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform the operations of the method of any preceding aspect.
According to an eighth aspect, there are provided one or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform the operations of the method of any preceding aspect.
Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.
The systems described in this specification can implement a brain emulation neural network having an architecture specified by a synaptic connectivity graph derived from a synaptic resolution image of the brain of a biological organism. The brains of biological organisms may be adapted by evolutionary pressures to be effective at solving certain tasks, e.g., classifying objects or generating robust object representations, and brain emulation neural networks may share this capacity to effectively solve tasks. In particular, compared to other neural networks, e.g., with manually specified neural network architectures, brain emulation neural networks may require less training data, fewer training iterations, or both, to effectively solve certain tasks. Moreover, brain emulation neural networks may perform certain machine learning tasks more effectively, e.g., with higher accuracy, than other neural networks.
The systems described in this specification can process a synaptic connectivity graph corresponding to a brain to predict the neuronal types (e.g., primary sensory type, visual type, olfactory type, memory type, and the like) of neurons in the brain. In particular, features can be computed for each node in the graph (e.g., the path length corresponding to the node and the number of edges connected to the node), and the node features can be used to classify certain nodes as corresponding to a type of neuron in the brain. A sub-graph of the overall graph corresponding to neurons that are predicted to be of a certain type can be identified, and a brain emulation neural network may be implemented with an architecture specified by the sub-graph, i.e., rather than the entire graph. Implementing a brain emulation neural network with an architecture specified by a sub-graph corresponding to neurons of a certain type may enable the brain emulation neural network to perform certain tasks more effectively while consuming fewer computational resources (e.g. memory and computing power). In one example, the brain emulation neural network may be configured to perform image processing tasks, and the architecture of the brain emulation neural network may be specified by a sub-graph corresponding to only the visual system of the brain (i.e., to visual type neurons).
The systems described in this specification can use a brain emulation neural network to train another neural network, referred to as a âstudentâ neural network, having a substantially less complex neural network architecture. More specifically, the student neural network may be trained to match outputs that are generated by the brain emulation neural network. The brains of many biological organisms have a large number of neurons, e.g., a fly brain may have on the order of ~10 5 neurons or more. Therefore, a brain emulation network may have a highly complex architecture, and processing data using a brain emulation neural network may be computationally expensive. After being trained, the student neural network may inherit the capacity of the brain emulation neural network to effectively solve certain tasks, while consuming fewer computational resources than the brain emulation neural network due to having a substantially less complex neural network architecture.
The systems described in this specification can use a brain emulation neural network in reservoir computing applications. In particular, a âreservoir computingâ neural network may be implemented with an architecture specified by a brain emulation sub-network followed by a âpredictionâ sub-network. Generally, the prediction sub-network may have a substantially less complex architecture than the brain emulation neural network, e.g., the prediction sub-network may consist of a single classification layer. During training of the reservoir computing neural network, only the weights of the prediction sub-network are trained, while the weights of the brain emulation neural network are considered static and are not trained. Generally, a brain emulation neural network may have a very large number of trainable parameters and a highly recurrent architecture. Therefore training the brain emulation neural network may be computationally-intensive and prone to failure, e.g., as a result of the model parameter values of the brain emulation neural network oscillating rather than converging to fixed values. The reservoir computing neural network described in this specification may harness the capacity of the brain emulation neural network, e.g., to generate representations that are effective for solving tasks, without requiring the brain emulation neural network to be trained.
This specification further describes techniques for using the synaptic connectivity graph to âseedâ (i.e., initialize) a search through a space of possible neural network architectures to identify an architecture that can be used to effectively perform a machine learning task. More specifically, the synaptic connectivity graph may be used to derive âcandidateâ graphs which specify corresponding neural network architectures, and the best performing of these architectures may be selected to perform the machine learning task. Seeding the neural architecture search process using the synaptic connectivity graph may facilitate the discovery of large numbers of biologically-inspired neural network architectures, some of which may be effective for performing machine learning tasks.
The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates an example of generating a brain emulation neural network based on a synaptic resolution image of the brain of a biological organism.
FIG. 2 shows an example data flow for generating a synaptic connectivity graph and a brain emulation neural network based on the brain of a biological organism.
FIG. 3 shows an example architecture mapping system.
FIG. 4 illustrates an example graph and an example sub-graph.
FIG. 5 shows an example adversarial training system.
FIG. 6 shows an example distillation training system.
FIG. 7 shows an example reservoir computing system.
FIG. 8 A shows an example architecture search system.
FIG. 8 B shows an example constraint satisfaction system.
FIG. 8 C shows an example evolutionary system.
FIG. 8 D shows an example optimization system.
FIG. 9 is a flow diagram of an example process for generating a brain emulation neural network.
FIG. 10 is a flow diagram of an example process for determining an artificial neural network architecture corresponding to a sub-graph of a synaptic connectivity graph.
FIG. 11 is a flow diagram of an example process for adversarial training of a student neural network using a brain emulation neural network.
FIG. 12 is a flow diagram of an example process for distillation training of a student neural network using a brain emulation neural network.
FIG. 13 is a flow diagram of an example process for processing data using a reservoir computing neural network that includes: (i) a brain emulation sub-network, and (ii) a prediction sub-network.
FIG. 14 is a flow diagram of an example process for seeding a neural architecture search procedure using a synaptic connectivity graph.
FIG. 15 is a block diagram of an example computer system.
Like reference numbers and designations in the various drawings indicate like elements.
DETAILED DESCRIPTION
FIG. 1 illustrates an example of generating an artificial (i.e., computer implemented) brain emulation neural network 100 based on a synaptic resolution image 102 of the brain 104 of a biological organism 106 , e.g., a fly. The synaptic resolution image 102 may be processed to generate a synaptic connectivity graph 108 , e.g., where each node of the graph 108 corresponds to a neuron in the brain 104 , and two nodes in the graph 108 are connected if the corresponding neurons in the brain 104 share a synaptic connection. The structure of the graph 108 may be used to specify the architecture of the brain emulation neural network 100 . For example, each node of the graph 108 may mapped to an artificial neuron, a neural network layer, or a group of neural network layers in the brain emulation neural network 100 . Further, each edge of the graph 108 may be mapped to a connection between artificial neurons, layers, or groups of layers in the brain emulation neural network 100 . The brain 104 of the biological organism 106 may be adapted by evolutionary pressures to be effective at solving certain tasks, e.g., classifying objects or generating robust object representations, and the brain emulation neural network 100 may share this capacity to effectively solve tasks. These features and other features are described in more detail below.
FIG. 2 shows an example data flow 200 for generating a synaptic connectivity graph 202 and a brain emulation neural network 204 based on the brain 206 of a biological organism. As used throughout this document, a brain may refer to any amount of nervous tissue from a nervous system of a biological organism, and nervous tissue may refer to any tissue that includes neurons (i.e., nerve cells). The biological organism may be, e.g., a worm, a fly, a mouse, a cat, or a human.
An imaging system 208 may be used to generate a synaptic resolution image 210 of the <figure-callout id="206" label="brain" filenames="US2
CLAIMS
Claims ( 21 )
1 . (canceled)
2 . A method performed by one or more data processing apparatus, the method comprising:
obtaining a three-dimensional (3D) synaptic resolution image of at least a portion of a brain of a biological organism; processing the 3D synaptic resolution image to generate data defining a set of weight values that jointly parameterize an artificial neural network architecture; generating an artificial neural network that has the artificial neural network architecture and that is parametrized by the set of weight values derived from the 3D synaptic resolution image of the brain; and processing a network input using the artificial neural network, in accordance with the set of weight values derived from the 3D synaptic resolution image of the brain, to generate a network output.
3 . The method of claim 2 , wherein processing the 3D synaptic resolution image to generate data defining the set of weight values that joint parametrize the artificial neural network architecture comprises:
processing the 3D synaptic resolution image of the brain to identify a plurality of synaptic connections depicted in the 3D synaptic resolution image of the brain; and processing the 3D synaptic resolution image of the brain to generate a respective biological weight value for each of the plurality of synaptic connections depicted in the 3D synaptic resolution image of the brain.
4 . The method of claim 3 , wherein each of the plurality of synaptic connections is between a respective first neuron and a respective second neuron in the brain of the biological organism, and
wherein processing the 3D synaptic resolution image of the brain to generate a respective biological weight value for each of the plurality of synaptic connections depicted in the 3D synaptic resolution image of the brain comprises, for each of the plurality of synaptic connections:
determining: (i) a first tolerance region in the 3D synaptic resolution image around a first neuron corresponding to the synaptic connection, and (ii) a second tolerance region in the 3D synaptic resolution image around a second neuron corresponding to the synaptic connection; and
determining the biological weight value for the synaptic connection based on an area of overlap between the first tolerance region and the second tolerance region.
5 . The method of claim 2 , further comprising processing the 3D synaptic resolution image of the brain to generate the artificial neural network architecture.
6 . The method of claim 5 , wherein processing the 3D synaptic resolution image of the brain to generate the artificial neural network architecture comprises:
processing the 3D synaptic resolution image of the brain to identify a plurality of biological neurons depicted in the 3D synaptic resolution image; and instantiating a respective artificial neuron in the artificial neural network architecture corresponding to each of the plurality of biological neurons.
7 . The method of claim 6 , further comprising:
processing the 3D synaptic resolution image of the brain to identify a plurality of synaptic connections depicted in the 3D synaptic resolution image; and determining a wiring between artificial neurons of the artificial neural network architecture based on the synaptic connections depicted in the 3D synaptic resolution image.
8 . The method of claim 7 , wherein determining the wiring between artificial neurons of the artificial neural network based on the synaptic connections depicted in the 3D synaptic resolution image comprises, for each of the plurality of synaptic connections:
mapping the synaptic connection to a corresponding connection between a corresponding pair of artificial neurons in the artificial neural network architecture.
9 . The method of claim 2 , wherein the artificial neural network architecture is further parametrized by a set of weight values that are not derived from the 3D synaptic resolution image of the brain.
10 . The method of claim 2 , further comprising training the artificial neural network using a machine learning training technique on a set of training data.
11 . The method of claim 2 , wherein the network input comprises image data.
12 . The method of claim 2 , wherein the network output comprises classification data that specifies a respective score for each of multiple classes.
13 . The method of claim 2 , wherein the biological organism is an animal.
14 . The method of claim 13 , wherein the biological organism is a fly.
15 . The method of claim 2 , wherein the 3D synaptic resolution image of the brain of the biological organism is an electron microscope image.
16 . A system comprising:
one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
obtaining a three-dimensional (3D) synaptic resolution image of at least a portion of a brain of a biological organism;
processing the 3D synaptic resolution image to generate data defining a set of weight values that jointly parameterize an artificial neural network architecture;
generating an artificial neural network that has the artificial neural network architecture and that is parametrized by the set of weight values derived from the 3D synaptic resolution image of the brain; and
processing a network input using the artificial neural network, in accordance with the set of weight values derived from the 3D synaptic resolution image of the brain, to generate a network output.
17 . One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
obtaining a three-dimensional (3D) synaptic resolution image of at least a portion of a brain of a biological organism; processing the 3D synaptic resolution image to generate data defining a set of weight values that jointly parameterize an artificial neural network architecture; generating an artificial neural network that has the artificial neural network architecture and that is parametrized by the set of weight values derived from the 3D synaptic resolution image of the brain; and processing a network input using the artificial neural network, in accordance with the set of weight values derived from the 3D synaptic resolution image of the brain, to generate a network output.
18 . The non-transitory computer storage media of claim 17 , wherein processing the 3D synaptic resolution image to generate data defining the set of weight values that joint parametrize the artificial neural network architecture comprises:
processing the 3D synaptic resolution image of the brain to identify a plurality of synaptic connections depicted in the 3D synaptic resolution image of the brain; and processing the 3D synaptic resolution image of the brain to generate a respective biological weight value for each of the plurality of synaptic connections depicted in the 3D synaptic resolution image of the brain.
19 . The non-transitory computer storage media of claim 18 , wherein each of the plurality of synaptic connections is between a respective first neuron and a respective second neuron in the brain of the biological organism, and
wherein processing the 3D synaptic resolution image of the brain to generate a respective biological weight value for each of the plurality of synaptic connections depicted in the 3D synaptic resolution image of the brain comprises, for each of the plurality of synaptic connections:
determining: (i) a first tolerance region in the 3D synaptic resolution image around a first neuron corresponding to the synaptic connection, and (ii) a second tolerance region in the 3D synaptic resolution image around a second neuron corresponding to the synaptic connection; and
determining the biological weight value for the synaptic connection based on an area of overlap between the first tolerance region and the second tolerance region.
20 . The non-transitory computer storage media of claim 17 , wherein the operations further comprise processing the 3D synaptic resolution image of the brain to generate the artificial neural network architecture.
21 . The non-transitory computer storage media of claim 20 , wherein processing the 3D synaptic resolution image of the brain to generate the artificial neural network architecture comprises:
processing the 3D synaptic resolution image of the brain to identify a plurality of biological neurons depicted in the 3D synaptic resolution image; and instantiating a respective artificial neuron in the artificial neural network architecture corresponding to each of the plurality of biological neurons.
US18/173,157
2019-12-31
2023-02-23
Artificial neural network architectures based on synaptic connectivity graphs
Abandoned
US20230229901A1
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US18/173,157
US20230229901A1
( en )
2019-12-31
2023-02-23
Artificial neural network architectures based on synaptic connectivity graphs
Applications Claiming Priority (2)
Application Number
Priority Date
Filing Date
Title
US16/731,396
US11593627B2
( en )
2019-12-31
2019-12-31
Artificial neural network architectures based on synaptic connectivity graphs
US18/173,157
US20230229901A1
( en )
2019-12-31
2023-02-23
Artificial neural network architectures based on synaptic connectivity graphs
Related Parent Applications (1)
Application Number
Title
Priority Date
Filing Date
US16/731,396
Continuation
US11593627B2
( en )
2019-12-31
2019-12-31
Artificial neural network architectures based on synaptic connectivity graphs
Publications (1)
Publication Number
Publication Date
US20230229901A1
true
US20230229901A1 ( en )
2023-07-20
Family
ID=76546295
Family Applications (2)
Application Number
Title
Priority Date
Filing Date
US16/731,396
Active
2041-05-08
US11593627B2
( en )
2019-12-31
2019-12-31
Artificial neural network architectures based on synaptic connectivity graphs
US18/173,157
Abandoned
US20230229901A1
( en )
2019-12-31
2023-02-23
Artificial neural network architectures based on synaptic connectivity graphs
Family Applications Before (1)
Application Number
Title
Priority Date
Filing Date
US16/731,396
Active
2041-05-08
US11593627B2
( en )
2019-12-31
2019-12-31
Artificial neural network architectures based on synaptic connectivity graphs
Country Status (2)
Country
Link
US
( 2 )
US11593627B2
( en )
WO
( 1 )
WO2021138092A1
( en )
Cited By (2)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20220284279A1
( en )
*
2021-03-03
2022-09-08
X Development Llc
Computational techniques for identifying the surface of a brain
US12026938B2
( en )
*
2019-07-12
2024-07-02
Huawei Technologies Co., Ltd.
Neural architecture search method and image processing method and apparatus
Families Citing this family (12)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US11593617B2
( en )
2019-12-31
2023-02-28
X Development Llc
Reservoir computing neural networks based on synaptic connectivity graphs
US11631000B2
( en )
2019-12-31
2023-04-18
X Development Llc
Training artificial neural networks based on synaptic connectivity graphs
US11568201B2
( en )
2019-12-31
2023-01-31
X Development Llc
Predicting neuron types based on synaptic connectivity graphs
US11620487B2
( en )
2019-12-31
2023-04-04
X Development Llc
Neural architecture search based on synaptic connectivity graphs
US11625611B2
( en )
2019-12-31
2023-04-11
X Development Llc
Training artificial neural networks based on synaptic connectivity graphs
US11783187B2
( en )
*
2020-03-04
2023-10-10
Here Global B.V.
Method, apparatus, and system for progressive training of evolving machine learning architectures
US20220414886A1
( en )
*
2021-06-28
2022-12-29
X Development Llc
Semantic image segmentation using contrastive channels
CN113762492B
( en )
*
2021-08-27
2024-03-01
åæµå¤§å¦
Image recognition system and method based on organic synaptic transistor artificial neural network
CN114403845B
( en )
*
2021-12-31
2025-05-27
æ·±å³å è¿ææ¯ç 究é¢
Dynamic addiction neural circuit generation method and system based on weakly supervised contrastive learning
WO2023123380A1
( en )
*
2021-12-31
2023-07-06
æ·±å³å è¿ææ¯ç 究é¢
Dynamic addiction neural circuit generation method and system based on weakly supervised contrastive learning
CN115905079A
( en )
2022-07-01
2023-04-04
é¿éå·´å·´ï¼ä¸å½ï¼æéå ¬å¸
System, computer-implemented method, and computer-readable storage medium
CN118378052B
( en )
*
2024-06-25
2024-09-24
é¹åå®éªå®¤
Signal analysis method, device, equipment and medium
Family Cites Families (51)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
WO1994010635A2
( en )
1992-11-02
1994-05-11
Boston University
Neural networks with subdivision
US5619619A
( en )
1993-03-11
1997-04-08
Kabushiki Kaisha Toshiba
Information recognition system and control system using same
US6542249B1
( en )
1999-07-20
2003-04-01
The University Of Western Ontario
Three-dimensional measurement method and apparatus
US7398259B2
( en )
2002-03-12
2008-07-08
Knowmtech, Llc
Training of a physical neural network
JPWO2010106587A1
( en )
2009-03-18
2012-09-13
ããã½ããã¯æ ªå¼ä¼ç¤¾
Neural network system
TWI406181B
( en )
2009-05-11
2013-08-21
Nat Univ Tsing Hua
A method for searching and constructing 3d image database
US9015093B1
( en )
2010-10-26
2015-04-21
Michael Lamport Commons
Intelligent control with hierarchical stacked neural networks
US8775341B1
( en )
2010-10-26
2014-07-08
Michael Lamport Commons
Intelligent control with hierarchical stacked neural networks
US20170372195A1
( en )
2012-10-19
2017-12-28
National Tsing Hua University
System for Finding Shortest Pathway between Neurons in A Neural Network
US20220036151A1
( en )
2012-10-19
2022-02-03
National Tsing Hua University
System for Finding Shortest Pathway between Neurons in Neuronal Linkage Pathways
KR20150016089A
( en )
2013-08-02
2015-02-11
ìë³ìµ
Neural network computing apparatus and system, and method thereof
US10019470B2
( en )
2013-10-16
2018-07-10
University Of Tennessee Research Foundation
Method and apparatus for constructing, using and reusing components and structures of an artifical neural network
WO2015067299A1
( en )
2013-11-05
2015-05-14
Brainlab Ag
Quantification of brain vulnerability
US20150170028A1
( en )
2013-12-13
2015-06-18
Qualcomm Incorporated
Neuronal diversity in spiking neural networks and pattern classification
US10198691B2
( en )
2014-06-19
2019-02-05
University Of Florida Research Foundation, Inc.
Memristive nanofiber neural networks
US10115054B2
( en )
2014-07-02
2018-10-30
International Business Machines Corporation
Classifying features using a neurosynaptic system
JP6446126B2
( en )
2015-04-17
2018-12-26
ï¼£ï½ï½ï½ï½ï½ ï¼³ï½æ ªå¼ä¼ç¤¾
Processing system and program
US10713562B2
( en )
2016-06-18
2020-07-14
International Business Machines Corporation
Neuromorphic memory circuit
US9646243B1
( en )
2016-09-12
2017-05-09
International Business Machines Corporation
Convolutional neural networks using resistive processing unit array
US10395165B2
( en )
2016-12-01
2019-08-27
Via Alliance Semiconductor Co., Ltd
Neural network unit with neural memory and array of neural processing units that collectively perform multi-word distance rotates of row of data received from neural memory
US10565500B2
( en )
2016-12-20
2020-02-18
Intel Corporation
Unsupervised learning using neuromorphic computing
JP6889270B2
( en )
2017-02-23
2021-06-18
ã°ã¼ã°ã« ã¨ã«ã¨ã«ã·ã¼ï¼§ï½ï½ï½ï½ï½ ＬＬＣ
Neural network architecture optimization
US10878313B2
( en )
2017-05-02
2020-12-29
Intel Corporation
Post synaptic potential-based learning rule
US20190005384A1
( en )
2017-06-29
2019-01-03
General Electric Company
Topology aware graph neural nets
CN111465946B
( en )
2017-10-27
2024-05-28
æ¸æ §ç§ææéå ¬å¸
Neural Network Architecture Search Using Hierarchical Representations
US12056605B2
( en )
2017-10-27
2024-08-06
Royal Bank Of Canada
System and method for improved neural network training
WO2019108923A1
( en )
2017-11-30
2019-06-06
Google Llc
Neural architecture search using a performance prediction neural network
DE102018201290A1
( en )
2018-01-29
2019-08-01
Osram Gmbh
METHOD FOR PRODUCING A COMPRESSED VERSION OF AN ARTIFICIAL NEURONAL NETWORK AND DEVICE AND STORAGE MEDIUM
CN118194960A
( en )
2018-02-02
2024-06-14
è°·ææéè´£ä»»å ¬å¸
Regularized Neural Network Architecture Search
CN109643229B
( en )
2018-04-17
2022-10-04
æ·±å³é²²äºä¿¡æ¯ç§ææéå ¬å¸
Application development method and platform of network model and computer readable storage medium
CN112368719A
( en )
2018-05-17
2021-02-12
å¥è·å ¬å¸
Gradient antagonism training of neural networks
US11205125B2
( en )
2018-06-29
2021-12-21
International Business Machines Corporation
Scheduler and simulator for an area-efficient, reconfigurable, energy-efficient, speed-efficient neural network
CA3110199A1
( en )
*
2018-08-21
2020-02-27
The Salk Institute For Biological Studies
Systems and methods for enhanced imaging and analysis
US20200129110A1
( en )
2018-10-24
2020-04-30
Translational Research Institute Pty Ltd As Trustee For Translational Research Institute Trust
Functional analysis of human brain using functional magnetic resonance imaging (fmri) for acute stress and post traumatic stress disorder (ptsd) monitoring neuroplasticity
US11010516B2
( en )
2018-11-09
2021-05-18
Nvidia Corp.
Deep learning based identification of difficult to test nodes
WO2020140073A1
( en )
2018-12-27
2020-07-02
Google Llc
Neural architecture search through a graph search space
US11531878B2
( en )
2019-02-19
2022-12-20
International Business Machines Corporation
Behavior prediction with dynamic adaptation to environmental conditions
US11544535B2
( en )
2019-03-08
2023-01-03
Adobe Inc.
Graph convolutional networks with motif-based attention
US20200364567A1
( en )
2019-05-17
2020-11-19
Samsung Electronics Co., Ltd.
Neural network device for selecting action corresponding to current state based on gaussian value distribution and action selecting method using the neural network device
US10997503B2
( en )
2019-06-20
2021-05-04
Google Llc
Computationally efficient neural network architecture search
KR102686100B1
( en )
2019-07-18
2024-07-19
ì¼ì±ëì¤íë ì´ ì£¼ìíì¬
Method of driving display panel and display apparatus for performing the method
US20210019599A1
( en )
2019-07-19
2021-01-21
Google Llc
Adaptive neural architecture search
US20210034813A1
( en )
2019-07-31
2021-02-04
3M Innovative Properties Company
Neural network model with evidence extraction
US11443162B2
( en )
2019-08-23
2022-09-13
Google Llc
Resource constrained neural network architecture search
US11568259B2
( en )
2019-10-15
2023-01-31
Zoox, Inc.
Cross batch normalization
US11625611B2
( en )
2019-12-31
2023-04-11
X Development Llc
Training artificial neural networks based on synaptic connectivity graphs
US11593617B2
( en )
2019-12-31
2023-02-28
X Development Llc
Reservoir computing neural networks based on synaptic connectivity graphs
US11620487B2
( en )
2019-12-31
2023-04-04
X Development Llc
Neural architecture search based on synaptic connectivity graphs
US11568201B2
( en )
2019-12-31
2023-01-31
X Development Llc
Predicting neuron types based on synaptic connectivity graphs
WO2022035829A1
( en )
2020-08-14
2022-02-17
X Development Llc
Processing satellite images using brain emulation neural networks
US20220051079A1
( en )
2020-08-14
2022-02-17
X Development Llc
Auto-encoding using neural network architectures based on synaptic connectivity graphs
2019
2019-12-31
US
US16/731,396
patent/US11593627B2/en
active
Active
2020
2020-12-18
WO
PCT/US2020/066153
patent/WO2021138092A1/en
not_active
Ceased
2023
2023-02-23
US
US18/173,157
patent/US20230229901A1/en
not_active
Abandoned
Cited By (2)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US12026938B2
( en )
*
2019-07-12
2024-07-02
Huawei Technologies Co., Ltd.
Neural architecture search method and image processing method and apparatus
US20220284279A1
( en )
*
2021-03-03
2022-09-08
X Development Llc
Computational techniques for identifying the surface of a brain
Also Published As
Publication number
Publication date
WO2021138092A1
( en )
2021-07-08
US20210201119A1
( en )
2021-07-01
US11593627B2
( en )
2023-02-28
Similar Documents
Publication
Publication Date
Title
US11620487B2
( en )
2023-04-04
Neural architecture search based on synaptic connectivity graphs
US20230229891A1
( en )
2023-07-20
Reservoir computing neural networks based on synaptic connectivity graphs
US11593627B2
( en )
2023-02-28
Artificial neural network architectures based on synaptic connectivity graphs
US11568201B2
( en )
2023-01-31
Predicting neuron types based on synaptic connectivity graphs
US11625611B2
( en )
2023-04-11
Training artificial neural networks based on synaptic connectivity graphs
US11631000B2
( en )
2023-04-18
Training artificial neural networks based on synaptic connectivity graphs
US20220188605A1
( en )
2022-06-16
Recurrent neural network architectures based on synaptic connectivity graphs
US20220051079A1
( en )
2022-02-17
Auto-encoding using neural network architectures based on synaptic connectivity graphs
Lee et al.
2022
Localization uncertainty estimation for anchor-free object detection
US20230142885A1
( en )
2023-05-11
Selecting neural network architectures based on community graphs
US20220050995A1
( en )
2022-02-17
Processing satellite images using brain emulation neural networks
US20230004791A1
( en )
2023-01-05
Compressed matrix representations of neural network architectures based on synaptic connectivity
US20230196059A1
( en )
2023-06-22
Attention-based brain emulation neural networks
US20220414433A1
( en )
2022-12-29
Automatically determining neural network architectures based on synaptic connectivity
US20230196541A1
( en )
2023-06-22
Defect detection using neural networks based on biological connectivity
Aa et al.
2017
Deep neural networks for image classification
US20230186622A1
( en )
2023-06-15
Processing remote sensing data using neural networks based on biological connectivity
US20230206059A1
( en )
2023-06-29
Training brain emulation neural networks using biologically-plausible algorithms
US20220414886A1
( en )
2022-12-29
Semantic image segmentation using contrastive channels
US20220414419A1
( en )
2022-12-29
Controlling agents interacting with an environment using brain emulation neural networks
Olawuyi et al.
2022
Plant diseases detection and classification using deep transfer learning
US20220414453A1
( en )
2022-12-29
Data augmentation using brain emulation neural networks
US20220391692A1
( en )
2022-12-08
Semantic understanding of dynamic imagery using brain emulation neural networks
US20230186059A1
( en )
2023-06-15
Neural networks based on hybridized synaptic connectivity graphs
US20220343134A1
( en )
2022-10-27
Convolutional neural network architectures based on synaptic connectivity
Legal Events
Date
Code
Title
Description
2023-02-23
AS
Assignment
Owner name : X DEVELOPMENT LLC, CALIFORNIA
Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LASZLO, SARAH ANN;WATSON, PHILIP EDWIN;SIGNING DATES FROM 20191231 TO 20200106;REEL/FRAME:062779/0335
2023-04-19
STPP
Information on status: patent application and granting procedure in general
Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION
2024-04-29
STPP
Information on status: patent application and granting procedure in general
Free format text : NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS
2024-08-14
STCB
Information on status: application discontinuation
Free format text : ABANDONED -- FAILURE TO PAY ISSUE FEE