Conceptio › Archive › Google Patents

Google Patentsopen access

Neural Network Representation Formats — Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. (US20250384299A1)

Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. · Google Patents

Google Patents · Patents · License: Open Access

patent, google patents, intellectual property, US20250384299A1, Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V., Stefan Matlage, en, 2025

ABSTRACT

Abstract

Data stream having a representation of a neural network encoded thereinto, the data stream including serialization parameter indicating a coding order at which neural network parameters, which define neuron interconnections of the neural network, are encoded into the data stream.

Description

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of copending U.S. patent application Ser. No. 17/711,569, filed Apr. 1, 2022, which in turn is a continuation of International Application No. PCT/2020/077352, filed Sep. 30, 2020, which is incorporated herein by reference in its entirety, and additionally claims priority from European Application No. EP 19200928.0, filed Oct. 1, 2019, which is incorporated herein by reference in its entirety.

The present application relates to concepts for Neural Network Representation Formats.

BACKGROUND OF THE INVENTION

Neural Networks (NN) have led to break-throughs in many applications nowadays:

object detection or classification in image/video data speech/keyword recognition in audio speech synthesis optical character recognition language translation and so on

However, the applicability in certain usage scenarios is still hampered by the sheer amount of data that is needed to represent NNs. In most cases, this data is comprised by two types of parameters, the weights and bias, that describe the connection between neurons. The weights are usually parameters that perform some type of linear transformation to the input values (e.g., dot product or convolution), or in other words, weight the neuron's inputs, and the bias are offsets that are added after the linear calculation, or in other words, offset the neuron's aggregation of inbound weighted messages. More specifically, these weights, biases and further parameter that characterize each connection between two of the potentially very large number of neurons (up to tens of millions) in each layer (up to hundreds) of the NN occupy the major portion of the data associated to a particular NN. Also, these parameters are typically consisting of sizable floating-point date types. These parameters are usually expressed as large tensors carrying all parameters of each layer. When applications involve frequent transmission/updates of the involved NNs, the data rate that may be used becomes a serious bottle neck. Therefore, efforts to reduce the coded size of NN representations by means of lossy compression of these matrices is a promising approach.

Typically, the parameter tensors are stored in container formats (ONNX(ONNX=Open Neural Network Exchange), Pytorch, TensorFlow, and the like) that carry all data (such as the above parameter matrices) and further properties (such as dimensions of the parameter tensors, type of layers, operations and so on) that that may be used to fully reconstruct the NN and execute it.

It would be advantageous to have a concept at hand which renders transmission/updates of machine learning predictors or, alternatively speaking, machine learning models such as a neural network more efficient such as more efficient in terms of conservation of inference quality with reducing, concurrently, a coded size of NN representations, computational inference complexity, complexity of describing or storing the NN representations, or which enables a more frequent transmission/update of a NN than currently or which even improves the inference quality for a certain task at hand and/or for a certain local input data statistic. Furthermore, it would be advantageous to provide a neural network representation, a derivation of such neural network representation and the usage of such neural network representation in performing neural network based prediction so that the usage of neural networks becomes more effective than currently.

SUMMARY

An embodiment may have a data stream having a representation of a neural network encoded thereinto, the data stream including a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference.

Another embodiment may have an apparatus for encoding a representation of a neural network into a data stream, wherein the apparatus is configured to provide the data stream with a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference.

Another embodiment may have an apparatus for decoding a representation of a neural network from a data stream, wherein the apparatus is configured to decode from the data stream a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference, and to use the numerical representation and bit size for representing the neural network parameters decoded from the data stream.

According to another embodiment, an apparatus for performing an inference using a neural network may have: an inventive apparatus for decoding a data stream, so as to derive from the data stream the neural network, and a processor configured to perform the inference based on the neural network.

Another embodiment may have a method for encoding a representation of a neural network into a data stream, providing the data stream with a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference.

According to another embodiment, a method for decoding a representation of a neural network from a data stream may have the step of: decoding from the data stream a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference, and to use the numerical representation and bit size for representing the neural network parameters decoded from the data stream.

Another embodiment may have a non-transitory digital storage medium having a computer program stored thereon to perform the methods for encoding and decoding a representation of a neural network when said computer program is run by a computer.

It is a basic idea underlying a first aspect of the present application that a usage of neural networks (NN) is rendered highly efficient, if a serialization parameter is encoded/decoded into/from a data stream having a representation of the NN encoded thereinto. The serialization parameter indicates a coding order at which NN parameters, which define neuron interconnections of the NN, are encoded into the data stream. The neuron interconnections might represent connections between neurons of different NN layers of the NN. In other words, a NN parameter might define a connection between a first neuron associated with a first layer of the NN and a second neuron associated with a second layer of the NN. A decoder might use the coding order to assign NN parameters serially decoded from the data stream to the neuron interconnections.

In particular, using the serialization parameter turns out to efficiently divide a bitstring into meaningful consecutive subsets of the NN parameters. The serialization parameter might indicate a grouping of the NN parameters allowing an efficient execution of the NN. This might be done dependent on application scenarios for the NN. For different application scenarios, an encoder might traverse the NN parameters using different coding orders. Thus, the NN parameters can be encoded using individual coding orders dependent on the application scenario of the NN and the decoder can reconstruct the NN parameters accordingly while decoding, because of the information provided by the serialization parameter. The NN parameters might represent entries of one or more parameter matrices or tensors, wherein the parameter matrices or tensors might be used for inference procedures. It was found that the one or more parameter matrices or tensors of the NN can be efficiently reconstructed by a decoder based on decoded NN parameters and the serialization parameter.

Thus, the serialization parameter allows the usage of different application specific coding orders allowing a flexible encoding and decoding with an improved efficiency. For instance, encoding parameters along different dimensions may benefit the resulting compression performance since the entropy coder may be able to better capture dependencies among them. In another example, it may be desirable to group parameters according to certain application specific criteria, i.e. what part of the input data they relate to or whether they can be jointly executed, so that they can be decoded/inferred in parallel. A further example is to encode the parameters following the General Matrix Matrix (GEMM) product scan order that support efficient memory allocation of the decoded parameters when performing a dot product operation (Andrew Kerr, 2017).

A further embodiment is directed to encoder-side chosen permutations of the data, e.g. in order to achieve, for instance, energy compaction of the NN parameter to be coded and subsequently process/serialize/code the resulting permutated data according to the resulting order. The permutation may, thus, sort the parameters so that same increase or so that same decrease steadily along the coding order.

In accordance with a second aspect of the present application, the inventors of the present application realized that a usage of neural networks, NN, is rendered highly efficient, if a numerical computation representation parameter is encoded/decoded into/from a data stream having a representation of the NN encoded thereinto. The numerical computation representation parameter indicates a numerical representation, e.g. among floating point or fixed point representation, and a bit size at which NN parameters of the NN, which are encoded into the data stream, are to be represented when using the NN for inference. An encoder is configured to encode the NN parameters. A decoder is configured to decode the NN parameters and might be configured to use the numerical representation and bit size for representing the NN parameters decoded from the data stream, DS.

This embodiment is based on the idea, that it may be advantageous to represent the NN parameters and activation values, which activation values result from a usage of the NN parameters at an inference using the NN, both with the same numerical representation and bit size. Based on the numerical computation representation parameter it is possible to compare efficiently the indicated numerical representation and bit size for the NN parameters with possible numerical representations and bit sizes for the activation values. This might be especially advantageous in case of the numerical computation representation parameter indicating a fixed point representation as numerical representation, since then, if both the NN parameters and the activation values can be represented in the fixed point representation, inference can be performed efficiently due to fixed-point arithmetic.

In accordance with a third aspect of the present application, the inventors of the present application realized that a usage of neural networks is rendered highly efficient, if a NN layer type parameter is encoded/decoded into/from a data stream having a representation of the NN encoded thereinto. The NN layer type parameter indicates a NN layer type, e.g., convolutional layer type or fully connected layer type, of a predetermined NN layer of the NN. The data stream is structured into one or more individually accessible portions, each individually accessible portion representing a corresponding NN layer of the NN. The predetermined NN layer represents one of the NN layer of the neural network. Optionally, for each of two or more predetermined NN layer of the NN, the NN layer type parameter is encoded/decoded into/from a data stream, wherein the NN layer type parameter can differ between at least some predetermined NN layer.

This embodiment is based on the idea, that it may be useful, that the data stream comprises the NN layer type parameter for NN layer, in order to, for instance, understand a meaning of the dimensions of a parameter tensor/matrix. Moreover, different layers may be treated differently while encoding in order to better capture the dependencies in the data and lead to a higher coding efficiency, e.g., by using different sets or modes of context models, information that may be crucial for the decoder to know prior to decoding.

Similarly, it may be advantageous to encode/decode into/from a data stream a type parameter indicting a parameter type of the NN parameters. The type parameter may indicate whether the NN parameters represent weights or bias. The data stream is structured into one or more individually accessible portions, each individually accessible portion representing a corresponding NN layer of the NN. An individually accessible portion representing a corresponding predetermined NN layer might be further structured into individually accessible sub-portions. Each individually accessible sub-portion is completely traversed by a coding order before a subsequent individually accessible sub-portion is traversed by the coding order. Into each individually accessible sub-portion, for example, NN parameters and a type parameter are encoded and can be decoded. NN parameter of a first individually accessible sub-portion may be of a different parameter type or of the same parameter type as NN parameter of a second individually accessible sub-portion. Different types of NN parameters associated with the same NN layer might be encoded/decoded into/from different individually accessible sub-portions associated with the same individually accessible portion. The distinction between the parameter types may be beneficial for encoding/decoding when, for instance, different types of dependencies can be used for each type of parameters, or if parallel decoding is wished, etc. It is, for example, possible to encode/decode different types of NN parameters associated with the same NN layer parallel. This enables a higher efficiency in encoding/decoding of the NN parameters and may also benefit the resulting compression performance since the entropy coder may be able to better capture dependencies among the NN parameters.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application relates to concepts for Neural Network Representation Formats.

BACKGROUND OF THE INVENTION

Neural Networks (NN) have led to break-throughs in many applications nowadays:

object detection or classification in image/video data speech/keyword recognition in audio speech synthesis optical character recognition language translation and so on

SUMMARY

In accordance with a fourth aspect of the present application, the inventors of the present application realized that a transmission/updating of neural networks is rendered highly efficient, if a pointer is encoded/decoded into/from a data stream having a representation of the NN encoded thereinto. This is due to the fact, that the data stream is structured into individually accessible portions and for each of one or more predetermined individually accessible portions, a pointer points to a beginning of the respective predetermined individually accessible portion. Not all individually accessible portions need to be predetermined individually accessible portions, but it is possible, that all individually accessible portions represent predetermined individually accessible portions. The one or more predetermined individually accessible portions might be set by default or dependent on an application of the NN encoded into the data stream. The pointer indicates, for example, the beginning of the respective predetermined individually accessible portion as data stream position in bytes or as an offset, e.g., a byte offset with respect to a beginning of the data stream or with respect to a beginning of a portion corresponding to a NN layer, to which portion the respective predetermined individually accessible portion belongs to. The pointer might be encoded/decoded into/from a header portion of the data stream. According to an embodiment, for each of the one or more predetermined individually accessible portions, the pointer is encoded/decoded into/from a header portion of the data stream, in case of the respective predetermined individually accessible portion representing a corresponding NN layer of the neural network or the pointer is encoded/decoded into/from a parameter set portion of a portion corresponding to a NN layer, in case of the respective predetermined individually accessible portion representing a NN portion of a NN layer of the NN. A NN portion of a NN layer of the NN might represent a baseline section of the respective NN layer or an advanced section of the respective layer. With the pointer it is possible to efficiently access the predetermined individually accessible portions of the data stream enabling, for example, to parallelize the layer processing or package the data stream into respective container formats. The pointer allows easier, faster and more adequate access to the predetermined individually accessible portions in order to facilitate applications that involve parallel or partial decoding and execution of NNs.

In accordance with a fifth aspect of the present application, the inventors of the present application realized that a transmission/updating of neural networks is rendered highly efficient, if a start code, a pointer and/or a data stream length parameter is encoded/decoded into/from an individually accessible sub-portion of a data stream having a representation of the NN encoded thereinto. The data stream is structured into one or more individually accessible portions, each individually accessible portion representing a corresponding NN layer of the neural network. Additionally, the data stream is, within one or more predetermined individually accessible portions, further structured into individually accessible sub-portions, each individually accessible sub-portion representing a corresponding NN portion of the respective NN layer of the neural network. An apparatus is configured to encode/decode into/from the data stream, for each of the one or more predetermined individually accessible sub-portions, a start code at which the respective predetermined individually accessible sub-portion begins, and/or a pointer pointing to a beginning of the respective predetermined individually accessible sub-portion, and/or a data stream length parameter indicating a data stream length of the respective predetermined individually accessible sub-portion for skipping the respective predetermined individually accessible sub-portion in parsing the DS. The start code, the pointer and/or the data stream length parameter enable an efficient access to the predetermined individually accessible sub-portions. This is especially beneficial for applications that may rely on grouping NN parameter within a NN layer in a specific configurable fashion as it can be beneficial to have the NN parameter decoded/processed/inferred partially or in parallel. Therefore, an individually accessible sub-portion wise access to an individually accessible portion can help to access desired data in parallel or leave out unnecessary data portions. It was found, that it is sufficient to indicate an individually accessible sub-portion using a start code. This is based on the finding, that an amount of data per NN layer, i.e. individually accessible portion, is usually less than in case NN layers are to be detected by start codes within the whole data stream. Nevertheless, it is also advantageous to use the pointer and/or the data stream length parameter to improve the access to an individually accessible sub-portion. According to an embodiment, the one or more individually accessible sub-portions within an individually accessible portion of the data stream are indicated by a pointer indicating a data stream position in bytes in a parameter set portion of the individually accessible portion. The data stream length parameter might indicate a run length of individually accessible sub-portions. The data stream length parameter might be encoded/decoded into/from a header portion of the data stream or into/from the parameter set portion of the individually accessible portion. The data stream length parameter might be used in order to facilitate cut out of the respective individually accessible sub-portion for the purpose of packaging the one or more individually accessible sub-portion in appropriate containers. According to an embodiment, an apparatus for decoding the data stream is configured to use, for one or more predetermined individually accessible sub-portions, the start code and/or the pointer and/or the data stream length parameter for accessing the data stream.

In accordance with a sixth aspect of the present application, the inventors of the present application realized that a usage of neural networks is rendered highly efficient, if a processing option parameter is encoded/decoded into/from a data stream having a representation of the NN encoded thereinto. The data stream is structured into individually accessible portions and for each of one or more predetermined individually accessible portions a processing option parameter indicates one or more processing options which have to be used or which may optionally be used when using the neural network for inference. The processing option parameter might indicate one processing option out of various processing options that also determine if and how a client would access the individually accessible portions (P) and/or the individually accessible sub-portions (SP), like, for each of P and/or SP, a parallel processing capability of the respective P or SP and/or a sample wise parallel processing capability of the respective P or SP and/or a channel wise parallel processing capability of the respective P or SP and/or a classification category wise parallel processing capability of the respective P or SP and/or other processing options. The processing option parameter allows a client appropriate decision making and thus a highly efficient usage of the NN.

In accordance with a seventh aspect of the present application, the inventors of the present application realized that a transmission/updating of neural networks is rendered highly efficient, if a reconstruction rule for dequantizing NN parameters depends on a NN portion the NN parameters belong to. The NN parameters, which NN parameters represent a neural network, are encoded into a data stream in a manner quantized onto quantization indices. An apparatus for decoding is configured to dequantize the quantization indices to reconstruct the NN parameters, e.g., using the reconstruction rule. The NN parameters are encoded into the data stream so that NN parameters in different NN portions of the NN are quantized differently, and the data stream indicates, for each of the NN portions, a reconstruction rule for dequantizing NN parameters relating to the respective NN portion. The apparatus for decoding is configured to use, for each of the NN portions, the reconstruction rule indicated by the data stream for the respective NN portion to dequantize the NN parameter in the respective NN portion. The NN portions, for example, comprise one or more NN layers of the NN and/or portions of an NN layer into which portions a predetermined NN layer of the NN is subdivided.

According to an embodiment, a first reconstruction rule for dequantizing NN parameters relating to a first NN portion are encoded into the data stream in a manner delta-coded relative to a second reconstruction rule for dequantizing NN parameters relating to a second NN portion. The first NN portion might comprise first NN layers and the second NN portion might comprise second layers, wherein the first NN layers differ from the second NN layers. Alternatively, the first NN portion might comprise first NN layers and the second NN portion might comprise portions of one of the first NN layers. In this alternative case, a reconstruction rule, e.g., the second reconstruction rule, related to NN parameters in a portion of a predetermined NN layer are delta-coded relative to a reconstruction rule, e.g., the first reconstruction rule, related to the predetermined NN layer. This special delta-coding of the reconstruction rules might allow to only use few bits for signalling the reconstruction rules and can result in an efficient transmission/updating of neural networks.

In accordance with an eighth aspect of the present application, the inventors of the present application realized that a transmission/updating of neural networks is rendered highly efficient, if a reconstruction rule for dequantizing NN parameters depends on a magnitude of quantization indices associated with the NN parameters. The NN parameters, which NN parameters represent a neural network, are encoded into a data stream in a manner quantized onto quantization indices. An apparatus for decoding is configured to dequantize the quantization indices to reconstruct the NN parameters, e.g., using the reconstruction rule. The data stream comprises, for indicating the reconstruction rule for dequantizing the NN parameters, a quantization step size parameter indicating a quantization step size, and a parameter set defining a quantization-index-to-reconstruction-level mapping. The reconstruction rule for NN parameters in a predetermined NN portion is defined by the quantization step size for quantization indices within a predetermined index interval, and the quantization-index-to-reconstruction-level mapping for quantization indices outside the predetermined index interval. For each NN parameter, a respective NN parameter associated with a quantization index within the predetermined index interval, for example, is reconstructed by multiplying the respective quantization index with the quantization step size and a respective NN parameter corresponding to a quantization index outside the predetermined index interval, for example, is reconstructed by mapping the respective quantization index onto a reconstruction level using the quantization-index-to-reconstruction-level mapping. The decoder might be configured to determine the quantization-index-to-reconstruction-level mapping based on the parameter set in the data stream. According to an embodiment, the parameter set defines the quantization-index-to-reconstruction-level mapping by pointing to a quantization-index-to-reconstruction-level mapping out of a set of quantization-index-to-reconstruction-level mappings, wherein the set of quantization-index-to-reconstruction-level mappings might not be part of the data stream, e.g., it might be saved at encoder side and decoder side. Defining the reconstruction rule based on a magnitude of quantization indices can result in a signalling of the reconstruction rule with few bits.

In accordance with a ninth aspect of the present application, the inventors of the present application realized that a transmission/updating of neural networks is rendered highly efficient, if an identification parameter is encoded/decoded into/from individually accessible portions of a data stream having a representation of the NN encoded thereinto. The data stream is structured into individually accessible portions and, for each of one or more predetermined individually accessible portions, an identification parameter for identifying the respective predetermined individually accessible portion is encoded/decoded into/from the data stream. The identification parameter might indicate a version of the predetermined individually accessible portion. This is especially advantageous in scenarios such as distributed learning, where many clients individually further train a NN and send relative NN updates back to a central entity. The identification parameter can be used to identify the NN of individual clients through a versioning scheme. Thereby, the central entity can identify the NN that an NN update is built upon. Additionally, or alternatively, the identification parameter might indicate whether the predetermined individually accessible portion is associated with a baseline part of the NN or with an advanced/enhanced/complete part of the NN. This is, for example, advantageous in use cases, such as scalable NNs, where a baseline part of an NN can be executed, for instance, in order to generate preliminary results, before the complete or enhanced NN is carried out to receive full results. Further, transmission errors or involuntary changes of a parameter tensor reconstructable based on NN parameters representing the NN are easily recognizable using the identification parameter. The identification parameter allows for each predetermined individually accessible portions to check integrity and make operations more error robust when it could be verified based on the NN characteristics.

In accordance with a tenth aspect of the present application, the inventors of the present application realized that a transmission/updating of neural networks is rendered highly efficient, if different versions of the NN are encoded/decoded into/from a data stream using delta-coding or using a compensation scheme. The data stream has a representation of an NN encoded thereinto in a layered manner so that different versions of the NN are encoded into the data stream. The data stream is structured into one or more individually accessible portions, each individually accessible portion relating to a corresponding version of the NN. The data stream has, for example, a first version of the NN encoded into a first portion delta-coded relative to a second version of the NN encoded into a second portion. Additionally, or alternatively, the data stream has, for example, a first version of the NN encoded into a first portion in form of one or more compensating NN portions each of which is to be, for performing an inference based on the first version of the NN, executed in addition to an execution of a corresponding NN portion of a second version of the NN encoded into a second portion, and wherein outputs of the respective compensating NN portion and corresponding NN portion are to be summed up. With these encoded versions of the NN in the data stream, a client, e.g., a decoder, can match its processing capabilities or may be able to do inference on the first version, e.g., a baseline, first before processing the second version, e.g., a more complex advanced NN. Furthermore, by applying/using the delta-coding and/or the compensation scheme, the different versions of the NN can be encoded into the DS with few bits.

In accordance with an eleventh aspect of the present application, the inventors of the present application realized that a usage of neural networks is rendered highly efficient, if supplemental data is encoded/decoded into/from individually accessible portions of a data stream having a representation of the NN encoded thereinto. The data stream is structured into individually accessible portions and the data stream comprises for each of one or more predetermined individually accessible portions a supplemental data for supplementing the representation of the NN. This supplemental data is usually not necessary for decoding/reconstruction/inference of the NN, however, it can be essential from an application point of view. Therefore, it is advantageous to mark this supplemental data as irrelevant for the decoding of the NN for the purpose of sole inference so that clients, e.g. decoders, which do not require the supplemental data, are able to skip this part of the data.

In accordance with a twelfth aspect of the present application, the inventors of the present application realized that a usage of neural networks is rendered highly efficient, if hierarchical control data is encoded/decoded into/from a data stream having a representation of the NN encoded thereinto. The data stream comprises hierarchical control data structured into a sequence of control data portions, wherein the control data portions provide information on the NN at increasing details along the sequence of control data portions. It is advantageous to structure the control data hierarchically, since a decoder might only need the control data until a certain level of detail and can thus skip the control data providing more details. Thus, depending on the use case and its knowledge of environment, different levels of control data may be useful and with the aforementioned scheme of presenting such control data enables an efficient access to the needed control data for different use cases.

Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. An embodiment is related to a computer program having a program code for performing, when running on a computer, such a method.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will be detailed subsequently referring to the appended drawings, which are not necessarily to scale-emphasis instead generally being placed upon illustrating the principles of the inventionâand in which:

FIG. 1 shows an example of an encoding/decoding pipeline for encoding/decoding a neural network;

FIG. 2 shows a neural network which might be encoded/decoded according to one of the embodiments;

FIG. 3 shows a serialization of parameter tensors of layers of a neural network, according to an embodiment;

FIG. 4 shows the usage of a serialization parameter for indicating how neural network parameters are serialized, according to an embodiment;

FIG. 5 shows an example for a single-output-channel convolutional layer;

FIG. 6 shows an example for a fully-connected layer;

FIG. 7 shows a set of n coding orders at which neural network parameters might be encoded, according to an embodiment;

FIG. 8 shows context-adaptive arithmetic coding of individually accessible portions or sub-portions, according to an embodiment;

FIG. 9 shows the usage of a numerical computation representation parameter, according to an embodiment;

FIG. 10 shows the usage of a neural network layer type parameter indicating a neural network layer type of a neural network layer of the neural network, according to an embodiment;

FIG. 11 shows a general embodiment of a data stream with pointer pointing to beginnings of individually accessible portions, according to an embodiment;

FIG. 12 shows a detailed embodiment of a data stream with pointer pointing to beginnings of individually accessible portions, according to an embodiment;

FIG. 13 shows the usage of start codes and/or pointer and/or data stream length parameter to enable an access to individually accessible sub-portions, according to an embodiment;

FIG. 14 a shows a sub-layer access using pointer, according to an embodiment;

FIG. 14 b shows a sub-layer access using start codes, according to an embodiment;

FIG. 15 shows exemplary types of random access as possible processing options for individually accessible portions, according to an embodiment;

FIG. 16 shows the usage of a processing option parameter, according to an embodiment;

FIG. 17 shows the usage of a neural network portion dependent reconstruction rule, according to an embodiment;

FIG. 18 shows a determination of a reconstruction rule based on quantization indices representing quantized neural network parameter, according to an embodiment;

FIG. 19 shows the usage of an identification parameter, according to an embodiment;

FIG. 20 shows an encoding/decoding of different versions of a neural network, according to an embodiment;

FIG. 21 shows a delta-coding of two versions of a neural network, wherein the two versions differ in their weights and/or biases, according to an embodiment;

FIG. 22 shows an alternative delta-coding of two versions of a neural network, wherein the two versions differ in their number of neurons or neuron interconnections, according to an embodiment;

FIG. 23 shows an encoding of different versions of a neural network using compensating neural network portions, according to an embodiment;

FIG. 24 a shows an embodiment of a data stream with supplemental data, according to an embodiment;

FIG. 24 b shows an alternative embodiment of a data stream with supplemental data, according to an embodiment; and

FIG. 25 shows an embodiment of a data stream with a sequence of control data portions.

DETAILED DESCRIPTION OF THE INVENTION

Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals even if occurring in different figures.

In the following description, a plurality of details is set forth to provide a more throughout explanation of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring embodiments of the present invention. In addition, features of the different embodiments described herein after may be combined with each other, unless specifically noted otherwise.

The following description of embodiments of the present application starts with a brief introduction and outline of embodiments of the present application in order to explain their advantages and how same achieve these advantages.

It was found, that in the current activities of coded representations of NN such as developed in the ongoing MPEG activity on NN compression, it can be beneficial to separate a model bitstream representing parameter tensors of multiple layers into smaller sub-bitstreams that contain the coded representation of the parameter tensors of individual layers, i.e. layer bitstreams. This can help in general when such model bitstreams need to be stored/loaded in context of a container format or in application scenarios that feature parallel decoding/execution of layers of the NN.

In the following, various examples are described which may assist in achieving an effective compression of a neural network, NN, and/or in improving an access to data representing the NN and thus resulting in an effective transmission/updating of the NN.

In order to ease the understanding of the following examples of the present application, the description starts with a presentation of possible encoders and decoders fitting thereto into which the subsequently outlined examples of the present application could be built.

FIG. 1 shows a simple sketch example of an encoding/decoding pipeline according to DeepCABAC and illustrates the inner operations of such a compression scheme. First, the weights 32 , e.g., the weights 321 to 326 , of the connections 22 , e.g., the connections 221 to 226 , between neurons 14 , 20 and/or 18 , e.g., between predecessor neurons 141 to 143 and intermediate neurons 201 and 202 , are formed into tensors, which are shown as matrices 30 in the example (step 1 in FIG. 1 ). In step 1 of FIG. 1 , for example, the weights 32 associated with a first layer of a neural Network 10 , NN, are formed into the matrix 30 . According to the embodiment shown in FIG. 1 , the columns of the matrix 30 are associated with the predecessor neurons 141 to 143 and the rows of the matrix 30 are associated with the intermediate neurons 201 and 202 , but it is clear that the formed matrix can alternatively represent an inversion of the illustrated matrix 30 .

Then, each NN parameter, e.g., the weights 32 , is encoded, e.g., quantized and entropy coded, e.g. using context-adaptive arithmetic coding 600 , as shown in steps 2 and 3 , following a particular scanning order, e.g., row-major order (left to right, top to bottom). As will be outlined in more detail below, it is also possible to use a different scanning order, i.e. coding order. The steps 2 and 3 are performed by an encoder 40 , i.e. an apparatus for encoding. The decoder 50 , i.e. an apparatus for decoding, follows the same process in reverse processing order steps. That is, firstly it decodes the list of integer representation of the encoded values, as shown in step 4 , and then reshapes the list into its tensor representation 30 â², as shown in step 5 . Finally, the tensor 30 â² is loaded into the network architecture 10 â², i.e. a reconstructed NN, as shown in step 6 . The reconstructed tensor 30 â² comprises reconstructed NN parameter, i.e. decoded NN parameter 32 â².

The NN 10 shown in FIG. 1 is only a simple NN with few neurons 14 , 20 and 18 . A neuron might, in the following also be understood as node, element, model element or dimension. Furthermore, the reference sign 10 might indicate a machine learning (ML) predictor or, in other words, a machine learning model such as a neural network.

With reference to FIG. 2 a neural network is described in more detail. In particular, FIG. 2 shows an ML predictor 10 comprising an input interface 12 with input nodes or elements 14 and an output interface 16 with output nodes or elements 18 . The input nodes/elements 14 receive the input data. In other words, the input data is applied thereonto. For instance, they may receive a picture with, for instance, each element 14 being associated with a pixel of the picture. Alternatively, the input data applied onto elements 14 may be a signal such as a one dimensional signal such as an audio signal, a sensor signal or the like. Even alternatively, the input data may represent a certain data set such as medical file data or the like. The number of input elements 14 may be any number and depends on the type of input data, for instance. The number of output nodes 18 may be one, as shown in FIG. 1 , or larger than one, as shown in FIG. 2 . Each output node or element 18 may be associated with a certain inference or prediction task. In particular, upon the ML predictor 10 being applied onto a certain input applied onto the ML predictor's 10 input interface 12 , the ML predictor 10 outputs at the output interface 16 the inference or prediction result wherein the activation, i.e. an activation value, resulting at each output node 18 may be indicative, for instance, of an answer to a certain question on the input data such as whether or not, or how likely, the input data has a certain characteristic such as whether a picture having been input contains a certain object such as a car, a person, a phase or the like.

Insofar, the input applied onto the input interface may also be interpreted as an activation, namely an activation applied onto each input node or element 14 .

Between the input nodes 14 and output node(s) 18 , the ML predictor 10 comprises further elements or nodes 20 which are, via connections 22 connected to predecessor nodes so as to receive activations from these predecessor nodes, and via one or more further connections 24 to successor nodes in order to forward to the successor nodes the activation, i.e. an activation value, of node 20 .

Predecessor nodes may be other internal nodes 20 of the ML predictor 10 , via which intermediate node 20 exemplarily depicted in FIG. 2 is indirectly connected to input nodes 14 , or may be an input node 14 directly, as shown in FIG. 1 , and the successor nodes may be other intermediate nodes of the ML predictor 10 , via which the exemplarily shown intermediate node 20 is connected to the output interface or output node, or may be an output node 28 directly, as shown in FIG. 1 .

The input nodes 14 , output nodes 18 and internal nodes 20 of ML predictor 10 may be associated or attributed to certain layers of the ML predictor 10 , but a layered structuring of the ML predictor 10 is optional and ML predictors onto which embodiments of the present application apply are not restricted to such layered networks. As far as the exemplary shown intermediate node 20 of ML predictor 10 is concerned, same contributes to the inference or prediction task of ML predictor 10 by forwarding activations, i.e. activation values, from the predecessor nodes received via connections 22 from input interface 12 via connections 24 to successor nodes towards output interface 16 . In doing so, node or element 20 computes its activation, i.e. activation value, forwarded via connections 24 towards the successor nodes based on the activations, i.e. activation values, at the input nodes 22 and the computation involves the computation of a weighted sum namely a sum having an addend for each connection 22 which, in turn, is a product between the input received from a respective predecessor node, namely its activation, and a weight associated with the connection 22 connecting the respective predecessor node and intermediate node 20 . Note that alternatively or more generally, the activation x forwarded via connections 24 from a node or element i, 20 , towards the successor nodes j by way of a mapping function m ij (x). Thus, each connection 22 as well as 24 may have a certain weight associated therewith, or alternatively, the result of mapping function m ij . Further parameters may be involved in the computation in the activation output by node 20 towards a certain successor node, optionally. In order to determine relevance scores for portions of the ML predictor 10 , activations resulting at an output node 18 upon having finished a certain prediction or inference task on a certain input at the input interface 12 may be used, or a predefined or interesting output activation of interest. This activation at each output node 18 is used as starting point for the relevance score determination, and the relevance is back propagated towards the input interface 12 . In particular, at each node of ML predictor 10 , such as node 20 , the relevance score is distributed towards the predecessor nodes such as via connections 22 in case of node 20 , distributed in a manner proportional to the aforementioned products associated with each predecessor node and contributing, via the weighted summation, to the activation of the current node the activation of which is to be backward propagated such as node 20 . That is, the relevance fraction back propagated from a certain node such as node 20 to a certain predecessor node thereof may be computed by multiplying the relevance of that node with a factor depending on a ratio between the activation received from that predecessor node times the weight using which the activation has contributed to the aforementioned sum of the respective node, divided by a value depending on a sum of all products between the activations of the predecessor nodes and the weights at which these activations have contributed to the weighted sum of the current node the relevance of which is to be back propagated.

In the manner described above, relevance scores for portions of the ML predictor 10 , for example, are determined on the basis of an activation of these portions as manifesting itself in one or more inferences performed by the ML predictor. The âportionsâ for which such a relevance score is determined may, as discussed above, be nodes or elements of the predictor 10 wherein, again it should be noted that the ML predictor 10 is not restricted to any layered ML network so that, for instance, the element 20 , for instance, may be any computation of an intermediate value as computed during the inference or prediction performed by predictor 10 . For instance, in the manner discussed above, the relevance score for element or node 20 is computed by aggregating or summing up the inbound relevance messages this node or element 20 receives from its successor nodes/elements which, in turn, distribute their relevance scores in the manner outlined above representatively with respect to node 20 .

The ML predictor 10 , i.e. a NN, as described with regard to FIG. 2 might be encoded into a data stream 45 using an encoder 40 described with regard to FIG. 1 and might be reconstructed/decoded from the data stream 45 using a decoder 50 described with regard to FIG. <b

CLAIMS

Claims ( 25 )

I/We claim:

1 . Data stream having a representation of a neural network encoded thereinto, the data stream comprising a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference.

2 . Apparatus for encoding a representation of a neural network into a data stream, wherein the apparatus is configured to provide the data stream with a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference.

3 . Apparatus for decoding a representation of a neural network from a data stream, wherein the apparatus is configured to decode from the data stream a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference, and to use the numerical representation and bit size for representing the neural network parameters decoded from the data stream.

4 . Apparatus of claim 3 , wherein the data stream, is structured into individually accessible sub-portions, each individually accessible sub-portion representing a corresponding neural network portion of the neural network, so that each individually accessible sub-portion is completely traversed by the coding order before a subsequent individually accessible sub-portion is traversed by the coding order, wherein the apparatus is configured to decode, from the data stream, for a predetermined individually accessible sub-portion the neural network parameter and a type parameter indicting a parameter type of the neural network parameter decoded from the predetermined individually accessible sub-portion.

5 . Apparatus of claim 4 , wherein the type parameter discriminates, at least, between neural network weights and neural network biases.

6 . Apparatus of claim 3 , wherein the data stream, is structured into one or more individually accessible portions, each one or more individually accessible portion representing a corresponding neural network layer of the neural network, and

wherein the apparatus is configured to decode, from the data stream, for a predetermined neural network layer, a neural network layer type parameter indicating a neural network layer type of the predetermined neural network layer of the neural network.

7 . Apparatus of claim 6 , wherein the neural network layer type parameter discriminates, at least, between a fully-connected and a convolutional layer type.

8 . Apparatus of claim 3 , wherein the apparatus is configured to decode a representation of a neural network from the data stream, wherein the data stream is structured into one or more individually accessible portions, each individually accessible portion representing a corresponding neural network layer of the neural network, and wherein the data stream is, within a predetermined portion, further structured into individually accessible sub-portions, each sub-portion representing a corresponding neural network portion of the respective neural network layer of the neural network, wherein the apparatus is configured to decode from the data stream, for each of one or more predetermined individually accessible sub-portions

a start code at which the respective predetermined individually accessible sub-portion begins, and/or a pointer pointing to a beginning of the respective predetermined individually accessible sub-portion, and/or a data stream length parameter indicating a data stream length of the respective predetermined individually accessible sub-portion for skipping the respective predetermined individually accessible sub-portion in parsing the data stream.

9 . Apparatus of claim 8 , wherein the apparatus is configured to decode, from the data stream, the representation of the neural network using context-adaptive arithmetic decoding and using context initialization at a start of each individually accessible portion and each individually accessible sub-portion.

10 . Apparatus of claim 3 , wherein the apparatus is configured to decode a representation of a neural network from a data stream, wherein the data stream is structured into individually accessible portions, each portion representing a corresponding neural network portion of the neural network, wherein the apparatus is configured to decode from the data stream, for each of one or more predetermined individually accessible portions, an identification parameter for identifying the respective predetermined individually accessible portion.

11 . Apparatus of claim 10 , wherein the identification parameter is related to the respective predetermined individually accessible portion via a hash function or error detection code or error correction code.

12 . Apparatus of claim 10 , wherein the apparatus is configured to decode, from the data stream, a higher-level identification parameter for identifying a collection of more than one predetermined individually accessible portion.

13 . Apparatus of claim 12 , wherein the higher-level identification parameter is related to the identification parameters of the more than one predetermined individually accessible portion via a hash function or error detection code or error correction code.

14 . Apparatus of claim 3 , wherein the apparatus is configured to decode a representation of a neural network from a data stream, wherein the data stream is structured into individually accessible portions, each portion representing a corresponding neural network portion of the neural network, wherein the apparatus is configured to decode from the data stream, for each of one or more predetermined individually accessible portions a supplemental data for supplementing the representation of the neural network.

15 . Apparatus of claim 14 , wherein the data stream indicates the supplemental data as being dispensable for inference based on the neural network.

16 . Apparatus of claim 14 , wherein the apparatus is configured to decode the supplemental data for supplementing the representation of the neural network for the one or more predetermined individually accessible portions from further individually accessible portions, wherein the data stream comprises for each of the one or more predetermined individually accessible portions a corresponding further predetermined individually accessible portion relating to the neural network portion to which the respective predetermined individually accessible portion corresponds.

17 . Apparatus of claim 14 , wherein the supplemental data relates to

relevance scores of neural network parameters, and/or perturbation robustness of neural network parameters.

18 . Apparatus of claim 3 , for decoding a representation of a neural network from a data stream, wherein the apparatus is configured to decode from the data stream hierarchical control data structured into a sequence of control data portions, wherein the control data portions provide information on the neural network at increasing details along the sequence of control data portions.

19 . Apparatus of claim 18 , wherein at least some of the control data portions provide information on the neural network which is partially redundant.

20 . Apparatus of claim 18 , wherein a first control data portion provides the information on the neural network by way of indicating a default neural network type implying default settings and a second control data portion comprises a parameter to indicate each of the default settings.

21 . Apparatus for performing an inference using a neural network, comprising

an apparatus for decoding a data stream according to claim 3 , so as to derive from the data stream the neural network, and a processor configured to perform the inference based on the neural network.

22 . Method for encoding a representation of a neural network into a data stream, providing the data stream with a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference.

23 . Method for decoding a representation of a neural network from a data stream, wherein the method comprises decoding from the data stream a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference, and to use the numerical representation and bit size for representing the neural network parameters decoded from the data stream.

24 . A non-transitory digital storage medium having a computer program stored thereon to perform the method for encoding a representation of a neural network into a data stream, providing the data stream with a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference,

when said computer program is run by a computer.

25 . A non-transitory digital storage medium having a computer program stored thereon to perform the method Method for decoding a representation of a neural network from a data stream, wherein the method comprises decoding from the data stream a numerical computation representation parameter indicating a numerical representation and bit size at which neural network parameters of the neural network, which are encoded into the data stream, are to be represented when using the neural network for inference, and to use the numerical representation and bit size for representing the neural network parameters decoded from the data stream,

when said computer program is run by a computer.

US19/303,981

2019-10-01

2025-08-19

Neural Network Representation Formats

Pending

US20250384299A1

( en )

Priority Applications (1)

Application Number

Priority Date

Filing Date

Title

US19/303,981

US20250384299A1

( en )

2019-10-01

2025-08-19

Neural Network Representation Formats

Applications Claiming Priority (5)

Application Number

Priority Date

Filing Date

Title

EP19200928

2019-10-01

EP19200928.0

2019-10-01

PCT/EP2020/077352

WO2021064013A2

( en )

2019-10-01

2020-09-30

Neural network representation formats

US17/711,569

US20220222541A1

( en )

2019-10-01

2022-04-01

Neural Network Representation Formats

US19/303,981

US20250384299A1

( en )

2019-10-01

2025-08-19

Neural Network Representation Formats

Related Parent Applications (1)

Application Number

Title

Priority Date

Filing Date

US17/711,569

Continuation

US20220222541A1

( en )

2019-10-01

2022-04-01

Neural Network Representation Formats

Publications (1)

Publication Number

Publication Date

US20250384299A1

true

US20250384299A1 ( en )

2025-12-18

Family

ID=72709374

Family Applications (5)

Application Number

Title

Priority Date

Filing Date

US17/711,569

Pending

US20220222541A1

( en )

2019-10-01

2022-04-01

Neural Network Representation Formats

US19/303,931

Pending

US20250384298A1

( en )

2019-10-01

2025-08-19

Neural Network Representation Formats

US19/304,022

Pending

US20250384300A1

( en )

2019-10-01

2025-08-19

Neural Network Representation Formats

US19/303,889

Pending

US20250384297A1

( en )

2019-10-01

2025-08-19

Neural Network Representation Formats

US19/303,981

Pending

US20250384299A1

( en )

2019-10-01

2025-08-19

Neural Network Representation Formats

Family Applications Before (4)

Application Number

Title

Priority Date

Filing Date

US17/711,569

Pending

US20220222541A1

( en )

2019-10-01

2022-04-01

Neural Network Representation Formats

US19/303,931

Pending

US20250384298A1

( en )

2019-10-01

2025-08-19

Neural Network Representation Formats

US19/304,022

Pending

US20250384300A1

( en )

2019-10-01

2025-08-19

Neural Network Representation Formats

US19/303,889

Pending

US20250384297A1

( en )

2019-10-01

2025-08-19

Neural Network Representation Formats

Country Status (7)

Country

Link

( 5 )

US20220222541A1

( en )

( 1 )

EP4038551A2

( en )

( 3 )

JP2022551266A

( en )

( 1 )

KR20220075407A

( en )

( 1 )

CN114761970A

( en )

( 11 )

TW202601460A

( en )

( 1 )

WO2021064013A2

( en )

Cited By (1)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20240330513A1

( en )

2023-03-31

2024-10-03

Irdeto B.V.

System and method for creating secured neural networks

Families Citing this family (16)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

JP7640552B2

( en )

2019-12-20

2025-03-05

Concepts for encoding parameters of neural networks

CN115720666A

( en )

2020-06-25

2023-02-28

è±è¿ªè¡ä»½æéå¬å¸

Method and apparatus for compression and training of neural networks

JP2022007503A

( en )

2020-06-26

2022-01-13

å¯å£«éæ ªå¼ä¼ç¤¾

Receiving device and decoding method

JP7755662B2

( en )

2021-04-16

2025-10-16

Apparatus, method and computer program for decoding neural network parameters using an updated model, and apparatus, method and computer program for encoding neural network parameters

EP4324097A1

( en )

2021-04-16

2024-02-21

Fraunhofer-Gesellschaft zur FÃ¶rderung der angewandten Forschung e.V.

Decoder, encoder, controller, method and computer program for updating neural network parameters using node information

US11729080B2

( en )

2021-05-12

2023-08-15

Vmware, Inc.

Agentless method to automatically detect low latency groups in containerized infrastructures

US11728826B2

( en )

2021-05-24

2023-08-15

Google Llc

Compression and decompression in hardware for data processing

FR3124342B1

( en )

2021-06-17

2024-01-12

Fond B Com

Methods and devices for decoding at least part of a data stream, computer program and associated data stream

US12293275B2

( en )

2021-07-06

2025-05-06

Beijing Dajia Internet Information Technology Co., Ltd.

Methods and apparatuses for high performance and accuracy fixed-point scale implementation

US12481877B2

( en )

2021-08-25

2025-11-25

Qualcomm Incorporated

Instance-adaptive image and video compression in a network parameter subspace using machine learning systems

JP2023103544A

( en )

2022-01-14

2023-07-27

ã·ã£ã¼ãæ ªå¼ä¼ç¤¾

Video decoding device

CN114896876A

( en )

2022-04-27

2022-08-12

éç»ç§æ(ä¸æµ·)æéå¬å¸

Quantization error compensation method, system, electronic device, and storage medium

IL317622A

( en )

2022-07-05

2025-02-01

Panasonic Ip Corp America

Decoder, encoder, decoding method, and encoding method

AU2023248075A1

( en )

2023-10-10

2025-04-24

Canon Kabushiki Kaisha

Method, apparatus and system for encoding and decoding a tensor

AU2023248076B2

( en )

2023-10-10

2026-01-22

Canon Kabushiki Kaisha

Method, apparatus and system for encoding and decoding a tensor

WO2025204828A1

( en )

2024-03-29

2025-10-02

Encoding device, decoding device, encoding method, and decoding method

Citations (1)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20180082181A1

( en )

2016-05-13

2018-03-22

Samsung Electronics, Co. Ltd.

Neural Network Reordering, Weight Compression, and Processing

Family Cites Families (18)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US6879725B2

( en )

2001-01-26

2005-04-12

International Business Machine Corporation

Method, system, and program for decoding a section from compressed data

US6775624B2

( en )

2001-10-19

2004-08-10

International Business Machines Corporation

Method and apparatus for estimating remaining life of a product

US20030174218A1

( en )

2002-03-14

2003-09-18

Battles Amy E.

System for capturing audio segments in a digital camera

CN1152531C

( en )

2002-04-23

2004-06-02

åä¸ºææ¯æéå¬å¸

Network Access Control Method for Fragmented Messages

US20070294500A1

( en )

2006-06-16

2007-12-20

Falco Michael A

Methods and system to provide references associated with data streams

RS64604B1

( en )

2011-06-16

2023-10-31

Ge Video Compression Llc

Entropy coding of motion vector differences

WO2013141665A1

( en )

2012-03-22

2013-09-26

ìì§ì ì ì£¼ìíì¬

Video encoding method, video decoding method and apparatus using same

CN108141225B

( en )

2016-07-14

2020-10-27

åä¸ºææ¯æéå¬å¸

General purpose data compression using SIMD engines

US12190231B2

( en )

2016-10-19

2025-01-07

Samsung Electronics Co., Ltd

Method and apparatus for neural network quantization

KR102526818B1

( en )

2017-07-07

2023-04-27

ë¯¸ì°ë¹ìë´í¤ ê°ë¶ìí¤ê°ì´ì¤

Data processing device, data processing method and storage medium

US11182695B1

( en )

2017-08-18

2021-11-23

Groupon, Inc.

Method, apparatus, and computer program product for machine learning model lifecycle management

US10599975B2

( en )

2017-12-15

2020-03-24

Uber Technologies, Inc.

Scalable parameter encoding of artificial neural networks obtained via an evolutionary process

KR102683757B1

( en )

2018-02-20

2024-07-10

ì¼ì±ì ìì£¼ìíì¬

The method of performing deep neural network learning and apparatus thereof

US11468316B2

( en )

2018-03-13

2022-10-11

Recogni Inc.

Cluster compression for compressing weights in neural networks

CN108985448B

( en )

2018-06-06

2020-11-17

åäº¬å¤§å¦

Neural network representation standard framework structure

CN109034378B

( en )

2018-09-04

2023-03-31

è¾è®¯ç§æï¼æ·±å³ï¼æéå¬å¸

Network representation generation method and device of neural network, storage medium and equipment

US12169786B1

( en )

2018-11-28

2024-12-17

Amazon Technologies, Inc.

Neural network accelerator with reconfigurable memory

US11488016B2

( en )

2019-01-23

2022-11-01

Google Llc

Look-up table based neural networks

2020

2020-09-30

TW114121318A

patent/TW202601460A/en

unknown

2020-09-30

JP2022520429A

patent/JP2022551266A/en

active

Pending

2020-09-30

TW114121298A

patent/TW202601457A/en

unknown

2020-09-30

TW109134251A

patent/TW202134958A/en

unknown

2020-09-30

TW114121322A

patent/TW202601464A/en

unknown

2020-09-30

TW114121320A

patent/TW202601462A/en

unknown

2020-09-30

TW114121321A

patent/TW202601463A/en

unknown

2020-09-30

CN202080083494.8A

patent/CN114761970A/en

active

Pending

2020-09-30

EP20785494.4A

patent/EP4038551A2/en

not_active

Withdrawn

2020-09-30

TW112113584A

patent/TWI900843B/en

active

2020-09-30

TW114121299A

patent/TW202601447A/en

unknown

2020-09-30

TW114121319A

patent/TW202601461A/en

unknown

2020-09-30

TW114121311A

patent/TW202601459A/en

unknown

2020-09-30

KR1020227014848A

patent/KR20220075407A/en

not_active

Ceased

2020-09-30

TW114121310A

patent/TW202601458A/en

unknown

2020-09-30

PCT/EP2020/077352

patent/WO2021064013A2/en

not_active

Ceased

2022

2022-04-01

US17/711,569

patent/US20220222541A1/en

active

Pending

2023

2023-10-10

JP2023175417A

patent/JP7614288B2/en

active

Active

2024

2024-12-26

JP2024230189A

patent/JP7835841B2/en

active

Active

2025

2025-08-19

US19/303,931

patent/US20250384298A1/en

active

Pending

2025-08-19

US19/304,022

patent/US20250384300A1/en

active

Pending

2025-08-19

US19/303,889

patent/US20250384297A1/en

active

Pending

2025-08-19

US19/303,981

patent/US20250384299A1/en

active

Pending

Patent Citations (1)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20180082181A1

( en )

2016-05-13

2018-03-22

Samsung Electronics, Co. Ltd.

Neural Network Reordering, Weight Compression, and Processing

Non-Patent Citations (2)

* Cited by examiner, â Cited by third party

Title

Neural Network Compression using Transform Coding and Clustering, Laude et al, https://arxiv.org/pdf/1805.07258 (Year: 2018)

R. Song, D. Liu, H. Li and F. Wu, " Neural network-based arithmetic coding of intra prediction modes in HEVC, " 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA, 2017, (Year: 2017)

Cited By (1)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20240330513A1

( en )

2023-03-31

2024-10-03

Irdeto B.V.

System and method for creating secured neural networks

Also Published As

Publication number

Publication date

CN114761970A

( en )

2022-07-15

US20250384297A1

( en )

2025-12-18

JP2023179645A

( en )

2023-12-19

TW202601462A

( en )

2026-01-01

TW202601461A

( en )

2026-01-01

JP2025063087A

( en )

2025-04-15

US20250384298A1

( en )

2025-12-18

EP4038551A2

( en )

2022-08-10

US20220222541A1

( en )

2022-07-14

TW202331600A

( en )

2023-08-01

JP2022551266A

( en )

2022-12-08

TW202134958A

( en )

2021-09-16

WO2021064013A3

( en )

2021-06-17

TW202601460A

( en )

2026-01-01

TW202601464A

( en )

2026-01-01

TW202601447A

( en )

2026-01-01

KR20220075407A

( en )

2022-06-08

WO2021064013A2

( en )

2021-04-08

TW202601459A

( en )

2026-01-01

TW202601463A

( en )

2026-01-01

TW202601458A

( en )

2026-01-01

TW202601457A

( en )

2026-01-01

TWI900843B

( en )

2025-10-11

US20250384300A1

( en )

2025-12-18

JP7614288B2

( en )

2025-01-15

JP7835841B2

( en )

2026-03-25

Neural Network Representation Formats — Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. (US20250384299A1)

Related documents