ABSTRACT
Abstract
Improving machine learning models in an artificial intelligence infrastructure includes: storing, within one or more storage systems of an artificial intelligence infrastructure, information describing a dataset and one or more transformations applied to the dataset resulting in a transformed dataset; and storing, within the one or more storage systems, information describing only portions of previous versions of a machine learning model that differ from a current version of the machine learning model, wherein the previous versions used the transformed dataset as input during one or more prior executions by the artificial intelligence infrastructure.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
This is a continuation application for patent entitled to a filing date and claiming the benefit of earlier-filed U.S. patent application Ser. No. 16/515,698, filed Jul. 18, 2019, herein incorporated by reference in its entirety, which claims priority to U.S. Pat. No. 10,360,214, issued Jul. 23, 2019, which claims priority from: U.S. Provisional Patent Application No. 62/574,534, filed Oct. 19, 2017, U.S. Provisional Patent Application No. 62/576,523, filed Oct. 24, 2017, U.S. Provisional Patent Application No. 62/579,057, filed Oct. 30, 2017, U.S. Provisional Patent Application No. 62/620,286, filed Jan. 22, 2018, U.S. Provisional Patent Application No. 62/648,368, filed Mar. 26, 2018, and U.S. Provisional Patent Application No. 62/650,736, filed Mar. 30, 2018.
BRIEF DESCRIPTION OF DRAWINGS
FIG. 1 A illustrates a first example system for data storage in accordance with some implementations.
FIG. 1 B illustrates a second example system for data storage in accordance with some implementations.
FIG. 1 C illustrates a third example system for data storage in accordance with some implementations.
FIG. 1 D illustrates a fourth example system for data storage in accordance with some implementations.
FIG. 2 A is a perspective view of a storage cluster with multiple storage nodes and internal storage coupled to each storage node to provide network attached storage, in accordance with some embodiments.
FIG. 2 B is a block diagram showing an interconnect switch coupling multiple storage nodes in accordance with some embodiments.
FIG. 2 C is a multiple level block diagram, showing contents of a storage node and contents of one of the non-volatile solid state storage units in accordance with some embodiments.
FIG. 2 D shows a storage server environment, which uses embodiments of the storage nodes and storage units of some previous figures in accordance with some embodiments.
FIG. 2 E is a blade hardware block diagram, showing a control plane, compute and storage planes, and authorities interacting with underlying physical resources, in accordance with some embodiments.
FIG. 2 F depicts elasticity software layers in blades of a storage cluster, in accordance with some embodiments.
FIG. 2 G depicts authorities and storage resources in blades of a storage cluster, in accordance with some embodiments.
FIG. 3 A sets forth a diagram of a storage system that is coupled for data communications with a cloud services provider in accordance with some embodiments of the present disclosure.
FIG. 3 B sets forth a diagram of a storage system in accordance with some embodiments of the present disclosure.
FIG. 4 sets forth a flow chart illustrating an example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 5 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 6 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 7 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 8 A sets forth a diagram illustrating an example computer architecture for implementing an artificial intelligence and machine learning infrastructure that is configured to fit within a single chassis according to some embodiments of the present disclosure.
FIG. 8 B sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 9 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 10 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 11 A sets forth a diagram illustrating an example artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 11 B sets forth a diagram illustrating an example computer architecture for implementing an artificial intelligence and machine learning infrastructure within a single chassis according to some embodiments of the present disclosure.
FIG. 11 C sets forth a diagram illustrating an example implementation of an artificial intelligence and machine learning infrastructure software stack according to some embodiments of the present disclosure.
FIG. 11 D sets forth a flow chart illustrating an example method for interconnecting a graphical processing unit layer and a storage layer of an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 12 A sets forth a flow chart illustrating an example method of monitoring an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 12 B sets forth a flow chart illustrating an example method of optimizing an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 13 sets forth a flow chart illustrating an example method of storage system query processing within an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 14 sets a forth flow chart illustrating an example method of storage system query processing within an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 15 sets a forth flow chart illustrating an example method of storage system query processing within an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 16 sets forth a flow chart illustrating an example method of data transformation offloading in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 17 sets forth a flow chart illustrating an additional example method of data transformation offloading in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 18 sets forth a flow chart illustrating an additional example method of data transformation offloading in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 19 sets forth a flow chart illustrating an additional example method of data transformation offloading in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 20 sets forth a flow chart illustrating an example method of data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 21 sets forth a flow chart illustrating an additional example method of data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 22 sets forth a flow chart illustrating an additional example method of data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 23 sets forth a flow chart illustrating an additional example method of data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 24 sets forth a flow chart illustrating an example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
FIG. 25 sets forth a flow chart illustrating an additional example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
FIG. 26 sets forth a flow chart illustrating an additional example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
FIG. 27 sets forth a flow chart illustrating an example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
FIG. 28 sets forth a flow chart illustrating an example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
FIG. 29 sets forth a flow chart illustrating an example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
DESCRIPTION OF EMBODIMENTS
Example methods, apparatus, and products for ensuring reproducibility in an artificial intelligence infrastructure in accordance with embodiments of the present disclosure are described with reference to the accompanying drawings, beginning with FIG. 1 A . FIG. 1 A illustrates an example system for data storage, in accordance with some implementations. System 100 (also referred to as âstorage systemâ herein) includes numerous elements for purposes of illustration rather than limitation. It may be noted that system 100 may include the same, more, or fewer elements configured in the same or different manner in other implementations.
System 100 includes a number of computing devices 164 A-B. Computing devices (also referred to as âclient devicesâ herein) may be embodied, for example, a server in a data center, a workstation, a personal computer, a notebook, or the like. Computing devices 164 A-B may be coupled for data communications to one or more storage arrays 102 A-B through a storage area network (âSANâ) 158 or a local area network (âLANâ) 160 .
The SAN 158 may be implemented with a variety of data communications fabrics, devices, and protocols. For example, the fabrics for SAN 158 may include Fibre Channel, Ethernet, Infiniband, Serial Attached Small Computer System Interface (âSASâ), or the like. Data communications protocols for use with SAN 158 may include Advanced Technology Attachment (âATAâ), Fibre Channel Protocol, Small Computer System Interface (âSCSIâ), Internet Small Computer System Interface (âiSCSIâ), HyperSCSI, Non-Volatile Memory Express (âNVMeâ) over Fabrics, or the like. It may be noted that SAN 158 is provided for illustration, rather than limitation. Other data communication couplings may be implemented between computing devices 164 A-B and storage arrays 102 A-B.
The LAN 160 may also be implemented with a variety of fabrics, devices, and protocols. For example, the fabrics for LAN 160 may include Ethernet (802.3), wireless (802.11), or the like. Data communication protocols for use in LAN 160 may include Transmission Control Protocol (âTCPâ), User Datagram Protocol (âUDPâ), Internet Protocol (âIPâ), HyperText Transfer Protocol (âHTTPâ), Wireless Access Protocol (âWAPâ), Handheld Device Transport Protocol (âHDTPâ), Session Initiation Protocol (âSIPâ), Real Time Protocol (âRTPâ), or the like.
Storage arrays 102 A-B may provide persistent data storage for the computing devices 164 A-B. Storage array 102 A may be contained in a chassis (not shown), and storage array 102 B may be contained in another chassis (not shown), in implementations. Storage array 102 A and 102 B may include one or more storage array controllers 110 A-D (also referred to as âcontrollerâ herein). A storage array controller 110 A-D may be embodied as a module of automated computing machinery comprising computer hardware, computer software, or a combination of computer hardware and software. In some implementations, the storage array controllers 110 A-D may be configured to carry out various storage tasks. Storage tasks may include writing d
CROSS-REFERENCE TO RELATED APPLICATIONS
This is a continuation application for patent entitled to a filing date and claiming the benefit of earlier-filed U.S. patent application Ser. No. 16/515,698, filed Jul. 18, 2019, herein incorporated by reference in its entirety, which claims priority to U.S. Pat. No. 10,360,214, issued Jul. 23, 2019, which claims priority from: U.S. Provisional Patent Application No. 62/574,534, filed Oct. 19, 2017, U.S. Provisional Patent Application No. 62/576,523, filed Oct. 24, 2017, U.S. Provisional Patent Application No. 62/579,057, filed Oct. 30, 2017, U.S. Provisional Patent Application No. 62/620,286, filed Jan. 22, 2018, U.S. Provisional Patent Application No. 62/648,368, filed Mar. 26, 2018, and U.S. Provisional Patent Application No. 62/650,736, filed Mar. 30, 2018.
BRIEF DESCRIPTION OF DRAWINGS
FIG. 1 A illustrates a first example system for data storage in accordance with some implementations.
FIG. 1 B illustrates a second example system for data storage in accordance with some implementations.
FIG. 1 C illustrates a third example system for data storage in accordance with some implementations.
FIG. 1 D illustrates a fourth example system for data storage in accordance with some implementations.
FIG. 2 A is a perspective view of a storage cluster with multiple storage nodes and internal storage coupled to each storage node to provide network attached storage, in accordance with some embodiments.
FIG. 2 B is a block diagram showing an interconnect switch coupling multiple storage nodes in accordance with some embodiments.
FIG. 2 C is a multiple level block diagram, showing contents of a storage node and contents of one of the non-volatile solid state storage units in accordance with some embodiments.
FIG. 2 D shows a storage server environment, which uses embodiments of the storage nodes and storage units of some previous figures in accordance with some embodiments.
FIG. 2 E is a blade hardware block diagram, showing a control plane, compute and storage planes, and authorities interacting with underlying physical resources, in accordance with some embodiments.
FIG. 2 F depicts elasticity software layers in blades of a storage cluster, in accordance with some embodiments.
FIG. 2 G depicts authorities and storage resources in blades of a storage cluster, in accordance with some embodiments.
FIG. 3 A sets forth a diagram of a storage system that is coupled for data communications with a cloud services provider in accordance with some embodiments of the present disclosure.
FIG. 3 B sets forth a diagram of a storage system in accordance with some embodiments of the present disclosure.
FIG. 4 sets forth a flow chart illustrating an example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 5 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 6 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 7 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 8 A sets forth a diagram illustrating an example computer architecture for implementing an artificial intelligence and machine learning infrastructure that is configured to fit within a single chassis according to some embodiments of the present disclosure.
FIG. 8 B sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 9 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 10 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure.
FIG. 11 A sets forth a diagram illustrating an example artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 11 B sets forth a diagram illustrating an example computer architecture for implementing an artificial intelligence and machine learning infrastructure within a single chassis according to some embodiments of the present disclosure.
FIG. 11 C sets forth a diagram illustrating an example implementation of an artificial intelligence and machine learning infrastructure software stack according to some embodiments of the present disclosure.
FIG. 11 D sets forth a flow chart illustrating an example method for interconnecting a graphical processing unit layer and a storage layer of an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 12 A sets forth a flow chart illustrating an example method of monitoring an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 12 B sets forth a flow chart illustrating an example method of optimizing an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 13 sets forth a flow chart illustrating an example method of storage system query processing within an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 14 sets a forth flow chart illustrating an example method of storage system query processing within an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 15 sets a forth flow chart illustrating an example method of storage system query processing within an artificial intelligence and machine learning infrastructure according to some embodiments of the present disclosure.
FIG. 16 sets forth a flow chart illustrating an example method of data transformation offloading in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 17 sets forth a flow chart illustrating an additional example method of data transformation offloading in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 18 sets forth a flow chart illustrating an additional example method of data transformation offloading in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 19 sets forth a flow chart illustrating an additional example method of data transformation offloading in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 20 sets forth a flow chart illustrating an example method of data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 21 sets forth a flow chart illustrating an additional example method of data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 22 sets forth a flow chart illustrating an additional example method of data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 23 sets forth a flow chart illustrating an additional example method of data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more GPU servers according to some embodiments of the present disclosure.
FIG. 24 sets forth a flow chart illustrating an example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
FIG. 25 sets forth a flow chart illustrating an additional example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
FIG. 26 sets forth a flow chart illustrating an additional example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
FIG. 27 sets forth a flow chart illustrating an example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
FIG. 28 sets forth a flow chart illustrating an example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
FIG. 29 sets forth a flow chart illustrating an example method of ensuring reproducibility in an artificial intelligence infrastructure according to some embodiments of the present disclosure.
DESCRIPTION OF EMBODIMENTS
Example methods, apparatus, and products for ensuring reproducibility in an artificial intelligence infrastructure in accordance with embodiments of the present disclosure are described with reference to the accompanying drawings, beginning with FIG. 1 A . FIG. 1 A illustrates an example system for data storage, in accordance with some implementations. System 100 (also referred to as âstorage systemâ herein) includes numerous elements for purposes of illustration rather than limitation. It may be noted that system 100 may include the same, more, or fewer elements configured in the same or different manner in other implementations.
System 100 includes a number of computing devices 164 A-B. Computing devices (also referred to as âclient devicesâ herein) may be embodied, for example, a server in a data center, a workstation, a personal computer, a notebook, or the like. Computing devices 164 A-B may be coupled for data communications to one or more storage arrays 102 A-B through a storage area network (âSANâ) 158 or a local area network (âLANâ) 160 .
The SAN 158 may be implemented with a variety of data communications fabrics, devices, and protocols. For example, the fabrics for SAN 158 may include Fibre Channel, Ethernet, Infiniband, Serial Attached Small Computer System Interface (âSASâ), or the like. Data communications protocols for use with SAN 158 may include Advanced Technology Attachment (âATAâ), Fibre Channel Protocol, Small Computer System Interface (âSCSIâ), Internet Small Computer System Interface (âiSCSIâ), HyperSCSI, Non-Volatile Memory Express (âNVMeâ) over Fabrics, or the like. It may be noted that SAN 158 is provided for illustration, rather than limitation. Other data communication couplings may be implemented between computing devices 164 A-B and storage arrays 102 A-B.
The LAN 160 may also be implemented with a variety of fabrics, devices, and protocols. For example, the fabrics for LAN 160 may include Ethernet (802.3), wireless (802.11), or the like. Data communication protocols for use in LAN 160 may include Transmission Control Protocol (âTCPâ), User Datagram Protocol (âUDPâ), Internet Protocol (âIPâ), HyperText Transfer Protocol (âHTTPâ), Wireless Access Protocol (âWAPâ), Handheld Device Transport Protocol (âHDTPâ), Session Initiation Protocol (âSIPâ), Real Time Protocol (âRTPâ), or the like.
Storage arrays 102 A-B may provide persistent data storage for the computing devices 164 A-B. Storage array 102 A may be contained in a chassis (not shown), and storage array 102 B may be contained in another chassis (not shown), in implementations. Storage array 102 A and 102 B may include one or more storage array controllers 110 A-D (also referred to as âcontrollerâ herein). A storage array controller 110 A-D may be embodied as a module of automated computing machinery comprising computer hardware, computer software, or a combination of computer hardware and software. In some implementations, the storage array controllers 110 A-D may be configured to carry out various storage tasks. Storage tasks may include writing data received from the computing devices 164 A-B to storage array 102 A-B, erasing data from storage array 102 A-B, retrieving data from storage array 102 A-B and providing data to computing devices 164 A-B, monitoring and reporting of disk utilization and performance, performing redundancy operations, such as Redundant Array of Independent Drives (âRAIDâ) or RAID-like data redundancy operations, compressing data, encrypting data, and so forth.
Storage array controller 110 A-D may be implemented in a variety of ways, including as a Field Programmable Gate Array (âFPGAâ), a Programmable Logic Chip (âPLCâ), an Application Specific Integrated Circuit (âASICâ), System-on-Chip (âSOCâ), or any computing device that includes discrete components such as a processing device, central processing unit, computer memory, or various adapters. Storage array controller 110 A-D may include, for example, a data communications adapter configured to support communications via the SAN 158 or LAN 160 . In some implementations, storage array controller 110 A-D may be independently coupled to the LAN 160 . In implementations, storage array controller 110 A-D may include an I/O controller or the like that couples the storage array controller 110 A-D for data communications, through a midplane (not shown), to a persistent storage resource 170 A-B (also referred to as a âstorage resourceâ herein). The persistent storage resource 170 A-B main include any number of storage drives 171 A-F (also referred to as âstorage devicesâ herein) and any number of non-volatile Random Access Memory (âNVRAMâ) devices (not shown).
In some implementations, the NVRAM devices of a persistent storage resource 170 A-B may be configured to receive, from the storage array controller 110 A-D, data to be stored in the storage drives 171 A-F. In some examples, the data may originate from computing devices 164 A-B. In some examples, writing data to the NVRAM device may be carried out more quickly than directly writing data to the storage drive 171 A-F. In implementations, the storage array controller 110 A-D may be configured to utilize the NVRAM devices as a quickly accessible buffer for data destined to be written to the storage drives 171 A-F. Latency for write requests using NVRAM devices as a buffer may be improved relative to a system in which a storage array controller 110 A-D writes data directly to the storage drives 171 A-F. In some implementations, the NVRAM devices may be implemented with computer memory in the form of high bandwidth, low latency RAM. The NVRAM device is referred to as ânon-volatileâ because the NVRAM device may receive or include a unique power source that maintains the state of the RAM after main power loss to the NVRAM device. Such a power source may be a battery, one or more capacitors, or the like. In response to a power loss, the NVRAM device may be configured to write the contents of the RAM to a persistent storage, such as the storage drives 171 A-F.
In implementations, storage drive 171 A-F may refer to any device configured to record data persistently, where âpersistentlyâ or âpersistentâ refers as to a device's ability to maintain recorded data after loss of power. In some implementations, storage drive 171 A-F may correspond to non-disk storage media. For example, the storage drive 171 A-F may be one or more solid-state drives (âSSDsâ), flash memory based storage, any type of solid-state non-volatile memory, or any other type of non-mechanical storage device. In other implementations, storage drive 171 A-F may include mechanical or spinning hard disk, such as hard-disk drives (âHDDâ).
In some implementations, the storage array controllers 110 A-D may be configured for offloading device management responsibilities from storage drive 171 A-F in storage array 102 A-B. For example, storage array controllers 110 A-D may manage control information that may describe the state of one or more memory blocks in the storage drives 171 A-F. The control information may indicate, for example, that a particular memory block has failed and should no longer be written to, that a particular memory block contains boot code for a storage array controller 110 A-D, the number of program-erase (âP/Eâ) cycles that have been performed on a particular memory block, the age of data stored in a particular memory block, the type of data that is stored in a particular memory block, and so forth. In some implementations, the control information may be stored with an associated memory block as metadata. In other implementations, the control information for the storage drives 171 A-F may be stored in one or more particular memory blocks of the storage drives 171 A-F that are selected by the storage array controller 110 A-D. The selected memory blocks may be tagged with an identifier indicating that the selected memory block contains control information. The identifier may be utilized by the storage array controllers 110 A-D in conjunction with storage drives 171 A-F to quickly identify the memory blocks that contain control information. For example, the storage controllers 110 A-D may issue a command to locate memory blocks that contain control information. It may be noted that control information may be so large that parts of the control information may be stored in multiple locations, that the control information may be stored in multiple locations for purposes of redundancy, for example, or that the control information may otherwise be distributed across multiple memory blocks in the storage drive 171 A-F.
In implementations, storage array controllers 110 A-D may offload device management responsibilities from storage drives 171 A-F of storage array 102 A-B by retrieving, from the storage drives 171 A-F, control information describing the state of one or more memory blocks in the storage drives 171 A-F. Retrieving the control information from the storage drives 171 A-F may be carried out, for example, by the storage array controller 110 A-D querying the storage drives 171 A-F for the location of control information for a particular storage drive 171 A-F. The storage drives 171 A-F may be configured to execute instructions that enable the storage drive 171 A-F to identify the location of the control information. The instructions may be executed by a controller (not shown) associated with or otherwise located on the storage drive 171 A-F and may cause the storage drive 171 A-F to scan a portion of each memory block to identify the memory blocks that store control information for the storage drives 171 A-F. The storage drives 171 A-F may respond by sending a response message to the storage array controller 110 A-D that includes the location of control information for the storage drive 171 A-F. Responsive to receiving the response message, storage array controllers 110 A-D may issue a request to read data stored at the address associated with the location of control information for the storage drives 171 A-F.
In other implementations, the storage array controllers 110 A-D may further offload device management responsibilities from storage drives 171 A-F by performing, in response to receiving the control information, a storage drive management operation. A storage drive management operation may include, for example, an operation that is typically performed by the storage drive 171 A-F (e.g., the controller (not shown) associated with a particular storage drive 171 A-F). A storage drive management operation may include, for example, ensuring that data is not written to failed memory blocks within the storage drive 171 A-F, ensuring that data is written to memory blocks within the storage drive 171 A-F in such a way that adequate wear leveling is achieved, and so forth.
In implementations, storage array 102 A-B may implement two or more storage array controllers 110 A-D. For example, storage array 102 A may include storage array controllers 110 A and storage array controllers 110 B. At a given instance, a single storage array controller 110 A-D (e.g., storage array controller 110 A) of a storage system 100 may be designated with primary status (also referred to as âprimary controllerâ herein), and other storage array controllers 110 A-D (e.g., storage array controller 110 A) may be designated with secondary status (also referred to as âsecondary controllerâ herein). The primary controller may have particular rights, such as permission to alter data in persistent storage resource 170 A-B (e.g., writing data to persistent storage resource 170 A-B). At least some of the rights of the primary controller may supersede the rights of the secondary controller. For instance, the secondary controller may not have permission to alter data in persistent storage resource 170 A-B when the primary controller has the right. The status of storage array controllers 110 A-D may change. For example, storage array controller 110 A may be designated with secondary status, and storage array controller 110 B may be designated with primary status.
In some implementations, a primary controller, such as storage array controller 110 A, may serve as the primary controller for one or more storage arrays 102 A-B, and a second controller, such as storage array controller 110 B, may serve as the secondary controller for the one or more storage arrays 102 A-B. For example, storage array controller 110 A may be the primary controller for storage array 102 A and storage array 102 B, and storage array controller 110 B may be the secondary controller for storage array 102 A and 102 B. In some implementations, storage array controllers 110 C and 110 D (also referred to as âstorage processing modulesâ) may neither have primary or secondary status. Storage array controllers 110 C and 110 D, implemented as storage processing modules, may act as a communication interface between the primary and secondary controllers (e.g., storage array controllers 110 A and 110 B, respectively) and storage array 102 B. For example, storage array controller 110 A of storage array 102 A may send a write request, via SAN 158 , to storage array 102 B. The write request may be received by both storage array controllers 110 C and 110 D of storage array 102 B. Storage array controllers 110 C and 110 D facilitate the communication, e.g., send the write request to the appropriate storage drive 171 A-F. It may be noted that in some implementations storage processing modules may be used to increase the number of storage drives controlled by the primary and secondary controllers.
In implementations, storage array controllers 110 A-D are communicatively coupled, via a midplane (not shown), to one or more storage drives 171 A-F and to one or more NVRAM devices (not shown) that are included as part of a storage array 102 A-B. The storage array controllers 110 A-D may be coupled to the midplane via one or more data communication links and the midplane may be coupled to the storage drives 171 A-F and the NVRAM devices via one or more data communications links. The data communications links described herein are collectively illustrated by data communications links 108 A-D and may include a Peripheral Component Interconnect Express (âPCIeâ) bus, for example.
FIG. 1 B illustrates an example system for data storage, in accordance with some implementations. Storage array controller 101 illustrated in FIG. 1 B may be similar to the storage array controllers 110 A-D described with respect to FIG. 1 A . In one example, storage array controller 101 may be similar to storage array controller 110 A or storage array controller 110 B. Storage array controller 101 includes numerous elements for purposes of illustration rather than limitation. It may be noted that storage array controller 101 may include the same, more, or fewer elements configured in the same or different manner in other implementations. It may be noted that elements of FIG. 1 A may be included below to help illustrate features of storage array controller 101 .
Storage array controller 101 may include one or more processing devices 104 and random access memory (âRAMâ) 111 . Processing device 104 (or controller 101 ) represents one or more general-purpose processing devices such as a microprocessor, central processing unit, or the like. More particularly, the processing device 104 (or controller 101 ) may be a complex instruction set computing (âCISCâ) microprocessor, reduced instruction set computing (âRISCâ) microprocessor, very long instruction word (âVLIWâ) microprocessor, or a processor implementing other instruction sets or processors implementing a combination of instruction sets. The processing device 104 (or controller 101 ) may also be one or more special-purpose processing devices such as an application specific integrated circuit (âASICâ), a field programmable gate array (âFPGAâ), a digital signal processor (âDSPâ), network processor, or the like.
The processing device 104 may be connected to the RAM 111 via a data communications link 106 , which may be embodied as a high speed memory bus such as a Double-Data Rate 4 (âDDR4â) bus. Stored in RAM 111 is an operating system 112 . In some implementations, instructions 113 are stored in RAM 111 . Instructions 113 may include computer program instructions for performing operations in in a direct-mapped flash storage system. In one embodiment, a direct-mapped flash storage system is one that that addresses data blocks within flash drives directly and without an address translation performed by the storage controllers of the flash drives.
In implementations, storage array controller 101 includes one or more host bus adapters 103 A-C that are coupled to the processing device 104 via a data communications link 105 A-C. In implementations, host bus adapters 103 A-C may be computer hardware that connects a host system (e.g., the storage array controller) to other network and storage arrays. In some examples, host bus adapters 103 A-C may be a Fibre Channel adapter that enables the storage array controller 101 to connect to a SAN, an Ethernet adapter that enables the storage array controller 101 to connect to a LAN, or the like. Host bus adapters 103 A-C may be coupled to the processing device 104 via a data communications link 105 A-C such as, for example, a PCIe bus.
In implementations, storage array controller 101 may include a host bus adapter 114 that is coupled to an expander 115 . The expander 115 may be used to attach a host system to a larger number of storage drives. The expander 115 may, for example, be a SAS expander utilized to enable the host bus adapter 114 to attach to storage drives in an implementation where the host bus adapter 114 is embodied as a SAS controller.
In implementations, storage array controller 101 may include a switch 116 coupled to the processing device 104 via a data communications link 109 . The switch 116 may be a computer hardware device that can create multiple endpoints out of a single endpoint, thereby enabling multiple devices to share a single endpoint. The switch 116 may, for example, be a PCIe switch that is coupled to a PCIe bus (e.g., data communications link 109 ) and presents multiple PCIe connection points to the midplane.
In implementations, storage array controller 101 includes a data communications link 107 for coupling the storage array controller 101 to other storage array controllers. In some examples, data communications link 107 may be a QuickPath Interconnect (QPI) interconnect.
A traditional storage system that uses traditional flash drives may implement a process across the flash drives that are part of the traditional storage system. For example, a higher level process of the storage system may initiate and control a process across the flash drives. However, a flash drive of the traditional storage system may include its own storage controller that also performs the process. Thus, for the traditional storage system, a higher level process (e.g., initiated by the storage system) and a lower level process (e.g., initiated by a storage controller of the storage system) may both be performed.
To resolve various deficiencies of a traditional storage system, operations may be performed by higher level processes and not by the lower level processes. For example, the flash storage system may include flash drives that do not include storage controllers that provide the process. Thus, the operating system of the flash storage system itself may initiate and control the process. This may be accomplished by a direct-mapped flash storage system that addresses data blocks within the flash drives directly and without an address translation performed by the storage controllers of the flash drives.
The operating system of the flash storage system may identify and maintain a list of allocation units across multiple flash drives of the flash storage system. The allocation units may be entire erase blocks or multiple erase blocks. The operating system may maintain a map or address range that directly maps addresses to erase blocks of the flash drives of the flash storage system.
Direct mapping to the erase blocks of the flash drives may be used to rewrite data and erase data. For example, the operations may be performed on one or more allocation units that include a first data and a second data where the first data is to be retained and the second data is no longer being used by the flash storage system. The operating system may initiate the process to write the first data to new locations within other allocation units and erasing the second data and marking the allocation units as being available for use for subsequent data. Thus, the process may only be performed by the higher level operating system of the flash storage system without an additional lower level process being performed by controllers of the flash drives.
Advantages of the process being performed only by the operating system of the flash storage system include increased reliability of the flash drives of the flash storage system as unnecessary or redundant write operations are not being performed during the process. One possible point of novelty here is the concept of initiating and controlling the process at the operating system of the flash storage system. In addition, the process can be controlled by the operating system across multiple flash drives. This is contrast to the process being performed by a storage controller of a flash drive.
A storage system can consist of two storage array controllers that share a set of drives for failover purposes, or it could consist of a single storage array controller that provides a storage service that utilizes multiple drives, or it could consist of a distributed network of storage array controllers each with some number of drives or some amount of Flash storage where the storage array controllers in the network collaborate to provide a complete storage service and collaborate on various aspects of a storage service including storage allocation and garbage collection.
FIG. 1 C illustrates a third example system 117 for data storage in accordance with some implementations. System 117 (also referred to as âstorage systemâ herein) includes numerous elements for purposes of illustration rather than limitation. It may be noted that system 117 may include the same, more, or fewer elements configured in the same or different manner in other implementations.
In one embodiment, system 117 includes a dual Peripheral Component Interconnect (PCP) flash storage device 118 with separately addressable fast write storage. System 117 may include a storage controller 119 . In one embodiment, storage controller 119 A-D may be a CPU, ASIC, FPGA, or any other circuitry that may implement control structures necessary according to the present disclosure. In one embodiment, system 117 includes flash memory devices (e.g., including flash memory devices 120 a - n ), operatively coupled to various channels of the storage device controller 119 . Flash memory devices 120 a - n , may be presented to the controller 119 A-D as an addressable collection of Flash pages, erase blocks, and/or control elements sufficient to allow the storage device controller 119 A-D to program and retrieve various aspects of the Flash. In one embodiment, storage device controller 119 A-D may perform operations on flash memory devices 120 A-N including storing and retrieving data content of pages, arranging and erasing any blocks, tracking statistics related to the use and reuse of Flash memory pages, erase blocks, and cells, tracking and predicting error codes and faults within the Flash memory, controlling voltage levels associated with programming and retrieving contents of Flash cells, etc.
In one embodiment, system 117 may include RAM 121 to store separately addressable fast-write data. In one embodiment, RAM 121 may be one or more separate discrete devices. In another embodiment, RAM 121 may be integrated into storage device controller 119 A-D or multiple storage device controllers. The RAM 121 may be utilized for other purposes as well, such as temporary program memory for a processing device (e.g., a CPU) in the storage device controller 119 .
In one embodiment, system 119 A-D may include a stored energy device 122 , such as a rechargeable battery or a capacitor. Stored energy device 122 may store energy sufficient to power the storage device controller 119 , some amount of the RAM (e.g., RAM 121 ), and some amount of Flash memory (e.g., Flash memory 120 a - 120 n ) for sufficient time to write the contents of RAM to Flash memory. In one embodiment, storage device controller 119 A-D may write the contents of RAM to Flash Memory if the storage device controller detects loss of external power.
In one embodiment, system 117 includes two data communications links 123 a , 123 b . In one embodiment, data communications links 123 a , 123 b may be PCI interfaces. In another embodiment, data communications links 123 a , 123 b may be based on other communications standards (e.g., HyperTransport, InfiniBand, etc.). Data communications links 123 a , 123 b may be based on non-volatile memory express (âNVMeâ) or NVMe over fabrics (âNVMfâ) specifications that allow external connection to the storage device controller 119 A-D from other components in the storage system 117 . It should be noted that data communications links may be interchangeably referred to herein as PCI buses for convenience.
System 117 may also include an external power source (not shown), which may be provided over one or both data communications links 123 a , 123 b , or which may be provided separately. An alternative embodiment includes a separate Flash memory (not shown) dedicated for use in storing the content of RAM 121 . The storage device controller 119 A-D may present a logical device over a PCI bus which may include an addressable fast-write logical device, or a distinct part of the logical address space of the storage device 118 , which may be presented as PCI memory or as persistent storage. In one embodiment, operations to store into the device are directed into the RAM 121 . On power failure, the storage device controller 119 A-D may write stored content associated with the addressable fast-write logical storage to Flash memory (e.g., Flash memory 120 a - n ) for long-term persistent storage.
In one embodiment, the logical device may include some presentation of some or all of the content of the Flash memory devices 120 a - n , where that presentation allows a storage system including a storage device 118 (e.g., storage system 117 ) to directly address Flash memory pages and directly reprogram erase blocks from storage system components that are external to the storage device through the PCI bus. The presentation may also allow one or more of the external components to control and retrieve other aspects of the Flash memory including some or all of: tracking statistics related to use and reuse of Flash memory pages, erase blocks, and cells across all the Flash memory devices; tracking and predicting error codes and faults within and across the Flash memory devices; controlling voltage levels associated with programming and retrieving contents of Flash cells; etc.
In one embodiment, the stored energy device 122 may be sufficient to ensure completion of in-progress operations to the Flash memory devices 107 a - 120 n stored energy device 122 may power storage device controller 119 A-D and associated Flash memory devices (e.g., 120 a - n ) for those operations, as well as for the storing of fast-write RAM to Flash memory. Stored energy device 122 may be used to store accumulated statistics and other parameters kept and tracked by the Flash memory devices 120 a - n and/or the storage device controller 119 . Separate capacitors or stored energy devices (such as smaller capacitors near or embedded within the Flash memory devices themselves) may be used for some or all of the operations described herein.
Various schemes may be used to track and optimize the life span of the stored energy component, such as adjusting voltage levels over time, partially discharging the storage energy device 122 to measure corresponding discharge characteristics, etc. If the available energy decreases over time, the effective available capacity of the addressable fast-write storage may be decreased to ensure that it can be written safely based on the currently available stored energy.
FIG. 1 D illustrates a third example system 124 for data storage in accordance with some implementations. In one embodiment, system 124 includes storage controllers 125 a , 125 b . In one embodiment, storage controllers 125 a , 125 b are operatively coupled to Dual PCI storage devices 119 a , 119 b and 119 c , 119 d , respectively. Storage controllers 125 a , 125 b may be operatively coupled (e.g., via a storage network 130 ) to some number of host computers 127 a - n.
In one embodiment, two storage controllers (e.g., 125 a and 125 b ) provide storage services, such as a SCS) block storage array, a file server, an object server, a database or data analytics service, etc. The storage controllers 125 a , 125 b may provide services through some number of network interfaces (e.g., 126 a - d ) to host computers 127 a - n outside of the storage system 124 . Storage controllers 125 a , 125 b may provide integrated services or an application entirely within the storage system 124 , forming a converged storage and compute system. The storage controllers 125 a , 125 b may utilize the fast write memory within or across storage devices 119 a - d to journal in progress operations to ensure the operations are not lost on a power failure, storage controller removal, storage controller or storage system shutdown, or some fault of one or more software or hardware components within the storage system 124 .
In one embodiment, controllers 125 a , 125 b operate as PCI masters to one or the other PCI buses 128 a , 128 b . In another embodiment, 128 a and 128 b may be based on other communications standards (e.g., HyperTransport, InfiniBand, etc.). Other storage system embodiments may operate storage controllers 125 a , 125 b as multi-masters for both PCI buses 128 a , 128 b . Alternately, a PCI/NVMe/NVMf switching infrastructure or fabric may connect multiple storage controllers. Some storage system embodiments may allow storage devices to communicate with each other directly rather than communicating only with storage controllers. In one embodiment, a storage device controller 119 a may be operable under direction from a storage controller 125 a to synthesize and transfer data to be stored into Flash memory devices from data that has been stored in RAM (e.g., RAM 121 of FIG. 1 C ). For example, a recalculated version of RAM content may be transferred after a storage controller has determined that an operation has fully committed across the storage system, or when fast-write memory on the device has reached a certain used capacity, or after a certain amount of time, to ensure improve safety of the data or to release addressable fast-write capacity for reuse. This mechanism may be used, for example, to avoid a second transfer over a bus (e.g., 128 a , 128 b ) from the storage controllers 125 a , 125 b . In one embodiment, a recalculation may include compressing data, attaching indexing or other metadata, combining multiple data segments together, performing erasure code calculations, etc.
In one embodiment, under direction from a storage controller 125 a , 125 b , a storage device controller 119 a , 119 b may be operable to calculate and transfer data to other storage devices from data stored in RAM (e.g., RAM 121 of FIG. 1 C ) without involvement of the storage controllers 125 a , 125 b . This operation may be used to mirror data stored in one controller 125 a to another controller 125 b , or it could be used to offload compression, data aggregation, and/or erasure coding calculations and transfers to storage devices to reduce load on storage controllers or the storage controller interface 129 a , 129 b to the PCI bus 128 a , 128 b.
A storage device controller 119 A-D may include mechanisms for implementing high availability primitives for use by other parts of a storage system external to the Dual PCI storage device 118 . For example, reservation or exclusion primitives may be provided so that, in a storage system with two storage controllers providing a highly available storage service, one storage controller may prevent the other storage controller from accessing or continuing to access the storage device. This could be used, for example, in cases where one controller detects that the other controller is not functioning properly or where the interconnect between the two storage controllers may itself not be functioning properly.
In one embodiment, a storage system for use with Dual PCI direct mapped storage devices with separately addressable fast write storage includes systems that manage erase blocks or groups of erase blocks as allocation units for storing data on behalf of the storage service, or for storing metadata (e.g., indexes, logs, etc.) associated with the storage service, or for proper management of the storage system itself. Flash pages, which may be a few kilobytes in size, may be written as data arrives or as the storage system is to persist data for long intervals of time (e.g., above a defined threshold of time). To commit data more quickly, or to reduce the number of writes to the Flash memory devices, the storage controllers may first write data into the separately addressable fast write storage on one more storage devices.
In one embodiment, the storage controllers 125 a , 125 b may initiate the use of erase blocks within and across storage devices (e.g., 118 ) in accordance with an age and expected remaining lifespan of the storage devices, or based on other statistics. The storage controllers 125 a , 125 b may initiate garbage collection and data migration data between storage devices in accordance with pages that are no longer needed as well as to manage Flash page and erase block lifespans and to manage overall system performance.
In one embodiment, the storage system 124 may utilize mirroring and/or erasure coding schemes as part of storing data into addressable fast write storage and/or as part of writing data into allocation units associated with erase blocks. Erasure codes may be used across storage devices, as well as within erase blocks or allocation units, or within and across Flash memory devices on a single storage device, to provide redundancy against single or multiple storage device failures or to protect against internal corruptions of Flash memory pages resulting from Flash memory operations or from degradation of Flash memory cells. Mirroring and erasure coding at various levels may be used to recover from multiple types of failures that occur separately or in combination.
The embodiments depicted with reference to FIGS. 2 A-G illustrate a storage cluster that stores user data, such as user data originating from one or more user or client systems or other sources external to the storage cluster. The storage cluster distributes user data across storage nodes housed within a chassis, or across multiple chassis, using erasure coding and redundant copies of metadata. Erasure coding refers to a method of data protection or reconstruction in which data is stored across a set of different locations, such as disks, storage nodes or geographic locations. Flash memory is one type of solid-state memory that may be integrated with the embodiments, although the embodiments may be extended to other types of solid-state memory or other storage medium, including non-solid state memory. Control of storage locations and workloads are distributed across the storage locations in a clustered peer-to-peer system. Tasks such as mediating communications between the various storage nodes, detecting when a storage node has become unavailable, and balancing I/Os (inputs and outputs) across the various storage nodes, are all handled on a distributed basis. Data is laid out or distributed across multiple storage nodes in data fragments or stripes that support data recovery in some embodiments. Ownership of data can be reassigned within a cluster, independent of input and output patterns. This architecture described in more detail below allows a storage node in the cluster to fail, with the system remaining operational, since the data can be reconstructed from other storage nodes and thus remain available for input and output operations. In various embodiments, a storage node may be referred to as a cluster node, a blade, or a server.
The storage cluster may be contained within a chassis, i.e., an enclosure housing one or more storage nodes. A mechanism to provide power to each storage node, such as a power distribution bus, and a communication mechanism, such as a communication bus that enables communication between the storage nodes are included within the chassis. The storage cluster can run as an independent system in one location according to some embodiments. In one embodiment, a chassis contains at least two instances of both the power distribution and the communication bus which may be enabled or disabled independently. The internal communication bus may be an Ethernet bus, however, other technologies such as PCIe, InfiniBand, and others, are equally suitable. The chassis provides a port for an external communication bus for enabling communication between multiple chassis, directly or through a switch, and with client systems. The external communication may use a technology such as Ethernet, InfiniBand, Fibre Channel, etc. In some embodiments, the external communication bus uses different communication bus technologies for inter-chassis and client communication. If a switch is deployed within or between chassis, the switch may act as a translation between multiple protocols or technologies. When multiple chassis are connected to define a storage cluster, the storage cluster may be accessed by a client using either proprietary interfaces or standard interfaces such as network file system (âNFSâ), common internet file system (âCIFSâ), small computer syst
CLAIMS
Claims ( 20 )
What is claimed is:
1. A method comprising:
storing, within one or more storage systems of an artificial intelligence infrastructure, information describing a dataset and one or more transformations applied to the dataset resulting in a transformed dataset;
identifying, by the artificial intelligence infrastructure, previous versions of a machine learning model that used the transformed dataset as input during one or more prior executions by the artificial intelligence infrastructure;
storing, within the one or more storage systems, information describing only differences between the previous versions of the machine learning model and a current version of the machine learning model;
retaining one or more first portions of the transformed dataset associated with the current version of the machine learning model at a first storage tier; and
moving one or more second portions of the transformed dataset associated with the previous versions of the machine learning model to a second storage tier.
2. The method of claim 1 wherein the storing, within the one or more storage systems of the artificial intelligence infrastructure, the information describing the dataset and the one or more transformations applied to the dataset resulting in the transformed dataset further comprises:
generating, by the artificial intelligence infrastructure applying a predetermined hash function to the dataset, the one or more transformations applied to the dataset, and the transformed dataset, a hash value; and
storing, within the one or more storage systems, the hash value.
3. The method of claim 1 wherein the storing, within the one or more storage systems, information describing only differences between previous versions of the machine learning model and a current version of the machine learning model further comprises:
generating, by the artificial intelligence infrastructure applying a predetermined hash function to the previous versions of the machine learning model and the transformed dataset, a hash value; and
storing, within the one or more storage systems, the hash value.
4. The method of claim 1 further comprising:
identifying differences between the current version of the machine learning model and the previous versions of the machine learning model.
5. The method of claim 1 further comprising:
determining, by the artificial intelligence infrastructure, whether data related to one or more of the previous versions of the machine learning model should be tiered off of the one or more storage systems; and
responsive to determining that the data related to the one or more of the previous versions of the machine learning model should be tiered off of the one or more storage systems:
storing the data related to the one or more of the previous versions of the machine learning model in lower-tier storage; and
removing, from the one or more storage systems, the data related to the one or more of the previous versions of the machine learning model.
6. The method of claim 1 further comprising identifying, from amongst the previous versions and the current version of the machine learning model, a preferred version of the machine learning model.
7. The method of claim 1 further comprising tracking an improvement of a particular version of the machine learning model over time.
8. An artificial intelligence infrastructure comprising:
one or more storage systems;
one or more graphical processing unit (âGPUâ) servers; and
a processing device, operatively coupled to the one or more storage systems and one or more GPU servers, the processing device configured to:
store, within the one or more storage systems, information describing a dataset and one or more transformations applied to the dataset resulting in a transformed dataset;
obtain, by the artificial intelligence infrastructure, identifiers for previous versions of a machine learning model that used the transformed dataset as input during one or more prior executions by the artificial intelligence infrastructure;
store, within the one or more storage systems, information describing only differences between the previous versions of the machine learning model and a current version of the machine learning model;
retain one or more first portions of the transformed dataset associated with the current version of the machine learning model at a first storage tier; and
move one or more second portions of the transformed dataset associated with the previous versions of the machine learning model to a second storage tier.
9. The artificial intelligence infrastructure of claim 8 wherein to store, within the one or more storage systems, the information describing the dataset and the one or more transformations applied to the dataset resulting in the transformed dataset the processing device is further configured to:
generate, by the artificial intelligence infrastructure applying a predetermined hash function to the one or more transformations applied to the dataset and the transformed dataset, a hash value; and
store, within the one or more storage systems, the hash value.
10. The artificial intelligence infrastructure of claim 8 wherein to store, within the one or more storage systems, information describing only differences between previous versions of the machine learning model and a current version of the machine learning model the processing device is further configured to:
generate, by the artificial intelligence infrastructure applying a predetermined hash function to the previous versions of the machine learning model, a hash value; and
store, within the one or more storage systems, the hash value.
11. The artificial intelligence infrastructure of claim 8 wherein the processing device is further configured to:
identity, by a unified management plane, differences between the current version of the machine learning model and the previous versions of the machine learning model.
12. The artificial intelligence infrastructure of claim 8 wherein the processing device is further configured to:
determining, by the artificial intelligence infrastructure, whether data related to one or more of the previous versions of the machine learning model should be tiered off of the one or more storage systems; and
responsive to determining that the data related to the one or more of the previous versions of the machine learning model should be tiered off of the one or more storage systems:
storing the data related to the one or more of the previous versions of the machine learning model in lower-tier storage; and
removing, from the one or more storage systems, the data related to the one or more of the previous versions of the machine learning model.
13. The artificial intelligence infrastructure of claim 8 wherein the processing device is further configured to:
identify, from amongst the previous versions and the current version of the machine learning model, a preferred version of the machine learning model.
14. The artificial intelligence infrastructure of claim 8 wherein wherein the processing device is further configured to tracking an improvement of a particular version of the machine learning model over time.
15. An apparatus comprising:
a memory; and
a processing device, operatively coupled with the memory, the processing device configured to:
store, within one or more storage systems of an artificial intelligence infrastructure, information describing a dataset and one or more transformations applied to the dataset resulting in a transformed dataset;
obtain, by the artificial intelligence infrastructure, identifiers for previous versions of a machine learning model that used the transformed dataset as input during one or more prior executions by the artificial intelligence infrastructure;
store, within the one or more storage systems, information describing only differences between the previous versions of the machine learning model and a current version of the machine learning model;
retain one or more first portions of the transformed dataset associated with the current version of the machine learning model at a first storage tier; and
move one or more second portions of the transformed dataset associated with the previous versions of the machine learning model to a second storage tier.
16. The apparatus of claim 15 wherein the processing device is further configured to:
identify, by a unified management plane, differences between the current version of the machine learning model and the previous versions of the machine learning model.
17. The apparatus of claim 15 wherein the processing device is further configured to:
determining, by the artificial intelligence infrastructure, whether data related to one or more of the previous versions of the machine learning model should be tiered off of the one or more storage systems; and
responsive to determining that the data related to the one or more of the previous versions of the machine learning model should be tiered off of the one or more storage systems:
storing the data related to the one or more of the previous versions of the machine learning model in lower-tier storage; and
removing, from the one or more storage systems, the data related to the one or more of the previous versions of the machine learning model.
18. The apparatus of claim 15 wherein the processing device is further configured to:
identify, from amongst the previous versions and the current version of the machine learning model, a preferred version of the machine learning model.
19. The apparatus of claim 15 wherein the processing device is further configured to:
track an improvement of a particular version of the machine learning model over time.
20. The apparatus of claim 15 wherein the processing device is further configured to:
generate, by the artificial intelligence infrastructure applying a predetermined hash function to the dataset, and the one or more transformations applied to the dataset, a hash value; and
store, within the one or more storage systems, the hash value.
US17/660,735
2017-10-19
2022-04-26
Machine learning models in an artificial intelligence infrastructure
Active
US12373428B2
( en )
Priority Applications (7)
Application Number
Priority Date
Filing Date
Title
US17/660,735
US12373428B2
( en )
2017-10-19
2022-04-26
Machine learning models in an artificial intelligence infrastructure
US19/257,844
US20250335437A1
( en )
2017-10-19
2025-07-02
Machine Learning Model Data Tiering
US19/373,053
US20260056947A1
( en )
2017-10-19
2025-10-29
Real-time data analytics and pipeline optimization in storage systems
US19/375,849
US20260056948A1
( en )
2017-10-19
2025-10-31
Efficient data delivery for artificial intelligence systems in distributed storage environments
US19/395,128
US20260072911A1
( en )
2017-10-19
2025-11-20
Pipeline monitoring and reconfiguration in machine learning systems
US19/423,524
US20260111420A1
( en )
2017-10-19
2025-12-17
Flash system with integrated log analytics and anomaly detection
US19/425,066
US20260127169A1
( en )
2017-10-19
2025-12-18
Artificial intelligence infrastructure with gpu-initiated storage operations
Applications Claiming Priority (9)
Application Number
Priority Date
Filing Date
Title
US201762574534P
2017-10-19
2017-10-19
US201762576523P
2017-10-24
2017-10-24
US201762579057P
2017-10-30
2017-10-30
US201862620286P
2018-01-22
2018-01-22
US201862648368P
2018-03-26
2018-03-26
US201862650736P
2018-03-30
2018-03-30
US16/045,814
US10360214B2
( en )
2017-10-19
2018-07-26
Ensuring reproducibility in an artificial intelligence infrastructure
US16/515,698
US11403290B1
( en )
2017-10-19
2019-07-18
Managing an artificial intelligence infrastructure
US17/660,735
US12373428B2
( en )
2017-10-19
2022-04-26
Machine learning models in an artificial intelligence infrastructure
Related Parent Applications (1)
Application Number
Title
Priority Date
Filing Date
US16/515,698
Continuation
US11403290B1
( en )
2017-10-19
2019-07-18
Managing an artificial intelligence infrastructure
Related Child Applications (1)
Application Number
Title
Priority Date
Filing Date
US19/257,844
Continuation
US20250335437A1
( en )
2017-10-19
2025-07-02
Machine Learning Model Data Tiering
Publications (2)
Publication Number
Publication Date
US20220253443A1
US20220253443A1 ( en )
2022-08-11
US12373428B2
true
US12373428B2 ( en )
2025-07-29
Family
ID=63452703
Family Applications (9)
Application Number
Title
Priority Date
Filing Date
US16/045,814
Active
US10360214B2
( en )
2017-10-19
2018-07-26
Ensuring reproducibility in an artificial intelligence infrastructure
US16/515,698
Active
2039-05-18
US11403290B1
( en )
2017-10-19
2019-07-18
Managing an artificial intelligence infrastructure
US17/660,735
Active
US12373428B2
( en )
2017-10-19
2022-04-26
Machine learning models in an artificial intelligence infrastructure
US19/257,844
Pending
US20250335437A1
( en )
2017-10-19
2025-07-02
Machine Learning Model Data Tiering
US19/373,053
Pending
US20260056947A1
( en )
2017-10-19
2025-10-29
Real-time data analytics and pipeline optimization in storage systems
US19/375,849
Pending
US20260056948A1
( en )
2017-10-19
2025-10-31
Efficient data delivery for artificial intelligence systems in distributed storage environments
US19/395,128
Pending
US20260072911A1
( en )
2017-10-19
2025-11-20
Pipeline monitoring and reconfiguration in machine learning systems
US19/423,524
Pending
US20260111420A1
( en )
2017-10-19
2025-12-17
Flash system with integrated log analytics and anomaly detection
US19/425,066
Pending
US20260127169A1
( en )
2017-10-19
2025-12-18
Artificial intelligence infrastructure with gpu-initiated storage operations
Family Applications Before (2)
Application Number
Title
Priority Date
Filing Date
US16/045,814
Active
US10360214B2
( en )
2017-10-19
2018-07-26
Ensuring reproducibility in an artificial intelligence infrastructure
US16/515,698
Active
2039-05-18
US11403290B1
( en )
2017-10-19
2019-07-18
Managing an artificial intelligence infrastructure
Family Applications After (6)
Application Number
Title
Priority Date
Filing Date
US19/257,844
Pending
US20250335437A1
( en )
2017-10-19
2025-07-02
Machine Learning Model Data Tiering
US19/373,053
Pending
US20260056947A1
( en )
2017-10-19
2025-10-29
Real-time data analytics and pipeline optimization in storage systems
US19/375,849
Pending
US20260056948A1
( en )
2017-10-19
2025-10-31
Efficient data delivery for artificial intelligence systems in distributed storage environments
US19/395,128
Pending
US20260072911A1
( en )
2017-10-19
2025-11-20
Pipeline monitoring and reconfiguration in machine learning systems
US19/423,524
Pending
US20260111420A1
( en )
2017-10-19
2025-12-17
Flash system with integrated log analytics and anomaly detection
US19/425,066
Pending
US20260127169A1
( en )
2017-10-19
2025-12-18
Artificial intelligence infrastructure with gpu-initiated storage operations
Country Status (4)
Country
Link
US
( 9 )
US10360214B2
( en )
EP
( 3 )
EP3862864B1
( en )
CN
( 2 )
CN111133409B
( en )
WO
( 1 )
WO2018218259A1
( en )
Families Citing this family (181)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
DE112015006953B4
( en )
*
2015-09-26
2025-07-10
Intel Corporation
TRAINING A VALID LANE
US12225049B2
( en )
2015-10-28
2025-02-11
Qomplx Llc
System and methods for integrating datasets and automating transformation workflows using a distributed computational graph
US10558646B2
( en )
*
2017-04-30
2020-02-11
International Business Machines Corporation
Cognitive deduplication-aware data placement in large scale storage systems
US11238164B2
( en )
*
2017-07-10
2022-02-01
Burstiq, Inc.
Secure adaptive data storage platform
US12184781B2
( en )
2017-07-10
2024-12-31
Burstiq, Inc.
Systems and methods for accessing digital assets in a blockchain using owner consent contracts
US11861423B1
( en )
2017-10-19
2024-01-02
Pure Storage, Inc.
Accelerating artificial intelligence (âAIâ) workflows
US12067466B2
( en )
2017-10-19
2024-08-20
Pure Storage, Inc.
Artificial intelligence and machine learning hyperscale infrastructure
US10671434B1
( en )
2017-10-19
2020-06-02
Pure Storage, Inc.
Storage based artificial intelligence infrastructure
US10360214B2
( en )
2017-10-19
2019-07-23
Pure Storage, Inc.
Ensuring reproducibility in an artificial intelligence infrastructure
US11494692B1
( en )
2018-03-26
2022-11-08
Pure Storage, Inc.
Hyperscale artificial intelligence and machine learning infrastructure
US11455168B1
( en )
2017-10-19
2022-09-27
Pure Storage, Inc.
Batch building for deep learning training workloads
US10951552B2
( en )
*
2017-10-30
2021-03-16
International Business Machines Corporation
Generation of a chatbot interface for an application programming interface
US10890898B2
( en )
*
2017-11-03
2021-01-12
Drishti Technologies, Inc.
Traceability systems and methods
US11769042B2
( en )
2018-02-08
2023-09-26
Western Digital Technologies, Inc.
Reconfigurable systolic neural network engine
US11741346B2
( en )
2018-02-08
2023-08-29
Western Digital Technologies, Inc.
Systolic neural network engine with crossover connection optimization
EP3528435B1
( en )
*
2018-02-16
2021-03-31
Juniper Networks, Inc.
Automated configuration and data collection during modeling of network devices
US11137926B1
( en )
*
2018-03-30
2021-10-05
Veritas Technologies Llc
Systems and methods for automatic storage tiering
US10789087B2
( en )
*
2018-05-31
2020-09-29
Hewlett Packard Enterprise Development Lp
Insight usage across computing nodes running containerized analytics
US11334329B2
( en )
2018-06-08
2022-05-17
Shanghai Cambricon Information Technology Co., Ltd.
General machine learning model, and model file generation and parsing method
CN111260018B
( en )
*
2018-11-30
2022-03-22
ä¸æµ·å¯æ¦çºªä¿¡æ¯ç§ææéå ¬å¸
Machine learning secondary model file generation method and device and storage medium
US10802719B2
( en )
*
2018-06-11
2020-10-13
Wipro Limited
Method and system for data compression and data storage optimization
US11115490B2
( en )
*
2018-07-31
2021-09-07
EMC IP Holding Company LLC
Host based read cache for san supporting NVMEF with E2E validation
JP6878369B2
( en )
*
2018-09-03
2021-05-26
æ ªå¼ä¼ç¤¾æ¥ç«è£½ä½æ
Volume allocation management device, volume allocation management method, and volume allocation management program
US10893120B2
( en )
*
2018-09-19
2021-01-12
International Business Machines Corporation
Data caching and data-aware placement to accelerate machine learning applications
CN109343833B
( en )
*
2018-09-20
2022-12-16
é¼å¯æºè½ç§ææéå ¬å¸
Data processing platform and data processing method
US11062042B1
( en )
2018-09-26
2021-07-13
Splunk Inc.
Authenticating data associated with a data intake and query system using a distributed ledger system
US12169987B2
( en )
2018-09-30
2024-12-17
Strong Force Tp Portfolio 2022, Llc
Intelligent transportation systems including digital twin interface for a passenger vehicle
US12380746B2
( en )
2018-09-30
2025-08-05
Strong Force Tp Portfolio 2022, Llc
Digital twin systems and methods for transportation systems
KR102797249B1
( en )
2018-10-05
2025-04-18
ì¼ì±ì ì주ìíì¬
Electronic apparatus and control method thereof
US11003999B1
( en )
2018-11-09
2021-05-11
Bottomline Technologies, Inc.
Customized automated account opening decisioning using machine learning
US11303454B2
( en )
2018-11-28
2022-04-12
International Business Machines Corporation
Producing and verifying computational determinations using a distributed ledger
US11139980B2
( en )
*
2018-11-28
2021-10-05
International Business Machines Corporation
Immutably storing computational determinations using distributed ledgers
CN109617992B
( en )
*
2018-12-29
2021-08-03
æå·è¶£é¾ç§ææéå ¬å¸
A method for dynamic election of edge computing nodes based on blockchain
EP4369229A3
( en )
*
2018-12-31
2024-09-25
INTEL Corporation
Securing systems employing artificial intelligence
CN109861789B
( en )
*
2019-01-15
2021-09-07
å京临è¿ç©ºé´é£è¡å¨ç³»ç»å·¥ç¨ç 究æ
A batch processing system of pipeline telemetry data adapting to slow recording and fast sending
US11507662B2
( en )
2019-02-04
2022-11-22
Sateesh Kumar Addepalli
Systems and methods of security for trusted artificial intelligence hardware processing
US11150720B2
( en )
2019-02-04
2021-10-19
Sateesh Kumar Addepalli
Systems and methods for power management of hardware utilizing virtual multilane architecture
US11544525B2
( en )
2019-02-04
2023-01-03
Sateesh Kumar Addepalli
Systems and methods for artificial intelligence with a flexible hardware processing framework
US11650961B2
( en )
2019-02-04
2023-05-16
Red Hat, Inc.
Managing replica unavailability in a distributed file system
US11477266B1
( en )
2019-02-14
2022-10-18
Sprint Communications Company L.P.
Data communication system to selectively and securely couple distributed ledgers with artificial intelligence (AI) engines
US11423454B2
( en )
2019-02-15
2022-08-23
Sateesh Kumar Addepalli
Real-time customizable AI model collaboration and marketplace service over a trusted AI model network
CN109785180A
( en )
*
2019-02-19
2019-05-21
å±±ä¸å¤§å¦
A scene perception system and method for digital twin workshop
US20200265509A1
( en )
*
2019-02-19
2020-08-20
Pathtronic Inc.
Secure and trusted multi-tenant service delivery platform for distributed multitenant-capable ai solution model compute processors
US11061738B2
( en )
*
2019-02-28
2021-07-13
Movidius Limited
Methods and apparatus to store and access multi dimensional data
US11182686B2
( en )
2019-03-01
2021-11-23
Samsung Electronics Co., Ltd
4T4R ternary weight cell with high on/off ratio background
US11409990B1
( en )
2019-03-01
2022-08-09
Bottomline Technologies (De) Inc.
Machine learning archive mechanism using immutable storage
WO2020189498A1
( en )
*
2019-03-15
2020-09-24
æ ªå¼ä¼ç¤¾ ï¼§ï½ ï½ ï½ ï¼§ï½ï½ï½ï½
Learning device, method and program
US11783176B2
( en )
*
2019-03-25
2023-10-10
Western Digital Technologies, Inc.
Enhanced storage device memory architecture for machine learning
US10929058B2
( en )
2019-03-25
2021-02-23
Western Digital Technologies, Inc.
Enhanced memory device architecture for machine learning
US11676063B2
( en )
*
2019-03-28
2023-06-13
International Business Machines Corporation
Exposing payload data from non-integrated machine learning systems
CN111832738B
( en )
*
2019-04-18
2024-01-09
ä¸ç§å¯æ¦çºªç§æè¡ä»½æéå ¬å¸
A data processing method and related products
USD956087S1
( en )
2019-04-23
2022-06-28
Bottomline Technologies, Inc
Display screen or portion thereof with a payment transaction graphical user interface
US11269859B1
( en )
*
2019-05-22
2022-03-08
Splunk Inc.
Correlating different types of data of a distributed ledger system
US11507562B1
( en )
2019-05-22
2022-11-22
Splunk Inc.
Associating data from different nodes of a distributed ledger system
US11663685B2
( en )
2019-06-18
2023-05-30
The Calany Holding S. Ã R.L.
System and method for providing digital reality experiences and decentralized transactions of real estate projects
US11650719B2
( en )
2019-06-18
2023-05-16
The Calany Holding S.Ã.R.L.
Virtual creation of real-world projects
US11687807B1
( en )
2019-06-26
2023-06-27
Bottomline Technologies, Inc.
Outcome creation based upon synthesis of history
US10970169B2
( en )
*
2019-06-27
2021-04-06
General Electric Company
Data configuration management system for an engine
CN112148663B
( en )
*
2019-06-28
2026-03-10
å为ææ¯æéå ¬å¸
Data exchange chip and server
US11635990B2
( en )
2019-07-01
2023-04-25
Nutanix, Inc.
Scalable centralized manager including examples of data pipeline deployment to an edge system
US11003501B2
( en )
*
2019-07-03
2021-05-11
Advanced New Technologies Co., Ltd.
Loading models on nodes having multiple model service frameworks
US11501881B2
( en )
2019-07-03
2022-11-15
Nutanix, Inc.
Apparatus and method for deploying a mobile device as a data source in an IoT system
US10963828B2
( en )
2019-07-19
2021-03-30
Capital One Services, Llc
Identifying and managing enterprise product availability
US10904100B1
( en )
*
2019-07-19
2021-01-26
Juniper Networks, Inc
Systems and method for replaying and debugging live states of network devices
US10810528B1
( en )
*
2019-07-19
2020-10-20
Capital One Services, Llc
Identifying and utilizing the availability of enterprise resources
US20220147430A1
( en )
*
2019-07-25
2022-05-12
Hewlett-Packard Development Company, L.P.
Workload performance prediction
US10956059B2
( en )
*
2019-07-31
2021-03-23
EMC IP Holding Company LLC
Classification of storage systems and users thereof using machine learning techniques
US11461676B2
( en )
2019-07-31
2022-10-04
EMC IP Holding Company LLC
Machine learning-based recommendation engine for storage system usage within an enterprise
CN110442389B
( en )
*
2019-08-07
2024-01-09
å京æå¾·ç³»ç»ææ¯æéå ¬å¸
A method for multiple desktop environments to share the use of GPU
CN110737517A
( en )
*
2019-08-14
2020-01-31
广西çµç½çµåè°åº¦æ§å¶ä¸å¿
electric power system cloud platform computing analysis micro-service resource scheduling method
CN115269717B
( en )
2019-08-22
2023-06-02
å为ææ¯æéå ¬å¸
Storage device, distributed storage system, and data processing method
US11514361B2
( en )
*
2019-08-30
2022-11-29
International Business Machines Corporation
Automated artificial intelligence radial visualization
EP3786781A1
( en )
*
2019-08-30
2021-03-03
Bull Sas
System to assist with the design of an artificial intelligence application, executable on distributed computer platforms
EP3786783A1
( en )
*
2019-08-30
2021-03-03
Bull SAS
System to assist with the design of an artificial intelligence application, executable on distributed computer platforms
EP3682340A4
( en )
2019-09-12
2020-12-02
Advanced New Technologies Co., Ltd.
LOG-STRUCTURED STORAGE SYSTEMS
CN111886582A
( en )
2019-09-12
2020-11-03
åæ°å è¿ææ¯æéå ¬å¸
Log structure storage system
SG11202002027TA
( en )
2019-09-12
2020-04-29
Alibaba Group Holding Ltd
Log-structured storage systems
US10942852B1
( en )
2019-09-12
2021-03-09
Advanced New Technologies Co., Ltd.
Log-structured storage systems
EP3669268A4
( en )
2019-09-12
2020-12-23
Alibaba Group Holding Limited
Log-structured storage systems
EP3695304B1
( en )
*
2019-09-12
2023-04-12
Advanced New Technologies Co., Ltd.
Log-structured storage systems
CN115398874A
( en )
*
2019-09-12
2022-11-25
åæ°å è¿ææ¯æéå ¬å¸
Log structure storage system
WO2019228572A2
( en )
2019-09-12
2019-12-05
Alibaba Group Holding Limited
Log-structured storage systems
CN111886591B
( en )
2019-09-12
2025-10-17
èèé¾ææ¯æéå ¬å¸
Log structured storage system
US12155731B2
( en )
2019-10-09
2024-11-26
Nutanix, Inc.
Platform-as-a-service deployment including service domains
CN110650609B
( en )
*
2019-10-10
2020-12-01
ç æµ·ä¸éç§ææéå ¬å¸
Cloud server of distributed storage
US11347613B2
( en )
2019-10-15
2022-05-31
UiPath, Inc.
Inserting probabilistic models in deterministic workflows for robotic process automation and supervisor system
CN110990256B
( en )
*
2019-10-29
2023-09-05
ä¸ç§»ï¼æå·ï¼ä¿¡æ¯ææ¯æéå ¬å¸
Open source code detection method, device and computer readable storage medium
US11240045B2
( en )
*
2019-10-30
2022-02-01
Red Hat, Inc.
Detection and prevention of unauthorized execution of severless functions
<a href="/pat