Kinetic drives are native key/value stores. This shifts the burden of maintaining the space mapping of a device from a file system to the drive itself. Applications need only put and get objects; they no longer need to guess at LBA layout or prescribe data location. This shift largely eliminates a very significant amount of drive IO that moves no data but rather represents metadata- and file system-related overhead. (In a recent benchmark, this overhead accounted for up to 92% of drive activity.) There is also incremental benefit here for scaling: as both device manufacturers and cloud datacenter operators ramp device capacity as aggressively as possible, the increased IO efficiency – and resulting net IO utilization – enables more balanced scaling of IO and capacity, in addition to absolute performance on a given device and across a Kinetic cluster. Incremental downstream performance gains come from the improved manageability enabled by the key/value semantic abstraction. For example, this abstraction allows for graceful handling of device failures, including partial failures, in some cases without the corresponding extensive rebuild times characteristic of large capacity drives.
The Kinetic platform is uniquely optimized for explosive-growth, scale-out datacenters. The Kinetic architecture with its disaggregation of storage from compute enables cloud datacenter operators to simply add storage as need for capacity grows. Additionally, the combined impact of Ethernet connectivity and the key/value API command structure enables incremental capacity to be scaled in a highly distributed manner with the replication of data directed from drive to drive, with minimal incremental system/capex cost.
Simplicity, Ease of Use / Adoption
Customers can build their own management applications or call the drive directly using the Kinetic API. The API is designed to enable rapid integration into a wide variety of storage software applications. Additionally, it is envisioned that Kinetic devices can be discovered dynamically, enabling adoption into existing datacenters and a heterogeneous environment evolution.
Kinetic drives are provided with a comprehensive user space library that allows applications to access the device directly. This library provides the complete interface to access the data and to manage the drive. It bypasses the normal operating system storage stack and lets the application to talk directly to the drive as if it were talking to another service in the datacenter. This process utilizes a typical application remote procedure call (RPC). This Kinetic platform currently provides libraries for Java, C++, C, Python, and Erlang, and other languages will be provided over time.
The Kinetic API allows applications to interact with the drive as if it were a typical Key/Value service on the network; it allows applications to put data in the form of keys and values to the drive and to get this data back by specifying just the key. As one would expect, keys and their values can be deleted. Additionally, the keys are ordered so that searching of the keys within ranges and finding the next and previous keys are possible.
Initial implementations of the Kinetic API include Swift and Riak CS, and others are in process. These systems allow thousands of drives to be managed as a single reliable storage cluster. With such third party management software, not only is the data stored reliably (using replication and/or erasure coding), but failed drives are also recovered transparently to the applications. There are also extensive drive management commands that allow the drive to report its health and to manage who is allowed to communicate with the drive.
The Kinetic platform allows implementation of new datacenter architectures. This is due to the fact that Kinetic drives interface directly with the applications, thereby eliminating an entire tier of hardware.
This technological advantage allows much denser storage racks, which impacts total cost of ownership in a number of different areas:
- Lower Capital Expenditure – Kinetic architectures allow the removal of storage servers from the datacenter. This translates directly into lower capital expenditure in building out datacenters. Alternatively, with the a comparable level of capital expenditure, customers can allocate much greater storage capacity in the same physical space.
- Labor – As the Kinetic architecture removes the need for storage servers, this then reduces the number of technicians required to maintain them. In addition, the denser storage enabled by the Kinetic architecture can reduce the number of technicians a datacenter needs to employ in general, leading to significant labor savings.
- Power consumption – The elimination of the tier of storage server architecture and more efficient rack density allows for fewer racks to stand up the same volume of storage. This reduces energy consumption.
- Uptime/technician error – The greater reliability of the Kinetic architecture with regards to automatic replication and failover reduces the number of errors related to the management of the storage datacenter. In addition, in-drive error management promises to reduce major technician incidents to the level of routine maintenance.
The increase in rack density provides another strong cost benefit for cases where physical real estate is a significant consideration, for example datacenters located in colocation facilities. The greater rack density means a significantly lower physical footprint for the datacenter, which translates directly into cost savings.
The security of storage services within the cloud datacenter is a difficult task. The Kinetic drive and interface library supports:
- Authentication – A full cryptographic authentication of servers that have access permission to the drive.
- Integrity – Full integrity check of the command and the data.
- Authorization – A clear set of roles by server as to what the application is allowed to do. Typical roles are read, read/write, management of the drive and management of the security in the drive.
- Transport Layer Security – For the security of very sensitive data and/or management commands, a full industry standard TLS suite is also provided.
This is a marked difference to other distributed storage systems where, inside the datacenter, traffic between services are not only unsecured, but also unauthenticated. This gives anyone who has access to the datacenter complete and unfettered access to the storage to read, modify and even delete all the data. In these situations, the security becomes the responsibility of the networking infrastructure and higher cost networking, separate network islands or complicated VLANs. The Kinetic security architecture allows low cost and flexible datacenter networking architectures.