Fujitsu Laboratories has developed Adaptive Replication Degree, a technology that automatically resolves problems caused by concentrated accessing of popular data items in a distributed storage system, preventing access-time slowdowns.
The new technology can instantly detect rise in popularity for a data item and automatically increase the number of replicas of it to level-off server loads.
Adaptive Replication Degree automates popularity spikes in a distributed storage which was earlier been done manually slowing down access times.
Distributed storage combines multiple servers into a single storage and increasing the number of servers improves storage capacity and performance, making this approach appropriate to store large data.
Distributed storage is used to store massive volumes of data, in which multiple hard drives, solid-state drives and other storage mechanisms are combined to be treated as a single storage.
In distributed storage simultaneously stores replicas of a single data item on multiple servers increases the reliability of data.
If there is a sharp increase in accesses to a particular stored data item, the load on the server storing it will increase, which may cause an upturn in user access times.
Adaptive Replication Degree automatically handles the process of detecting concentrated accesses and varying the number of replicas, so the process requires no manual intervention.
The new technology simplifies the access concentrations by approximately 70%, improving access times by tenfold. It will also stabilise operations on ICT systems where access patterns are difficult to predict.
It also incorporates a popularity-estimation engine that estimates popularity weighted by how recently it was accessed using a small amount of memory to detect sudden access concentrations.
Adaptive Replication Degree includes a new technique that analyses how highly concentrated access occurs, which causes the number of replicas to fluctuate, based on the frequency of access during periods of concentrated access.
The number of replicas to add is automatically determined when a popular data item is detected. This technique uses two threshold values, one indicating that a spike is underway, another as a sign of an impending spike, to detect periods of concentrated access.
The higher the access frequency during those periods, the more replicas are created, so that replicas are increased in proportion to the magnitude of a traffic spike.
Fujitsu Laboratories is continuing to test and improve the performance of this technology to apply it to products and services in 2013.
