In the coming year, the digital transformation trend will continue to accelerate as companies increasingly push to become digital enterprises, writes Nikita Ivanov, Founder & CTO, GridGain Systems.
Many companies recognise that digital technologies can do far more than drive marginal increases in efficiency. They can enable companies to reinvent their business models, change their relationships with customers, and disrupt their industries. Existing companies would do well to understand that if they don’t become a data-driven digital enterprise, they will likely lose out to a company that is one.
Look at the changes in the Fortune 500 companies due to digital transformation. By 2017, just 12 percent of companies listed in the Fortune 500 in 1955 were still on the list. This was largely the result of digitally-driven tech startups transforming the competitive environment while the legacy Fortune 500 companies ignored or failed to keep pace with these changes.
In-Memory Computing
In-memory computing will continue to be a key enabler of this digital transformation trend.
The technology is now mainstream as the cost of memory has come down and mature vendor solutions have been proven in production environments to deliver the promised benefits. In addition, implementing in-memory computing solutions has become increasingly manageable as vendors provide native integrations with other commonly used solutions like Apache Kafka and Apache Spark and support common protocols, such as SQL, Java and .NET.
The number of technical professionals with experience implementing distributed computing solutions has also increased dramatically. In-memory computing is no longer only for big companies with large IT budgets.
Over the past several years, we have seen companies across a broad range of industries increasingly adopt in-memory computing platforms to achieve the application performance and scalability they need to achieve their digital transformation or omnichannel customer experience goals:
- American Airlines is using in-memory computing to accelerate response times, automate processes and meet SLAs for business applications that process huge datasets from multiple sources.
- eTherapeutics is using an in-memory computing platform to complete in just a few hours, or even minutes, computational drug discovery projects that used to take weeks – and the company is now able to explore many more potential drugs than it previously could in the same amount of time.
- Finastra, which provides financial services software, is using in-memory computing for real-time processing of massive amounts of trade and transaction data, eliminating bottlenecks and enabling next-generation financial services.
- ING is using in-memory computing to support growth, meet increased customer demand for mobility and provide an improved user experience even as it continues to depend on legacy mainframe computers.
In 2019, two new exciting developments in in-memory computing will enable companies to expand and refine their digital transformation or omnichannel customer experience initiatives. The first development is in machine learning and artificial intelligence. In-memory computing platform vendors are responding to the massive interest in machine-driven decision making by adding machine learning capabilities to their in-memory computing platforms.
The performance and scalability of in-memory computing platforms are already enabling new computing approaches, such as hybrid transactional/analytical processing (HTAP), also known as hybrid operational/analytical processing (HOAP), to unlock the value of data in real-time and drive business results. HTAP eliminates the need for separate operational and analytical databases – and the need to use a time-consuming extract, transform and load (ETL) process to move data from the operational database to the analytical database.
With machine learning libraries integrated with the in-memory computing platform, companies will be able to create solutions that support “in-process HTAP” (a Gartner term), which means their machine learning models will be continually retrained in real-time based on new operational data. In-process HTAP enables optimal decision making based on machine learning models that evolve in real-time. In-process HTAP supports mission-critical applications such as fraud detection, credit approvals, price setting, and vehicle and package routing. As a cost-effective solution for enabling in-process HTAP, in-memory computing will allow more companies to utilise machine learning across an ever-broader array of use cases.
Further, new integrations between in-memory computing platforms and deep learning systems, such as TensorFlow, will allow companies to feed their in-memory computing platform data directly into deep learning platforms, eliminating the need to create and maintain a separate analytical architecture connected via ETL to their operational database. This strategy will dramatically reduce the cost and complexity of using operational data to train processor-intensive artificial intelligence models.
The second important in-memory computing industry development that will impact 2019 is in-memory-computing-platform-as-a-service (imcPaaS). We have already seen the database world move rapidly towards a database-platform-as-a-service (dbPaaS) model, in which databases are consumed as a service from cloud providers. This trend will apply to in-memory computing solutions as well.
According to 451 Research, by 2019, 60 percent of workloads will be deployed in cloud environments, including on-premises private clouds, hosted private clouds, IaaS and SaaS. By 2022, says Gartner, application platform as a service (aPaaS) deployments will drive two-thirds of new in-memory computing data grid (IMDG) implementations, up from one-third in 2018. Since these cloud environments are the way of the future, organisations developing digital transformation or omnichannel customer experience initiatives will need to ensure their applications easily scale across cloud-based infrastructure. They will also need to ensure real-time visibility into their data, no matter where it resides, to support their operational, regulatory or analytical requirements.
imcPaaS will enable end-user companies to easily consume in-memory computing platforms as PaaS solutions on major cloud services such as AWS, Microsoft Azure, Oracle Cloud, Huawei Cloud and more. We already see leading companies across a range of industries, from financial services to online business services to transportation and logistics, deploying their in-memory computing platforms on private and public clouds for large-scale, mission-critical use cases. And in-memory computing vendors are already making their products available as dbPaaS or imcPaaS solutions with SQL APIs for compatibility and ease of use. In 2019 vendors will likely increase the functionality of these solutions and add new capabilities to their imcPaaS solutions.
To remain competitive, companies that are not already digital enterprises have no choice but to accelerate their digital transformations and become data-driven businesses. As a key enabler of digital transformation, in-memory computing technologies will continue to receive increasing focus across a wide range of industries. In addition to the two key technical trends noted above, 2019 will likely be the year when in-memory computing becomes a part of every fast data discussion as companies leverage all their enterprise data in real-time to drive business value.
