What Is In-Memory Analytics?
Today, data lives everywhere. Its volume, velocity and variety are increasing beyond all expectations. Harnessing data analytics has already helped many leading brands move beyond traditional business intelligence to real-time analytics for greater efficiencies, risk avoidance and enhanced revenue through customer-tailored offerings. Businesses that are slow to tap value from data with analytics solutions can put themselves at a significant competitive disadvantage.
Speed is the key requirement for an IT infrastructure that can support analytics-driven decision-making. The business value of decision-support solutions often depends on being able to deliver results at least thousands of times faster than conventional solutions. Achieving this lofty goal requires taking a new approach to processing: In-memory computing.
The concept of in-memory computing is simple. In the conventional approach to processing data, the data resides on a hard disk in the system or attached by a network. When needed, it’s called into the local system memory (today known as RAM), and from there moves to the CPU. The long seek times for data residing on disks often can become a bottleneck.
With in-memory computing, data is stored directly in system memory. This architectural approach dramatically reduces latency by eliminating the time spent seeking data on the disk and then shuttling it closer to the CPU. Today, in-memory computing relies on DRAM memory, which is expensive, making it not cost-efficient for large volumes of data. However, evolving persistent memory technologies offer a solution by combining high-capacity, affordability and data persistence with near-DRAM levels of performance.
In-memory analytics often has two other important technical components that increase the performance of the software.
Columnar data storage: Instead of the traditional two-dimensional structuring of data (rows and columns), in-memory analytics data has a one-dimensional, linear structure.
Massively parallel processing: In-memory analytics makes full use of multi-core, multi-thread processor capabilities, which are freed to operate on the data given the reduced access latencies.
The Maturing Business Intelligence Portfolio
Business analytics, like many IT initiatives, can become even more valuable to an enterprise as organizations gain experience and operational maturity about delivering solutions. More traditional or conventional approaches, such as descriptive and diagnostic analytics, that tell a business what happened: “where we were” instead of “where we could go.”
The next step on the maturity scale, predictive analytics, looks forward. It replaces a seat-of-the-pants approach to decision making with one that’s disciplined and data-driven. Predictive analytics operates in real time. Often, it extends its reach to people who are on the front lines making constant low-level decisions, for example which pallets to load into which container.
These small decisions aren’t extremely important in themselves, In the aggregate, however, they can make a big difference to the bottom line, either through cost avoidance or increased revenue. Over time, predictive analytics will enable businesses to automate processes that are now manual so that they move at “compute speed.”
In later stages of the maturity model, prescriptive analytics, explores what-if scenarios on larger time scales and projects possible outcomes. Prescriptive analytics might be used, for example, to determine the optimal location for a new retail outlet.
All of these forward-looking approaches make use of data within the organization—sometimes including transactional data—as well as many different forms of data available from third party aggregators.
In-memory analytics solutions may not replace conventional data warehouses but can enhance an organization’s total decision support capability. It’s possible to get started with in-memory analytics before engaging in a wholesale re-platform of your business.
Business Value of Analytics
The sources of data with business value are endless: Data from factory sensors, from multiple retail channels, from social media, even from weather satellites and other third-party feeds. New developments like smart cities and the Internet of Things will only add to the load. Companies can't ignore this data if they want to remain competitive. Properly analyzed, it can increase sales by predicting the up sell most likely to succeed, cut distribution costs with smarter routing and inventory management, reduce manufacturing costs and improve quality with sophisticated root cause analysis—again, the list is virtually endless.
Sometimes, the path to actionable information derived from this flood of data is simply finding patterns in what already happened. In other instances, real-time results are needed to improve the customer experience, stop a malware exploit or prevent fraudulent use of a credit card, to cite a couple of examples.
The barriers to its adoption are falling. All the major IT vendors offer analytics solutions, and there are numerous vertical solutions as well. The number of data scientists with the requisite skill sets to both use and support sophisticated analytics is growing. Also, many companies are working to “democratize” the use of analytics through simpler interfaces and built-in algorithms. The publicity surrounding analytics, along with its solid business case, have made funding easier to obtain.
The bottom line is that there is clear business value in analytics. Numerous brands are already using in-memory analytics to boost revenue and cut costs. Those who don’t pursue these operational advantages are at risk of competitive disadvantage.
Analytics in Action
In-memory analytics is a proven, game-changing technology that is having a huge impact right now on every aspect of business and organizational management, including manufacturing, supply chain management, human resources, marketing, distribution, finance and more.
For many organizations, the key benefit of in-memory analytics is the ability to process vast quantities of data fast enough so that the resulting insights become a difference maker. Pattern recognition involving large amounts of data is a key use case. The IRS*, for example, analyzes tax returns as they are being processed to identify patterns of mistakes or problems. The result has been interventions that stopped the IRS from erroneously refunding several hundred million dollars.
Predictive analytics is perhaps the most useful application of in-memory technology. At UPS*, predictive models for delivery operations are responsible for reductions in miles driven, which saves the company money and reduces the overall company carbon footprint.
Predictive analytics is particularly effective in retail. A retailer, for example, has the ability to initiate an in-memory analytics project to produce targeted marketing campaigns, resulting in reduced costs. Any industry can likely benefit from approaches such as this.
Step-by-Step Guide to Getting Started
Here is a five-step process for getting started with in-memory analytics.
- Identify pain points. Consult with business unit leaders to create a list of pain points that would be difficult or impossible to solve with existing systems. This list should be prioritized based on those items that align with existing strategies, hold promise for new insights, are within the skill capabilities of the IT organization, and have a solid business case. With some iteration, the end result should be a clear list of objectives and the resources to achieve them.
- Research and familiarize yourself with the available analytics solutions on the market. In light of that knowledge, evaluate your current infrastructure. It’s important to understand where the data to be analyzed will be coming from, who owns it, and what measures will be necessary to ensure data quality and security.
- Identify and cultivate the skills your team will need. Hire new talent or plan to outsource some tasks if necessary. In many cases, new employees will be coming with skill sets that match your needs.
- Establish technology requirements above and beyond what’s currently in place. In-memory analytics requires modern hardware, including compute, storage and networking infrastructure. You will also need to determine which analytical queries and algorithms you’ll need to generate to achieve the desired outputs, and then decide how those outputs can be presented in engaging ways. Look to both proprietary and open source solutions for your software, as a wealth of options exist.
- Create the final use cases or the project. Determine what data will be used, and map out data flows. Then develop a test environment for a production version.
Intel in Analytics: Hardware and Beyond
Intel offers the industry’s broadest and platform for in-memory analytics, with significant capability to scale with increasing workloads. It is capable of supporting a variety of diverse analytic workloads, including real-time, in-memory databases, scale-out Spark* deployments, high performance computing (HPC) and machine learning. It incorporates compute, storage, memory, fabric and networking technologies, all optimized for “work better together” performance where the whole is greater than the sum of the parts.
The result is a flexible infrastructure, with built-in security that delivers the high performance needed to meet today’s needs while forming a solid, trustworthy foundation for the future.
Intel® architecture gives IT organizations a consistent baseline across their infrastructure, with a predictable path for scaling analytics initiatives over time and a broad product offering, which means there is no need to support multiple architectures. It also offers a consistent software-programming model for developers, enabling them to focus on enhancing performance and features.
Intel® architecture is supported by a rich ecosystem of hardware and software partners. Intel actively collaborates with these partners on an ongoing basis to help optimize their product performance on Intel® architecture.
With Intel as an analytics partner, organizations have the flexibility to choose an open source software platform or one of the industry-leading commercial platforms, such as those from SAS, SAP, Oracle, IBM and Microsoft, and many others.
With its history of success, Intel is a rich source of information about what it takes to succeed with an in-memory analytics initiative.
Intel is primarily known for its processors, and for many, the Intel® Xeon® processor Scalable family is synonymous with in-memory analytics. The full story, however, is much, much broader, and well worth investigating. Click here to learn more about how Intel can help your organization to develop an in-memory analytics strategy.