The Technologies Behind Advanced Analytics in Commodity Management

Based on Google searches, there has been increasing awareness around advanced analytics and predictive analytics for the last several years. The need for analytics is being driven by user demand and enabled by technical capability. What sets analytics apart from yesterday’s business intelligence tools are the unique insights users gain.

“Gartner defines advanced analytics as the analysis of all kinds of data using sophisticated quantitative methods (for example, statistics, descriptive and predictive data mining, simulation and optimization) to produce insights that traditional approaches to business intelligence (BI) — such as query and reporting — are unlikely to discover.”

— Gartner
  Magic Quadrant for Advanced Analytics Platforms
February 2015

No longer is ETRM and CTRM software adequate for companies exposed to commodity market risk. While the ability to capture the data that surrounds each transaction or engagement is still required, that functionality alone is not enough. As margins tighten and conditions get tougher, today's commodities businesses require advanced analytics.

The Limitations of Yesterday's Systems

What we are seeing today is a shift from simple query and reporting to advanced analytics. Although analytics has existed for many years, its use was limited by access to data and technology. Not all users had access to all of a company’s data and with yesterday’s BI tools, there were limitations on how much of that data could be put into a data warehouse.

When bringing data into a data warehouse, there is significant effort involved in preparing and cleaning that data. Due to the effort involved, companies will pick and choose only the most important data to bring into a data warehouse so they can limit already overly burdensome lengthy implementation times. So yesterday’s analytics were performed on an incomplete data set.

Today's Technology Advancements

Data analysis techniques have existed for many years, such as regression, forecasting, optimization, and simulation. But the technology advancements in distributed data storage and in-memory computing have now made these methods accessible to all businesses.

• Structured and semi structured data for longer time spans can be stored and accessed quickly.
• Much higher speeds are achieved to process data.
• Insights into other decision variables can be computed and made accessible over the cloud.

Several technology innovations are paving the way for the most effective analytics:

• In-memory data grids (IMDG) are providing unprecedented processing speeds; advanced calculations are performed in minutes instead of hours. IMDGs can support hundreds of thousands of in-memory data updates per second, and they can be clustered and scaled in ways that support large quantities of data.
• Schema-on-read technology allows data to start flowing into the analytics systems in its original raw form, and parsed only at process time. This enables users to be able to ask any question, not just those baked into a predefined data model. Schema-on-read allows for versatile organization of data and is ideally suited for working with large sets of raw data.
• Dynamic visualization enables users to ask iterative questions to get timely answers. These interactive displays enable users to view large amounts of data to discover patterns and make faster and more accurate decisions.
• Machine learning evolved from the study of pattern recognition and computational learning theory in artificial intelligence. It explores the study and construction of algorithms that can learn from and make predictions on data. Machine learning is being used to make predictions such as cash flow predictions based on factors around invoices and counterparties.
• Predictive analytics is the branch of data mining concerned with the prediction of future probabilities and trends. Predictive analytics can be used to run complex forecasting models and scenarios to answer the “what if” questions; for example, what happens to my bottom line if market prices go up/down?

(View this and other presentations from Eka on SlideShare.)

Companies looking to adopt analytics will typically consider these options:

• Purchasing a generic platform that requires knowledgeable users, such as data scientists and analysts, to build the analytics pertinent to that company.
• Hiring a service provider to build a customized analytics solution.
• Purchasing a domain specific solution.

Domain specific solutions, such as Eka’s Commodity Analytics Cloud built specifically for the commodities markets, are enabling commodities companies to gain advanced analytics without investing millions of dollars and untold man-hours.

»»» Download brochure on Eka's analytics platform for commodities. «««

Volatility is the new normal for any business for which the wholesale price of commodities is a major input cost. Commodities companies have a need for fact-based decision making.

In addition to the need users have for better decision making tools, the technology now exists to support big data. Managing commodities has always been a big data business. By its very nature, commodity trading creates thousands of individual data points. Behind all these data points are decisions to be made around supplier, quantity, quality, type of storage, type of transport, etc. Taken together these individual dimensions create a significant amount of complexity – and produce huge volumes of data. The companies that are best equipped to use that data will have a competitive advantage.

However, the promise of big data and the insight that it can deliver can only be realized with significant analytics capabilities to turn raw numbers into actionable information. This is a significant step on the evolution of commodity management systems: from data capture to data analytics. The ability to analyze information to create predictive models allows firms to develop accurate, repeatable formulas that take into account market conditions to identify optimal scenarios.

Eka’s Commodity Analytics Cloud

Eka's Commodity Analytics Cloud is an advanced analytics solution that brings commodity specific analytics to all business users. The platform delivers industry-specific apps that cover P&L, procurement, risk, and supply chain. Unlike generic business intelligence tools, Eka's advanced analytics platform has been built with the specific needs of commodities companies in mind.