Economic Value from Analytics

A proposal to standardise the approach.

Photo by Erik Mclean on Unsplash

Background

Of late, I’ve found myself discussing with senior practitioners of data analytics / data science (i.e. Analytics for short) about the attributed value of their work within their organisations. There has been a clear shift in the past 20 years where major organisations have become comfortable, and in fact have insisted, on having the economic value (EV) of Analytics reflected in their annual reports. The ability to measure this value and impact has led to Analytics having a seat at the table. It’s given the function accountability and ownership. And this is a good thing.

But economic value can be a double-edged sword. The need to constantly justify, and deliver EV, can end up distorting the best intentions of the Analytics function. And so I dedicate my 99th article to unpacking my thoughts on Analytics EV from a 1st principle perspective, and this might shape the conversation with their Finance brethren.

(I write a weekly series of articles where I call out bad thinking and bad practices in data analytics / data science which you can find here.)

Definition

As is usual, let me start with definitions. What exactly is Economic Value, and how does it relate to the work of data analytics / data science? Google’s search AI says that “Economic value represents the benefit a good or service provides to an individual or company, often measured by how much someone is willing to pay for it.” This is as succinct a definition as any, and so I will go with it. The interesting phrases are “… benefit of a good or service …” and “… how much someone is willing to pay for it …”. These 2 phrases indicate that EV is subjective, both in terms of perceived benefits and the price someone is willing to pay for it.

For most organisations, EV, as applied to data analytics / data science, is further characterised by defining the benefits as (a) efficiency, (b) outcome due to better decisions, and © the creation of new data products & services. These benefits are typically measured in terms of incremental $, cost savings ($), and cost avoidance ($). It’s important here to note that EV is NOT equivalent to P&L. For example, cost avoidance does not appear on the accounting books. This difference often creates tension with Finance, as they are the ones responsible for reporting EV.

Another essential consideration in Analytics EV is that we are attributing the value created by the manpower in the Analytics function. It is a way to justify the cost of the function and to quantify its capabilities. This has the following corollary: there is no such thing as recurring EV. EV should be measured based on a forward 12-month horizon and no more. Having continuous recurring EV beyond the 12-month horizon suggest that we can extract benefit with NO manpower input or cost, which is generally untrue.

Analytics in all its Glory

The common misconception is that Analytics is all about finding insights, whatever the word “insight” might mean for different stakeholders. The reality is that Analytics work sits at the intersection of front- and back-office. While Analytics work is often creative and involves the act of discovery, there are aspects of Analytics work that are also very much like Operations. And Analytics is also frequently positioned as supplementary IT. In abbreviated terms, let’s just say that any of the work that Analytics does can be classified into (a) R&D, (b) Ops, and © IT.

Examples of Analytics R&D work: diagnostic analytics, market research, predictive modelling. These kinds of work result in better decisioning.

Examples of Analytics IT work: creating data pipelines, automation, production integration of AI/ML models. These kinds of work result in engineered solutions.

Examples of Analytics Ops work: report and dashboard maintenance, computational execution (e.g. campaign lead extraction), process fine-tuning. These kinds of work result in improved process efficiencies.

Let’s consider a common piece of work that Analytics does — business intelligence reports and dashboards. Conceptualising the metrics and designing the look of the dash is Analytics R&D work. Constructing the actual dashboard is Analytics IT work. Maintaining and fine-tuning the dashboard over time is Analytics Ops work. Each of these 3 types of Analytics work has unique attributable EV, and we should take the sum of it.

Standardising EV for Analytics

At present, no standards in EV attribution exists for Analytics. The Analytics function often gets into a tussle with the Finance team on the appropriate and acceptable measures. Much of EV don’t show up as “green $” in the accounting books, a cause of much grief to the Finance team.

I propose the following:

1. Analytics R&D work to be measured in incremental $. We can measure the incremental $ as the potential $ impact of the findings (i.e. insights) if implemented multiplied by a discount rate to factor in implementation friction. The potential $ impact can be formally estimated through test-control or champion-challenger PoCs and then scaled up, or it can be informally estimated using logic arguments.
2. Analytics IT work to be measured as cost savings. We can measure the cost savings based on the equivalent work being outsourced to an IT agency., and what they might charge using their man-day rates and how long they might take. The cost savings come from cheaper internal costing and/or faster turnaround time.
3. Analytics Ops work to be measured as cost avoidance for the Analytics function. This is the key. We should NOT be measuring cost avoidance that the stakeholders accrue from the work of the Analytics function. We use cost avoidance and instead of cost savings when it comes to efficiency improvements because it relates to manpower utilisation. Manpower is a fixed cost. Efficiency improvements do not reduce this cost, but rather, it allows for increased throughput handling, thereby reducing the need to incur proportional cost increases; i.e. cost avoidance. We can measure the cost avoidance based on time savings expressed as $ or FTE (full-time equivalent; manpower).

The Case of Gen AI

Many organisations are looking to implement a bunch of Gen AI use cases, to signal to the market that they are not laggards in the data and digital transformation race. However, measuring EV from Gen AI use cases has become problematic. Why? Gen AI is typically a productivity play, where many of the use cases are internal-oriented and employee-facing, and the common approach is to measure the EV from an FTE save perspective. But most stakeholders are not comfortable with that approach given the current climate where organisations are increasingly looking to replace jobs with AI. Any cost avoidance expressed as FTE might feel like a target on one’s back to eventually reduce FTE, and turn those cost avoidance numbers into cost savings.

At the heart of this problem is measuring Gen AI EV as Analytics Ops work. Based on my proposal above, I would argue that Gen AI EV should be measured as Analytics R&D and Analytics IT work. Is the intent of the Gen AI use case to improve decision-making? If not, then regardless of the reason for the use case, we should view it as cost savings, computed based on the $ saves from not engaging an external consultant or IT vendor to build and implement the solution.

Conclusion

Organisations today don’t have a concern justifying the existence of their Operations units or the Risk Management units. They are not tasked to produce EV. Analytics, on the other hand, continue to justify their worth. The danger is when Analytics EV is mis-directed and all consuming, resulting in the Analytics function chasing the wrong focus and metrics. In our modern data and digital world, Analytics is the essential glue that holds the decision ecosystem together. Maintaining that BAU is not cost-free, nor is it decision-free — keeping to the status quo is also a decision that requires the right information and the right efficiency, all of which continues to be supported by the Analytics function.