Those Who Can, Segment
The art of sensemaking.
Background
Years ago, I was interviewing a veteran leader in the data analytics / data science practice for my PhD thesis, and he told me something quite intuitive and yet extraordinary: “The ability to segment and classify is what differentiates a great analyst from a regular one.”
In my long career, the best analysts that I’ve worked with all have a strong tendency towards simplification-through-segmentation; creating groupings of things based on a set of perceived similarity of attributes. It’s an instinctive way of focusing on the “20% of data that gives you 80% of the information signals”.
For the purpose of this article, I will use the term “segmentation” and “classification” interchangeably for the high-level task of “grouping”, although I recognise that in the data analytics domain, they mean different methods. But what exactly is segmentation? Why do we segment? How should we segment? And so I dedicate my 58th article to unpacking this topic further.
(I write a weekly series of articles where I call out bad thinking and bad practices in data analytics / data science which you can find here.)
What is Segmentation?
Segmentation is the foundation cornerstone of Marketing. And the Marketing discipline is great at extracting meaning from a noisy environment. The ability to segment is therefore a foundational cognitive requirement to be a good data analyst — because the job entails distilling information signals from naturally noisy data.
Segmentation is the art of amplifying information signals through the creation of meaningful groupings. It requires the critical ability to recognise attributes that can be used to underpin those groupings. By grouping items together, we are concentrating the inherent information signals through data reduction and simplification. Often, the attributes may not be obvious or explicit, and those who can see or elicit the right attributes will logically have a high competency in data sensemaking. Data sensemaking is simply the ability to apply a frame or perspective to interpret meaningful information from the data, and to show how the various data elements inter-relate to each other. I’ve written about it in my recent article on how to perform diagnostic analytics.
Why Do We Segment?
A good segmentation is about problem-finding or problem-defining. A good segmentation illuminates and clarifies; it removes ambiguity by allowing one to have an interpretation to a phenomenon of interest. In fact, it’s one of those things that AI cannot do (today). At its core, segmentation is about distribution analysis. We are looking for confirmation (intuitive) or surprise (counter-intuitive) in the resulting distribution. The infamous SWOT analysis is such a segmentation. While it has its detractors, one cannot deny the simple elegance of this segmentation methodology in moving an ambiguous strategy-oriented conversation forward.
I find segmentation to be a great catalytic starting point to any data-driven project. In my consulting work, I am often confronted by client requests that come across as broad sweeping statements of intent; they are poorly framed, if at all, and lack specificity. Finding a way to “crystallise” the requests often start with a segmentation effort. I was once asked to help an organisation grow their business by way of data-driven solutions. A useful way to start was to segment their business approach on acquisition, deepening and retention to figure out where the “hits and misses” are, and where the concentration of opportunities might lie. The trick is in defining the boundaries of the segments, and then within those bounded segments, to further create sub-categories of relevance. This often requires a fair amount of domain knowledge.
Segmentation can also be considered a heuristic for predictive analytics. Sometimes we are looking for correlation and rank ordering in the segments. Marketing often employs this approach to make sense of their target universe. The right segmentation can help you pierce through the veil of opportunities and risks.
How Do We Segment?
You can perform segmentation using a nested approach — a single dimension to start, and then further sub-segmenting within. Or you could use the common 2-dimensional approach. Consider the classic 3x3 (aka 9-box) segmentation approach. We see it in a variety of situations — e.g. Potential (H/M/L) vs Performance (H/M/L) in HR, Risk vs Revenue in business strategy, Pricing vs Utility in Marketing. The 9-box is a powerful way to describe a landscape of interest. The audience gets it!
Now you may be wondering why 3x3 and not 2x2 or 4x4? Miller’s Law states that humans have a cognitive load of 7±2, meaning we can process on average 7 pieces of information simultaneously, with 9 at the upper limit. A 4x4 segmentation will result in 16 boxes of information that is simply overwhelming. On the flip side, a 2x2 segmentation often feels too broad-stroked and not fine-grained enough to be meaningful; one of the reasons why the SWOT analysis feels too “high-level”.
You can even consider a 3-dimensional approach — a 2x2x2 segmentation (8-box). The trick with such segmentation approach is the equivalence of the dimensions. For example, revenue and risk are both outcomes, and hence equivalent. Potential and performance are latent and explicit manifestation of abilities, and hence equivalent. I have seen mistakes made in such 3x3 segmentation where one dimension is an input and the other is an outcome. In such an instance, one is simply doing a direct correlation analysis, rather than a segmentation.
Ultimately, segmentation is sensemaking. It’s iterative. There is no single prescribed methodology. There is a huge creative aspect to it, anchored on experience.
Conclusion
This brings us back to the claim: “The ability to segment and classify is what differentiates a great analyst from a regular one.” How true is that? Given that segmentation is deeply connected with data sensemaking, problem-finding and problem-defining, it does seem to be a valid claim. And so, when you are next looking to evaluate your data analytics / data science talent, it may be useful to construct an ambiguous use case and see how they might approach it, and whether their goto thinking is anchored on segmentation.
