The Limitations of Big Data

In recent years, data has gained a prominent role in Competitive Intelligence and Market (CI&M) activities. Related careers, such as data analysts and data scientists, have secured positions in intelligence departments.

Indeed, it is a fundamental activity, especially in industries that handle large volumes of data to generate insights about consumers. Big data, one of the buzz words at the turn of the millennium, became a common expression and is no longer a lecture topic at events – AI took its place.

Today, there is a myriad of platforms working with large volumes of data. At Link Estratégia, we receive dozens of messages per week from lead generation companies, all working with high volumes of open data (databases from the Federal Revenue Service, state and county boards, etc.). trabalhando com alto volume de dados abertos (bases da Receita Federal, Juntas Comerciais, etc).

There are also platforms that use customer information to analyze profiles, increase engagement, and boost retention. It is common, for example, in drugstores and supermarkets, where there is always a discount when customers grant access to their data.

Apparel retailers have also widely adopted this model, asking customers to register on a website or app to receive some extra benefit, either discounts, refunds, or participation in VIPs groups.

But like any tool, big data has its limitations, especially when there is a scarcity of data. In niche B2B competitive environments, for instance, the contribution is restricted. Even in B2C environments, depending on the product, the use of big data can be frustrated by the lack of data.

A less glamorous product, like dishwashing detergent, is unlikely to generate a relevant volume of comments on the internet. It is also not very easy to get buyers of this type of product to give their data in exchange for promotions.

But it is not just data access issues that hinders the use of big data. The type of insight that these tools and methods provide is also limited. Sometimes, good qualitative research, with in-depth interviews, is more suitable.

To better understand the value drivers that lead a customer to a particular product, for example, a focused approach may be more appropriate. In-depth interviews with a group that represents the ideal customer profile can guide the adjustment of features, price, packaging, sales narrative, and various aspects of market strategy.

In some cases, there is the possibility of a deeper approach. In another post, we talked about the framework created by Bouquet, Barsoux, and Wade in ALIEN Thinking: The Unconventional Path to BreakThrough Ideas and about the book Small Data by Martin Lindstrom.

These two ways of analyzing consumer behavior go beyond in-depth interviews and elicitation of information. And they go far beyond analysis exclusively based on large volumes of data.

Related Posts

30

Dec
All Subjects, Demystifying Intelligence, Market and Competitive Intelligence

Best Practices for Enhancing M&CI Project Impact

Market and Competitive Intelligence is one of the most powerful tools for reducing uncertainty and driving strategic decisions; however, to be effective, it is essential to overcome resistance, calibrate execution, and gain the trust of decision-makers.

Indicator Monitoring within Scenario Analysis

27

Nov
Demystifying Intelligence, Intelligence Techniques, Market and Competitive Intelligence

Understanding How Indicator Monitoring Works Within Scenario Analysis

Have you ever heard of an analytical technique called Indicator Monitoring? This Market and Competitive Intelligence (M&CI) method is used alongside Scenario Analysis to identify which projected scenarios are most likely to materialize