When data misleads: The high cost of telling a half-baked story

Can a statistic be completely accurate and still tell the wrong story? How easily it is to create white lies, and how people can use data and information to shift the narrative or goal posts.

Data can be cherry-picked and selectively presented to support a particular argument, while bits of information that may contradict or weaken that argument are quietly left out. Nothing about the gathered data or statistics is necessarily untrue, but rather, the problem is that the statistic alone is not telling the whole story.

Take sustainable products as an example. Our research often finds that the majority of consumers agree and look positively at sustainable and more environmentally responsible products in the market. Sustainable products touch on heartstrings and are a matter of importance for many of us, and rightfully so. It naturally paints us a picture of being one of the “good guys”. Even if you believed otherwise, or cared little about the environment, you would be hard-pressed to answer otherwise and merely just support it in a lukewarm manner.

So, these sustainable products are thus a compelling reason for businesses to invest more heavily in sustainable products, right?

If you were a business seeking to produce sustainable products and have your own sustainable balance sheet, would you ponder further if you deep dived into our conducted research, to find out that only 60% of those consumers are willing to pay more for sustainable products, and the majority of them would accept no more than a 10% premium?

It is thus true that sustainable products are widely accepted, and the data is not false, but what has changed is our understanding of what consumers actually say they want, and what they would actually actively do. The bigger picture opened new avenues of thought and insight.

The gap of data transparency

For See Toh Wai Yu, CEO of Central Force Insights and President of the Marketing Research Society of Malaysia (MRSM), this is becoming an increasingly important conversation for the research and insights industry. Oftentimes, when one gets their hands on a set of data and statistics, it is the deep dive and transparency into how said data is interpreted, presented and ultimately used that matters.

We have never had access to more information than we do today. In fact, today’s age of information overload places emphasis on cutting out the fat (noise) in order to get down into the meat and bones (facts and outcomes).

Businesses have access to customer databases, governments have administrative statistics, researchers have increasingly sophisticated tools for gathering information, and artificial intelligence (AI) can process and summarise big data within seconds.

Yet having more data does not necessarily mean that we have a better understanding of what is happening. The problem often begins somewhere between the research and the narrative. Let’s be frank, headlines and clickbait draw attention and clicks, but the underlying is so murky and important context can disappear. We may see a percentage without knowing who was surveyed, how the respondents were selected, what questions were asked, when the research was conducted, or what circumstances may have influenced the findings.

Furthermore, Wai Yu describes this issue of transparency as requiring the responsibility not simply to produce an accurate figure, but to give people enough information to understand what that underlying figure actually represents.

Through the honest and ethical application of methodologies, it is important to present findings in their proper context, and resist the urge and temptation to make a much broader claim, shout a headline for more clicks and views, or twist a narrative to upsell a goal and objective. This also means being prepared to accept what the research tells us, even when it is not what we expected to hear.

Evidence before assumption

Throughout his professional career in research, when Wai Yu and his team encounter a data point, one of the first questions they ask is, “Is that true, or how true is that?” Wai Yu advocates the importance of asking questions and developing good reasoning.

Far be it to train yourself to be an always skeptical person, but this deceptively simple question gears the mind to not take everything at face value. If we truly seek to know more about a subject or news headline, it is imperative to seek further credible data sources before making a judgment or decision.

So, how can we tell whether research findings are reliable? Start by asking simple enough questions: Who was surveyed, how many people participated, how the respondents were selected, and what the research was actually designed to measure? Was it a specific demographic, or does it genuinely represent a wider population? When was the research conducted, and what was happening at that particular point in time? Is there bias? Who funded the research, and what exactly was the research intended to measure?

Wai Yu gave a useful example from a recent nationwide opinion poll conducted by Central Force Insights. One of the findings attracted attention because a particular young political figure came up ahead, receiving 29% of the votes, when respondents were asked who they would prefer as the next Prime Minister.

That 29% was an accurate finding, but the more interesting part of the research was that a substantial proportion of those respondents came from the 18–25 age group, meaning that the demographic composition of the result provided an important layer of context to the headline figure.

Without that context, a reader might simply see 29% and form one interpretation. With the context, the same 29% tells us something considerably more nuanced about generational preferences and the way the result should be understood.

Research should challenge assumptions, not simply confirm them 

Avoid seeking moments of ‘sendiri shiok’, which is Malaysian slang for “feeling good about oneself”. Research is at its best when it gives a brand and organisation the information that it did not already know, and sometimes that means telling hard truths and revealing things that one would rather not hear.

Good research comes with first scrutinising the research itself, examining the methodologies, data collection and analytics to ensure that the findings are sound. Once confident that the work has been conducted properly, there should be no hesitation to present the reality of what this research has found, nor to curry favour, even when it contradicts what a client expected or hoped to hear.

This uncomfortable finding can actually create opportunities and solutions, reveals weaknesses in product design, a gap between customer expectations and experience, an emerging reputational concern, or an assumption about the market that is no longer holding true or relevant today.

More information makes responsibility even more important

The rise of AI has made the issue more complicated. AI is an incredibly useful tool for processing information, identifying patterns and helping people work with data. But AI remains a tool, and the quality of its output is inevitably influenced by the quality of the information available to it.

If unreliable or inaccurate information is used as part of the foundation, AI does not magically correct the problem. It may simply reproduce that information in a faster, more convincing and potentially more widely distributed form. And when the information does not exist, there is another danger: AI may still produce an answer when the more responsible response would be to acknowledge that there is not enough evidence.

For Wai Yu, that is why people need to retain a healthy degree of scrutiny even as technology becomes more sophisticated. When we encounter information online, across social media feeds, through search engines or through AI overviews, we should not simply accept it because it has been presented confidently. We should ask where it came from, how it was produced and whether there is enough evidence to support the conclusion being made.

Reinforcing what matters: values and reputation

For us here at Central Force Insights, these principles have particular meaning as we recently celebrated our 30th anniversary. Three decades in the research and insights industry is not simply a measure of longevity, but also of handling data and information that can influence business decisions, public policy, brands, communities and people’s lives.

That comes with a weighted responsibility to treat information with care and we continue to reinforce the principles that continue to matter: the importance of ethics and research integrity, evidence before assumption, and information with responsibility. As an ESOMAR Corporate Member, reflecting its connection to the international professional community and the standards that underpin responsible research practice, along with Wai Yu’s role as President of MRSM gives Central Force Insights a broader platform to advocate for stronger ethics, transparency and professional integrity across Malaysia’s research industry.

The next time you see that bold headline or thumbnail on a YouTube or TikTok video, understand oftentimes that there is more than meets the eye (R.I.P. Peter Cullen, the voice and heart of Optimus Prime).

Ultimately, the value of research is not in how impressive a number looks on a slide or spreadsheet, nor in how conveniently it supports a preferred or feel-good narrative. Its value lies in helping people understand what is truly happening well enough to make better decisions.

Watch the conversation behind the story

The ideas explored in this article were also discussed by See Toh Wai Yu on The Nation on BERNAMA TV. Watch as he shares why research should challenge assumptions rather than simply confirm them, and why data transparency and context are essential to making better decisions.