#28. Primary data and human fallibility

May 20, 2024

The Ackmaniac

Last month, in a palatial Palm Springs Trump haven, Eric Cortellesa of Time sat down with Truth Social executive Donald Trump (also running for President). Cortellesa's mission: uncover Trump's presidential plans so Time can lay that out objectively. Black and white. Lots of direct quotes from Trump in the interview, so you know what he *actually* said. We can now all have a common understanding of what we’re talking about.

But it didn’t work, of course. It was never going to. And it has nothing to do with Time or Trump. It was never going to work because humans cannot tell objective stories.

This week (University) President-slayer Bill Ackman tweeted this:

His point is: don’t trust the version of the truth you get from the media (Time, in this case). Any story will reflect how the author/outlet wants you to read it. If you want to know the truth, go to the source. He ends his tweet with “That’s why I vastly prefer podcasts and other long form interviews, and ideally an in-person meeting, when trying to get a sense of someone.” Great work, Bill. He’s right. It’s more illuminating to hear directly from somebody rather than through a secondhand source. But he’s also wrong.

—————

Data Translation ≠ Data. Data Translation > Data.

Assume you’re reviewing a prospective investment and want some non-spin truth. So you use Ackman’s primary-est way to get information: an in-person interview. It’s still not primary data. The information you get will go through two horribly biased lenses before you internalize it: your interviewee’s experience and your own experience. Any information coming from a person (or something derived by a person) will be colored with their experience of the event, the world, and the interview. Once the words come out of their mouths, you are going to further color them with your experience of the event, this person, the world, and the interview. Those two things will always happen. Always. And not because either of you hate the truth, but because both of you are human.

But maybe screw humans, you have satellite data streaming to your computer to answer your question. That sounds primary. This information will still pass through two very similar lenses: the algorithm (because you’re not going pixel by pixel yourself, my friend) and your own annoying brain again.

Earnings data isn’t primary data. Financial statements are not primary data. Interviews are not primary data. Anything that can be made to “look” better or worse is not data. That information was once raw data that has been transmuted to be usable at scale. And all data transmutations are built by people (or are people) who, like you and me, fail in our effort to be objective.

Nearly every piece of data we scrutinize has been made ingestible by some “data translator” (a person or an algorithm). There’s too much data for us to meaningfully use it otherwise. Entire companies are funded every day whose sole purpose is to generate new data that never existed before. Of course we’re generating more data every day, but the sheer scale is mind blowing. Pick your anecdote: 90% of the world’s data was created in the last two years and the amount doubles every two years; it takes us four days to generate as much data today as we did in all of 2010; every internet user globally created over 1.5 megabyte of data per second alive last year.

That’s why we almost never want primary data. We would drown in the sheer volume. We need Anthropic and OpenAI and all the translators we can get.

So we rely on flawed translaters to make the best decisions we can. While the desire to *be* more objective or write counterbalancing algorithms is understandable, it will always fall far short of the desired perfection (👋Google’s racially diverse Nazis). The closest we can get to eliminating human limitation is understanding it.

—————

Human fallibility

It’s impossible to perform truly primary research on every question. And if you did, it’s impossible to not instill your own biases.

The takeaway, though, isn’t to nihilisticly reject information. It’s to understand that incomplete people cannot build complete systems. If you want to get as close as possible to a truth, don't try to wring objectivity out of a person. Expand your aperture instead.

I’d never suggest *not* speaking to somebody directly if given the opportunity. However, better than squeezing truth-blood out of a person-stone, interview 10 people around that person. Read other people’s biased perspectives on the person. Listen to two unrelated people discuss. Hear that person tell somebody else about the same thing.

Getting closer to a truth isn't about getting closer to the source and hoping for objectivity. It is incapable of being objective. Getting closer to the truth means expanding your surface area around a given topic or person. That’s why interviewers do references. That’s why interviewees should ask multiple people at a prospective company the same question. Rather than holding a company/person/algorithm to unrealistic expectations, treat the web of artifacts around it like a puzzle. Each piece is complimentary to the limitations of the others.

Or maybe don’t. I’m biased as hell and felt like writing this today.