When KPIs Turn Maladaptive: "Carrots Unrestrained"

Do you have opinions regarding the usefulness of key performance indicators (KPIs), good or bad? I do too, mostly not-good, and all of them are strongly-held. Somewhere around ten-ish years ago, I was in 'metrics hell', and I promised myself I'd find time to write this once I finally got out of it.

It took ten years of almost writing the rant I wanted to write until I finally zoomed out enough on the history around them to grasp how we ended up in this seventh level of performance optimized hell. I knew that if I understood how, I'd be closer to understanding why, and possibly closer to helping leave the work world a little better than I found it.

So, I researched, and researched, and finally feel like I found the answer I was seeking. And, well, I'll do my best to share it in a way that doesn't pack in any more boring business history than necessary, and I'll do my best to keep it upbeat and witty while I poke the very existence of these life-monopolizing little ass hats with very sharp sticks! Let's begin with a simple challenge:

If given a goal (A), how do we objectively define KPIs to keep us progressing toward A as we gain momentum? What keeps the entropy from momentum alone from being what drives us to complete the goal?

Do we need to have that conversation for relatively simple goals? If not, then When should we objectively define what those KPIs should be? That sounds like it would cause some strain to lift, so maybe we can save some effort by asking Can we objectively define what KPIs should be?

Looking back on 30 years of being in tech from the first dot-com boom of the 90s and now working on open source alternatives to big AI, I thought back on so many wonderful ways companies turned from gold to poop through KPI idolization. In my observation, the top comorbidity in failed developer-facing startups has been, by far, blind worship of the the all mighty key performance indicator (KPI). This is a call to spot and stop madness if and when you see it.

We need some context from history to illustrate how we ended up with these sadistic little bastards ruling our professional existence, so let's start by going back in time to 1954 to get a malt shake. In fact, in 1954, a rather revolutionary book called The Practice Of Management (archive link) first hit the shelves which lowered the altitude of business reviews by focusing most on objectives rather than gut feelings and intuition, which is how they sailed by the metaphorical stars back then with very little qualitative data to steer by.

Peter Drucker's book planted the seeds that became the modern KPI, years before anyone was using the term metric outside of engineering.

We need to adjust our perspectives a bit, so they're not alien to 1954 business culture. For instance, in 2025, most people would consider decision-making to be a process. In 1954, it was a practice and one best suited for a sole practitioner who had a set of gut feelings that could be trusted for the job. Experience could, and often did come later, for those that could prove their instincts trustworthy. This is one of the reasons that executive teams were primarily made up people from nearly-identical backgrounds.

We're also dealing with a market that has a considerably longer attention span than what we're used to today, as well as one that gets its information almost strictly from its family, friends and neighbors first.

Feelings: the age before KPI dashboards

KPIs weren't developed to solve over-arching multi-dimensional problems (as one might infer based on how they're so liberally applied), but rather as an artifact of new knowledge being available. Before we had computers and thousands of market data points to consider at every turn, we talked about growth in qualitative, almost moral terms, not as a switchboard of yes/no answers to metrics we either did or did not meet.

"Are we making good widgets?", we'd ask, as we looked at how long they sat on shelves before selling and what people had to say about them. We looked at production in terms of how many we can make in a shift, and consider workers happier when that number is higher.

Of course we knew that other market trends we couldn't understand were affecting us, and we knew that we were chronically tone-deaf to what people actually wanted out of our products, just like everyone else. And ignorance, at least until the late 1960s and FORTRAN, was a very gentle bliss.

The trick was being risky enough to make million dollar decisions based on biased anecdotes without being wrong. But, back then, this just felt natural.

Signal: the age of mainframes to primitive data lakes

It feels strange to refer to something computational as primitive, but computers have been with is longer than many might think.

Companies have always had the wisdom of some market data; Big Pharma used to send representatives to local pharmacies in order to get sentiment from customers shopping their brands, as did big tobacco, also in pharmacies! But I digress; companies have always had numbers beyond sales and costs to pilot by.

Computers offered a far more accurate way of tabulating and correlating data that was much kinder than manual counting to humans, who are very ill-suited to repetitive tasks without committing errors. We don't have a concept of granular user profiles yet, but companies are getting information about television ratings, periodical subscribers, several census, consumer reports and other sources that don't reduce the same way for easy tabulation.

As you'd expect, the loudest and most recent things in the room were the things that captured the most attention. Companies then developed a sliding and scaling window that they were capable of paying attention from, as they became even more oblivious to their markets than ever before, ironically, in the face of access to unprecedented levels of available information.

It wasn't until the late 1970s that computational power was democratized enough for middle-managers to be able to utilize machines they could run on their desks to access the catalogs of data that companies had, but getting the data itself was still a manual (as in load the tape reels) process that required several highly-paid people's cooperation every time you did it. My Stepfather worked for Exxon in Public Relations at this time and it was his job to pull the data to pull reports. He pulled his back out, plenty, because usable reports could take an entire fiscal quarter to pull and narrate.

When you look back on five decades of any kind of marketing, you're looking back on five decades worth of differences tech made when it came to informing teams on what should go into the creative, and whose perspectives should be in charge of its narrative.

Startup companies could afford small VAX machines to run their own models, crunch their own numbers, or import and merge their own records. But the picture is still vague; sales data is still slow to filter back into the already slow models, and decision making has gotten way more expensive without getting any objectively easier.

It's 1979 when John F. Rockart, a professor at MIT Sloan School of Management suggested that executives focus on limited set of critical metrics rather than trying to pay attention to every growing source of meaningful signal.

Then, something called The Internet happened. Things got way more complicated than just watching how long stuff sat on shelves; we had to figure out where the shelves were all over again.

Optimization: the age of micromanagement

Humans have a gargantuan capacity for cooperation; our ability to "host" our professional roles independently from our personal ethos and, to varying extents, ego, is key to bringing our cumulative intellect to bear on complex problems.

The truth of the matter is, we like being a "{insert your ant role here}" in a very objectified way, as if we're actors playing parts on stage, because it lets us shed "work feelings" as soon as we leave for the day; well, at least, it used to. Now it feels more like a leash than a symbiotic tether.

Optimization began, almost out of necessity, as an after-hours activity: both in conception and execution, because it wasn't permitted to have any adverse affects on the business. Executives knew that simply watching things affected their outcome on an instinctual level well before physicists documented the phenomenon.

I've spent the better part of a decade wondering exactly when we began accepting direction on such demanding levels; levels that verge on defying our very basic autonomy as human beings. From our 1954 perspective, being awake and working six hours early three days a week just to survive grilling when arbitrary numbers on a dashboard would be completely irregular and not accepted.

It's common for even entry-level jobs today. Optimization, at least from what I concluded, has to start at the breakdown of autonomy, or at least how much people can reasonably expect; but it's not just because we need to fall in line with specific metrics, it's because we need to fall in love with having someone else's way for sixteen hours a day.

For the purposes of this essay, which is to explore how we might have arrived at a hell scape of insatiable mechanized masters (and I'm not even referring to AI yet), optimization was mostly about re-defining work culture while "moving fast and breaking things" in an effort to learn by doing.

It happens gradually, like erosion. You can't really look for where it was suddenly one way and then turned into another. But much like the universe we live in, the observant and spiritual among us can often pick out some sense of intentionality. It wasn't an evil plot to change corporate culture to resemble indentured slavery, it really was just the result of a series of opportunities.

So while yes, this age did see a lot of serious study on ripple effects that tiny changes in input can have on organizations, its most significant contribution was the realization that business culture norms had to change to normalize a much more demanding climate for real strides to be possible.

To this day, incompetent people fail upwards by blaming a lack of employee cooperation for their own intrinsic project shortcomings.

Engagement: the age of eyeballs & "soft" metrics

When my mother was pregnant with me, a R.J. Reynolds representative at the pharmacy where she shopped informed her that a mentholated cigarette would help soothe breathing ailments associated with pregnancy; she smoked More Menthol afterwards up until she was hospitalized for COPD the first time.

"Soft metrics" began to be a umbrella term that referred to anything that tracked sentiment or feelings in a more passive sense. Tobacco companies knew pregnant women had health concerns, so they made sure they had information available to them both where and when it mattered to a sale.

This was an example of how a company engaged with its customer base in response to changing sentiment, but when we think about the word engagement as a metric, we almost universally think about something describing customers engaging with the company in some way (its app, other users, feedback channels, and so forth).

Because, as you might have guessed, modern social media companies and tobacco companies have something in common: they need to keep their users varyingly addicted to their product in order for their business models to flourish.

Engagement isn't always evil; you could very responsibly track, adjust to, and deeply care about how much users engage with feature C over feature D if they both do essentially the same thing different ways (just to name an example). Tracking engagement as something that should always be engorging is, in almost every situation, stupid to do in a vacuum, and unfortunately all too often, that's where it's done.

Modern KPIs: the age of hyperfocus up and to the right

Richard Li, the CEO of Ambassador Labs once said something really simple, but I think often overlooked by many:

A startup's greatest asset is speed.

He wasn't saying that everyone should run around on some kind of fast forward mode doing lots of randomly-beneficial things very quickly; that would help, but it doesn't move you against a goal.

What he meant was, startups can be much faster at doing very simple things than their lumbering, often process-laden more established competition. Startups can cut checks faster, hire people faster, make deals on their feet without a month of stakeholder meetings, buy software without lengthy approval processes, and more.

What this means is smaller companies can be surgical in spotting and exploiting opportunities that can make or break them, based on stories weaved together from a lot of metrics and then sold to investors. These startups are essentially betrothed to the game plans that got them funded, which are built on, you guessed it, a few dozen open-worded KPIs.

At every quarterly business review, you've either got to have a green chart, or a brand new (and better) narrative with a brand new fool-proof metric-driven path to success. Think of the plan and the KPIs being like the mylar components of a complicated decorative balloon. If it's all planned out and engineered correctly, a sudden infusion of air (err, cash) should balloon it up just perfectly.

At modern startups, north-star KPIs are beyond important, they're absolutely existential. Deviation means an entire perfectly-orchestrated plan could all come crashing to a halt. What you're told has to happen that quarter must happen, or the company has to re-kindle favor with investors.

Individual per project KPIs are every bit as existential, just not as visible to investors because they really don't want to see that level of minutia. And, when you're dreading going into a meeting to report a specific number will cause you to meet overall goals, thus causing a cascading effect, you'll literally contemplate soul-selling just to make the damn color turn green.

This isn't what modern companies want; this isn't the landscape that lends to the great work/life balance everyone wants. In some ways it's just how some feel they need to structure to survive. But, people are pushing back. Well, unemployed people mostly, who aren't burdened anymore with a desire to not draw attention to themselves by complaining.

It sucks and you just have to get through it; that's what the paychecks are there to help cope with. But sometimes it's just not enough ... which is a great segue into the next topic!

Knowing when you've left the rails

There are definite symptoms that you can watch out for! This includes those who are working on 1 - 2 person teams, or in companies of just one or two people. The pattern of maladaptation is the same and is usually sequential:

Loss of agency or narrative: You're not really aware of anything other than what you're doing, and you have no agency to change that. The over-arching narrative of where everyone was headed is now a repeating loop of re-measuring and affecting a metric. You're not aware of how you impact others, or you've been told not to care.
Quantify ... ALL THE THINGS!: You're being asked to translate how events in the market, community or industry impact a matrix of indicators, or dashboard, in a way that doesn't care about nuance. You're being asked to over-simplify the enormously complex in a way that affects people's jobs and lives without any acknowledgement that's what's happening. In aggressive and advanced cases, you're asked to do this in retaliation for suggesting idolization of metrics might be affecting judgement.
Forget outcomes! Feed the beast!: And the beasts are metrics. This is particularly dangerous if in a corrective tailspin; you forget you're flying a crashing aircraft because you're so fixated on how the blinking lights change based on how you move. Tunnel vision is far too pedestrian a term.
Who cares?: Suddenly, the FUD merchants don't sound so much like conspiracy nuts anymore and people begin to feel like they've stopped mattering in whatever it is the whole is becoming, or has become. People check out, and some never rebound.

When you see all of these things present and you're not reeling from a recent C-level shakeup, it means you're heading toward a C-level shakeup, so take hope! You could share this with a decision maker, but situations like this seldom manifest when there's responsible adults in charge. I don't mean to be a doom sayer, but if you're here, get your CV and professional network in order.

Sadly, the 4 steps I outlined above is all too often the "ballad of the down round" or "song of the series-C" (I'm here all week). Let's look at ways this has actually come to pass, shall we?

Conclusion

My goal in writing this was to make what I've learned and figured out available to people who are, perhaps right now, dealing with the dread that is the night before a meeting where you know you have to say you haven't met your metrics.

It's a little bit of validation that I hope helps someone decide to push back a little bit more in a 1:1 meeting and help land an idolization flight before it gets too high and suddenly drops.

If you're in management, I want you to feel more confident to speak up when things looked propped up on unreal metrics and might land on needless layoffs.

And I wanted to do it all in a way that was a little funny, a little informative, a little motivational and around the length of a job commute. And, well, perhaps the cathartic venting was fun, too, but this was about being there for someone else in a way that I wished I'd found something there for me. Work culture sucks really bad right now, and it's only because I'm broadly out of it that I can be so frank about what's wrong with it.

So if you're reading this while taking a break from a job where the only acknowledgement you get is a green spreadsheet cell and not shamed that week, this is validation from an old neck beard that you're not what's wrong with this picture and it doesn't have to be this way.

Bonus anecdotes: popular KPI idolization blunders

First, I think almost every company has experienced this; some just soared to far greater heights while compromised with it than others, so it was rather well documented and observed. I'm not picking on any company in particular.

Stack Overflow: "Developer Story"

I must disclose that I was an employee of Stack Overflow from the end of 2010 to the end of 2020, even though all information I'm about to share was well-documented on the company's meta site at the time.

Stack Overflow had a difficult time luring technical job seekers to its platform. It was the number one destination on the planet for programmers in a panic watching their code melt down, but it wasn't yet solidified as a place where one could get a job, even though that was one of the original functions planned for the site by its founders.

Stack Overflow launched something they called a developer story, which was just a quick overview of someone's accomplishments and places they worked. It was a place to talk all about one's self and catered well to it, but it just wasn't getting any attention.

Their goal of completions hanging in the toilet, someone decided to try redirecting new sign ups to fill out a story before being redirected to do whatever they came to do. When this happened, there was a 99.9% chance every new sign up arrived to do one thing: ask an urgent question urgently, so this just resulted in confusing them even more and even worse question quality.

But, it moved the developer story completion metric so significantly that 1/3 of the team's KPIs depended on it to stay green, so that's how it stayed, for a very long time. Meanwhile, the site was flooded with low quality questions, spam moderators could not remove, and very few interesting developer stories.

The jobs product was eventually abandoned altogether in favor of a Teams product.

Twitter: "Daily Active Users"

Daily Active Users (DAU) and mDAU, meaning "monetizeable" daily active users, is still a north star metric for many social platforms, or platforms that rely mostly on user-generated content to give their user base as a whole new things to do and explore.

There's nothing wrong with measuring this and considering rolling back recent changes if it suddenly plummets seemingly in response to something you do.

The problem is how this quickly transforms to an over-obsession with engagement, which is problematic because it almost invariably increases to requiring unhealthy levels for success to have been met. This is how people get hurt if trust & safety isn't considered prominently in the process, which at Twitter, was usually not the case.

I left Twitter completely shortly after Musk's takeover, so I wasn't really around for the whole active minutes transition. And after reading how it went, I'm glad. I'm still really sore about the wonderful disabled community that used to exist on Twitter, so I'm cutting this one a little short, and there's not much else to say about it anyway.

YouTube: "Watch Time"

YouTube simultaneously struggled with and created the state of the art around understanding everything you possibly can about a video short of having a human paid by YouTube watch it and complete a questionnaire. They've been selectively transparent about rankings and results, especially where advertiser concerns come into play, but they give some advice and produce some reports with data.

One of the biggest bungles of their entire product history is the signal of watch time, or how many minutes eyeballs have been entertained by a video. YouTube's theory is that the more something is watched to (or close to) the end, the more likely it is to be at least relevant.

Today, it basically measures being included in playlists that tend to run in the background while people work on other things. To add insult to injury, many of the videos with long watch times are artificially inflated with random junk so they have longer watch times 🤡

MSI is, essentially, a grow light for misinformation. Facebook didn't set out to build that (seriously, there aren't armies of Darth Vader coders who want to create an empire over there, Facebook is built on the labor of people who thought they were doing altruistic stuff). I'm being very clear in saying that current and ex Facebook employees aren't to blame for our current information hell scape, at least not until you get to the C-level.

But, MSI is why misleading click-bait is spit out so zealously by FB and similar platforms - because people tend to interact more intensely when provoked. Facebook needed to come up with a way to decide what posts were better to prioritize in feeds so that people were likely to spend more time reading and reacting to their feeds, thus earning the company more money.

Facebook gives weighted meaning to certain kinds of interactions, and one can understand the logic pretty well on some level. Imagine a meeting where someone says something like this:

But if someone actually quotes a post, and writes more than two sentences with capitalization and punctuation before sharing it (a signal that they especially care about how it's received) - should we assign more weight to that interaction? How should that transcend into the overall weight?

They were asking the right questions at each segment of the rather large problem they had chopped up to work on; the problem is they were thinking in a vacuum. Soon enough, people started writing entire books on how to exploit this behavior to basically get any message to echo as loudly as you want there.

Bonus points if it's unbelievable and presented as truth, because that just gets it even more friends of friends views and traction from people jumping in the fray to call bullshit, but inadvertently just propelling the post even further.

Internally leaked documents (AKA the Facebook papers) showed that employees had dire concerns over misleading and harmful content being amplified. Facebook, well, <gestures at Meta> remains ... mostly the same, while they talk about how hard they're working on it.