My OKR playbook: how I picked the right metrics for different kinds of products

Over the years I've worked on three very different kinds of products, and each one quietly forced me to throw out whatever metric playbook I'd brought with me. There's no universal dashboard. What you measure depends entirely on who's using the thing, whether they had any say in using it, and what you're actually trying to change about their day. Here's how I thought about all of them.

First: tools for front-line workers

These are the apps nobody asks for and everybody uses anyway. I'm talking about the service technicians who fix Coca-Cola machines at McDonald's and get guidance through their support app, and the various Starbucks apps that baristas tap through during a shift. The workforce doesn't get a vote here. They're going to use the tool whether they love it or hate it, because it's part of the job.

And the user base is huge, which means you can't just walk over and ask everyone how it's going. So the usual vanity metrics, like daily active users, hours spent in the app, or conversion rates, are basically noise. Of course people are in the app. They have no choice. Instead, you have to get clear on the one thing that matters: what task am I automating, or what process am I making better? Then you measure against that, and nothing else.

Take Playbuilder, a barista-facing iPad app I worked on. It looks at past sales, projects future sales by channel and product type, and then tells the partners on the floor what each of them should focus on to serve customers as quick as possible. For something like that, here's what I actually tracked:

• Time saved once partners started acting on the app's productivity suggestions. Saved time is saved labor, and saved labor is a dollar amount. That's the whole game.
• Paper process elimination. Every paper step you kill is a real, line-item cost savings.
• User sentiment, because even a captive audience tells you something when they grumble.

Second: tools for content creation and prep

The second bucket is the teams getting materials ready behind the scenes. Marketing teams pulling campaign assets together. Teams building menus for print and digital. They don't have a ton of choice about the tools either, but the upside is they're far easier to actually talk to. You can sit with them.

My favorite example is the marketers getting menus ready to print. It's a meticulous, slow process: design, marketing copy, coffee and food information, pricing that can swing wildly from store to store, and on top of all of it, the legal requirements. Every bit of that is labor. So the KPIs I'd push to improve were:

• Time to prepare menus for the next promotion, hours that translate straight into dollars.
• Reprint cost, the money spent reprinting when something goes out the door with an error.
• Features you can unlock, the things that aren't possible today but would generate revenue the moment you enable them. That revenue is your OKR.

That last one is worth slowing down on, because it's the one people forget. Here's a story from my Coca-Cola days. We built a deceptively simple feature that let us bill for smaller repairs that previously slipped through the cracks. Better invoicing, basically. That one little capability ended up generating a serious amount of money for the company. Nobody would've found it on a standard usage dashboard. You find it by asking what the product can't do yet, and what that's costing you.

Third: customer-facing applications

Then there's the third bucket, the apps your customers hold in their own hands. Think the Starbucks app. This is where you finally get to care about the ultimate business metrics, the ones executives actually lose sleep over:

• Conversion rate
• Average ticket
• Revenue per session and more

This whole tier is about making sales, plain and simple. But I'd also fold in a couple of happiness metrics, because a customer-facing app that converts today but annoys people will quietly bleed you tomorrow. So I keep an eye on:

• System Usability Scale (SUS)
• Customer sentiment

Fourth: platforms

Well... This is a separate conversation to have. Let's talk about it different time. https://www.veravolk.com/artic...

So how do you actually measure all this?

Identifying the KPIs is the easy part. Capturing them honestly is where the work lives.

Time savings usually breaks into two pieces. The first is process automation, the work that happens out in the world, off-screen. For that you do time studies in the field, measuring before and after, and you can also lean on self-reporting too, especially for corporate content teams. The second is system improvement, the time you cut from inside the app itself. Analytics is king here: Datadog, Mixpanel, Google Analytics, Application Insights, take your pick.

The easiest thing to measure, hands down, is physical materials savings, like less paper, fewer menu reprints, less coffee beans wasted. You don't need anything clever for that. Just get your hands on the purchasing data and the story tells itself.

Conversion rates and average ticket? Analytics again.

User feedback depends entirely on who you're talking to. For internal content tools, it's wonderfully simple: go talk to people. They're right there. For the broader retail and consumer audiences, where you can't possibly reach everyone, I like dropping survey tools right into the app. There's no shortage of good ones on the market.

Hunting for the data, hoarding it, picking it apart until it confesses something useful: that's where I shine. It started as a byproduct of my actual job, but somewhere along the way I got really, really good at it.