Using OKRs

I use OKRs as the mechanism for team’s to encode their quarterly goals. Here’s a guide on writing good ones that can be measured.

Objectives

Objectives are aspirational goals that tie closely to a key business, product, or tech objective for the organization.

Objectives are frequently multi-quarter goals.

Good examples:

• Redesign product to delight our customers
• Drive internal adoption of Town Hall live-event communications
• Achieve cross-platform parity across Windows and Mac for backend development
• Migrate infrastructure to the cloud
• Lay a scalable foundation for Machine Learning
• Modernize web client tech stack

Examples that could be improved:

Leverage ML platform in production for sending daily update emails This is pretty specific and too prescriptive for an objective (might even be too prescriptive for a KR), unless the end game is using ML for emails. If the goal is actually to improve the daily update emails (independent of ML), the Objective should be more along the lines of “Significantly improve daily email or better something about the actual goal of “Drive engagement through daily email”. If the goal is to eventually productionize ML, and this is a necessary step, the objective could be something like “Validate ML platform to improve engagement”, etc

Help active and inactive users discover conversations happening within their network This is ok, but something more direct like “Improve discovery of conversations within a user’s network”, which will naturally lend towards KRs that can quantify that impact

Key Results

Each objective contains multiple key results. Key results are quantifiable outcomes whose achievement will help realize the linked Objective.

A Key Result is not merely a task.

A Key Result is NOT something you do. It is not an activity. It is something that happened because of the tasks that the team completed. Frame the KR in terms of the business or user value you expect to deliver.

Note: Key results need to complete within a quarter.

Each Key Result must have a metric (can be binary) and be quantifiable. (Use S.M.A.R.T. goals!)

Good Examples:

• ML-powered email experience showing 10% lift in MAU vs. control
• 50% of all web API traffic running through Azure Front Door
• Team-wide Dogfood of Rebar Project Milestone 1 available on iOS and Android

When writing a KR, you should be able to answer “YES” to all of the following questions:

1. Will it be completely unambiguous to all parties at the end of the quarter if the team succeeded or failed to achieve the OKR?
2. Does the Key Result require non-trivial effort (and won’t happen naturally)?
3. Did you write the KR with enough flexibility to change course and still make progress towards the objective and fulfill the spirit of the OKR? For example, you might decide to change technologies partway through the quarter or find a different way to hit a key metric (e.g., realize that you can achieve your goals with a different product experience than you originally envisioned at the beginning of the quarter). You don't want the KR to be so prescriptive that you don't have room to accommodate some changes to plan.
4. If you achieve the KR, will you make progress towards the objective?
5. If your KR has business metrics attached, have you accounted for enough time in the quarter to measure the impact of the work?

Examples of KRs that could use improvement:

A Data Scientist can add a new feature without a lot of data engineering assistance

This is hard to quantify (see rule #1). Better would be something like: “with less than N hours of data engineering team effort” or “X% reduction in time for data engineering to add new features for Data Science”

Leverage weekly discovery algorithm to rank threads in daily digest emails

This could use reframing, e.g. “Weekly discovery algorithm powering ranking of threads in Daily Digest emails rolled out 100% world-wide”. However, it might be running afoul of #3 as well; depending on the objective, we might not want to tie so closely to specific algorithm and specific email in the KR to give the team more flexibility to course correct, e.g. “ML algorithm powering ranking in at least one engagement email rolled out 100% world-wide”. Better yet would to include some expected improvement on metrics if we had confidence in a baseline.

Scoring Criteria on Key Results

At the end of the quarter, it is critical we ambiguously know if a team has met their commitment or not. To help clarify this, we want to specify the exact target for the team. At the same time, we want to leave room to accommodate stretch. OKRs typically are scored on a 0-1 scale and 0.7 is considered success, so we will borrow that here:

• Achieving >= 0.7 is success on the OKR. 0.7-1.0 can be used for stretch goals when appropriate.
• Achieving < 0.7 means we did not meet the OKR

Unless specified otherwise in the KR itself (e.g. “by Nov 1”), the timeline for delivery of each Key Result is at the end of quarter.

Make sure the KR text and the 0.7 score are aligned so it is easy to understand the target at a glance.

Examples

KR: ML-powered email experience showing 10% lift in MAU vs. control

How to score: linear, 0.7 = 10%

KR: 50% of all production web API traffic running through Azure Front Door

How to score:

• 0.3: 1% of all web API traffic
• 0.7: 50% of all web API traffic
• 1.0: 100% of all web API traffic

KR: Team-wide Dogfood of Rebar Project 1 on iOS and Android

How to score: [link to the Rebar project doc which lists P1s / gates to ship]

• 0.7 = Team Dogfood
• 1.0 = Company-wide dogfood

KR: 100% turndown of legacy data center

How to score: Binary (we want to finish this completely to get credit).