How do you measure whether one-on-ones are effective?

Dongyoung Kim, CPO

One-on-one effectiveness can't be captured in a single number. Look at three layers separately: how much they happen (frequency), how they go (conversation quality), and what they leave behind (follow-through), and take the team member's experience, not the leader's, as the standard. In practice, reviewing five metrics each quarter is enough: completion rate, team member experience, agenda ownership, speaking ratio, and action-item completion rate.

Why one-on-one effectiveness is hard to measure

This article is for HR people who, after rolling out one-on-ones, have to answer “Are our one-on-ones working?”, and for team leads who want to review their own team's. If you're still designing the program, read How do you roll out a one-on-one program? first.

First, leaders and team members remember the same meeting differently. In a SoapBox survey of just over 1,000 managers and employees in the US and Canada at the end of 2016, 75% of managers believed their reports left one-on-ones more motivated, but only 58% of employees said they did. On follow-up, 28% of managers said there was none, versus over half of employees (SoapBox, 2017). Leaders' self-assessment alone overstates the effect.

Second, conversations are confidential by default. The moment HR opens the content, team members stop speaking candidly. So look at operating data and anonymous experience, not content.

Third, the causal link between one-on-ones and performance doesn't show up over short periods. The same survey says it does not attempt to show the link between performance and one-on-ones. That's why you look at leading indicators (are good conversations happening?) before outcome indicators such as turnover and engagement.

Five metrics for one-on-one effectiveness

LayerMetricHow to measureWhat to watch for
Frequency1. Completion rateShare of scheduled one-on-ones that actually happened (excluding cancellations and indefinite postponements)Whether it drops only for certain teams or periods
Quality2. Team member experienceQuarterly anonymous survey, 3-5 items, 5-point scaleThe gap with leaders' self-assessment
Quality3. Agenda ownershipShare of meetings with at least one topic the team member addedWhether it has become a status check for the leader
Quality4. Speaking ratioHow much of the conversation the team member didWhether the leader does most of the talking
Follow-through5. Action-item completion rateShare of agreed items completed by the next one-on-oneWhether the leader's items pile up
  1. Completion rate is the baseline. If one-on-ones are the first thing dropped in busy periods, team members conclude the time doesn't matter to their leader. But a high completion rate with low scores elsewhere means one-on-ones are running only as a formality.
  2. Team member experience is the most important metric. Ask leaders the same items and compare the two scores, and you also see how accurately leaders see themselves.
  3. Agenda ownership shows whose time the one-on-one is. If team members never add topics, the leader is probably just asking for status. In the SoapBox survey, 58% of managers and 61% of employees said they don't share an agenda before one-on-ones, and an agenda was named as the best way to improve them.
  4. Speaking ratio is an indirect measure of whose conversation it is. There's no right ratio, but if the leader consistently dominates, it's a sign the conversation is drifting toward instructions and reporting. Recording-based tools show it without counting by hand.
  5. Action-item completion rate shows whether conversations lead to real change. Whether the leader's items (“I'll look into it”) get done affects trust more than the team member's. How to set them is in How do you set and follow up one-on-one action items?.

Ready-to-use one-on-one survey questions

Run it once a quarter, anonymously, and keep it under five minutes. Use a 5-point scale (1 strongly disagree to 5 strongly agree).

Core items for team members (5)

  1. My one-on-ones happen as scheduled, consistently.
  2. I can bring up the topics I want to discuss in one-on-ones.
  3. I can speak candidly about uncomfortable things in one-on-ones.
  4. What we agree in one-on-ones is generally done before the next conversation.
  5. One-on-ones help my work and growth.

Open-ended (choose one or two)

  • Which one-on-one conversation last quarter helped you most?
  • If you could change one thing about one-on-ones, what would it be?

Matching items for leaders: ask the same five items rephrased as “I think my team members feel that…”. The items with the biggest gap from team members' scores become the first topics for leader training.

How to review one-on-one operations

  1. Set a quarterly baseline. Don't judge the first quarter's numbers as good or bad; use them only as a baseline. Compare with your own last quarter, not with other companies.
  2. Look at team level. To keep individual answers from showing, combine teams with few respondents (for example, fewer than three) with the parent organization when you share results.
  3. Give results back to leaders first. Use them as a mirror for leaders' conversation habits, not as review material. Have each leader pick their lowest-scoring item as next quarter's improvement goal.
  4. Don't change the questions. Change them and you lose the trend. Keep the same items for at least a year.

What to watch when reading the results

  • Don't look at content. If HR opens conversation records in the name of operating metrics, the psychological safety one-on-ones depend on collapses. Announce in advance what will be measured.
  • Don't tie the numbers directly to leaders' reviews. Once leaders know they count, you get one-on-ones run just to fill in the record.
  • High completion with low experience scores means a quality problem. Check against the eight mistakes in Why do one-on-ones fail?.
  • High experience scores with low action completion means comfortable conversations that don't lead to change. Start by building the habit of agreeing on tasks and deadlines at the wrap-up.

Frequently Asked Questions

Q.When do one-on-ones start to show results?
There's no general timeline. Leading indicators such as completion rate and agenda ownership can change within a quarter. It's safest to look at outcome indicators such as turnover and engagement over at least a year.
Q.How often should we run the one-on-one survey?
We recommend once a quarter. Monthly leads to survey fatigue, and twice a year takes too long between spotting a problem and fixing it.
Q.Can the number of one-on-ones be a KPI?
If the count alone is the KPI, short, perfunctory meetings multiply. Keep the count as a minimum requirement and look at team member experience alongside it.
Q.What if team members don't answer the survey candidly?
State how anonymity is protected (who sees results, and at what level) in the survey invitation, and share what changed because of last quarter's results. Seeing their answers lead to change is what makes people answer candidly.

ORBLIT

How Orblit helps

To make it easier to put this article into practice, Orblit helps in these ways.

  • 01Metrics that build up without counting

    Record the conversation and you get an AI summary, action items (in progress or done), the leader-to-member speaking ratio, and feedback for the leader. Metrics accumulate as you run one-on-ones, with no separate measurement work.

  • 02Organization-wide completion at a glance

    The meeting dashboard shows one-on-one progress and member status across the organization, and once you set a one-on-one cycle, email and push notifications keep recurring meetings on track.

  • 03Content only for those who need it

    Data is separated by workspace and access is managed by role, so HR can see operating metrics while conversation content stays with those who need it.

Sources

  1. The One-on-One Study 2017 · SoapBox · 2017

Read next

How do you roll out a one-on-one program? Six steps for HR

Roll out a one-on-one program by setting its purpose and minimum standards, having executives and a pilot team experience it first, and then extending it company-wide. A single leader training and a distributed guideline are not enough: when things get busy, one-on-ones are the first thing dropped. HR's role is not to hold the conversations but to create the standards, tools, and learning that let leaders hold them consistently. Measure the effect by what changed after the conversations, not by completion rates.

Dongyoung Kim, CPO

Why do one-on-ones fail?

One-on-ones usually fail in preparation, conversation, and follow-up more than in the leader's goodwill. Walking in without an agenda, spending the whole time on status, and ending without commitments are the typical cases. Here are the signs that one-on-ones are stuck, eight things leaders commonly miss at each stage, and what to change right away.

Dongyoung Kim, CPO

How do you set and follow up one-on-one action items?

A one-on-one action item is the next step agreed at the end of the conversation, and one sentence should say who will do what by when. Set only one or two per one-on-one, and write down what the leader will do as well as the team member. Then spend the first five minutes of the next one-on-one checking last time's action items. Keep to these three and you will have far fewer “same conversation every time” one-on-ones.

Dongyoung Kim, CPO