The default path to engineering management is promotion of the strongest engineer on a team. Sometimes that works. Often what transfers is a strong conviction about how the work should be done, paired with no framework for getting it done through other people — producing a manager who is still functionally the team's best engineer, and a team that has quietly stopped growing.
Hiring managers externally is not obviously better unless you know what you are testing for. Here is what I test, and how.
1. Can they be specific about someone else's growth?
The single most reliable signal. I ask: tell me about an engineer who got materially better while working for you. What were they like before, what did you do, what changed?
Strong candidates go straight to a person. They describe a specific gap, what they tried, what did not work first, and how they knew it had landed. Weaker candidates describe what their team shipped — which answers a different question entirely.
People who develop others remember the individuals. People who managed delivery remember the deliverables.
The follow-up matters as much: who on your team was hardest to help, and did you get there? An honest "no, and here is what I think I got wrong" is a much better answer than a tidy success story.
2. Do they disagree well?
I will state a position I actually hold, and push. Something like: most performance work should sit with the squads, not a central team.
What I am watching for is not whether they agree. It is whether they can hold a position under pressure from someone senior, concede the parts that are genuinely right, and stay specific rather than retreating into abstraction.
A manager who cannot push back on me will not protect their team from me either. That is not a hypothetical failure — it is how bad roadmap commitments get made and how burnout starts.
3. Do they think in systems or in incidents?
I ask about something that went wrong. Then: what made it possible?
The tell is where they go. Incident thinkers describe the fix. Systems thinkers describe the conditions — the missing test, the review that was rushed because of a deadline nobody pushed back on, the alert that had been noisy for weeks so everyone ignored it.
At scale, the second kind of thinking is the job. A director cannot inspect every decision; they can only shape what makes good decisions likely.
A question that works
"What do you do in your first month that you would not be able to do in your sixth?" Good managers know their outsider window is short and finite, and have a plan for spending it on the things that get harder to see once you have acclimatised.
4. Can they hold ambiguity?
Roadmaps at platform scale are permanently incomplete. Priorities move because the business moves. A manager who needs certainty before acting transmits their anxiety straight down to the team, and you feel it in retention long before you see it in delivery.
I probe this by describing a genuinely unresolved situation and asking what they would do on Monday. The answer I want is not a plan; it is a sequence of small, reversible moves that reduce the uncertainty. People who need the full picture first will say so.
What I stopped testing for
Current technical depth. It matters that they can hold a credible technical conversation and smell an unrealistic estimate. It does not matter whether they could still pass your senior engineer loop, and testing for it selects for people who have not yet let go of the IC identity — which is the exact failure mode you are trying to avoid.
Process knowledge. Anyone can learn your ceremonies. Nobody can be taught to care about the people in a fortnight.
The reference call
Ask one question of a former report: would you work for them again, and what would you want to be different?
The pause before the answer tells you most of what you need to know.