What You're Actually Screening For When You Screen for 'AI Skills'

When you write "AI skills" on a job description, you are pointing at two different things and calling them one. The first is Fluency: what a candidate can make AI do. The second is Direction: whether they can steer it, catch it when it is confidently wrong, and override it. Fluency is now universal. Direction is rare, and it is the thing you were reaching for. This piece is about learning to see the two apart. Once you can, your whole screen changes.

Let me be fair to the phrase before I take it apart. "AI skills" is not a lazy line. It is a hiring manager feeling a real shift and reaching for the nearest word. Something in the work has changed, the old descriptions do not capture it, and "must have AI skills" is the honest attempt to name it. The trouble is that it names the easy half and goes quiet on the hard one. So you screen for the thing everyone has. You stay blind to the thing almost no one is tested on.

The short version:

• "AI skills" bundles two capabilities that behave completely differently. Fluency is what you can make AI do, and in 2026 it is table stakes. Direction is whether you can steer the output, notice when it is wrong, and override it. Direction is the real predictor.
• Screening only for Fluency sorts nobody. When almost every candidate can prompt a model into a clean draft, "can they use AI" is a question the whole shortlist answers yes to. A screen everyone passes is not a screen.
• Direction shows up nowhere on a CV. Wharton researchers found people followed AI's answer around 80% of the time even when it was wrong, and grew more confident doing it. Whether a candidate is in that 80% is exactly what you need to know and exactly what the résumé hides.
• The fix is not a harder Fluency test. It is a second axis. Once you separate what a candidate can make AI do from whether they can tell it where to go, the column you were always trying to hire for stands out on its own.

---

Fluency: what a candidate can make AI do

Start with the half that is easy to see, because it is the half your process already measures.

Fluency is operational competence with the tool. A fluent candidate knows the interfaces, has prompts they reach for, gets a usable first draft in seconds, and can chain a few tools together without friction. Watch one work and you are watching skill, because it is skill, of a kind.

Here is what has happened to that skill, though. It has become ordinary. A course teaches it in an afternoon. A teenager has it by default. Fluency used to be a differentiator the way knowing Excel was a differentiator in 1995. Then everyone learned Excel. Knowing it stopped telling you anything.

That is the state of Fluency now. Necessary, universal, and therefore useless as a sort. When I say "AI skills means nothing," which I argued in full in why "must have AI skills" means nothing, this is the half I am pointing at. Not that Fluency is worthless. That measuring it separates no one, because the whole room clears the bar. Ask a candidate "do you use AI in your work?" and every one of them says yes, confidently, with an example. You learn that they have met the tool, which you knew before they walked in. The question was theatre. Everyone knew their lines.

Direction: whether they can steer it

Now the half your process does not see.

Direction is what a candidate does with the output once the tool has produced it. It is a bundle of related capabilities, and they all share one shape: they are the ways a person imposes their own judgment on the machine's answer instead of accepting it. Can they tell where the output should go, not just what it says? Can they feel when it has drifted somewhere wrong? Can they catch a claim that is confident and false? And when the model gives them the obvious answer, can they generate a better one and override it?

None of that happens in the demo. It happens after. The demo is Fluency. Direction is the quiet work that starts the moment the clean draft appears and the candidate has to decide whether to trust it.

This is the capability that predicts whether a hire is any good. The value of Fluency is capped. Once a candidate can operate the tool smoothly, more smoothness adds little, because the draft was coming out of the tool regardless. The value of Direction has no cap. A candidate who steers well catches the expensive mistake and gets sharper as the work gets harder. On routine work the two look identical. On the work you are actually hiring for, they diverge completely.

Direction is also the thing AI cannot supply on the candidate's behalf, which is the whole reason it survives as a signal. Fluency can be borrowed from the model. Direction has to come from the person, because directing the model well requires knowing something the model does not. That is why a candidate can fake a portfolio, a take-home, or a writing sample, as I walked through in every hiring signal AI can now fake, and cannot fake the moment the answer in front of them is wrong.

Why the two get confused, and why it costs you

They get confused because on the surface they present as the same activity. Watch a candidate build something impressive with a model and you are watching Fluency, and it is easy to read the polish of the result as evidence of judgment behind it. It is not. The polish came from the tool.

The confusion has a cost, and it is a specific one. A fluent candidate with no Direction is not merely a weaker version of a good hire. They are a distinct and more dangerous kind, because they are fast, confident, and wrong in ways nobody can see coming. They ship the model's mistakes at the model's speed, with the model's certainty attached, and feel more sure the whole time.

This is not me being cynical about people. It is a measured tendency. In a Wharton study titled Thinking, Fast, Slow, and Artificial, researchers Steven Shaw and Gideon Nave ran three experiments with 1,372 people on reasoning tasks. Accuracy rose from 46% with no AI to 71% when the AI was right. Then, when the AI was wrong, accuracy fell to 31.5%, below the 46% people managed with no help at all, and they followed the wrong answer around 80% of the time, confidence climbing as accuracy dropped. Shaw and Nave call it cognitive surrender: the mind hands the thinking to the machine and borrows its certainty without checking whether it is right. We traced how that works its way into the actual quality of the output in cognitive surrender and output quality.

Read that as a hiring fact and it lands hard. Roughly four in five people, put in front of a confident wrong answer, take it. Direction is the capability that puts a candidate in the other fifth. Screening for Fluency tells you nothing about which fifth of the room a person sits in. It measures the wrong axis and calls it done.

Seeing the two axes clearly

Put them side by side and the picture stops being blurry.

Axis	What it means	What it looks like in a candidate	Does it sort candidates?
Fluency	What the candidate can make AI do	Fast first draft, knows the tools, chains them smoothly, answers "do you use AI?" with ease	No. Almost everyone now clears the bar, so it separates no one
Direction	Whether the candidate can steer the output, catch it when it's wrong, and override it	Reads the output and hesitates, checks the load-bearing claim, generates an alternative to the obvious answer, tells the model no	Yes. It is rare, invisible on paper, and the actual predictor of a good hire

The table is the argument in two rows. Fluency is the column your process measures and the one that sorts nobody. Direction is the column your process ignores and the only one that does the sorting. Every hiring manager who has written "must have AI skills" was pointing at the second column with a phrase that only describes the first.

What Direction looks like in the room

Fluency you already know when you see it: speed and comfort, a quick draft, easy talk about which model does what. Good signs, all shared by most of the shortlist, none worth weighting. Direction is the one you have to learn to spot.

It looks like friction, in the good sense. The candidate gets the clean output and does not immediately move on. They pause. Something in it does not sit right, so they check the one claim everything else is resting on. When the model hands them the obvious framing, they say "or we could look at it this way" and produce something the model would not have. Ask them about a time the machine was confidently wrong and they caught it, and they tell you exactly what felt off, what they checked, and what they rebuilt. The candidate who only has Fluency answers that same question with a story about how fast they shipped. One is describing judgment. The other is describing throughput and hoping you will not notice the difference.

That gap, what a person does when the model is confident and wrong, is the entire hiring decision, and it is the case I made in full in stop hiring AI users, start hiring AI Operators. A user operates the tool. An Operator operates the outcome.

Why screening only for Fluency sorts nobody

A screen works by dividing the room. That only functions when the thing you are testing is unevenly distributed. Test for something everyone has and the screen divides nothing, it just stamps the whole room approved and hands you back the same shortlist you started with.

Fluency is now evenly distributed. That is not a complaint about candidates, it is a fact about the tool. When "AI skills" means Fluency, and it almost always does, the screen you built to be selective has quietly become a formality. It feels rigorous because it mentions AI, which feels current. It sorts no one, because the capability it checks is the one the market already handed to everybody. And the demand keeps climbing: AI-related skills now appear in roughly 2.5% of all US job postings and rising. The postings multiply while the thing they are trying to screen for gets no easier to see.

To make the screen sort again, you add the second axis. You stop asking what a candidate can make AI do and start building a moment where the AI is wrong, then watch what they do. That is the one test Fluency cannot pass on Direction's behalf, because passing it requires the exact judgment you are trying to hire.

What to do with this on Monday

You do not need to tear up your process. Demote one thing and add another.

Demote every question that measures Fluency. "Which tools do you use," "walk me through your AI workflow" — keep them if you like, but score them near zero, because the whole shortlist passes and you learn nothing that sorts them. Treat the answers as confirmation the person is not a novice, and move on.

Add one thing that measures Direction. Hand the candidate a real task with an AI output already attached, one that is clean, plausible, and quietly wrong. Say nothing about the error. The candidate high on Fluency and low on Direction accepts the gift and builds on it. The one who has both gets an itch, checks the thing everyone else assumed, finds the crack, and rebuilds the part that was rotten. Same tool, same task, same twenty minutes, opposite outcome. You have just measured the axis your old process could not see.

That is the correction. Not a harder test of what candidates can make AI do, but a real test of whether they can tell it where to go.

Frequently asked questions

What are "AI skills," really? The phrase bundles two capabilities that behave differently. Fluency is what a candidate can make AI do — operate the tool, get a fast draft, chain a few models together. Direction is whether they can steer the output, catch it when it is confidently wrong, and override it. Fluency is now universal. Direction is rare, and it is the capability the phrase is reaching for without saying so.

What is the difference between AI fluency and AI direction? Fluency is operational: it is what happens while the tool produces the output. Direction is judgment: it is what happens after, when the candidate decides whether to trust the output, where it should go, and whether to override it. Fluency can be learned in an afternoon and is capped in value. Direction has to come from the person and compounds as the work gets harder.

Why doesn't screening for "AI skills" work? Because it almost always means Fluency, and Fluency is now evenly distributed across candidates. A screen only sorts a room when the capability it tests is unevenly held. When nearly everyone can prompt a model into a clean draft, testing for that ability passes the whole shortlist and separates no one.

How do I test for Direction in an interview? Build a moment where the AI is wrong. Give the candidate a real task with a polished, plausible, quietly incorrect AI output attached, and say nothing about the error. Watch whether they catch it. A candidate with Direction checks the load-bearing claim, finds the crack, and rebuilds. One without it builds on top of the mistake. The test works because faking it would require the exact judgment you are measuring.

Isn't the fluent candidate still a safer bet than one who's slow with the tools? Not necessarily. The slow candidate is a visible, manageable weakness — you can see the gap and close it. The fluent candidate with no Direction is an invisible one. Wharton's Shaw and Nave found people followed a confident wrong answer around 80% of the time, growing more sure as they got it wrong. A fluent hire in that 80% ships the model's mistakes fast and looks finished doing it.

Does this mean Fluency doesn't matter at all? No. Fluency is necessary — a candidate who cannot operate the tool at all is genuinely behind. It just is not a differentiator anymore, because almost everyone has it. Treat Fluency as a floor to clear, not a signal to weight. The differentiation, and the hire, live entirely in Direction.

---

Ivanooo built the AI Operator Profile to measure the axis your interview can't see: not what a candidate can make AI do, but whether they can direct it when it is wrong. If "must have AI skills" is on your job description, Direction is the thing you were trying to say.