How AI is turning every job interview into a coding interview

When Peter Grafe, the CEO of marketing firm BlueAlpha, brings potential hires into the firm’s offices for the paid multiday work trials, he runs them through the usual processes, skimming over their resume and asking them about their prior work experience.

But recently, he’s also started asking them to do something else: fire up Claude Code and build something.

That might be expected for software engineers at the company, but Grafe says even candidates for commercial roles are now expected to be open to using AI coding tools to automate routine work, speed up research, and execute tasks faster.

At BlueAlpha, applicants are given real assignments drawn from the company’s day-to-day work, asked to plan their approach, and then told to implement it. “The task they get, if you weren’t using Claude Code, for example, would be very hard to complete,” he said.

What Grafe is doing at BlueAlpha may well be at the vanguard of a shift in how we hire people — but it’s not unique. “We embed a practical project into every role we hire for, technical and nontechnical alike, and we’re explicit that candidates are expected to use AI tools,” said Jacob Bennett, CEO and cofounder of Crux Analytics. 

Instead of simply talking through strategy, some CMOs, investors, and operators are now being asked to use AI tools live — or during a tight take-home window — to create something in front of interviewers. A number of other firms do the same, while Nicole DeTommaso, a principal at Harlem Capital, says that anecdotally, she’s seen practically every potential candidate looking to join a venture capital firm being asked to show their prowess with AI coding tools.

DeTommaso wrote that one candidate was asked to build an AI agent that could produce automated research about industries within a working week that could reliably brief partners on a sector before they invested. Another needed to use the likes of Claude Code and Codex to vibe code a dashboard to show information about portfolio companies.

“You are not told which tools to use or how to go about it. You are just expected to figure it out,” she wrote. “And increasingly, what you can actually show in an interview matters more than what’s on your resume.”

Ironically, the push for applicants to prove they can use AI is a measure introduced in response to the torrent of AI-enhanced applications employers are fielding, Grafe said. “We put up a job description and within two days, we had 200 job applications,” he said. “Something like 95% were not qualified.”

Asking applicants to put up or shut up helps filter out some of those who have overexaggerated their experience using AI in companies like BlueAlpha, which has been an early adopter of the tech and has plans to give every employee a personal coding bot or agent. “We bought a bunch of Mac minis,” he said.

But it’s also be a way to figure out whether someone will be a good fit — and can use the technology that they’re likely to see in their day-to-day job. “You’ve got to get them in, in person,” he said, “whether it’s for a day or just a couple of hours. Because you really start to understand how they’re thinking.”

That’s also why Crux Analytics uses AI tests in its hiring process. “What we’re actually evaluating is less the output itself and more how they think about deploying these tools,” said Bennett. “Where did they use them? Where didn’t they, and why? The candidates who stand out are those who reach for AI in genuinely sensible places rather than just everywhere.”

So-called skills-based hiring is seen as a better way to separate the wheat from the chaff by employers, per a recent survey by software firm TestGorilla. The survey found that three in every four employers now use skill tests, with 84% satisfied that they’re making good hires using these tests. Some 71% of companies reckon that skills tests are a better predictor of hiring the right person than reading resumes.

BlueAlpha’s shift began about six months ago and has improved the company’s speed of iteration, said Grafe. AI is a force multiplier for smart, tactical people who can pair strategy with fast execution, but not a replacement for the judgement of his workers, he added. 

There’s mixed evidence on whether AI use actually makes people faster at their jobs. A 2025 field study by METR, an AI analysis organization, of experienced open-source developers found that allowing AI tools made them take 19% longer on realistic coding tasks, even though they expected the opposite.

That might explain the current vogue for showing off your vibe coding skills. But whether it’ll stick as a useful demonstration of how well someone might fit into a company and usefully contribute is yet to be seen, according to long-tenured experts who have analyzed how companies tick, including Cooper.

“This all depends on context and the sort of role you are hoping to fill,” said Stefan Stern, a visiting professor in management practice at Bayes Business School. “Clearly some ‘day 1 AI effectiveness’ is going to be needed in some jobs. But in others, new skills could be taught and acquired.”  

Chasing the current trend for vibe coding might help capture interesting talent for the here and now, reckons Stern, but “it would be a very confident employer who can say now what specific AI competence they are going to need from people even six months from now,” he said. As he pithily put it, “Often when hiring, attitude is a more important consideration than today’s aptitude.”

Cooper worries that a myopic focus on AI prowess could end up capturing people who are fully Claude-pilled, but “they may discard somebody who would be actually excellent to do the job and are prepared to learn,” he said. “To have that as one aspect of recruitment is a good idea, but not to put that as a top priority.”

Grafe is all too aware of overreliance on a single data point when deciding who to hire and who not to. But he also thinks that more hirers will inevitably follow in his footsteps of testing people’s comfort levels with AI coding tools before they end up in employment — mostly because he thinks it’ll be inevitable that every employee uses AI. “Ultimately, all these AI tools don’t give you another brain,” he said. “You still need to think about your own things… but they do give you speed.”