By Amanda Richardson
A YC-backed founder called me last week with some good questions. His 20-person startup hadn’t hired an engineer in over a year. Now they’re opening reqs again, and instead of just dusting off the old loop, he wanted to understand how interviews had changed and if candidates were ready for AI interviews. His process — one coding challenge, one system design, one product conversation — is this still the right way to find great engineers? He’d started experimenting with new AI-focused interview questions and a few candidates seemed thrown by them. So he came to me with some questions: are candidates ready for AI use in interviews? Is this common? How do you craft this new process to reveal skills? Is AI just scaring good people off?
These are the questions every technical recruiter and dev leader is asking right now, whether or not you’ve noticed it yet. If your last technical hire happened before mid-2025, here’s the thing: the interview you ran back then probably isn’t wrong, but it’s almost certainly incomplete. And the dev leaders and recruiters getting ahead of this aren’t the ones who’ve figured it all out — they’re the ones asking, like he did, whether their process is actually surfacing the best skills for today’s market or missing the important skills all together.
What actually changed over the past year or two
Engineers mostly aren’t writing code in an IDE anymore. They’re prompting. Coding agents plan, edit across entire repos, and execute multi-step tasks — which means “watch someone solve a problem” now means watching someone direct a model, not type syntax from memory. The frontier companies have already moved: agentic interview formats are rolling out at Meta, Google and others. Full file-tree repos, real bugs, real feature requests, and a candidate who’s expected to prompt, orient, and debug their way through it — no algorithms-on-a-whiteboard or Cracking the Coding Interview nostalgia required.
That shift brings a lot of new questions. Lines of code, token counts, “did they finish fast” — is that a clean signal? Is a candidate who burns fewer tokens more skilled, or just more cautious? Is speed the differentiator now, or a trap? Nobody has fully agreed on the rubric yet, which is exactly why so many loops feel out of date: they’re still measuring for a world where output was the whole story.
The fundamentals didn’t disappear — they got harder to see
Here’s the part that should be reassuring: this isn’t really about AI at all. It’s still the same test – are you smart and do you get things done. AI tooling doesn’t replace that; it just adds a new surface where you can watch it happen (or not happen). The candidates who stand out are the ones who ask sharp questions, know when to trust the model and when to override it, and can still explain why the code works. The ones who flounder tend to flounder in a very familiar way — they just do it inside a prompt window instead of an editor.
And despite what the complexities might imply, most hiring decisions in practice aren’t close calls. It’s rarely choosing between two candidates – one an 87 versus an 85. It’s an 80 and a 20, and it’s obvious within the interview conversation. Don’t let AI theater distract you from that.
Do’s and don’ts for the AI-era loop
So if you’re brushing off your old process – what should you aim for?
Don’t run a single, generic “coding interview” and call it comprehensive.
Leading companies are now splitting technical rounds into distinct signals: one round with AI tooling explicitly not allowed (are the core skills actually there?), one round where AI tooling is required (can they direct a model effectively?), and a choice on a third interview where you can assess a candidate’s tool choice. Some even do a systems design interview layered on top. One interview can no longer tell you everything – you need multiple steps.
Don’t assume every candidate is fluent with AI tools by default.
Adoption is wildly uneven — plenty of strong engineers simply haven’t had access to the tooling. Testing for AI fluency ensures you are hiring the skills you need on your team right now.
Don’t confuse “nervous about unfamiliar tooling” with “not a good fit.”
A candidate who panics because they don’t know your specific agent setup isn’t giving you a skills signal — they’re giving you a UX signal. Take the time to prepare candidates on what to expect and ensure you’re getting interview tooling that looks like the tooling does in the real world.
Do decide deliberately whether you’re pinning every candidate to the same model or letting them choose.
And know what you’re actually testing either way. Model choice itself has become a legitimate interview question: does the candidate know what’s current, and can they reason about the tradeoffs?
Do give candidates a practice sandbox before the real interview.
This is the direct answer to that founder’s original worry — it means a “no” is because of skill, not because someone panicked over tooling they’d never seen. You’re not scaring off good candidates; you’re giving them a fair shot to show you who they actually are.
Do expect (and allow) your interviews to get shorter.
When orientation takes minutes instead of an hour, you don’t need 90-minute blocks to get a real signal — you need a few sharp questions.
The bottom line
Hiring is hard. It just got a new layer of hard, because the tools candidates use to do the job are now part of what you’re evaluating — whether you meant to test for that or not. The companies getting this right aren’t the ones with the flashiest AI questions in their loop. They’re the ones asking the same question that founder asked us: are we actually finding the best people who can work in the real world environment and tooling we use at our company? Optimize your tooling and questions. Get intentional about what each round is measuring.
If your team hasn’t touched its interview process in the last year, it’s worth a conversation before your next req goes out. Talk to our team at CoderPad — we work with everyone from frontier AI labs to Fortune 500 companies to start ups on their way to being the next top tech company on exactly this problem, and we’re happy to share what’s actually working right now.