Building a Better Future for Data Teams with Live Technical Interviews

How should data teams work smarter in the future — and can better technical interviews help them do it? 

These were just some of the many thought-provoking questions I recently talked through with Boris Jabes of Census on his podcast, The Sequel Show. We covered everything from the overlap between interviewing and creating collaborative tech organizations to what a perfect KPI for data teams might look like.

Whether you’re a data analyst, an engineering manager, or are just interested in the changing face of interviewing, there’s something for everyone in this conversation. Here are some of the highlights.

Say goodbye to whiteboards and riddles. The future of technical interviewing is collaborative.

As CoderPad’s Senior Analytics Manager, I live and breathe technical interviews. Yet I think we’re constantly learning all that a live assessment can really do — for engineers, managers, and data practitioners alike.

Early on in our chat, Boris posed what might seem like a simple question: “What, exactly, does a technical interview encompass?”

Boris and I have experience with organizations of all sizes — ranging from Microsoft all the way down to seed startups — and there has been a clear evolution of tech interviews over time. 

In the past, interviewers might have simply assigned sets of questions with strict criteria for answers. (Until recently, some big tech companies even used riddles in their interviews.) Then came real-time whiteboard interviews. While these did help to get a sense of a candidate’s problem-solving style, they too often felt like a one-sided test.

Now, we’ve got something more useful. I outlined to Boris how, with CoderPad’s interview platform, we strive to remove bias and let candidates work in their own style. (That’s also part of why our live interviews support more than 30 programming languages.)

“The most useful part of a live technical interview is that — if done correctly — it can assess if a candidate matches the real, day-to-day responsibilities of a position. It puts interviewers and candidates on the same team. And with an analytics engineer, for instance, what better way is there to assess their true skills than by opening up a SQL pad and seeing what they can do?”

Most often, the person interviewing a potential engineer or data analyst isn’t destined to be their boss, but instead a collaborator. So what’s the point of making interviews feel adversarial? 

Interviewing is a two-way street. First, you need to decide if this candidate can do the job. But everyone involved is also assessing whether the person on the other end is someone they’d enjoy working with. Let’s face it: the old style of interviewing didn’t help anyone figure that out. 

Live interviews can minimize stress at scale.

During our chat, Boris noted a fascinating tactic that he uses for assessing candidates at his current company, Census. In order to get a sense of how a late-stage interviewee might fit in with the broader team, he organizes a built-in hackathon.

As he describes, this strategy reveals much more than just someone’s technical skillset. More importantly, “It’s about just working together. In a hackathon, we can’t hide who we are as a company and, hopefully, it’s clear by the end of it that we’re on the same side as a candidate.”

The reason I found this so compelling is that it offers an organic way to get around one of the biggest problems in modern tech interviews: stress. Building the time to develop a rapport with someone, joke around a bit, and see their skills outside the confines of an assignment is invaluable. I view this as especially key with data specialists, who often end up floating between teams and need to feel comfortable working with broad sets of stakeholders.

Unfortunately, Boris’s method is hard to implement if you’re hiring at scale. Conducting a hackathon in every interview is time-intensive—taking up from four hours to a full day. But the core benefit of this style–fostering a collaborative environment between the interviewer and the candidate–is exactly how we at CoderPad see the future of technical interviewing. 

A collaborative CoderPad interview lets engineers and data analysts approach problems in their own way across languages. I outlined the advantages of our system like this:

“Once we know someone is writing in R or Python or SQL, we can see what insights they produce and how they arrived there. This also mitigates bias from interviewers; with data from across hiring processes, it becomes less about seeing if a candidate answers how I, the interviewer, want them to. Instead, we can compare their process with other candidates.”

Now, collaboration can be baked in and all parties have something in common from the beginning. And it’s not just candidates who prefer this method; we’ve also gotten feedback from interviewers noting that interactive functions make prep far easier. Above all our hope is that, even if interviewing isn’t always fun, it doesn’t need to be painful. 

Make exploration part of your data team’s process.

When it comes to modern data teams, it’s easy to get caught thinking exclusively about day-to-day performance metrics. But Boris framed the deeper why? for data professionals in an interesting way:

“Data analysts have a choice to go to a consultancy, to work in-house, to go any number of directions. But I think what really draws them in is a huge opportunity to eventually answer questions that are currently fundamentally unanswered in the industry. That’s super exciting.” 

At CoderPad, we’re motivated by an ambitious goal — improving technical interviewing — but our data team is in a unique position to ask questions in real-time along the way. So what data points do we collect? Our primary KPIs might seem simple on the surface:

• How often are people using the product?

• Are they enjoying it?

Yet within these measures there are more interesting questions, chief among them, “What isn’t working?” And that’s where we can see a throughline all the way from interviewing to internal product development. Let me explain. 

When you compare data science to engineering, data’s focus lies more in hypothesis exploration.

This type of exploration requires you to prioritize active listening and take the time to evaluate the concerns of different stakeholders. Our models for data teams rely on unearthing as much context as possible — and that’s why it’s crucial to use live interviews to see how ICs interact with others and use the context they’re given.

To finish things off,  Boris and I examined a more abstract idea: the ultimate KPI for data teams. In his view, the most useful KPI would be the ability to measure an organization’s rate of learning. That might include how often you run experiments or even how receptive teams are to change — processes that can only be accomplished with great data teams.

This idea really got me thinking. As I said to Boris, considering the rate of learning would be a perfect way to assess an organization’s health at a macro level. That’s because what it describes “is necessarily a collaborative effort,” something I believe all companies can improve upon.

Want to dive deeper into the intersection of data teams and technical interviews? Take a listen to the full episode here.