Our Guide to Creating Language-Agnostic Programming Interview Questions

May 27, 2022 • Interviewing

By Ken Swearengen

Our customers regularly ask us about the best way to tell if candidates will be able to perform well on their engineering teams.

One of the best pieces of advice we give when asked this question is: make your question more about business practices and less about skills with specific programming languages.

When you’re measuring their ability to answer a business case rather than something they could simply Google, you’re getting a better measure of their actual success on the job.

That’s why many companies use language-agnostic interview questions that allow anyone with programming experience to answer them in a programming language they feel comfortable with.

Pascal Zajac, Vice President of Software Engineering at BigCommerce, believes that allowing candidates to “show their skills in the language they’re comfortable with is crucial. It gives us greater confidence [that] we’ve seen their best self.”

Enormous benefits from going language-agnostic

Allowing candidates to choose their assessment language comes with a list of benefits, including:

Increased candidate pool – When you’re not just limiting yourself to Java engineers or React developers, you can drastically increase the number of candidates who can take – and pass – your assessment.
Improved candidate experience – Candidates will have a more pleasant experience when they can code in a familiar language rather than one they’re forced to use.
Better understand a candidate’s problem-solving process – When candidates can focus on solving the issue at hand rather than the nuances of coding in an unfamiliar language, you can better see how they can work through a problem.
Question reusability – You won’t need to rewrite the question for other languages.

Despite the benefits of language-agnostic programming questions, implementing them can be a little tricky.

So we’ll show you precisely what you need to do to make them work for you.

Elements of a good language-agnostic question

Some critical themes in the example above make this a good language-agnostic question:

It’s “generic” enough that a candidate can code it in any language – For example, API and data visualization libraries are available in just about every language.
It’s based on an actual business case – nothing turns a candidate off more than having to do a project that’s completely unrelated to the work they’d be doing on the job.
It’s not an algorithm question – algorithm questions are notorious for not really testing anything of importance. Instead, they just test a candidate’s ability to memorize a not-often-used piece of university-level trivia.
It explains why knowing how to do this task is important– in this case, it’s a standard feature request that directly helps our customers.
It emphasizes collaboration – By encouraging candidates to communicate issues they encounter, you create a greater rapport with them and improve the interview process. Additionally, the questions they ask will help clarify what areas they’d need help with if they worked with you and what parts of the question you could communicate better.
It has criteria for success – The candidate knows what they need to do to complete the task.
All the resources are readily available – This includes links to documentation, request/response code samples, or database schemas that a candidate would need to complete the task.

Language-agnostic programming interview question template with example

One of the best types of questions that lends itself to a language-agnostic format is the use of APIs to process and/or present data.

For example, at CoderPad, we have an API that allows users to communicate with different pad-related resources (like the pad itself, the question bank, and other data related to pad usage).

We regularly get requests from customers asking to view data related to their CoderPad usage, so for us, an excellent business-related question for our candidates looks something like this.

Give Context

Start with a brief introduction to the product or feature they’ll be working with. Remember you want your question to be relevant to your business. What’s the product or feature that the candidate will be interacting with?

The CoderPad experience revolves around the “pad,” the interactive environment that interviewers use to assess candidate skills. We try to make it as close to an actual on-the-job coding experience as possible.

Describe why and how customers use this feature. Put the candidate in your customer’s shoes – why is this product or feature important to the customer?

Our customers often request new ways to visualize the data of their pad usage to help them improve their recruitment efforts.

The assignment

Make sure you include the end goal of the assessment. What results should the candidate produce from their assessment? Pass some test cases? Create a good-looking UI? Process some data? Let them know here!

For this particular exercise, you will use the CoderPad API to determine the most used programming languages for an organization and create a simple chart to display that information in real-time.

Important points to keep in mind – you want to avoid any “gotchas” that may come across as cheap interview techniques. Are there any nuances of working with your product or code base that the candidate should be familiar with? This is the place to note that.

You can do this in any language you like. We also don’t care how fancy the chart or graph looks – the important thing is that it’s accurate and easy to understand.

Supporting information

Supporting information like helpful links and text. Are you using a database or file? Is there a helpful request/response JSON available for API questions?

You can find the API documentation – including how to access your API key – here. We recommend using the GET /api/organization/pads endpoint to access the language information for each pad.

Tips for successful completion of the task. Any other information that would help the candidate complete the task – like encouraging them to communicate with you!

We highly encourage you to ask questions on anything that confuses you during the project – we want this project to showcase your skills and our ability to help you do your best work as well!
We also recommend taking advantage of the Focus Time feature at the start of the project to help you collect your thoughts before the interview begins.
Good luck!

Reviewing

When reviewing a language-agnostic assessment, your goal is more to analyze the thought process rather than give a right/wrong judgment.

For example, with the question above, we’d look at the following:

Did a candidate decide to go for a manual refresh or a polling system to get data from the API?
If they utilized polling, how often did they set it to?
Did they take API rate limiting into consideration?
Did they decide to handle HTTP status codes of 400, 500, etc., for generic error handling?

Ask them about their decisions regarding these questions. Why’d they pick a manual refresh over a polling system? Why or why not handle HTTP status codes?

None of these are inherently right or wrong answers, but they provide insights into the candidate’s problem-solving skills and ability to rationalize their decisions. You can see what they sacrifice for their results to see if it aligns with your team’s values.

For example, if you’re running an eCommerce app, does the candidate prioritize speed, loading, and time to interaction? If your product has stringent design requirements, is the candidate building pixel-perfect UIs and using best practices for CSS and accessibility?

Use your team’s values and priorities to assess how the candidate’s priorities would fit with your team.

Summary

It’s time to start interviewing better. With over 30 languages to choose from, you can create a pleasant and customizable coding experience for your candidates with CoderPad.

If you’re looking for more interview guides, check out our list of interviewing resources: