Case Study

How Clio Created a Fair Hiring Process for the Modern, AI-Enabled Developer with CoderPad

Clio, the global leader in legal technology, sought to revamp its engineering hiring process to align with the evolving skillset demands of modern developers. By integrating CoderPad into its workflows, Clio not only enhanced the candidate experience but also improved the reliability and inclusivity of its technical assessments.

The Challenge

Clio’s previous technical interviews involved a coding exercise facilitated by screen-sharing the candidate’s local development environment over meeting software. This approach worked well for a number of years, but there were opportunities for improvement:

  • Solvable by an LLM: The exercise could be trivially solved by a large language model (LLM). The nature of the local development environment limited the upper bound of exercise complexity, inhibiting Clio’s ability to distinguish the developer’s coding skill from AI intervention.
  • False Negatives: The algorithmic nature of the exercise ran the risk of disproportionately filtering out candidates from alternative pathways, such as bootcamps, who were capable developers but who lacked recent formal data structure and algorithms education.
  • Binary Pass/Fail: The requisite simplicity of the exercise led to a binary pass/fail signal that didn’t provide interviewers with a nuanced understanding of candidates’ skill levels. As a result, it was difficult to gauge if a candidate would be successful in a differently-levelled role.
  • Lack of Depth: The exercise failed to assess critical on-the-job capabilities, such as understanding legacy code or adapting new requirements to existing constraints.
  • Interviewer Constraints: The screen-sharing setup restricted interviewers’ ability to interact with the code collaboratively, limiting opportunities for meaningful engagement.

The Solution

The new interview process, built on CoderPad, has been redesigned with an eye towards Clio’s values of “design for diversity” and “win and help win”. Namely, Clio seeks to screen candidates in, not out, by providing a consistent and equitable experience for everyone.

Leveraging the CoderPad IDE

  • CoderPad enables real-time collaboration, ensuring candidates and interviewers can work together on coding exercises seamlessly.
  • By eliminating the need for local setups, CoderPad provides a level playing field with its industry standard, configurable IDE.

Fostering Accessibility with Progressive Challenges

  • Gradual introduction of increasingly complex requirements allows candidates to showcase their best skills while reducing interview pressure. Candidates can score easy wins early on, which improves their level of confidence overall.
  • Replaced binary pass/fail outcomes with assessments based on the candidate’s ability to progress through requirements, offering improved insight into their skill level.

Empowering Candidates with Choice

  • Clio introduces multiple problem types designed to elevate candidate agency and better showcase skills from varied candidate backgrounds and expertise.
  • Candidates select problems aligned with their strengths, improving accessibility and enabling them to present their best selves.

Assessing How Engineers Think and Leverage AI

  • The exercise is intentionally designed to resist simplistic AI-driven solutions, requiring deeper critical thinking and contextual understanding.
  • Candidates are encouraged to use AI tools thoughtfully as part of their workflow, with the assessment focusing on their ability to guide and enhance these tools effectively.

The Current Process

Clio’s reimagined interview process with CoderPad is now a structured and consistent approach:

  1. Initial Screening: A talent acquisition specialist conducts an initial call to assess basic qualifications and fit.
  2. The Onsite Panel (Conducted Virtually):
    • Pairing Interview with CoderPad:
       A collaborative coding session where candidates tackle progressive challenges.
    • System Design Interview:
       A whiteboarding exercise focused on high-level architectural problem-solving.
    • Technical Leadership Interview:
       A session that evaluates retrospective technical decisions.
    • Culture Screen:
       A discussion to gauge values alignment and cultural fit.

The Impact at Clio 

Tom Heinan, Director of Engineering at Clio, shared reflections on the company’s evolving approach to hiring—one that places greater emphasis on skills rather than resumes. This shift reflects Clio’s commitment to creating a fairer, more effective process that identifies top talent based on ability, not pedigree.

Though still in its early stages, Tom is optimistic about the long-term impact of this reimagined process on hiring outcomes.

Our interview process is evolving. The role of a developer is changing, and so are the tools they rely on. With CoderPad, we’re laying the groundwork for a more inclusive, skills-based approach that allows us to tap into exceptional talent from a broader range of backgrounds and experiences.

CoderPad code interview all

Conclusion

Clio’s work with CoderPad demonstrates the power of reimagining technical hiring to reflect the evolving role of software engineers. By shifting the focus from screening candidates out to screening them in, Clio strives to maintain a fair, consistent, and inclusive process that celebrates diverse strengths and prepares the organization to meet the future of development head-on.