menu Documentation menu

AI Fluency environment

The AI Fluency environment is a CoderPad Interview environment for assessing how candidates use AI to complete realistic, non-coding work. Candidates work in a shared document-style workspace with built-in AI Assist, while interviewers can observe how they prompt, iterate, evaluate AI output, and apply judgment in real time.

Use AI Fluency for roles where the day-to-day work involves writing, analysis, planning, customer communication, or decision-making with AI rather than writing code. Common use cases include non-coding roles like sales, marketing, customer success, support, and operations. 

AI Fluency is best suited for live interviews, where the interviewer can observe the candidate’s workflow and discuss or ask follow-up questions in real time. It can also be used for take-home exercises.

Build an AI Fluency question

For the best candidate experience, create the question before the interview and preload the materials the candidate will need.

A strong AI Fluency question should include:

  • Task instructions — what the candidate should do and what they should create.
  • Business context — the role, scenario, customer, campaign, or problem they are working with.
  • Reference materials — docs, data, or files the candidate should reference or analyze for the task.
  • Time guidance — how long the candidate should spend on the task.
  • Evaluation guidance — what interviewers should look for during the session.

Avoid starting candidates from a blank document. AI Fluency works best when candidates have realistic materials to review, synthesize, and use in their work.

Example: for a BDR interview, you might provide a fictional account summary, CRM notes, campaign performance data, and product context, then ask the candidate to draft personalized outreach using AI Assist.

Supported files

AI Fluency currently supports:

  • CSV files
  • Markdown files
  • HTML files (generated by AI assist)

Excel files are not currently supported. If candidates need to work with spreadsheet-style data, convert the file to CSV before adding it to the pad.

Create an AI Fluency pad

To create an AI Fluency pad:

  1. Go to the dashboard.
  2. Create a new pad.
  3. Open the languages menu.
  4. Select AI Fluency under Nontechnical.

You can use an AI Fluency pad as a standalone non-coding interview, or as part of a broader interview process that also includes coding pads.

Run an AI Fluency interview

During a live interview, the candidate and interviewer join the pad using the interview link.

The candidate can:

  • Read the task instructions.
  • Review the provided files and reference materials.
  • Use AI Assist to ask questions, plan, draft, edit, or critique their work.
  • Use AI to generate single file HTML dashboards.
  • Write their final response in the document editor.

The interviewer can:

  • Observe the candidate’s document edits and AI usage in real time.
  • Ask follow-up questions as the candidate works.
  • Use built-in video calling andEnable AI notetaker to generate notes from the conversation.
  • Take private interviewer notes that are hidden from the candidate.

Everything the candidate does in the pad is captured for review, including prompts, AI responses, document edits, and comments. Video calls are not recorded.

Use AI Assist

AI Assist is built into the pad, so candidates do not need to leave the interview environment or use a separate AI tool. The candidate can choose between various leading models. 

The AI Assist panel supports:

  • Ask — ask the AI a question or request without directly modifying the document.
  • Edit — have the AI make edits to the document or generate single file HTML dashboards
  • Plan — have the AI help structure the work before writing.

Because AI Assist is embedded, reviewers can see how the candidate used AI during the session instead of only reviewing the final work.

Review the session

After the interview, use playback to review the candidate’s work and AI usage.

Reviewers can inspect:

  • Candidate prompts.
  • AI responses.
  • Document edits.
  • Private interviewer notes, if available.
  • AI notetaker notes, if enabled.

Use the review to evaluate both the final output and the candidate’s process. For example, reviewers may look at whether the candidate gave AI enough context, verified AI output, improved weak responses, used source materials accurately, and applied good judgment before finalizing their work, plus the quality of the final product.

Interviewers can use their private notes and the playback record to complete the ATS scorecard after the session.

Best practices

  • Create the question before the interview and preload the files the candidate needs to reference.
  • Use realistic, role-specific tasks instead of abstract questions about how the candidate uses AI.
  • Give candidates enough context and supporting documents to show how they work with information, not just how they prompt.
  • Tell candidates what their final work product should be, but leave room for them to decide how to use AI.
  • Evaluate both the final work and the process behind it: prompting, judgment, verification, editing, and communication.
  • Create a scoring rubric in advance and use it for consistent evaluation and comparison. 
  • Use AI Fluency for an interview round where candidates already complete a scenario, work sample, or role-specific exercise.