Creating custom questions

Select an item from the list for more information on custom question creation:

Create a custom question in the question editor

To create your own custom question, click Question Editor in the left navigation menu.  

If this is the first question you have created, click Create your first question.

If you already have a list of questions, click New question in the top right corner.

In the New Question pop up, select a Question type. There are four options:

• Multiple choice question.
• Text question.
• Programming exercise.
• Project exercise.

Select the question language(s) and click CREATE.

Create multiple choice questions

• Input the Title (the candidate does not see the title).
• Write out the question in the Statement box on the left.
• Text options include common styling options, code inputs, LaTeX formulas, and adding images and attachments.
• On the right, input the answer options.
• Indicate the correct answer(s) with the switch.
• Select Multiple answers possible if there is more than one answer.
• Select Randomize order during the test to randomize the answer list order if required.
• The candidate must select ALL valid answers, and no other answers, to receive the maximum points available, otherwise marks are proportional.

Settings

Add the following details:

• The domain (i.e. technology or programming language) of your question where you can select an existing domain or create a new domain. 
• The skill to test.
• The question difficulty.
• Timer options.
• Total points available.
• Whether to add the question to automatically generated tests.
• Whether to authorize Screen to add this question to the Screen question library.

Click SAVE to save the question.

Create text questions

Input the Title and write out the question in the Statement box below.

Add the Settings as before.

Select (1) Automatic or (2) Manual validation in the Validation section.

1. Automatic validation

Select the skill and input the answer to the question. Candidate answers must match exactly, but are case insensitive.

✅ Check the Regular expression box and input a regex code that allows a more flexible range of answers. Check the Oracle Java Regex description for more information.

2. Manual validation

Input a Label for the report. Select the Skill and Weight. The Points value auto-populates.

After a candidate completes a test with a question that needs manual validation, you will receive an email inviting you to manually validate the answer to the question. The system then calculates a final score.

Create programming exercise

In the New Question pop up, select Programming exercise from the Question type dropdown list.

Next, select a specific Programming language.

✅ For multiple languages select custom question setting Multiple (Language-independent) at the bottom of the list. See the multiple language section below for more information.

The Question template dropdown contains existing Screen questions that you can use as a basis for your own.

You can edit the details of the existing question, including adding a zip file, before choosing the settings.

In Settings, input the points for the question.

✅ The domain field may have more options depending on the language selected.

If you want, you can add your question to the Screen library by checking the authorization box. Hover over the information circle to see the details:

You’ll also have the option to add custom files in the External dependency section. By clicking the Add external dependency button, you can add CSV files, text files, JSON files, or any other kind of files you’d like the candidate to work with.

⚠️ Your dependency files must be uploaded as `.zip` format.

Scroll down to see the Initial candidate answer code and the Initial candidate test code input boxes. This is the editable code your candidate sees at the start of the test.

The Code validator section stores validation code which runs against the candidate’s solution in order to assess it.

Underneath the Code validator is a Validator Mapping section where you can set the criteria for validation.

✅ The method field in the Validator Mapping must match the method name in the Code validator.

You can also try out a Possible solution and test your code. Click Preview to test the question:

You will see the question as the candidate sees it:

After testing and submitting, click Save to save the question.

Multiple language question

You can create a multiple language programming exercise by selecting the Multiple (Language-independent) option at the bottom of the Programming language list when creating the exercise.

Create your question by manually inputting the details as before, but without selecting any specific language content.

You’ll then need to fill out the starter code in the Default code generator, also known as the stub generator. You’ll use a special syntax to create a list of commands that will then be translated into starter code for whatever language the candidate selects. Select View documentation above the code generator window to learn more about the commands available to you.

Next, you will also see a section called Tests. Here, you can add tests that the candidate will be able to run during their assessment. These tests are visible to the candidate.

You also have the option of adding Validators, which perform the same function as tests but are not visible to the test taker.

✅All Multiple (Language-independent) programming exercises that you create are available to add to Language Independent tests.

Create project exercise

✅ Project exercises are usually longer than other kinds of questions.

Select Project exercise and click Create in the New Question window.

Input a title and a statement for your project question.

In Settings, choose difficulty, timer, and points:

Then, click + Add your project to upload the project information that serves as a starting point for the candidate. All file types are acceptable.

Add the Evaluation Criteria you want to use, then Preview and/or Save.

✅Project exercises will need to be manually validated.

Add translations

Whenever you create a new question, if you select more than one language, you will find tabbed areas for inputting the text in other languages.

Custom question video tutorial

For more information on creating custom questions, check out this Creating custom question with CoderPad Screen tutorial video: