learnability matters

Every design has some basic thumb rules and guidelines. The objective of this article is to quickly jot down the guidelines for creating assessment questions in an elearning program. Remember, these are just tips and not rules. Each company has different versions of these rules. Here’s a quick list of what we follow.

Assessments are learning checks that track the learner’s progress. This can be in the form of exercises, practice sessions, quizzes, activities, evaluation, pre-test questions, check your understandings, and post assessment questions. It is important to frame assessment questions to test the learner’s progress. What role do assessments play in the learning process?

• First and foremost, assessments make learners think! They ensure that the learner continuously interacts with the content.
• Assessment questions stimulate the learner’s brain. The attention span of an average human is 8-10 seconds. Every 15 minutes, introduce an activity to maintain interest. This also acts as a breather for the learner.
• Challenging and innovative assessments increase the learner’s motivation to proceed further.
• Assessments reinforce learning. The learner is made to think about what he/she has just read. Attempting an exercise ensures increased retention rates.
• Assessments break myths and misconceptions. If the learner has formed an incorrect relationship between two concepts, assessments help clarify this.
• Assessments boost the learner’s ego. Assessments are designed such that the learners feel good about themselves.

There are several types of assessment questions. Some of these are multiple choice questions (MCQs), multiple multiple choice questions (MMCQs), drag and drop, match the following, true or false, fill in the blanks, simple games, crosswords, and so on. When developing assessments or exercises, it is important to ensure the following:

1. Assessment questions should be directly linked to the learning objectives and the enabling objectives.
2. It should have at least one exercise for each objective.
3. The assessments questions should be relevant and within in the defined scope.
4. Remember to use the Bloom’s taxonomy while framing assessment questions. If the objective reads ‘Identify the causes of global poverty.’, remember that the learner has to identify not apply this information.
5. Always provide clear instructions about what you expect the learner to do. Else, ensure that the affordance is good and the scent of information is strong.
6. The questions stem should be provide information required for the learner to complete the exercise and not hints or clues.
7. Design a realistic question stem with plausible options.
8. Do not time the exercise. This only adds to the pressure. Let the learner take the exercise in his/her own sweet time.

Given below are examples showing how the objectives can be converted into assessment questions.

Guidelines for MCQs:

• Avoid responses that are interrelated. One option should not be similar to another.
• Avoid negatively stated items: “Which of the following is not covered under child psychology?” It is easy to miss the negative word “not.” If you use negatives, make the negative qualifier bold to ensure that the learner sees it.
• Ensure that all responses are equal in length. Avoid making the correct answer longer than the incorrect answer. It becomes a give-away.
• Avoid the use of “none of the above” and “all of the above.”
• Avoid repeating words used in the question stem in the responses. Using the same word in an option will ensure that learner select that specific response.
• Use plausible and realistic distractors. The distractors should belong to a family. If it stands out, it would be a give away.
• Create grammatically parallel items to avoid giving away the correct response. For example: In four options, do not start three of them with verbs and one with a noun.

Lets us look at some guidelines for True and False:

• Do not use definitive words such as “only,” “none,” and “always.” Learners may choose false. Also, avoid uncertain words such as “might,” “can,” or “may.” Typically, the learner ends up choosing true. We do not want to mislead the learner.
• Do not use negatively stated items because they are confusing to interpret. “Thomas Jefferson did not write the Declaration of Independence.” True or False?
• Learners have a tendency to choose “true,” therefore, design at least 60% of question to be “false” to minimize guesswork.
• Use precise words (100, 20%, half) rather than vague or ambiguous language (young, small, many).

Given below are some guidelines for Drag and Drops:

• The question step should very clearly state the headings under which the options can be placed.
• The place where the options can be dropped should be visible.
• Drag and drops should always have a Reset button.
• Once the options are dropped, we should be able to reposition them.
• The options should belong to the same family. For example, if the are descriptions of steps followed in a process. All options should indicate this.

If you noticed, we have not included key-ins. We typically avoid key-in questions as they are not challenging to the learner and it is extremely difficult to identify whether the learner is correct or incorrect. If the learner types carbon carbon double bond instead of carbon-carbon double bond, it would show the answer as incorrect. Or, if the learner makes a spelling mistake (example ‘manger’ instead of ‘manager’). This is not fair to the learner and may exasperate him/her.

Some interesting ways of forming assessments:

1. Create multi-level assessments: Assessment questions can be designed such that there are series of questions at three levels: knowledge level, comprehension level, and application level.
   1. Knowledge level : Given below are a few images. Drag and drop the images of oils and fats under the respective headings.
   2. Comprehension Level: Chipmunks and ground squirrels hibernate during winter. How do they survive without food for long durations?
   3. Application level: Oil is used as a raw material in the manufacture of soaps. Identify the oils that are used to make soaps. Also, prepare soaps using an oil in your laboratory. Share your finding with with your friends by comparing your soap with those prepared by others.
2. Create scenario-based questions: An assessment question can provide a brief scenario based on which the learner has to respond. The learner uses the information provided in the scenario to answer the question.
   • Scenario: Gautam’s goal setting style
     • Arranges a meeting with Vishnu, Rahul, and Harveer at Cafe Coffee Day
     • Briefs the team by sharing all details to get maximum buy-in
     • Discusses the project plan and outlines the key deliverables
     • Explains the interdependencies with Sidhant’s team members
     • Listens to suggestions on the project plan and suggest that he would discuss the same with Vikram and Sidhant
     • Receives a request to revisit the timelines but he suggests that the dates are non-negotiable
   • Question: What could Gautam have done differently?
     • He should have showed flexibility with regards to the timelines.
     • He should have explained each team member’s goal.
     • He should have held the meeting in a formal setup.
     • He should have mailed the project details beforehand.
3. Create innovative assessment questions: Assessments can be designed with creativity (will blog this in detail soon). Based on the objective of the assessment, you can design the assessment question. The objective of the following assessment was make the learner aware of the several types of responses to conflict situation. The exercises involves a series of questions that the learner answers based on his/her logic. At the end, based on the typical responses made, the learner is informed about the category under which his/her basic responses fall. Unlike other assessments, there are are no right or wronger answer. Given below is an example of a situation.
   • Introduction: You may have encountered numerous conflict situations at your workplace. At times, you may have to respond to certain situations. Given below are certain complaints from your team members. Reflect on the situations and choose your response from the options provided.
   • Situation: The situation between Anu and Sheila has gotten worse. They have both complained to you as you are their reporting manager. The entire team is not able to focus on work because of this feud. Anu and Sheila are in your office and they insist that they longer wish to work in close quarters. How will you respond to this?
     • “Anu, you will be assigned to another team. Hope you feel better now.”
     • “I do not want to get involved in this cat fight. Both of you should sort this out.”
     • “Let me see if we can all discuss and solve this problem.”
     • “I don’t want to hear another word about this issue. Get to Work.”
     • “Julie, you can work with me till I arrive at a solution.”

Hope these tips were useful! As I mentioned, these are only tips and not rigid rules. Within this, we encourage our writers to be innovative. If you have any tips that you wish to share, feel free to do so.

Share this