Strategies and Tips Regarding Writing Multiple Choice Questions
What are multiple choice questions?
Multiple choice questions are questions usually used on a test or assessment where a question is presented with usually 4 or 5 options. Test-takers are instructed to choose the “best” choice among the answers.
Why are some advantages for multiple choice questions?
Multiple choice exams have been used for many decades in both the academic and non-academic worlds, with uses ranging from standardized testing to personality questionnaires. When thinking about whether you should be using multiple choice questions as part of your assessment arsenal, many questions will arise. One main question is the advantages of multiple choice questions, which include:
- Versatility: Multiple choice exams can be designed to test a variety of levels of understanding. Furthermore, MC exams can be extremely practical, allowing for many questions to be asked on a single assessment tool, as well as reducing the amount of resources needed for grading and statistical analyses.
- Reliability: Removes scorer reliability, and allows for the possibility of multiple questions to be asked on a given topic
- Validity: There has been a lot of evidence showing the validity of MC questions. The fact that it generally takes less time to answer a MC question than a written one allows for more questions to be asked, increasing the validity of the assessment tool.
What are some disadvantages of multiple choice questions
- Students have the ability to guess. A student who never studies will be able to perform at chance due to the nature of providing options and answers.
- Constructing effective stems and lures can be difficult, which may lead students to pick up subtle differences in the answers aiding them in their ability to guess.
- Exposing students to false information (lures) can distract them and cause interference, which may not have happened if you just asked them the questions and provided them space to answer
- Multiple choice questions do not allow for individual interpretation, as well as constricts the content to the context of the question
Possible misconceptions regarding multiple choice questions
1. Multiple choice questions only test rote memory
While rote memory is something that multiple choice questions excel at assessing, these types of questions also have the ability to test higher order thinking. It has been demonstrated that constructing effective questions and answers, you are able to test critical thinking, problem solving, and even reduce the ability of students to guess
2. Multiple choice questions are easier than written answers
There is anecdotal evidence (many times from students) that multiple choice questions are easier. Some of the reasoning behind it is that instead of having to think critically about the material, as long as you can recognize the correct key terms, you will be able to select the correct answer. However, this is a misunderstanding of multiple choice questions, and is generally reflected by poor multiple choice writing practice. These types of questions can be written to make them more difficult and to test the same amount of knowledge as if they were essay questions.
Strategies on how to construct effective multiple choice questions
1. Start with your own learning outcomes
When we construct any type of assessment, we always have to think about what our learning objectives are. Different assessment tools are better at getting at different types of learning. By understanding what you want your students to demonstrate, you are better able to pick the most appropriate assessment tool for the job.
2. Think about how you want to construct the question (the stem)
Stems are essentially the questions that students are given. Stems can be written in a variety of different ways, and these differences can produce intentional or sometimes unintentional outcomes. Some questions that might help with creating a statistically reliable and valid stem include:
- What knowledge/skill do I want my students to demonstrate by answering this question?
- Does my stem make sense? Will the language confuse students?
- What ‘level of processing’ do I want to address? Some questions may be geared to measure rote memory, while other questions might be aimed at measuring the students’ ability to synthesize or deconstruct an idea/argument.
By thinking about the purpose of each stem, you will be able to be more mindful of how you construct your questions. Some other strategies that might be helpful as you think about your stem are:
- Is the stem meaningful and does it propose a problem
- Is there irrelevant information? By removing as much irrelevant information, you reduce the amount of unnecessary cognitive load, which will increase the validity of the question.
- Does the language produce a barrier for English Language/Multi Language learners? Are there words that are not important that might be a stumbling block to non-native English speakers?
- Are you using negative statements? Negative statements can create unnecessary cognitive load (especially for English Language Learners). Negative statements should be avoided UNLESS the outcome requires it (for example: identification of hazardous materials might benefit from negative phrasing)
3. Next think about your alternative answers (lures)
Writing good stems is only half the battle when constructing effective multiple choice questions. The other half is reliant on writing effective alternative answers (or lures). One concern that many people have regarding multiple choice assessments are that there is a factor of guessing. By constructing effective lures, you minimize any advantage some students might have by just being “good test-takers”. Some strategies that might help in constructing effective lures are:
- Alternatives should all be plausible. Increasing the plausibility of the lures decreases the chance of getting the correct answer based on inferring information through context.
- Alternatives should be as homogenous in content as possible. This is very similar to the point above, which is that creating homogenous content reduces the ability to infer any correct answers based on question-content/structure and not course content.
- Alternatives should be stated clearly and in similar grammatical and syntactical styling. Having similar grammar reduces unnecessary cognitive load and reduces any advantages that students might get from inferring correct answers based on grammar and style.
- Alternatives should be mutually exclusive. Ensuring that alternatives are mutually exclusive reduces any confusion regarding which might be the correct answer.
- Avoid unnecessary complexity
4. Ask yourself what level of thinking you want to assess
As mentioned above, multiple choice questions have the capabilities of measuring not only rote memory, but also application and critical thinking. Rote memory multiple choice questions are usually the easiest to write, but sometimes thinking about how we can test higher level thinking with multiple choice can be difficult. Some strategies that might be helpful in constructing more higher level thinking questions are:
- Is the information provided in their notes? If so, this might be a more rote memory type of question
- Can the question be asked in a new way or in a different context? This might help with measuring students’ ability to apply their understanding of the material
- Does the question require the synthesis or comparison of multiple pieces of information? If so, this usually allows for the assessment of critical thinking or problem solving skills.
What about “all of the above” and “none of the above”
Researchers over the years have investigated the role of answers like “all of the above” and “none of the above”. Most agree that “none of the above” questions increase cognitive load (Tollefson, 1987), and don’t actually assess what the students know, but only test that they know the options are NOT correct. Recently, this research has also extended to “all of the above”, showing that only under very limited circumstances do answers like “all of the above” produce strong assessment validity (Paneerselvam & Callendar, 2016).
How do we test higher order thinking with multiple choice questions:
One of the main criticisms of multiple choice questions it that they tend to be perceived as shallower compared to short/long answers or even other forms of assessments (like essays or labs). However, researchers over the years have suggested that, through thoughtful construction, multiple choice exams can test higher order cognitive functions (Considine, Botti & Thomas, 2005; Morison & Walsh, 2004). Below are some ways of constructing multiple choice questions that test these higher levels of thinking:
- Creating questions that require more than one piece of knowledge
- Creating questions that require the integration across facts/ideas/lectures
- Creating questions that require application of an idea to a new or novel context
- Creating answers/lures that are not taken directly from text
- Creating answers/lures that require a high level of discrimination