Understanding the advantages, disadvantages, and pitfalls of different types of assessment questions
Intended Learning Outcomes
- Understand the advantages and disadvantages of different types of assessment questions.
- Create assessment questions that are clear and concise, and engage different levels of student cognition.
Facilitation Notes
Begin with a discussion of the purpose of assessment. You can do this exercise as a free form discussion or use an anonymous polling platform. Review the concept of formative vs. summative assessments.
Introduce the process of question design as one where the teacher needs to define what they want the student to do (i.e. what skill should they demonstrate). A helpful framework for this discussion is Bloom’s taxonomy, which helps to classify different levels of cognition. Emphasize that all levels of cognition are important ranging from foundational knowledge to high level processing. Verbs associated with Bloom’s taxonomy are extremely helpful in guiding question writing.
Ask folks to brainstorm questions they could ask surrounding a given topic. The topic provided here is viruses, but you can modify this to fit your needs. Invite TAs to pick a level of Bloom’s taxonomy and write down a rough idea for a question related to the topic. Provide a couple examples before sending them off to brainstorm. Invite as many folks to share back with the group as are willing.
Next, explore different question types using the slides provided. Emphasize the advantages and disadvantages of each type and highlight the tips for construction (excerpted from Teaching at Its Best: A Research-Based Resource for College Instructors by Linda B. Nilson). There are also examples given for each, which you may use or modify to fit your field. Consider emphasizing the importance of streamlining questions to reduce cognitive load on students, which is a common theme throughout all question types.
- Multiple choice example: Have folks compare and contrast the two versions of the question as they are written. What is good and bad about each? How could you modify the question to be more descriptive and streamlined? (source: Vanderbilt University, Center for Teaching Writing good multiple choice questions by Cynthia J. Brame)
- Multiple choice pitfalls: Ask folks to identify pitfalls in the examples shown (or others of your choosing). Encourage TAs to eliminate throwaway options, remove options that are subsets of others (and instead implement a select all/checkbox format), and change the question type to short answer if options are too long or nuanced to be captured in a multiple choice question. (source: Vanderbilt University, Center for Teaching Writing good multiple choice questions by Cynthia J. Brame)
- True false example: Three progressively better options are provided for making this example more clear and cognitively challenging. Consider presenting them one at a time. The first question is somewhat ambiguous (what reach are we talking about?) and has a problematic verbal cue (“always”). Improve this by adding specificity and asking students to promote analysis over strict recall, achieved in question 2. Increase the challenge for students further by adding multiple statements that they have to evaluate (question 3).
- Short answer example: Draw an example from one of the courses in the department that one or more of your TAs teach. Make sure to include the prior knowledge that students would be expected to have in answering the question.
Finally, leave time (at least 15 min.) for TAs to design and receive feedback on a question that they draft. They may do this on any topic of their choosing. Peer feedback in pairs typically works best for this activity so that there is a lower barrier and plenty of time to share.
Close with additional tips for question design and an Exit Ticket that prompts folks to think about what they are taking away and what they still have questions about.
