What is the capital of North Dakota? This question is a factoid: you know it or you don’t. The Coen brothers may have prompted you to believe the answer is “Fargo”, but a quick Google search will confirm that the correct answer is “Bismarck”. This is a “first-order” question. Multiple choice exams in high school and college rely predominantly on this style of question. The exams generated by testing factoid knowledge are easy to create and easy to grade. With almost no room for interpretation, first-order tests are simple.
The use of first-order questions in medical school and graduate school is of little value. Professional students have an incredible capacity to memorize arcane facts and figures. For this reason, multiple choice test questions at the graduate level try to increasingly emphasize “second-order” thinking. What do people in North Dakota like to eat? This question requires a different level of understanding than simply memorizing a fact. The point, and certainly the hope, of second-order questions is that they can only be answered if you are well-versed in the subject. The question may require knowledge of the heritage (Norwegian), the weather (freezing cold), or the palate (nothing too spicy) of the region. These factors, taken in aggregate, may allow a test taker to understand why “Tater Tot Hot Dish” would be the correct answer. Second-order questions try to see if you can pass for a local.
The difference between first-order questions and second-order questions can cause a great deal of frustration for test takers. If you have had success on multiple choice exams that were predominantly first-order questions you may incorrectly assume that simple memorization is the key to success in medical school. This approach is fraught with problems. Second-order questions aim to highlight issues that can not be answered by simple factoids; test takers must be able to assimilate a large amount of knowledge and apply it to an unusual situation. First-order questions require the rapid-fire certainty of speed dating; second-order questions invoke the gentle understanding needed to navigate a lovers’ quarrel.
While second-order questions, it is argued, are more useful than factoids, there is a far superior way to evaluate an examinee’s comprehension. Having students explain concepts, whether in writing or through discussions, is a far better measure of establishing one’s mastery of material. Such “third-order” questions give no room for a lucky guess (just pick “C”) or obtaining pirated questions from an online question bank. Fortunately, third-order testing is gaining traction in medical education. Medical students must now pass a Clinical Skills exam that involves medical students actually talking to patients. Surgeons in many specialties must pass an oral exam (“Certifying Exam”) that involves discussing challenging clinical scenarios with a panel of experts. Such examples hint that medical education realizes the importance of communicating content mastery and offers hope that future doctors will not simply be multiple choice test automatons.
To complete the North Dakota example, a third-order question would involve asking someone to describe making Tater Tot Hot Dish. If the examinee is able to describe turning a bag of tater tots, 1 pound of ground beef, and two cans of condensed soup (cream of mushroom and cream of celery) into blizzard-soothing, soul-warming, post-call goodness, then they truly know something unique about the Peace Garden State.
Life Is Not A Multiple Choice Exam 
Leave a Reply