Experimentation

When I give presentations about teaching, I always urge the audience members to experiment as much as possible. It is hard to make improvements if you are not willing to try new things. I am always reminded of Albert Einstein’s definition of insanity: doing the same thing over and over again and expecting different results.

Some experiments work and some experiments don’t work. That is just the nature of the game. However, you will never find the winners if you are not willing to risk some losers. Playing it safe is no fun (and provides no benefit).

I tried an experiment with my final exam about two weeks ago. Even now, I am still not sure whether it was a winner or a loser but I found it interesting. I like the fact that I am still thinking about it.

What is the purpose of a final exam? I can think of two reasons. First, it gives the students one last opportunity to influence their grades. There is something about having hope for improvement that keeps students working until the end. Second, the final exam forces the students to review the material and, hopefully, get it better set in their understanding. In other words, they learn more.

I like giving my students an opportunity to improve their grades but my main reason for believing in final exams is that I really want them to leave the semester with all of the knowledge fresh in their minds. The final exam should encourage them to tie all of the material from the semester into a cohesive whole.

Unfortunately, I have often been disappointed in the results of final exams. Students seem overwhelmed by the huge amount of material and flit back and forth during their studies over the various topics without really getting a strong grip on any of it. They just don’t always learn as much as I want from their preparation.

So, at the end of my Intermediate Accounting II test this past semester, I wrote out 49 multiple-choice questions that covered everything that we had discussed that I thought was essential. I tried to gear each question to take about 4 minutes to solve. Although they were designed to be multiple-choice questions, I did not include any answers—just the questions.

Ten days before the final exam, I distributed these questions to my students along with the following speech: “Here is your final exam. These are the 49 questions that I would really love for you to be able to answer on the final exam. When you arrive for the final exam, you will have three hours to answer these questions. I will only make three changes from what you see here:

“1 – I will change the order of the questions.

“2 – I will add four multiple-choice answers to each question along with a “none of the above” answer.

“3 – Most importantly, for each question, I will change one or more of the variables in the question. For example, if the cost is $400,000, I might change that to $500,000. If the life is 5 years, I might change that to 10 years. If the interest rate is 8 percent, I might change that to 10 percent. If the blue method is used, I might change that to the red method. But the question will be fundamentally the same. If you can answer these questions, you should be able to answer all questions on the test.

“If you make sure you can work these 49 questions over the next ten days, you should make 100. But you must understand the problem so well that my changing of the variables will not really slow you down. I realize these are very difficult questions, but they cover the essentials that I want you to be able to work. You’ve got ten days to get these 49 under control.”

I quite honestly was not sure what was going to happen. In the end, the A students missed about 6 of the questions and got 43 correct. The B students missed about 13 and got 36 correct. The C students missed about 20 and got 29 correct. (The D and F students missed more, as you might imagine).

If I had given this test without the pre-test, I am convinced that most would have missed 50 to 100 percent more than they did. Students had clearly gone over the pre-test and learned to work many of the questions. They knew where to focus their attention. However, the number of missed questions was still higher than I had anticipated. Okay, these were 49 extremely tough questions about leases, pensions, cash flows, bonds, deferred taxes, and the like. But I really expected someone to become obsessed and learn them all backwards and forwards and make 100. That didn’t happen. Even with ten days, they just didn’t have enough time for that.

What interested me the most was that this type test had little impact on overall grades. Of all my students, the final exam grade made by 68 percent was within five points of their overall average for the semester. Students with an 82 average made about 82 and students with a 95 average made about 95. Only 32 percent had more than a 5 point difference between this test from their final average. I really had expected a greater number of students to show a greater change.

But, the basic question is still the same-did the students learn more in their studying? That was what I was trying to accomplish. And, I think they did that. Or, at least, I am encouraged enough to try it again. Maybe, this time with 40 questions instead of 49. Maybe, you just can’t do 49 complex questions in three hours even with a ten-day head start.

That’s my most recent experiment and how it worked. What was yours?