EVERYONE CHANGES OVER TIME

Happy holidays to all the teachers out there.   This blog just went over 67,800 page views.   I have been thrilled throughout the year to have the chance to interact with so many wonderful educators.  

**

When I give teaching presentations around the country, I am often asked how my teaching has changed during the past 42 years.   Because I stress working for 5 percent improvement each year, that particular question is certainly a legitimate one.   Invariably someone will jump up and ask:   Okay, how are you managing to improve over time?

I actually have a couple of different answers for that question.   But, there is one response that I always give:   During the past decade, I have come to spend a lot more time writing my test questions.   I used to throw tests together hurridly with one goal:   to be fair.   Now, though, I view testing as a much more important element of my class environment, one that requires a significant amount of preparation time. 

I occasionally argue in this blog that the way you test students is the way they will learn.   No matter what you tell them, if you test memorization, they will memorize.   If your tests are purely mechanical, they will only learn the mechanics.   If you only test at a superficial level, they will only learn material to that same shallow depth.   I believe this is the absolute truth.   The type of tests you create has a huge influence on the type of learning that the students do.

I am always shocked by how many well intentioned faculty members turn testing over to a textbook test bank.   I want to run screaming into the night when I hear that.   In my opinion, an overworked graduate student who does not know you or your students is not in any position to write a legitimate test for your students.   When writing this blog, I sometimes discuss what I would do if I were king of education.   Burning all test banks would be one of my first royal acts.

Yes, I know you are extremely busy.   But abdicating this valuable task to a person who might never have taught a single class (or a class like yours) makes no sense.   Any test in your class should be designed for your students based on what you have covered and based on what you want them to know.   It should not be composed of randomly selected questions written by some mysterious stranger.   To me, using a test bank is like asking Mickey Mouse to pinch hit for Babe Ruth.    You are giving away an essential element of the course to someone who might not be up to the task.

Over the decades, I have worked very hard to learn how to write good questions.   During those years, I have written some questions that were horrible.   But, I have learned much from that experience.  

--The first thing I learned about test writing was that a question that everyone could answer was useless.  

--The second thing that I learned was that a question that no one could answer was also useless.  

As with any task, you practice and you look at the results and you get better.   You don’t hand off an essential part of your course to a test bank.

As everyone who has read this blog for long probably knows, one of the things I started doing about 8 years ago was allowing students to bring handwritten notes to every test.   That immediately stopped me from writing questions that required memorization because the students had all that material written down and in front of them.

That was a good start but that was not enough.   Allowing notes pushed me in the right direction but it did not get me to the tests I wanted.   It takes practice and study.  

About 3 weeks ago, I wrote a 75 minute test for my introduction to Financial Accounting class here at the University of Richmond.  This test was the last one of the semester (prior to the final exam).  By that time, I surely believed that everyone in the class had come to understand what I wanted them to accomplish.   So, I wanted to test the material in such a way as to see how deeply they really did understand it.  

I wrote 12 multiple-choice questions designed to take about 4-8 minutes each.   For accounting tests that are often numerically based, I like multiple-choice questions because I can give 6-8 potential answers and, therefore, limit the possibility of a lucky guess.  

In writing the first four of these questions, I tried to envision what an A student could figure out but that a B student could not.   In other words, I wanted these four questions to show me the point between Good and Excellent.   These were tough.   For those questions, I really didn’t worry about the C, D, or F students.   These questions were designed specifically to see if I could divide the A students from the B students.

The next four questions were created to divide the B students from the C students.   They were easier questions but a student would have to have a Good level of understanding to figure them out.  I knew the A students could work these questions and I knew the D students could not work them.   These four were written to split the B students from the C students.

The final four questions were created to divide the C students from those with a lesser level of understanding.   They were easier but still not easy.  I wanted to see who deserved a C and who did not.   If a student could get those four questions correct, that (to me) was average work.   Those students deserved at least a C.   But, if a student could not get those four, they really had failed to achieve a basic level of understanding worthy of a C.  

Then, I shuffled the 12 questions and gave them to my students.

How did this test work out in practice?   Pretty well.   When it was over, I put the papers in order from best to worse to see if I was comfortable with the results.   I genuinely felt like I could tell the A students from the B students from the C students from everyone else.   And, isn’t that a primary reason for giving a test?

Okay, I had to create a pretty interesting curve to get the grades to line up with what I thought I was seeing.   But I am the teacher for this class.  That evaluation should be mine.  I tell my students early in the semester that I do not grade on raw percentages.   Getting 66 percent of the questions correct should not automatically be a D.   In fact, in many cases, getting 66 percent of the questions correct might well be a very impressive performance.   It depends on the difficulty of the questions.

After the first test, students will often ask something like, “I only got four questions out of 12 correct and I still got a C, how can that be?”   My answer is simple “by answering those four questions, you have shown me how much you have understood and I thought that level of understanding deserved a C.”

If I take an adequate amount of time and write good questions, I believe I can gain a good evaluation of the knowledge of the student.   And I usually find they will work harder after that to achieve a deeper level of understanding because they begin to see what I am after.   The way you test is the way your students will learn.