WHAT IS YOUR PURPOSE FOR GIVING A TEST?

I have written about one essay on teaching per month for the past 10 years or so and I still seem to have more to say.  If you would like to receive a short email whenever I post a new essay, drop me an email at Jhoyle@richmond.edu.  I won't use the email for any other purpose.
*****

A friend (Dr. Paul Clikeman) and I are leading a faculty discussion in our building next month on testing.  It is a great topic for conversation because it is such a regular and critical interaction between every student and their teacher.  More arguments surely occur over test grades than anything else in the daily teaching process.

There are many serious questions that our faculty group can discuss during this time.
--Should teachers give objective tests or subjective tests?
--Should teachers consider Bloom’s taxonomy when writing test questions?
--Should speed in completing the test be a factor in the grading?  (In other words, should slower students be penalized for working slowly and, possibly, not finishing all the questions?)
--How can the teacher prevent cheating?
--Should students be allowed to have notes or other aids during a test?
--Should grades be distributed on a predetermined curve?
--How should a teacher determine if a student really deserves an A?
--Is there really a difference between a B+ and A- (or between a C+ and a B-)?
--Should teachers make previous tests available to students?
--Should teachers reuse tests?
--Should teachers use questions from a test bank produced by a textbook publisher?
--How can teachers grade written essay answers fairly and consistently?
--Should teachers pass out an answer sheet?
--How can teachers be sure questions do not have inherent flaws?
--Is it better to give a lot of good grades or a lot of bad grades?  Some people have high means whereas others have low means.  Is one preferable?  
--Is testing even necessary?  Should teachers stick to papers and presentations for grading purposes and avoid tests completely?

Those are all great questions but there is one question that I think must come before all of these questions and it is the point of this essay.  Why do you give tests in the first place?  The answer might seem obvious but your response will affect the answers to almost all of the above questions.

You really need to be comfortable with why you give examinations before you attempt to write any exam questions.  

I tried to come up with several reasons for giving an exam.  I am going to list these randomly.  I would like for you to pick the top one for you because I think that will help as you consider how to write better exam questions in the future.

Here are possible answers to my basic question in no particular order.

A –Tests provide evidence for grades.  A teacher has to give grades.  They really do not need to serve any other purpose.  They can be used to show the student why a certain grade was given.

B – Tests can be used to motivate students.  Almost every teacher, at some point, has threatened, “This will be on the next test,” just to get the students to wake up and pay more attention.  It provides a bit of urgency for the students.

C – Tests are good challenges.  Without tests, many students would tend to be mediocre.  Students are human and human beings need the challenge provided by a test in order to do their best work.  If a test is not appropriately challenging, the best students will feel betrayed – they worked hard for nothing (and they won’t make that mistake a second time).  

D – Tests help guide students by showing them what you want them to learn.  If you want them to memorize, then you test them on memorization.  If you want them to develop critical thinking skills, then you must test them on their development of critical thinking skills.  Students pay attention to test questions and catch on to what you are trying to accomplish.  

E – Tests provide the teacher with a good means of self-evaluation.  Student answers show the teacher what has been learned to date so the teacher can judge whether that is sufficient and take corrective actions if the results are not as hoped.  For example, if many students miss the same question, either the question was written poorly or the students were not properly prepared for it.  

F - Tests help students to determine how they are doing in relation to the others in the class.  For example, a low grade on a test can indicate that the student needs to try different study habits or invest more hours to catch up with the other students.

Okay, my guess is that you are saying to yourself, “Well, 3 or 4 of those answers apply to me to some extent.  I have several important reasons for testing.”  

Be that as it may, I think every teacher should take the plunge and have one primary reason clearly in mind for every test.  It is hard to write questions properly if you do not have a definite reason for the test in mind.

Speaking strictly for me, I do have one goal for every test and that is (D) – I mean for my tests to help guide the students by showing them what I want them to learn.  I don't want their to be any ambiguity.  I have long had the motto, “The way you test is the way the students will learn.”  Consequently, I attempt to tie every test question to my overall goal for the semester.

On the first day of class (and often thereafter), I tell my students that my class has only one purpose, “I am going to present you with weird, odd, and unusual situations and then help you figure out how to resolve those logically according to accepted accounting principles.”  We do that every day.  Then, in every question on every test, I try to present my students with weird, odd, and unusual situations and see if they can now logically figure out on their own how to resolve those according to accepted accounting principles.  

Of course, I don't teach history, English, or biology, but I think I could adapt that goal if I did teach other subjects.

I must admit that I really like establishing a tie between my class goal and my tests.  

Yes, I have other reasons to give tests.  For example, I like that they can motivate and challenge the students.  However, the primary reason, by far, that I give exam questions is to guide the students toward what I want them to learn.

Why do you give tests?  Have a clear answer in mind and you might find that the testing process becomes a more beneficial component of your teaching.

THREE TIPS TO HELP YOUR TESTS CREATE BETTER STUDENT LEARNING

Quick note:   If you are in need of a little inspiration as you begin a new school year, at the video link below, I tell four stories about teaching that I hope will get every teacher excited about another year in the classroom.  On good days, it is a truly marvelous and wonderful profession.  And, even on bad days, it can still be extremely rewarding.  I hope the video reminds you of that fact.  

https://www.youtube.com/watch?v=nT428yjJ0Ls&t=176s

**

Thoughts on Testing:  I am currently writing tests for my students so testing is on my mind.  Our first examination of the semester is tomorrow.  As I enter my 48^th year in the classroom, I am convinced that the writing and grading of tests is a true pain in the neck.  In fact, testing stopped being fun for me while Richard Nixon was president of the United States.

However, I am even more certain that testing can be (and should be) an extremely positive influence on student learning.  For that reason, I still invest an enormous amount of time in thinking and constructing examinations.  

In one of the first speeches I ever delivered about teaching years ago, I made the observation that,

                   “The way you test is the way your students will learn.” 

If you test students on memorization, they will work diligently to memorize every word you utter no matter how you teach them.  However, if you test their critical thinking skills, your students will quickly realize that memorization is not helpful and begin to focus their attention on critical thinking. 

If that is true (and I believe it is), then you cannot simply rely on a publisher’s test bank for your test questions.  For years, I have argued vehemently against the use of test banks.  I think they have done more harm to college learning than virtually anything else I know.  

Consequently, I want to provide you with three testing tips from my own experience as a teacher.   All of them have helped me over the years in the teaching of my students.  Perhaps, you will find one or two that are helpful to you. 

(1) – Notes – I always allow my students to bring two sheets of notes to every test.  They have to be hand-written because I want each student to do their own thinking about what is important, what material I am likely to cover and what questions I might ask.  I believe students need to learn to assess the relative importance of what we discuss in class.  By limiting the quantity of notes, students cannot simply judge all the material as equally important and just write down everything.  They must decide what is essential.

For me, that assessment is helpful for student learning.  However, that is not the primary reason for allowing notes.  I let students bring in notes because it sends a clear message (to the students AND to me) that I am not going to test them on memorization.  How can I test memorization if they have notes?  This puts an obvious burden on me to come up with reasonable questions that go beyond memorization.  I have often said that I never learned how to write good test questions until I began to allow students to have notes.  That step forced me to come up with better questions, ones requiring student understanding that went deeper than memorization.  When I look back on my evolution as a teacher, this decision to allow notes was a hugely important step.  Student learning improved because I began to challenge them with better questions. 

Try it once – see if it works for you.

(2) – Testing Circle – Every semester, during the last class before our first test, I spend 5 minutes talking with my students about my “testing circle.”  I draw a big circle on the board and tell them, “This represents everything we have discussed this semester to date.  I keep good records.  I know the topics our class has examined.  I will write about 1/3 of the test questions directly from this material because it covers what we talked about in class.  I want to see if you were here and awake, that you paid attention and understood.   If you get these questions correct, then you must have followed the daily conversation with a basic degree of understanding.  That strikes me as approaching average work which is evidence that you are getting close to a C.”

I then draw an X roughly 3 inches outside of the circle and explain, “This X represents test questions that are connected to our class coverage but go beyond what we analyzed in class.  I want to see if you can take your class knowledge and extend it to solve something new, something a bit more complex.  It will take analysis and thinking but you can do it.  If you understand the class material well enough, you can use that fundamental knowledge to figure out legitimate answers for this second group of questions.  To me, success at this level starts to look like Good work and begins to show me that you might deserve a B (or, at least, you are heading that way).  It indicates that you can make use of your knowledge and that is important to me.”

I then draw an X roughly 12 inches outside the circle.  “This final X again represents test questions that are connected to our class coverage, but they are more complex.  They will require an excellent understanding and thinking to determine the key elements of that connection so that you can come up with a solution.  I do not know that anyone can get them all correct, but I want to see if you can solve some of these questions because that starts to feel like Excellent and Outstanding work.  That level of understanding begins to show me that you are capable of earning an A.  Virtually every practice question I have sent to you by email this semester was either a B level or an A level question.  I took what we had done in class and tried to show you how that basic knowledge could be extended to solve more complicated issues.”

Once again, I am stressing that I have little interest in memorization.  I am also doing something that I am not sure enough teachers do.  I am showing the students visually what critical thinking is and why it is important.  We talk a lot about critical thinking in education but few students seem to have any clue what it is and why it is important.  That to me is a serious weakness of much of college education.  We say we are doing it but we do not even explain what it is.  For me, critical thinking is developing a level of understanding that is so solid that the student can use it as the foundation to solve questions and issues that are ever more complicated.  

I often tell my students that the course will never be finished until they can solve all the possible problems based on their own knowledge and, therefore, have no further need of me. 

(3) - Answer Sheets – For me, every test has two purposes:  (1) to help assess the students’ knowledge for grading purposes and (2) to improve the students’ understanding so they will do better in coming classes and on the next examination.   I believe students should show improvement on every subsequent examination. 

Consequently, I email my students an answer sheet within a few hours of every exam.   Not surprisingly, that is the time they are most interested in (a) the knowledge they needed to know and (b) how I wanted them to attack and answer each question.   Students will often pour over answer sheets with more intensity than they ever invest in the class.  They focus their attention like a laser on any questions they miss.  For me, that has always seemed like the perfect moment for improving student knowledge and that is what an answer sheet encourages.  I often have more interesting conversations about the answer sheets than I do about class material.  

For each test, I type up an answer sheet that starts with, “Here is how I would have looked at each question.  Here is how I would have started building an answer.   Here is the answer that I hope you were able to derive.”   No matter how complicated the question, I want the answers to seem logical and sequential.  Education is often the taking of a seemingly random and confused bunch of information and organizing it into knowledge that is logical and sequential.

Of course, there is a troubling disadvantage with distributing answer sheets—you cannot reuse test questions semester after semester.  The questions and answers are “out there” in the student world.   I know this is probably sacrilegious to say but I think teachers should write new questions for each semester.  The old questions become stale and dated.  New questions help keep you focused on what you want your students to know.  Like exercise, the writing of test questions is not fun but it is good for you.     

Answer sheets can help the students.   Again, try it once and see if it works.

Fourth Tip:   And, here is a fourth tip absolutely free of charge.  Send the answer sheets you use one semester to your students the next semester.  Once again, that will help them see that you are not asking for any memorization.  The answer sheets will show them what you mean when you say,  "I want to see if you can take your class knowledge and extend it to solve something new, something a bit more complex."  Students often believe a teacher is bluffing until they see evidence.   At that point, they will forget about memorization and start looking for ways to use the knowledge they have attained.

**

None of these ideas is perfect.  I can see potential flaws easily enough.  Nevertheless, in my own personal experience, they have each helped my students develop stronger and deeper learning.  If you are looking to experiment in your teaching, testing is an excellent place to start.  I think you will find opportunities for quick improvement.

When done well, your questions and answers can have a positive effect on the knowledge and understanding of your students.  Better testing can make for better learning.    

I AM CHEERING FOR YOU

I often think about teaching in comparison to being a coach.  Both teachers and coaches work with a group of people in hopes that those people will accomplish some task particularly well (often under pressure).   There is an ongoing learning experience where individuals in both groups gradually improve (hopefully) over time.   In each case, the whole process culminates in some type of test – sports teams play a game that they hope to win whereas students take an examination where they hope to excel.   In sports, the coach is trying to maximize the team’s chances for a victory.   In education, the teacher is trying to maximize the amount of every student’s understanding so that each person can do as well on the exam as possible.  

I spent my weekend writing a long, complex test for 41 of my students.   I know it will be a challenge to each one.   There is nothing easy about any of these questions but they have worked hard and they are capable of success.  I would really like to maximize the chances for success.

When I finished writing the test, I decided to sit down to watch a little bit of the basketball games in March Madness.   It is hard to avoid these games at this time of the year.

As I watched the teams play, I was struck by how much time and energy the coaches had expended in hopes of getting each player to do their very best.  The best coaches seemed to have taken nothing for granted.   They had done everything possible to help the players perform well.  Hmm, I found that interesting – they had done everything possible to help the players perform well.   Had I done as much for my own students?

I started thinking about my students and the test that they were surely preparing for at that moment.  We had spent an enormous amount of time working on the material but I wasn’t sure that I had helped them to be as psychologically prepared for the test as possible.   Is that my role?   Am I purely a teacher of material?   Or, if I want my students to really do well, do I have more of an obligation than that?  

One of the things I don’t like about testing is that it tends to put the teacher in an adversarial relationship with the students.   We are the coach but we are also the judge and that creates a bit of separation in the minds of both parties.   I’ve always wanted my students to know I was on their side.   I think that helps their learning.

After the last game was over last night, I decided that there really was a little bit more that I could do for my students to help them do their best on the test today.  Instead of going to bed, I wrote them one final email, not about the subject matter but rather about doing their best.   I imagine that a great basketball coach might have done something like this.   And, in all seriousness, are those games on television one bit more important than the success of your own students?  You might disagree but I think not.   One of the first steps in being a better teacher is to recognize the importance of your role and in doing it as well as you possibly can.

Here is the email that I sent out.  I have no idea whether it increased anyone’s grade even one point but, for me, it was worth a try.   At least, I wanted my students to know that I was cheering for them.  I did truly want them to do well.   Before your next test, you might try something similar.   If nothing else, I think it is good for the student-teacher relationship for them to know that you really do want them to learn and succeed.

To my students:

You will have your second test in roughly 12 - 13 hours.   I know I have said all of this before but I want to say it again as you mentally prepare yourself for the battle.

Most importantly, I doubt seriously that I am going to ask you anything that is not already in your head.   Seriously, I wrote each question with one comment to myself, "I think this is in their heads -- it is not really impossible/bizarre/unworkable.  I think they'll know this."  

So, I think the whole key to the test is getting the material out of your head smoothly and onto the paper.   That's all I want you to worry about in these last few hours -- getting the knowledge out of your head smoothly and onto the paper.  

To do that, you know what I'm going to recommend first -- get a normal night's sleep.   Being tired is one of the worst things you can do on a difficult test.   No one functions well when they are tired.   If you normally get 7 hours of sleep, then go for 7 hours of sleep.

Second, stay calm.   I know the questions are going to look bizarre at first.  Take a deep breath and tell yourself, "he wrote these questions knowing us and believing that we can work them.  Getting rattled is not going to help.  Let me read it carefully."  

Third, have confidence.   You are bright people who have made your way into this university, into this school, and into this class.   That didn't happen by accident.   Yes, the material is complicated but it is not that complicated.   Don't blow it out of all proportions.

Fourth, keep your concentration.   I always tell my students, "if the building catches on fire, you don't want to notice until some fireman picks you up and carries you from the room."    I don't care what happens in room 223 tomorrow morning, nothing but that test should make any difference to you.

Fifth, if you get stuck on a question, don't waste a lot of time on it.   Go find another question that you might know better and come back to the "stuck" question at the end of the time.  

Finally, be careful.   I'm always shocked/dismayed by how many points great students just throw away by doing careless things.   If you don't know a question, that's fine, I can live with that.   But don't just hand over points by making careless errors.  

I know you (not the person beside you but YOU) are capable of doing great.   I'll be cheering for you!!!!!!   Go get it!!!!!    Make it happen!!!!!!

HELP YOUR STUDENTS FOCUS ON THE QUESTIONS

On Friday morning, April 8, 2016, I will be speaking at the Ohio region meeting of the American Accounting Association on the topic of “Perspectives of High-Quality Teaching.”   If you are in the Cleveland area, I hope you will show up.  I would love to meet you and chat about teaching.   Here’s the URL.

http://aaahq.org/meetings/2016/ohio-midyear-meeting

**

I am teaching three classes this semester—two Intermediate Accounting II classes and one Financial Accounting class.  I have 78 students.   I realize that some of you teach hundreds but, for me, 78 is a fairly large number.  That provides some challenges when trying to get each student to do outstanding work.

I always give three tests every semester and a final exam.   I believe that gives the students a chance to show me what they have really learned.   After the first two tests, I usually have a good group (50-60 percent) who are doing well and have an excellent chance of getting an A or a B (hopefully).   They are strong and talented students who put in the work consistently for every class.

But I have another group (40-50 percent) who do not seem able to break through.  In some cases, the students are simply not working.   In other cases, accounting does not come easily to them.   We all have different talents.   In truth, though, a vast majority of my students are working relatively hard and seem more than capable of making an A or B.   Well, then, what assistance can I offer this second group of students to help them move from C’s and D’s to A’s and B’s?  

In many ways, isn’t that one of our most important jobs—helping students who are struggling to figure out how to become outstanding?   And, isn’t that where the victories are the most satisfying?   Getting a bright, hard-working student to make an A feels good but I always realize that they could have probably done it without me.  I don’t deserve too much credit.   Getting a student who has a C or a D with only 3-4 weeks left in the semester to make an A or a B seems like teaching at its best.

So, I take it as a personal challenge to get my “under B” group to do better.  First, you have to get them out of a “C” mentality.   After two low test grades, it is easy to become discouraged and start to think of yourself as no better than an average student.   That’s nonsense.   That’s absolute baloney.    Everyone can do better.   I am convinced of that.   I like to remind them that they still have well over half of their grade to be determined.   In my classes, the last regular test and the final exam make up approximately 57 percent of their overall grade.   Although the semester seems to be drawing to a close, they are not even at half time yet as far as their grade is concerned.   They still have plenty of time left to make an A or B but they do need to make some adjustments and they need to make them immediately.  I need to impress on them that they can do better but there is some urgency.  Without urgency, change is tough.

As probably everyone who reads this blog knows, I use the Socratic Method.   My class is filled with questions that the students work to answer.   I am training them (I hope) to learn how to “figure out” answers for themselves.  My giving them answers and information is not nearly as beneficial as them getting the information and figuring out the answers for themselves.

When a student comes by to ask for help here in this last month, I like to ask that person to start writing one test problem after each class.  I want to see one problem that they think I might ask on a test based on the material we covered.   I want them to start focusing on how the material can be turned into questions.   In the book Make It Stick, the authors assert that students often over-estimate what they have learned.   I think that is probably true.   I also think it is true that students focus on answering the questions they have already seen and not on the questions they are going to see. 

What I find fascinating is that, even after having two of my tests, students often write poor questions.   For the most part, they simply take the questions that I ask in class and change a few words or numbers.   I think that is how many of them have been trained in high school.   The teacher says something.   The student writes it down.   The student hands it back on the test.   The student gets an A.   That does not work in my class.   I ask them to take material and do something different with it.   I sometimes think that the reason they are not making an A or B is that they don’t truly understand how they are going to be tested.

When they send me their questions, I often point out “that sounds like what I asked in class.  I’m probably not going to ask that same question again.   What would that prove?   How could I twist the question to make it different and see what you really understand?”   Usually, on a second (or maybe third) attempt, the questions start looking like one of my test questions.   The student starts making a break through—not on the answer side but on the question side.

At that point, when they start to understand the nature of the questions they are going to see, then coming up with legitimate answers becomes a more realistic goal. 

If you are having students who do not seem to be able to “break through” into the A and B range, you might try that.   After every class, ask them to write a question that you might ask on a test.   Then, if they do not do a very good job of that, help them see what more you might be expecting from them.   Get them to focus on the questions before they worry too much about the answers.  

I sent my Financial Accounting students a practice problem this morning.   Sure enough, I took what we had done in class and added something a bit different.   I challenged them to “figure it out.”   And then I tried to make the point more clearly:   “And, as you are getting ready for the third test start asking yourself two questions:  (1) Can I do the standard problem?  (2) How can the problem be extended to make it more challenging?   That's when education gets exciting.”

Maybe focusing on the questions will help your C and D students move up to A’s and B’s here in the last few weeks of the semester.  That’s a victory for everyone.

HOW TO WRITE A TEST

 

A good friend emailed me a few days ago and asked for some suggestions on writing a test.  Students were mostly getting good grades on her tests but she wasn’t sure that they were learning as much and as deeply as she wanted.

I think testing is a teaching topic that we do not talk about frequently enough.   Too often, we are probably afraid of exposing our weaknesses.   Few people are trained to write good test questions.   She was creating an accounting test but I believe the basic discussions around testing are pretty much universal across the various disciplines.   Here were the thoughts that I sent to her.

Rule 1 – Testing is just about the most important thing a teacher does each semester because it sets the tone for the students.   It tells them what you really want and expect from them.   If you test memorization, they will learn to memorize.   If you test mechanics, they will learn nothing but mechanics.   Think about what you want them to learn and then test that way.   Here’s where you need some type of mission statement – “I am teaching my students to  . . . “

Rule 2 – Because it is so very important, never turn your testing over to a busy grad student in some unknown college in some unknown place.   That is who writes most test banks.   For what you know, those people might well know less about excellent teaching and testing than my cat.    If you were a basketball coach at one of the local universities, would you turn the practice sessions over to the janitor because you were uncertain about running them?   That is nonsense.   But you turn your testing over to someone who doesn’t know your school or your class or your students.   Learning good testing takes some practice but you can always do it better than the grad student at an unknown college in an unknown place.

Rule 3 – A test has one major purpose – to differentiate the A students from the B students and so on.   Differentiation is the purpose.    And, of course, to have the students believe that your differentiation was fair.   1/3 of the questions should be workable by the A, B, and C students but not the D and F students.   1/3 of the questions should be workable by the A and B students but not the C, D, and F students.   1/3 of the questions should only be workable by the A students – that is how they prove they are A students.   That is how you make them feel good about themselves.

Rule 4 – If you use a test bank, always realize that most test banks are for sale on the Internet.   The students often buy them for practice purposes.   No test bank is absolutely safe as far as I know.

Rule 5 – Always be willing to curve.   I tell my students that I grade the tests and then I assess what is excellent work, good work, average work, and so on.   I next curve those tests (and only those tests) that deserve to be an A so that they get an A.   That is where my professorial judgment comes in to play.   I decide what is excellent, what is good, and so on.   If I judge a 78 to be excellent, I curve that to an A.   If I judge a 95 to be good (a different test obviously), I curve that to a B.

Rule 6 – I am a big believer in the wonder of puzzles.   Where possible, I try to write test questions that are basically puzzles.   I also believe where possible that questions should resemble real life.   These are 20 year old adults – they are old enough to vote and old enough to go to war.   Don’t make test questions look like test questions from their high school days.   Make them look like real life with some kind of twisted puzzle logic.   Questions that incorporate “what if” are usually good as are questions that ask “how would this have changed” or “how would you decide between these two options?”  

**

I recently gave a test in Financial Accounting and another test in Intermediate Accounting II here at the Robins School of Business at the University of Richmond.   If you would like to get a copy of either of those tests (just to see what I do), drop me a note at Jhoyle@richmond.edu.

**

Because I am writing above about puzzles, here is a quick story that I liked.   A good friend of mine recently gave me the book Mindset – The New Psychology of Success by Carol S. Dweck.   I opened the book to the first page and was fascinated to read the following which seemed to have come directly from my own experience:

“When I was a young researcher, just starting out, something happened that changed my life.  I was obsessed with understanding how people cope with failures, and I decided to study it by watching how students grapple with hard problems.   So I brought children one at a time to a room in their school, made them comfortable, and then gave them a series of puzzles to solve.   The first ones were fairly easy, but the next ones were hard.   As the students grunted, perspired, and toiled, I watched their strategies and probed what they were thinking and feeling.   I expected differences among children in how they coped with the difficulty, but I saw something I never expected.

“Confronted with the hard puzzles, one ten-year-old boy pulled up his chair, rubbed his hands together, smacked his lips, and cried out, ‘I love a challenge.’   Another, sweating away on these puzzles, looked up with a pleased expression and said with authority, ‘You know, I was hoping this would be informative.”

Yeah, puzzles can make class and teaching a whole new ball game.

CREATING A FINAL EXAMINATION

“Too often, we settle for dreams that merely scratch the surface of our abilities and then wonder why we are dissatisfied with the results.”

From the book:   Don’t Just Dream about Success:   Stack the Odds in Your Favor

By:   Joe Hoyle (to be published in January 2014)

Returning from Thanksgiving break, virtually all college teachers start looking forward to creating and then grading final exams.   It is a necessary part of the job but it is also an event that can impact the education of each student rather significantly.   Below is a rewritten version of a blog entry that I posted 3 ½ years ago.   I thought it made a good point back then.   I believe the same thing today.

**

Why do you give a final exam to your students?

What do you hope to accomplish?

I have talked with many professors over the years and their strategies for final examinations vary greatly from one to the next.  Here are several typical strategies:
--It is a comprehensive three-hour examination on the material from the entire semester with a major grade component.    This seemed to be the strategy most of my teachers employed when I attended college back in the 60s.
--It is a one-hour test covering just the material since the last hourly exam with no added weight in comparison to other tests.
--It is basically ignored because the student’s work for the entire semester should be more important than what they can do on one day at the end of the semester.
--It is a little harder than a one-hour test but a student can only improve his or her grade; the final exam cannot hurt the course grade.

I do know that as I walk through our building during final exam week most students seem to leave well before the time limit is reached. I am not sure that many final exams are still three hours in length.

I have always been interested in the final exams that are given by some law school professors.  The entire grade for the semester is based on what the student can do on the final exam questions.  Nothing else counts.  The rationale is that, if you are going to be a lawyer, you need to be at your very best every day that you walk into court no matter what is thrown at you.  There can be no down days.

However, I am teaching 19-20 year old sophomores and juniors in college and not 25-30 year old law students.  I am afraid that I would have students facing nervous breakdowns if I put the entire grade for the semester on the final exam.

So, what are my goals for a final exam? Psychologically, what am I trying to accomplish?
---I want students to stay emotionally involved with my course all the way through the last day of the semester.  I am not interested in them quitting early.  Thus, the final exam has to count enough to make it worth their time to keep working.  In my classes, the final exam is roughly 35 percent of the overall course grade.  I have found that this is enough to keep them emotionally involved until the very end.
---I want students who do poorly on the first (and, even, second) test of the semester to have a chance to improve their grades.  If a student makes a C or a D (or an F) on the first test, it can be very disheartening.  It is easy to lose hope.  I do everything I can to keep them from giving up.  I like to be able to say “if you can show me that you can learn this material, you still have a lot of your grade left to earn on the comprehensive final examination.”  Nothing pleases me more than for a student to make a low C or a D on the first test and then come roaring back to make an A for the course.  That is hard to do unless the final examination has a pretty serious weight attached to it.
---Likewise, I don’t like students who do well on the first test to get complacent and think they have an A in the bag.  I want to be able to tell them:   “Good job on this first test but you need to realize that there is a lot of semester left and you need to keep up this level of work from beginning to end.”
---I want students to understand the material well enough that they can still answer questions from throughout the semester at the very end.  If we cover a topic in September, I think they should be able to answer a reasonable question on that topic in early December.  Because I want to stress understanding more than memorization, I don’t think that is too much to ask.

As a result, I do give a comprehensive final examination and I do grade it and that grade (for better or worse) counts roughly 35 percent of the overall course grade.  In my introductory financial accounting course, I want the first student to leave after 2 hours and the last student to finish at 3 hours. I like it when about half stay virtually the whole time.

For intermediate accounting, I want the first person to finish in three hours but everyone else is relatively close to being finished. The material in that course is so complicated that I don’t see how a final exam can take much less than three hours.

In writing the exam, I line up all the topics for the entire semester on a sheet of paper and pick one pretty much at random (deferred income tax assets, for example). I then ask myself—if one of my students is at a job in six-months and this topic is raised, what should I expect an A student to be able to remember after five minutes of review?  In all honesty, I would love to ask “what should I expect an A student to be able to remember immediately” but I don’t think that is realistic.  Students forget material quickly (even accounting). 42 years of teaching has shown me that students never remember quite as much as I might hope.

Based on the answer to the question of what I want them to remember in six months, I write a problem to test if they hold that level of knowledge.  I never want to ask an easy question because that proves nothing.   But, there is little reason to ask an impossibly hard question.   Writing a question that no one can answer will not provide me with any usable information.   
I estimate how long that first question will take the A student to answer and go back to my list and select another topic for another question.   When I have filled up my time allotment in this way, I quit.  

However, I then immediately construct an answer sheet.   I do not want to get to the final exam site and discover that a question cannot be answered because of missing information or that a typo is going to throw the students off track or that several questions are really easier than I had anticipated.   It is hard to fix a final exam once the test has been distributed.   The answer sheet is essential because it allows me to evaluate each question as well as the entire exam.   I cannot even guess how many mistakes over the years I have resolved in advance because I force myself to create an answer sheet.

Setting up the final exam in this way keeps the students (I hope) thinking about my course all the way until the end of the semester. I want accounting to be on their minds until the semester is completed.   And, it gives them a reasonable last chance to make up for any poor grades they might have earned during the semester.

What’s your philosophy? Why do you give a final exam?   How do you set it up?

HERE'S YOUR SUMMER ASSIGNMENT

As I have mentioned before, on August 7, 2013, I will be giving a 75 minute presentation about this blog at the Annual Convention of the American Accounting Association in Anaheim, California.  If you are going to be at the convention, I hope you’ll stop by and chat.   I’d love to meet each and every one of the people who read these essays.  I’ll be there for several days so let’s get together over coffee and talk about teaching.  

In preparing for this talk, I have gone back and looked at some of the 170 posts that I have written over the years.   To me, it is interesting how many of these posts push teachers (including me) to think about the last day of class and what the teachers want their students to have accomplished.   Until you visualize what you want your students to do by the end of the semester, it is hard to know how to guide them.   Over and over, you see that theme in my thoughts on teaching.   You can accomplish very little without knowing specifically what you want to accomplish.   Notice the word is not “vaguely” but “specifically.”  

One of my very favorite posts was “Training Dogs” which I put up on this site on March 4, 2010.   (It is available at http://joehoyle-teaching.blogspot.com/2010/03/training-dogs.html ).   This essay talks about how people go about training dogs and makes what I thought was an obvious point:   training students uses the same basic techniques that you would use in training dogs.   It seems easy when you talk about dogs; in fact, it seems like basic common sense.   I am not sure why it appears so much more complicated when working with college students.

In this earlier essay, I ask readers to come up with 8 steps for training dogs.   They are all obvious but my very first one is:   “Have a firm understanding of what you want them to accomplish.”   Again, that seems so self-evident.   You cannot teach a dog to roll over if you are not sure what you want the dog to do.  

Unless the goal is clear in your mind, the dog will just be confused.  

Unless the goal is clear in your mind, the student will just be confused.  

I think a lot of teachers who struggle are simply not sure of their goal.   Maybe vaguely they are but not specifically.

Okay, how do you put this into practice?   I was working with one of our younger teachers this week and I gave him an assignment that might be helpful to you.   I told him to spend several hours over the summer writing a “hypothetical but absolutely perfect final exam” for his fall classes.  In other words, if the students did exactly what he wanted them to do for the entire semester, what would he want them to show him on the final exam?   That seems like a legitimate assignment.   If the students learned exactly what he had hoped for, how would they be able to show that on the final exam?    It seems to me that there really needs to be a connection between those final questions and what they ultimately need to know.

So, don’t write your final exam at the end of the semester to ascertain grades.

Instead, write your final exam now before the semester even begins as a guide for what you want your students to be able to do by the end.   Don’t be vague – make those questions as specific as you possibly can.   That's the real purpose -- to add specificity to your goals.

Okay, you probably won’t ever give this exam.   That’s not the point.   The point is to help YOU (not them) to visualize what you want them to be able to accomplish at a high quality level.

Visualize the perfect semester where every class goes great and every assignment is done well and every reading is carefully analyzed.   How would the students be able to demonstrate that they had done really well for you?  

Surely, you would not fill up this hypothetical exam with memorization questions.   That just cannot be your ultimate goal.  Well, if that’s not the goal, what is your goal?   What would be several perfect questions to allow students to show you the intense depth of their understanding?

I have an idea that this might be a very interesting summer project.   I have held before in these essays that most teachers spend way too little time thinking about testing and how it can be used to help pattern student learning.   Here’s a chance to give some deep thought to that process.

By the time school starts back in the fall, can you have a hypothetical final exam created that would enable your students to show what they know and understand?   If so, that can really serve as a road map for how you attack each day of the entire semester.   Anything that won’t get the students moving toward the point where they can answer those hypothetical final exam questions will quickly start to seem trivial to you.   And, that’s probably a good sign.

WHAT WAS YOUR BEST?

 

In my previous blog posting, I talked about the creation of faculty learning communities as a way to generate conversation about various aspects of teaching/learning/education.   At that time, I brought up one of my favorite topics.   If you have read this blog for long, you are aware that I am a firm believer that the way you test will strongly influence the way your students learn.   If you want to create a different class environment, you need to test differently.

Therefore, in discussing faculty learning communities, I strongly feel that every aspect of testing should be a topic of serious conversation by people who want to become better teachers.

So, today, I have a question for you.   I would bet it is a question that you have never been asked before no matter how long you have taught.   And, I would argue that it is a question we should be asking all the time.   In fact, I think we should have a national contest built around this one question.   I believe the answers might well improve college education (which is not a small statement to make).

Here’s the question.   Think before you answer. 

Whether you teach history, political science, math, accounting, or the like, in your testing during the past semester (or academic year), what was the very best question you asked your class on a test?   If testing matters so much, then we should all have some really good questions that we are especially proud of having written.   Why did you feel that particular question was so good?

Excellent test questions set the tone for your class:   this is what I consider important, this is the way I want you to learn, this is the kind of thinking you should be doing, this is what I want from you rather than memorization.   Students need guidance – nothing guides them quite like what they believe will be on the tests.

Okay, so what was my best question for the past semester?   As most readers of this blog probably know, I teach accounting at the Robins School of Business at the University of Richmond.    We have very bright students who are willing to do as much or as little as you ask of them.   It is all about asking.   In testing, I want them to know that I am going to ask them for a lot of serious thinking.

One of my courses is Intermediate Accounting II.   Most of my students believe that accounting is basically the memorization of set rules that they must apply to particular situations.   Many of them are left-brained and love the comfort of those rules.   They are not necessarily happy that I want them to think outside of the box.

However, my experience has been that, in real life, accountants are thrown into odd situations almost every day and must use all of their brain cells to figure out what is going on so that they can determine what response is needed.  

You can learn the rules for being a medical doctor but the actual application is much more stressful (and interesting).

On my first test this past semester, I wanted to break the students away from the memorization of rules.   So, when they opened the test, they discovered that they had an accounting client on the planet Kryptoplasm.   The businesses on that planet use a unique set of accounting rules (referred to as Krypto-GAAP).   I then presented a variety of situations and described the basic rules found in Krypto-GAAP.  I then asked the students to determine the impact of converting the financial statements prepared on that planet into financial statements that could be used in the US (based on US GAAP).   For each situation, they had to tell if reported net income would go up or down, whether the reported liabilities would go up or down, and so on.

The students had never seen anything like this which is what I wanted.  

What was I trying to accomplish?

--I wanted the students to read the questions carefully.   They could not anticipate the accounting rules on the other planet so they had to read the words and think about what those people were doing.   I think the ability to read and think through what you are being told is vitally important in solving problems.

--I wanted to downplay the importance of memorization.   No matter what you tell a student they will believe that they can prosper by memorizing everything you have said.   I wanted no questions that simply asked them to replicate a mechanical rule.

--I wanted them to make judgments as to which rules should have been applied.   By describing the weird things that were happening on this planet, they had to step back and think about how those events should be reported.   The problem was more than just manipulating numbers.

--Despite being set on a faraway planet, I wanted the students to be placed in a real life situation.   Having a client do weird things is no stretch of the imagination.   Too many tests have nothing to do with real life and reinforce the student’s suspicions that college classes are just student exercises.

Did I like the results?

Yes, in fact, I liked this question so very well that my second test used the same format (without warning the students).

Yes, in fact, the standard joke in my classes quickly came to be “what would they do in Krypto-GAAP?” which alerted the students to the fact that accounting rules are not set in stone but simply selected at a point in time and place as the most appropriate method.

Did the test question work?   I have no hard data but I do know that I had a really good semester in Intermediate Accounting II.   My students quickly came to be open to talking about accounting in interesting and theoretical ways.   We learned the rules but we didn’t become obsessed with the rules.   I think one of the major reasons why the semester went so well was because of the message that I sent out in that first test (and then followed up on in the second test).   This question helped show the students the kind of thinking I wanted them to be able to do.   The way you test is the way they will learn.

So, what was your best question of the past semester?