Tuesday, October 28, 2014

HOW TO WRITE A TEST

 

A good friend emailed me a few days ago and asked for some suggestions on writing a test.  Students were mostly getting good grades on her tests but she wasn’t sure that they were learning as much and as deeply as she wanted.

I think testing is a teaching topic that we do not talk about frequently enough.   Too often, we are probably afraid of exposing our weaknesses.   Few people are trained to write good test questions.   She was creating an accounting test but I believe the basic discussions around testing are pretty much universal across the various disciplines.   Here were the thoughts that I sent to her.

Rule 1 – Testing is just about the most important thing a teacher does each semester because it sets the tone for the students.   It tells them what you really want and expect from them.   If you test memorization, they will learn to memorize.   If you test mechanics, they will learn nothing but mechanics.   Think about what you want them to learn and then test that way.   Here’s where you need some type of mission statement – “I am teaching my students to  . . . “

Rule 2 – Because it is so very important, never turn your testing over to a busy grad student in some unknown college in some unknown place.   That is who writes most test banks.   For what you know, those people might well know less about excellent teaching and testing than my cat.    If you were a basketball coach at one of the local universities, would you turn the practice sessions over to the janitor because you were uncertain about running them?   That is nonsense.   But you turn your testing over to someone who doesn’t know your school or your class or your students.   Learning good testing takes some practice but you can always do it better than the grad student at an unknown college in an unknown place.

Rule 3 – A test has one major purpose – to differentiate the A students from the B students and so on.   Differentiation is the purpose.    And, of course, to have the students believe that your differentiation was fair.   1/3 of the questions should be workable by the A, B, and C students but not the D and F students.   1/3 of the questions should be workable by the A and B students but not the C, D, and F students.   1/3 of the questions should only be workable by the A students – that is how they prove they are A students.   That is how you make them feel good about themselves.

Rule 4 – If you use a test bank, always realize that most test banks are for sale on the Internet.   The students often buy them for practice purposes.   No test bank is absolutely safe as far as I know.

Rule 5 – Always be willing to curve.   I tell my students that I grade the tests and then I assess what is excellent work, good work, average work, and so on.   I next curve those tests (and only those tests) that deserve to be an A so that they get an A.   That is where my professorial judgment comes in to play.   I decide what is excellent, what is good, and so on.   If I judge a 78 to be excellent, I curve that to an A.   If I judge a 95 to be good (a different test obviously), I curve that to a B.

Rule 6 – I am a big believer in the wonder of puzzles.   Where possible, I try to write test questions that are basically puzzles.   I also believe where possible that questions should resemble real life.   These are 20 year old adults – they are old enough to vote and old enough to go to war.   Don’t make test questions look like test questions from their high school days.   Make them look like real life with some kind of twisted puzzle logic.   Questions that incorporate “what if” are usually good as are questions that ask “how would this have changed” or “how would you decide between these two options?”  
**

I recently gave a test in Financial Accounting and another test in Intermediate Accounting II here at the Robins School of Business at the University of Richmond.   If you would like to get a copy of either of those tests (just to see what I do), drop me a note at Jhoyle@richmond.edu.

**

Because I am writing above about puzzles, here is a quick story that I liked.   A good friend of mine recently gave me the book Mindset – The New Psychology of Success by Carol S. Dweck.   I opened the book to the first page and was fascinated to read the following which seemed to have come directly from my own experience:

“When I was a young researcher, just starting out, something happened that changed my life.  I was obsessed with understanding how people cope with failures, and I decided to study it by watching how students grapple with hard problems.   So I brought children one at a time to a room in their school, made them comfortable, and then gave them a series of puzzles to solve.   The first ones were fairly easy, but the next ones were hard.   As the students grunted, perspired, and toiled, I watched their strategies and probed what they were thinking and feeling.   I expected differences among children in how they coped with the difficulty, but I saw something I never expected.

“Confronted with the hard puzzles, one ten-year-old boy pulled up his chair, rubbed his hands together, smacked his lips, and cried out, ‘I love a challenge.’   Another, sweating away on these puzzles, looked up with a pleased expression and said with authority, ‘You know, I was hoping this would be informative.”

Yeah, puzzles can make class and teaching a whole new ball game.

Monday, October 13, 2014

SELLING THE NEED FOR WORK



In my previous blog posting, I talked about motivation – are you a football coach or a scout leader?   I received several emails asking how I motivate students under either of those approaches.   Well, no motivational style works perfectly on every occasion but I think you need to (a) really communicate clearly to your students at all times and (b) sell the course to them.    Every course and every friend and every club is demanding every minute of a student’s time.   Why should they pick your course to focus their attentions?   I think that is where communication becomes vital and I think that communication has to have some element of marketing in it.   If you believe in the importance of your course, then you need to help them understand what needs to be done.

Consequently, here is an email that I sent to my students today.

 
To:   Accounting Students

From:   JH

At the beginning of the semester, I made the comment that a successful class is like a dance that is well done.   If I do half of the work and you do half of the work, then we can accomplish great things.   But if either of us does less than half of the work, then the dance is never going to go very well no matter how much the other party is working.      

You have every right to assess whether I (or any other teacher) is doing half of the work.   If not, you should complain.

And, in the same manner, I periodically assess how you are doing.   We are on fall break.   It is a good time for an assessment.   Our second test is in just a few days.   I know how you did on the first test.   I’m really interested now in where you are heading on our second test.

My guess is that you view this class as a class—maybe a little more important or a little less important than others, but really nothing different than a class.

I view this class as an opportunity.   It is one where you can add some knowledge to your brain that might prove helpful one day.   It is an opportunity that might make you a bit sharper at some time in the future, more astute, a better decision-maker, a wiser and more successful person.

So, over the last couple of days, I have gone over the seating chart, person by person assessing whether you are making good use of this opportunity.    Are you doing your half of the dance?   Truthfully, as a whole, I am pretty well pleased.   No group is perfect but a number of you are clearly doing your half.   In general, I have few complaints.   Unfortunately, we live in a specific world and not in a general world.

Here’s how I kind of assess students when I am thinking about each one of you.

--Enthusiasm.   There is little that is worse in teaching than to walk in to a classroom and have students who clearly would prefer physical torture over your class.   The body language tells it all:   “I hate this and I am going to hate it no matter what you try to do.”   The view from my perspective can sometimes be disheartening.   Fortunately, most students give the teacher the benefit of the doubt and that is more than fair.

--Consistency.   One of the hardest things for any student to do is to be ready to go each and every day.   It is the rare A+ student who walks in every day ready for the debate.  Many students believe that being prepared 2 days out of 3 is darn good but that’s just a 66 percent average and that’s a D.

--Interest.   Some students take notes like they are majoring in stenography.   They have no interest in the material but they are going to copy it down word for word so they cram it into their brains.   Other students actually express an interest in what we are discussing.   You can just see it in their faces.   I guess they are less judgmental.   They step back and try to figure out why the material might be interesting or, at least, important.   That doesn’t mean they are going to major in it.   They just try to look beyond the curtain to see what might be interesting to know.   College material is not an obstacle course to be survived but rather an adventure in learning.

--Ambition:   I have said before and I’ll probably say again, I think the world needs more ambitious people and that should start with young people like you folks.   Too many people settle for average/mediocre from the time they are young until the day they die.   I am always interested in seeing who truly wants to do well just because they want to do well.   Anyone can do well if they have some external force applying motivational pressure.   It is only the chosen few who have the “fire in their bellies” that enables them to motivate themselves just because they want to be winners.

--Preparation, Level One:   Okay, I give out daily sheets with questions.   Many of those questions really come directly from the assigned reading.   I am always interested in seeing how many students are at least willing to go to the trouble to read the text and answer those questions.   There is really no thinking involved.  It is just a matter of being persistent enough to locate the material in the book.   I am always ready to pull out my hair (?) when a student can’t answer a question that should have taken 5 minutes to find.

--Preparation, Level Two:   This always shows the students who probably should think about becoming a major.   It is all about taking material and going beyond just the basics.   How does the answer to one question lead you to the answer to a second, much more difficult question?   In many ways, that is the essence of this course.   Can you take basic material and use it to answer new and more challenging questions?   That is the one aspect of this course that takes an entire semester to develop.   If we do it right, that is the skill that will become stronger and stronger over the next weeks.

I could assess students on other things but this, to me, is pretty much what education is all about:   enthusiasm, consistency, interest, ambition, basic preparation (Level One), and more in depth preparation (Level Two).   Give me those and we’ll have a dance that even Fred Astaire and Ginger Rogers couldn’t have beaten.


Thursday, October 2, 2014

MOTIVATION--ARE YOU A FOOTBALL COACH OR A SCOUT LEADER?



The Aspen Institute Business & Society Program recently asked me to write a blog entry on educational innovation.   You can check out my thoughts on that topic at:


**

A new school year has begun.   I hope your classes are all off to a great start.  

One of the discussions that I have with myself at the beginning of every new school year is about my role as a motivator for my students.   Over the years, I have had a number of conversations with college professors who adamantly assert that student motivation is not their responsibility.  Their feelings certainly have some validity:   “These students are adults.  By this time, it is their responsibility to provide their own motivation.   If they don’t want to learn the material, then they should not be in my class.   In fact, I am not sure they should be in college if they are not interested.   I am not going to treat them like third graders who have to be coaxed into learning arithmetic, geography, and the like.  I am here to explain the materials and help them understand the complexities but I am not a cheerleader.   That is not my job.   When a student signs up for my class, they are saying that they are willing to do the work necessary because they want to learn the material.   It is not my responsibility to be a motivator.”  

In an ideal world, I might well agree with that philosophy.   In that world, students would walk into class ready to learn and constantly beg to be pushed further.  

At least in my class, it is not an ideal world.   My students are very bright but they have a number of other classes as well as job interviews and a wide array of extracurricular activities.  The fact that I actually want them to learn and understand the material can get lost in the helter-skelter existence of a college student.  

And, to tell the truth, every college teacher is a motivator in some way simply by their mere presence in the classroom.   Some teachers might motivate students to stretch themselves beyond their abilities.   Others might encourage the students to do little or no work.   Teachers cannot disassociate themselves from motivation.   Instead, they can decide how they want that motivation to impact the students and their work.  

I always believe that two different types of motivation are available for college professors.   I refer to the first as the “football coach.”   The professor walks into the room and starts pushing students onward.    “I want everyone to learn this material; therefore, I am going to tell you exactly what I want you to do and will expect you to do it.   I have designed every step that I want you to follow and I will push you to do them all and do them well because I want to see great results.”   Picture a football coach like Bear Bryant of the University of Alabama or Vince Lombardi of the Green Bay Packers.   By pushing with enthusiasm and conviction, they motivated their players to greatness.

The “football coach” version of motivation can be very successful.   Think back to all the great teachers you have known over the years and I would expect some, possibly many, to have followed this strategy.   The teacher provides the ambition and the energy and views poor grades as losses to be avoided.   The word that I often associate with the football coach-teacher is “demanding.”   They demand the best from their students and push them forward to achieve that goal. 

Why are there not more college teachers who adopt the football coach approach?   It is a lot of work and takes a considerable amount of energy and some students resist being pushed along even if it is for their own good.  

The other type of classroom motivation that I see is the “scout leader” approach.    This person has a lot of patience and will work carefully with students for hours to make sure they understand the material.   However, the scout leader wants the education to be student-centered.   The goal is for the student to do the work because the student has come to see its importance and not because of being forced.    The scout leader views the role of the teacher as one of guidance.    “If you want to learn this material, I will be glad to help you in every way I can.   I’ll show you why the material is important but the decision to actually do the work has to be with you as the student.    It is your life.   I can show you how to start a fire but, after that, you have to decide whether you want to practice enough to actually be able to start a fire.”

Once again, if you think about the great teachers you have known over the years, almost certainly some have been scout leaders.    They will often be described as “kind” and “caring.”   They are patient guides who enable a student to be successful if that is what the student chooses to do.   There is often a love of learning that is conveyed from teacher to student.

Okay, here are a few questions to ponder.  

--If you were a student in college once again, which type of motivation would you prefer?   Did you like professors who pushed you toward success or professors who guided and enabled you but left the decisions about what path to take up to you?
--In your building, who is the best football coach teacher and who is the best scout leader teacher?   It is good to have role models who can show you how a particular approach can be used to achieve success.
--If I asked your students whether you are more like a football coach or more like a scout leader, what would they tell me?   Self-assess.   What kind of motivator are you?
--How satisfied are you with your motivational strategy?   Is it working as you would like?   Are you getting the results that you want?    That is the ultimate question.   If the results are not what you want, how can you tweak your motivation?   Do you need to push more or guide more?   How do you motivate?

Interesting questions to consider by any teacher.

When I talk about motivation at presentations, someone in the audience will invariably ask me whether I am a football coach or a scout leader.   My response is always the same because I have thought about this for many years.   Teachers in college have very limited time to accomplish their goals.   In a normal semester, I am only with my students for 150 minutes per week for about 14 weeks.  I have to get my students up and running very quickly.  

I start out each new semester as a football coach.   I tell the students exactly what I want them to do and demand that they comply.   I want my students always well prepared for class so I require preparation.   I want my students to develop stronger critical thinking skills so I prepare tough questions that I require them to work in order to reach logical solutions.

However, almost from the first day, I begin to slide over toward the scout leader model.   I want the students to become more responsibility for their own learning.   I only want to be a football coach for the first few weeks.   Gradually, I move into more of a guidance model.   The transition cannot be too quick or the students will become confused.   However, with practice, the pushing and demanding can morph into guiding and enabling.  

For me, teacher-centered education is okay to get the group started well but needs to become student-centered education by the last part of the semester.   I want each student to get off to a great start and that is easier for me to do as a football coach.   By the end of the semester, I want the students getting actively into their own education—not because I demand it but because that is what they want to do.  

But that is just how I like to work.   You have to decide for yourself whether you are suited for the “football coach” model or the “scout leader” model or possibly something in-between.   If any person is going to walk into a classroom as a teacher, some level of motivation (either for good or bad) is going to take place.    You ought to consider what type of motivation is best for you and for your students.