AP Reading in KC

I've been gone for awhile, away from a computer, one of three such "cyber-sabbaticals" I will take this summer. For the last 9 days, I've been in the grand Midwestern town of Kansas City, Missouri grading math problems. As a member of the elite, national force of 800 college professors and high school math teachers selected to grade the 350,000 AP calculus tests, I spent 7 consecutive days grading tests for about 8 hours each day (returning each evening to my great view from the 14th floor of my hotel overlooking the city--see above.)

The free-response questions we were grading consisted of 6 questions, each with multiple parts. Because the route to a correct answer can vary, not to mention the verbal reasoning and justification, Educational Testing Services (ETS) hires us math teachers to interpret the answers and award no, partial, or full credit appropriately. To ensure that each "reader," as we're called, arrives at the same score for each problem (out of a possible 9 points each,) we are briefed extensively on the grading standards for each problem. Sitting in a large auditorium, the "question leader" goes over the philosophical intent of the problem, quickly glides through the typical, most ideal version of the problem and how to score it, then goes to the exceptions.

Looking at several samples of students work, we are trained to see particularly interesting versions of the problem and how to score them uniformly. In my case, I'm listening to the question leader with one ear, my "table partner," sitting next to me with the other ear, while I frantically write down all special cases on my notebook paper, all while my mind is processing everything. After we've worked through several "dry runs" together of grading sample papers, hopefully all arriving at the same score, we head back to our grading rooms, which are nothing more than tables set up in a huge convention center with only curtained partitions between groups of tables.

At the table, we sit in groups of 16 in a rectangle, each grading the same problem. At the head of our table are the more experienced readers known as table leaders, who are there to make sure the rest of us are doing our job and to answer questions about dubious methods. And so the actual grading begins. We are each given a folder containing 25 test booklets, on which we are not allowed to write. We take each booklet, flip to the appropriate problem, then mentally "read" the student's work for each section in that problem, keeping a mental tally of the points (up to 9) that they have earned. We then record that total on a bubble-sheet that corresponds to that particular test in that particular folder that came from that particular box. . .

Once we work through all 25 booklets, we trade in the folder for another folder with 25 more booklets. And so the process continues ad infinitum. . . . or until that problem has been graded for all 350,000 booklets (about 2 days!!) Some readers naturally read faster than others, and with increased speed comes the potential for human error. Each reader is paired up with a table partner, whose primary role is to be a person to bounce ideas off of and to help you interpret "shady" methods. You spend a lot of time talking to this person over the course of the week, and I have been fortunate to have had amazing partners the last two years (believe me, not everyone is compatible with the person they are randomly paired up with!)

As you can imagine, there are checks and balances. A very important role of the table leader is to "back read" at least the first completed folder for each problem of everyone at their table. This means that they go through all 25 tests and score them themselves, comparing their arrived-at scores to yours. If there is a discrepancy, they point it out to you and ask you to justify your score. If they are moved by your argument, they honor your professional judgment and leave the score. If it's an obvious violation of the standards, they record their own score (which is a strike against you.) Some readers are so paranoid when they earn a "strike," thinking that their pay will be docked, they'll have to stay later, or that it might ultimately hurt their chances to some day be promoted to table leaders themselves. I've no such ambitions, so I just move at my own pace, doing my own thing. I've been fortunate the last two years to have had the very nicest table leaders who have been gracious and diplomatic in telling me that I've read a problem too leniently or too strictly.

I calculated my average pace to have been one folder every 15 minutes, which is 100 problems each hour, or roughly 800 problems each day. Over the 7-day grading frenzy, I've estimated that I read/grading 5,600 problems. With 350,000 tests each containing 6 problems to grade, that comes to 2,100,000 total problems to be graded by 800 people, bringing the average to be read by each reader to 2,625 problems per person. Needless to say, I graded my fair share. Also needless to say, I'm glad it's over and I'm glad I'm back home. But being a math teacher who enjoys grading papers (for money!) with 799 of the best, most enthusiastic mathematicians from around the nation, I'm looking forward to doing it all over again next year.