The surprisingly reflective article in today’s New York Times (“Hurdles Emerge in Rising Effort to Rate Teachers,” December 27, 2010) asks a question that statisticians often overlook in the evaluation of teachers. Who taught this kid?
John White, Deputy Chancellor of the New York City Schools, addressed the problem of teachers getting ranked on subjects they did not teach or being omitted from the rankings altogether. This is more of an institutional problem, but it raises the more consequential issue of how to determine whether a teacher has been truly responsible for a student’s learning.
Mr. White reassured the Times that “before the next rankings was released, teachers would be able to review class lists to verify which students they taught, a practice that generally did not happen in the past” (A13). But a more complex question that I rarely hear addressed by statisticians is: How many school days should a student be in class before he or she can be considered a student of that teacher? Is 90 out of 180 days enough? 70 out of 180? 50 days?
In city secondary schools this is not an academic question. I can not cite individual attendance records for New York City schools, but 50% or lower attendance would not be irregular in an urban high school. Should a teacher’s effectiveness be judged from the performance of a student who is present less than half the time?
Statisticians always assume that variables like school attendance will even out, if a large population is studied. Everyone will have attendance issues in their classes. But school attendance closely correlates with the socioeconomic status of secondary school students, because poverty often affects how much students have to work or baby-sit for caretakers who have to work. So, unless each class has a random distribution of students from various socioeconomic classes, their rate of attendance will not be randomly distributed.
Every teacher knows which classes in a school will present attendance or behavior problems, and they will use every device in their repertoire to avoid teaching them. But someone will teach them, and they will be challenged by the discontinuity of learning and wearied by the endless make-up sessions that absent students demand. They will work harder with less results, and they will be judged inferior for the test performance of students who were not in school enough days to really learn their subject.
As a writing teacher, I’ve been acutely aware of what happens to a student who does not attend school for ten consecutive days. Consistent with good writing pedagogy, I might assign a major writing assignment over two weeks, allowing students time for brainstorming, drafting, conferencing with peers and instructor, proofreading and final publication. Writing teachers understand that critical instruction, including grammar lessons, takes place in the midst of this process. Students learn very little from the final draft, returned with dozens of red marks and a “D.” They learn about as much as a traffic officer learns from the wreck of an accident after the fact.
If a student misses five out of the ten days that a writing assignment was in progress, he or she will probably miss the critical instruction that occurs during the process. It is not a question of grabbing the assignment sheet and catching up at home. It is a question of producing a draft and getting appropriate feedback before it is ultimately due and graded. It is a question of understanding why the writing that received a “B” last year can not qualify for a “C” this year. It is a question of how to address a sentence structure problem that has persisted for four years, without sitting down with that student for fifteen minutes before the final draft is due. This is why you can hardly teach a student with marginal literacy, when he or she is absent 50% or more of the school year.
I’m sure teachers of other subjects, for which learning is a seamless process, could testify to this problem. I wouldn’t presume to speak for them, but I am sure that discontinuity of learning is a problem across subjects.
People who administer schools seem to have forgotten that when learning is interrupted, it is very difficult to grab the threads again when the absent student returns to school. Your heart sinks when a student returns from a two-week absence and asks at the beginning of class, “What did I miss?” Then you tell the student he or she will have to return at the end of school to catch up on missed work, and the student forlornly reports, ” I have to go home to babysit for my mom. She works the afternoon shift at the hospital.” So you hand the student the assignment and steal a few minutes during class for a briefing that took twenty minutes one day of class and five minutes follow-up each day of the process. Only this students gets two minutes.
Not to get too sentimental about disadvantaged students, this vignette serves to show the impact of attendance on learning. The fewer days students attend your class, the less impact you will have you their learning. Sounds logical enough, but where is that considered in the byzantine complexity of value-added assessment? How do we decide who, if anyone, taught this kid?