The most troubling question about the study “Academically Adrift: Limited Learning on College Campuses” is that Bob Herbert, New York Times columnist, like most of his colleagues, does not for a minute question whether the “critical thinking” of college students has been adequately measured by whatever assessment was administered.

It is indeed troubling that “Thirty-six per cent of the students said they studied alone less than five hours a week.” And perhaps alarming that the same students are pulling an average 3.16 GPA.  There are some disclaimers that might be made about “studying alone,” because colleges encourage study groups and collaborative effort is considered a crucial skill of the marketplace.  Did they ask how many hours were spent studying in groups?

But more troubling is the undisputed claim that “after the first two years of college, 45 per cent of the students made no significant improvement in skills related to critical thinking, complex reasoning, and communication” and that two years later the percentage had only improved to 36 per cent.

Does anyone know how these critical skills were assessed? Has anyone taken such a test, in which “critical thinking, complex reasoning and communication” were validly measured? I have not investigated the testing instrument used in this study, but I would think someone would, before proclaiming that college students are dumber than they used to be.

Suppose the assessment of critical reasoning was the time and accuracy it took to fill in a crossword puzzle with the content knowledge expected of college students? Suppose it was a version of the Miller Analogies test, with content expected of a liberal education (be sure to cover art history and music) ? Suppose it was a thirty-minute essay question asking for the causes of terrorism in the Western world? Ask yourself, college graduates, do you want your critical reasoning skills assessed on any one of these tests?

The irony is the utter neglect of critical evaluation of a study that purports to measure the critical reasoning of college students.  How can we claim to know such things without knowing the nature of the assessment? The news media are the most uncritical arbiters of news about education in the literate world. They accept every test at face value. Heaven forbid we might test journalists this way.

I admit I am disturbed by the findings of Richard Arum and Josipa Roksa, because, regardless of the validity of their thinking assessments, where there is smoke, there is fire.  I try to learn what I can from studies that probe the learning habits of college students.

But I think it either hypocritical or delusional that the news media in general and Bob Herbert, in particular, would report on studies of student competence and knowledge, without asking if the tests were valid.  This is a classic case of uncritical thinking.